New Zealand lies astride two tectonic plates, the Pacific and the Australian. This gives us the Taupo Volcanic Zone in the North Island. In the North, the Pacific plate is subducting (diving under) the Australian plate. Material from this which melts at depth is the cause of the TVZ above.
In the SW of the South Island, the Australian Plate dives very steeply under the pacific one. The region in between these two opposing subductions has caused the Southern Alps to rise. Left to itself without erosion, Mt Cook/ Aoraki would be some 20km taller than it is.
The above picture gives a sound basis for understanding the development of the geographic features in New Zealand. Climate, and long standing sea level fluctuations go a long way to underlie the remaining features.