Despite the name, Mile High Stadium's name does not refer to the stadium's elevation. The name honors Denver's nickname: "The Mile High City."
Denver earned its moniker because the 15th step of the Colorado State Capitol building sits exactly 5,280 feet (one mile) above sea level. A brass marker on this step commemorates the precise elevation. The nickname dates back to the late 19th century and became official city branding.
The stadium's actual elevation is approximately 5,200 feet—close to but not exactly one mile. The "Mile High" name celebrates the city's identity, not the building's precise altitude.
The question's premise about NYC skyscrapers is irrelevant: building height ≠ ground elevation. Denver's elevation comes from its location on the High Plains at the Front Range of the Rocky Mountains, not from tall structures. One World Trade Center's roof is ~1,368 feet tall, but its base is near sea level—its top reaches only ~1,368 feet elevation, far below Denver's ground level.