Does the RER from Paris to Versailles travel above ground?

No, the RER from Paris to Versailles does not travel above ground. The majority of the route is underground, except for a short section between Saint-Quentin-en-Yvelines and Versailles-Chantiers stations.