Start downtown with the RiverPlace fountains. Located just off Main Street, these are series of water jets and sprayers that are part of a cascading series of steps leading to the Reedy River.
There are also several state parks in Greenville that have lakes for swimming. The two best (and most accessible) are the ones at Table Rock State Park and Paris Mountain, which may be the better of the two because it is located about five miles from downtown.
If lakes aren't your thing, then check out some of the pools in the county's recreation district. The best is Discovery Island, which has water slides, a lazy river and a splash pool for younger kids. One bit advice is to call ahead. The crowds ( and lines) get long on the weekends.
If you like something more secluded, try the Saluda River runs on the western edge of Greenville Count, and has plenty of shoals and inlets accessible from main roads. However, watch out for canoers and kayakers on this river.
