The question refers to the U.S. state of Georgia, not the country of Georgia. Georgia's capital has indeed shifted from coastal cities to the interior over its history:
The moves were driven by westward population expansion, the desire for a more centrally located government accessible to all citizens, and the rise of Atlanta as a transportation and commercial center after the Civil War.