The phrase is often used to express a sense of nostalgia for a lost era, or to compare something to the great civilizations of the past. It can also be used as a warning against the dangers of hubris, and the idea that no civilization is invincible.
In the context of the poem, Byron is reflecting on the fall of the Venetian Republic, and how it once rivaled the great civilizations of Greece and Rome. He is lamenting the loss of its glory, and expressing his fears that other great civilizations could suffer a similar fate.
The quotation 'glory that was Greece grandeur that was Rome' is a powerful reminder of the transience of power and the importance of cherishing our cultural heritage.