The Celts did not abandon England in 410 AD. By that time, the Romano-British population (largely Celtic peoples Romanized over centuries) was firmly established.
The Roman legions withdrew from Britain in 410 AD after nearly 400 years of occupation. Emperor Honorius told British cities to "look to their own defenses."
With Roman authority gone, Germanic tribes (Angles, Saxons, Jutes) began migrating and settling in larger numbers, eventually forming Anglo-Saxon kingdoms. The native Britons were displaced westward (Wales, Cornwall) or assimilated.
It was the Romans who left, not the Celtic population. The Celtic peoples remained, though their political structures fragmented.