Who started back-to-Africa movement?

The phrase was used as early as 1859 with reference to various abolitionists; and it is usually used with relation to African American intellectuals during the late 19th and early 20th century for instance Marcus Garvey (1919), and Alexander Crummell in relation to his work Africa and America: Addresses and Discourses (1891). W. E. B. Du Bois coined the term later and used it frequently.