Where did the dirt and rocks come from in New York city to increase size of Ellis island?

Ellis Island was expanded using fill from nearby Liberty Island, not with dirt and rocks from New York City.