Start with something substantial to give form to the bottom or base of the duffle bag. Shoes, boots or slippers serve this purpose well.
Roll and pack bulky things, such as sweaters or sweatshirts. Follow with lighter items, including pants, short and T-shirts, all rolled as well.
Fill empty spaces as you go with underwear, socks and other soft items.
Pack your toiletries in plastic bags with zip tops, and put them near the top of the duffle. These bags prevent liquids from spilling onto your clothes, and they are also handy for carrying loose items.