Yes, the land that became New Jersey was initially included in the grant to the Duke of York (later James II) in 1664, but it was almost immediately separated from New York's governance.
King Charles II granted his brother, the Duke of York, a vast territory stretching from the Delaware River to the Connecticut River. This encompassed New Netherland (New Amsterdam) and the lands that would become New Jersey. However, before the Duke took formal possession, he conveyed the land between the Hudson and Delaware Rivers to two loyal courtiers: Lord John Berkeley and Sir George Carteret. They named it New Jersey after Carteret's defense of the Isle of Jersey in the English Civil War.
The two provinces had separate governments, capitals (Perth Amboy and Burlington), and land systems, leading to confusion and disputes.
In 1702, the proprietors of both East and West Jersey surrendered their governing rights to the Crown. Queen Anne united them into a single Royal Colony of New Jersey, governed by a royal governor (shared with New York until 1738, then separate) and an elected assembly.
While the Duke of York's original charter technically included the region, New Jersey was never functionally administered as part of the colony of New York. It was carved out as a distinct proprietary colony within months of the English takeover from the Dutch.