According to Chalmers, the founder of this great family came from England in the reign of David the First (1124-53), and obtained the lands of Gordon. He left two sons, Richard, and Adam, who had the territory of Gordon, with the lands of Fanys.  The elder son, Richard de Gordon, died in 1200. He was succeeded by his son, Sir Thomas de Gordon and his son, also named Thomas, died in 1285, and his only daughter, Alicia, married her cousin Adam de Gordon.

His grandson, Sir Adam de Gordon, Lord of Gordon, one of the most eminent men of his time, was the progenitor of most of the great families of the name in Scotland. In reward for his faithful services, Bruce granted to him and his heirs the noble lordship of Strathbogie, in Aberdeenshire, by the forfeiture of David de Strathbogie, Earl of Athole.  Sir Adam fixed his residence there, and gave these lands and lordship the name of Huntly, from a village of that name in the western extremity of Gordon parish. From their northern domain, the family afterwards acquire the titles of Lord, Earl, and Marquis of Huntly, and the latter is now their chief title. Sir Adam was slain, fighting bravely in the vanguard of the Scotch army at the battle of Halidonhill, July 12, 1333. By Annabella, his wife, supposed to have been a daughter of David de Strathbolgie above mentioned, he had four sons and a daughter. The eldest son, Sir Alexander, succeeded him. The second son, William, was ancestor of the Viscounts of Kenmure.

Sir Adam, lord of Gordon, fell at the battle of Homildon, 14th September 1402. By his wife, Elizabeth, daughter of Sir William Keith, Great Marischal of Scotland, he had an only child, Elizabeth Gordon, who succeeded him.  She married Alexander Seton, second son of Sir William Seton of Seton, and he took his wife's name was styled lord of Gordon and Huntly. He left two sons, the younger of whom became ancestor of the Setons of Meldrum.  Alexander, the elder, was in 1449 created Earl of Huntly. George, 6th Earl, was created Marquis of Huntly by King James in 1599. George, 4th Marquis, was made Duke of Gordon in 1684. George, 5th Duke, died without issue in 1836. At his death the title of Duke of Gordon became extinct, and the Marquisate of Huntly devolved on George, Earl of Aboyne.