The centerpiece of the Seton Family was the Palace of Seton, which had stood on the same spot on the Seton Baronial Lands for upwards of eight hundred years.

The original castle was a defensive one, a square tower built during the time of Seier de Lens/de Seton sometime around 1061-66.  Later, this was enlarged to become a more commodious residence, however it retained it's overall military character with French styling.  Having been repeatedly embattled, and much destroyed, it wasn't until after the Rough Wooing by England's King Henry VIII that it became known as the famed, "Palace of Seton".  It was George, 6th Lord Seton, under James V, who was responsible for the re-creation, and Mary de Guise, the French wife of Scotland's King James V, was often present.  Prior to this, the Seton's had been much involved in the affairs of Scotland's Royal Family, having the privilege of their presence on many occasions, over successive generations, with the family's munificent tastes being much sought after by the Scotland's Monarch's as a place of relaxation and refuge.

For this reason was the Seton House rebuilt into a magnificent Palatial Structure by the 6th Lord Seton, and continued upon by his son, the famed George, 7th Lord Seton, as well as their descent, the Earls of Winton.