Until 1408, The Lairds of Touch were Frasers. The estate was aquired by a branch of the Setons in Aberdeen, by a son of the 1st Earl of Huntly, and the tower is thought to have been built in the 15th Century. Originally it would have been larger and there would have been no windows on the ground floor.

In the 16th Century, part of the tower was pulled down and the house extended to make a more substantial, fortified house. The North side of the top floor was added in the 17th Century.

Construction of the South front was started in 1758 and the plans may have been prepared by William Adam, although there is no record of this. The oval staircase is a brilliant design that might well have been the work of William's son, Robert Adam. The Buchanans, became owners of the estate in 1928 but the Seton coat of arms can still be seen on the pediment of the South front.

The Setons of Touch were supporters of the Jacobites and were the Heriditary Armour Bearers for the Scottish Kings. In 1745, on his way to the Battle of Prestonpans, Prince Charles Edward Stuart stayed at Touch.  When fleeing after Culloden, he is said to have found refuge in the cave under the waterfall in Touch Glen. It would have been far too dangerous to stay in the house with Seton's Jacobite sympathies well known.

The old road from Stirling to Glasgow passed within 200 yards of the front of Touch House and until the middle of the 18th Century, the road skirted an impassable swamp. After the failed 45 Rebellion, Hugh Seton and other local lairds, brought families down from the Highlands to work on the mammoth task of draining the Carse of Stirling. Ditches were dug to float the peat from this bogland down to the River Forth and eventually out to sea. Around 60 square miles were reclaimed, exposing rich, clay soil beneath.

The estate covers 4,500 acres of woodland and farms. The 5 carse farms were created after the reclamation in the 18th Century. The walled garden and the woodland garden contain over 150 different species of rhododendrons and azaleas, as well as many other interesting trees, shrubs and herbaceous plants. Forth Valley Orienteers, the Boy Scouts and the army use the estate and there are occasional guided walks with the Countryside Rangers. The public are welcome to make use of the the estate and Touch House has been open to the public on Doors Open Days.