Girdled by a wide moat, Broughton Castle was originally built in 1306 by Sir John de Broughton, who lies buried in a painted tomb in the 14th-century parish church. Always a fortified manor rather than a true castle, the building was much enlarged by the Fiennes family in Tudor times.

With house, gatehouse and church spire all harmonised by the same lichen-dappled stone, the grouping is wonderfully picturesque, and Broughton has treasures to match its appearance. In England's Thousand Best Houses, by Sir Simon Jenkins, it features in the top 20 sites honoured with full, five-star rating.

Lying among low, green hills about three miles from Banbury, Broughton is somewhat secluded from the modern world, which perhaps contributes to its popularity as a film location. Back in 1976 it furnished a backdrop for Tony Richardson's Joseph Andrews in which the front lawn was graced by a parade of topless nuns! Broughton was later used in The Scarlet Pimpernel and The Madness of King George. But the castle's star performance was perhaps in the multi-award winning Shakespeare in Love (1998) where it played family home to Shakespeare's paramour, Viola, played by Gwyneth Paltrow.

The Oak Room, Great Hall and Gatehouse all appear in the film, and the balcony scene, in which Shakespeare proclaims his love, was shot at Broughton too though the balcony itself had to be specially built out of timber.

Pleasingly, Shakespeare himself was played by the actor Joseph Fiennes, related to the family who have lived at Broughton Castle since Tudor times. He and his twin brother Jacob used to play there when they were young.

Stone-carved corbel heads in the Groined Passage survive from the original mediaeval building, the timeworn faces perhaps representing family members, or the masons who built the castle. Other images also catch the eye: a musician blowing a horn; a muzzled bear; a bat-like creature; some writhing semblance of a dragon.

Here too is a Green Man with leaves streaming from his mouth, a pagan motif apparently symbolising the triumph of green life over winter and death.

Creatures of the mediaeval otherworld, they speak of primal shades that still haunted minds in an age of Christian faith. A staircase leads up from the passage to a private chapel dating from 1331, when a licence was granted to John de Broughton to hold Divine Service at his home.

The handsome Gatehouse was built towards the end of the 14th century after the Castle was bought by William of Wykeham, Bishop of Winchester. Not long afterwards, Broughton acquired crenellations or battlements at the top of the walls.

Work evidently began around 1406, for royal permission was needed to crenellate a building and in that year Sir Thomas Wykeham received the necessary licence. Though the battlements were basically decorative, they could serve for defence if required. Inside the gatehouse are narrow slits through which archers could shoot their arrows.

In 1448 Broughton Castle passed by marriage to the Fiennes family, Lords Saye and Sele, who transformed the mediaeval manor into a Tudor mansion, nearly doubling its size in the process. Two new storeys were built and state rooms were added, with new bay windows at the front and stair towers at the back.

Steeped in the romance of nobility, Broughton might be assumed to have been naturally Cavalier during the Civil War. But not so: North Oxfordshire had strong Puritan sympathies, and nearby Banbury was a hotbed of zealotry. William Fiennes, the 8th Lord Saye and Sele, was himself a convinced Puritan and Parliamentarian. In the reign of Charles I, he hosted many secret meetings at his secluded Castle, to coordinate opposition to the King.

When war broke out William and his sons garrisoned their castle-home for Parliament, also raising a regiment of blue-coats and four troops of horse who fought at nearby Edgehill in 1642. After the battle, however, Broughton was besieged, battered and taken by the Royalists. Cannonballs, recovered from the moat, are on display in the Council Chamber where William had hosted his secret meetings.

We know little of William's personality except from the writings of Royalist adversaries. One of them called him a seriously subtil peece, and always averse to the Court wayes, something out of pertinaciousnesse'. The reputation for canniness survives in his nickname: Old Subtlety.' He was no extremist. When fortunes turned against the King, he would play a role in trying to bridge differences between Royalist and Parliamentary factions. William disapproved of the execution of Charles I and retired under the Commonwealth from all political activity. On the Restoration of the monarchy he was pardoned by Charles II and made a privy councillor.

William's portrait hangs in Broughton's Great Hall whose bare masonry walls are also decorated with arms and armour from the Civil War. Elsewhere are many fine relics of Tudor and Stuart times: fireplaces, portraits and plaster ceilings. Queen Anne's Room takes its name from the wife of James I, who slept here in 1604. The surprise feature is a squint in the northeast corner, looking down into the mediaeval chapel.

The King's Chamber is named after James I himself, and has a tremendous renaissance chimneypiece with a carved overmantel showing dryads dancing round an oak. Its impact is matched by the busy, splashing colours of 18th-century, handpainted Chinese wallpaper. Enter the room and you experience a total immersion in exotica, climaxed by the arcs of an astonishing cantilivered bed. This was made of oak as recently as 1992 to theme in with wallpaper and plasterwork.

Family portraits line the walls of the 18th-century Gothick Long Gallery, and memorabilia in the Great Parlour includes a top hat presented to Cecil Fiennes, a Victorian son of the house. During a cricket game in 1859 he bowled out three batsmen with successive balls - one of the earliest recorded hat tricks'.

The tour ends in the Oak Room downstairs where an unusual internal porch bears the Latin inscription Quod olim fuit meminisse minime iuvat' (There is no pleasure in the memory of the past). Was it placed there by Old Subtlety at the time of the Restoration, as an epitaph on the troubled years of war? That can be the only explanation, for in reality there is infinite pleasure to be had in the memories woven into the soul of this house.

Broughton is not the largest stately home in Oxfordshire but it is for many people the best, a place rich in its heritage yet wearing its pedigree lightly. Never over-commercialised, it remains very much the family home of the present Lord and Lady Saye and Sele who I found chatting amiably and informally to visitors on the day of my arrival. One of the guides told me: It is unique, really, and I think we all feel the same excitement when we arrive to open up. It is such a privilege to introduce people to the house.' Broughton Castle is open from 2pm-5pm on Wednesdays and Sundays between 30 April and 15 September; also Thursdays in July and August; and August Bank Holiday Monday. For further information visit www.broughtoncastle.com or phone 01295 262624.

The room that hath no ears' mystery feature The English Civil War was planned in top secret meetings at Broughton. Its owner, William Fiennes, was a prominent Parliamentarian, a reported oracle of the Puritans' and a very Godfather of the Party.' In 1640 and 1641 he hosted many hush-hush debates about taking up arms in what is now called the Council Chamber. Dubbed the room that hath no ears' it was a small, low chamber high up at corner of the house, reached by a staircase leading directly from the grounds. The conspirators arrived by night, unseen by servants or guests.

John Hampden, John Pym, Lord Brooke, Sir Henry Vane and many other top Parliament men assembled here under cover of darkness. When they were of a complete number,' wrote diarist Anthony Wood, there would be great noises and talkings heard among them, to the admiration of those who lived in the house, yet could they never discover my lord's companions.' It may be that rumours of the secret comings and goings inspired persistent belief in an underground passage leading away from the Castle. Folklore has variously ascribed its hidden entrance to quarry workings as far away as Bloxham, and Bretch Hill outside Banbury.

I heard visitors discussing the supposed tunnel during my own visit to Broughton, and the administrator, Julia Moorhouse, told me that the house has received emails from America on the subject. Someone had traced their family history back to Broughton; one of their ancestors was supposed to have attended the secret meetings. They were convinced that the tunnel was real. But where would the tunnel go if not under the moat? At 6ft it's too deep, and it's been here too long.' The mind boggles at the engineering resources which would have been needed to build such a subterranean passage, and to maintain it from flooding. But the truth is that there is no tunnel. This is one house with a real enough secret history; it needs no myth to enhance its fascination.