The Queen was possibly enjoying the company of John Chinaman as I passed through the royal family’s Norfolk estate in the days between Christmas and the new year. Mr Chinaman? Who he? Why, he’s the golden statue of Buddha in the grounds of Sandringham House, referred to in this style by Queen Elizabeth the Queen Mother in a letter (February 5, 1992) to “My Darling Lilibet”. It is an “affectionate nickname”, according to William Shawcross, the editor of Counting One’s Blessings: The Selected Letters of Queen Elizabeth The Queen Mother (Macmillan, £25). Others might take a different view. Still, at least it wasn’t “Old Slitty Eyes”.

How gleefully and gloriously politically incorrect the Queen Mum was is confirmed on page after page of this richly enjoyable book. In 1980, for instance, by which time the fur trade was generally considered shameful, she writes to Lilibet: “And now the FUR COAT! Never in my wildest dreams did I think that I would actually own one! [Really? Couldn’t afford one, I suppose.] You are such a kind and loving angel & I am deeply deeply grateful.” To Princess Margaret she writes on May 22, 1960, from Government House, Lusaka: “I went to Barotseland, which is quite delightful. Everyone falls on their knees when they see one — not grovelling but enormously natural & polite.”

As Richard Littlejohn would say, you couldn’t make it up.

The Queen Mother was in my mind more than once during my post-Christmas trip to North Norfolk in which I returned to haunts not visited for three decades and more. Our base was King’s Lynn, whose antique charm I had not explored since January 1973 when I attended an exhibition of paintings by the Bloomsbury artist Duncan Grant at the Fermoy Gallery (named in honour of Ruth, Lady Fermoy, Woman of the Bedchamber to the Queen Mother and founder, in 1951, of the King’s Lynn Festival). The QM opened the show and lent two Grant paintings from her collection, both studies of St Paul’s Cathedral, a favourite subject of the artist.

On hearing of my travel plans, friends reacted with incredulity, dismissing Lynn as a ghastly place ruined by expansion of the port and the arrival of newcomers, many from Eastern Europe. One pal said: “It’s the dump you end up in if you fall asleep on the Cambridge train.”

Wikipedia is not encouraging where prospective visitors are concerned, advising that “tourism in King’s Lynn is a minor industry and attracts a relatively tiny amount [sic] of tourists each year”.

Why this should be is hard to fathom, since there is plenty to look at and enjoy, including many superb buildings reflecting the town’s prosperous mercantile past.

One of them is the Custom House directly behind me in the photograph above. Built in 1683 to the design of Henry Bell, the building now functions as a tourist information centre, in which role it was opened by the Prince of Wales in 1999.

In front of it, facing out towards the Great Ouse, is a splendid statue of Lynn’s most famous son, Captain George Vancouver (1757-98). An officer of the Royal Navy, he is best known for his 1791-95 expedition, which explored and charted North America’s north western Pacific coast regions. The Canadian city of Vancouver (among much else) is named in his honour.

Readers might note that the attractive house in the centre of the row on the right is on the market. Sudbury House, Grade II* listed, will set you back £329,950, astonishing as this seems from the perspective of Oxfordshire.

We stayed at the Globe Hotel at one corner of the huge Tuesday Market Place. This was partly because I’d always been curious about what a Wetherspoon hotel was like but also because of a family connection with it. In its days as a Berni Inn, my mother, visiting for a late lunch, popped to the loo and emerged to find the place had closed around her. Not quite an old lady locked in a lavatory. The place was clean, if not exactly luxurious; the staff helpful and friendly.

More interesting as a building is Wetherspoon’s other Lynn pub, the nearby Lattice House, which dates back in part to the early 15th century. It was restored and returned to pub use (for the first time since 1919) by West Norfolk Council, which appears to be an admirable guardian of the town’s architectural heritage.

Aside from its own merits, Lynn is an excellent base from which to travel to more fashionable seaside haunts. I would exclude from these the village of Heacham, where I enjoyed a family holiday in 1957 (and, inter alia, had my first taste of a cheese and onion crisp). Then the landscape was littered with buildings damaged in the devastating floods of four years earlier. Now it’s a mess of horrid chalets.

No such damage has been done in Wolferton, which is set amid rhododendron-lined lanes on the Sandringham estate. I camped here with the Boys’ Brigade in 1966 and watched England’s victory in the World Cup final on a flickering black-and-white television in the local pub. At least I think it was a pub; today there is only a social club.

In the early days of her reign, the Queen alighted at a beautiful station here when she went on her Norfolk holidays. She and her family travelled in the royal train, with Edward Thompson’s B2 steam locomotive No. 61671 Royal Sovereign at its head. Now the railway line is closed, the lovely Victorian station building is a private house and our much-loved monarch takes a service train to King’s Lynn.

How times change . . .