The Black Door: Spies, Secret Intelligence and British Prime Ministers

‘Scandals are the nastiest experiences in politics,’ noted Bernard Donoughue, senior policy adviser to both Wilson and Callaghan. In September 1977, reflecting on the latest press revelations about alleged attempts by the intelligence services to subvert or even overthrow the prime minister, the so-called ‘Wilson plot’, he added that these affairs brought together ‘scared politicians, hysterical and self–righteous civil servants and hypocritical lying journalists’.15 Intelligence scandals have long caused sleepless nights in the flat above 10 Downing Street. Almost as soon as he entered office, Attlee was confronted with revelations about the ‘atom spies’, Alan Nunn May and Klaus Fuchs, two wartime scientists employed by the British who had seemingly passed the secrets of the West’s new wonder weapon to Moscow. When Stalin detonated an atomic bomb ahead of the British in August 1949, there were awkward public questions in Parliament, made worse by the mysterious flight of Donald Maclean and Guy Burgess to the Soviet Union in 1951. Security scandals had an alarming habit of appearing out of a clear blue sky, as Anthony Eden discovered just a few years later. In 1956, he clumsily attempted to bat away questions about the mysterious death of the navy frogman ‘Buster’ Crabb, who during a state visit to Britain by the Soviet leadership had embarked on a hazardous secret operation that MI6 had failed to clear properly with the political authorities. The Crabb fiasco triggered a major review of prime ministerial clearance for clandestine operations and new mechanisms for assessing attendant political risk.

The 1963 Profumo affair was a defining moment for Downing Street. It not only helped prompt Macmillan to terminate his premiership early, it also heralded a new attitude by the press towards matters of secrecy. Hitherto, the long shadow of the Second World War had ensured a degree of circumspection and security-consciousness by journalists when addressing intelligence matters. But by the 1960s, propelled by a range of scandals around CIA special operations, the press was hungry for stories about espionage. The shooting down of Gary Powers’ U-2 spy plane in May 1960 reverberated personally for Macmillan, collapsing a Cold War summit with Khrushchev that he had worked hard to create. This was followed by the CIA’s disastrous Bay of Pigs escapade, which marred the first months of Kennedy’s presidency and raised questions about his personal judgement. Profumo too had an Anglo–American dimension, with anxious FBI officers arriving in London to probe the delicate question of whether Kennedy had met Christine Keeler. For Macmillan, always keen to project the persona of Edwardian stability, the increasing connections made by the press between premiers, politics and the secret world became deeply disconcerting. It was a wind of change he did not welcome.

Harold Wilson had enjoyed tormenting Macmillan over Profumo. As leader of the opposition in 1963, he recognised that this episode was about sex and high society as well as secrets – a gift in terms of media coverage. Entering Downing Street in 1964, he was determined to protect himself against the same fate, and noted that his predecessor, Alec Douglas-Home, had already established a Security Commission to which he could refer malodorous matters when they came before Parliament, giving the prime minister the welcome appearance of actually doing something. For Downing Street, the tactic of creating long-running inquiries into secret matters gradually became a central mechanism for containing toxic security issues. But Wilson also wanted his own ‘security enforcer’, and appointed his paymaster general, George Wigg, to sniff out potential scandals within government. Cabinet ministers and MI5 loathed the freewheeling security inquiries that Wigg made into the most sensitive parts of Whitehall. All this failed to protect Wilson from hideous embarrassment over the interception of telegrams during the ‘D-Notice affair’ in 1967, and then the public flaunting of MI6 secrets by Kim Philby in his waspish memoirs published in 1968.

Labour leaders faced a peculiar predicament. They often found themselves publicly defending domestic security services that they privately feared might be seeking to undermine them. Attlee faced the awkward position of being a socialist prime minister at the onset of the Cold War. The Labour Party and MI5 were not, at this point, natural bedfellows, and Attlee had to balance issues of positive vetting with backbench accusations of launching an anti-communist witch-hunt. Attlee was always conscious of the tension between intelligence, security and liberty. He agonised over investigation into the personal backgrounds of Whitehall officials, and introduced it only under the American threat of ceasing nuclear cooperation, passing this into law as almost the last act of his administration. By the 1970s, Labour prime ministers were inspecting MI5 files on their own MPs and wondering what level of risk was involved in appointing them to government. It was now known by Downing Street that some MPs – like John Stonehouse – had actually worked for Eastern bloc intelligence services. Indeed, one of the longest-serving MPs in the House of Commons, Labour’s Bob Edwards, who represented Wolverhampton South-East until 1987, was a fully-paid-up KGB agent. Other MPs had worked closely and enthusiastically with the CIA or Mossad.16

Jack Straw, one of the most prominent cabinet ministers of recent years, recalls his initial security vetting when he first joined the government. ‘A man in a mac, with a skin disease which meant he could not shake my hand … came to interview me … for three hours.’ Later the same man came back for another three hours, and in this second interview he suddenly leaned across the desk, looked his subject in the eye and asked, ‘Mr Straw, do you like men?’ This reflected the fact that historically, several people who had spied for the Soviets had been trapped by sexual blackmail. In 1974, when Straw was an adviser at the Department of Health and Social Security, his file was already two inches thick, and from their questions it was clear that MI5 had been collecting material on his family members since he was fifteen. Straw reflected on the scale of the surveillance operation that this implied. But, he recalls, this ‘neither surprised nor shocked me’. He saw it as part of everyday life on the strange planet that was Cold War Britain, where the KGB had to be kept at bay. Years later, when Straw became home secretary, MI5 was nervous that he might wish to see his own file. But he did not request this privilege, taking the view that he had no more right to see this secret material than any other citizen – and he gave the same response to Peter Mandelson, who was characteristically eager to peruse his own MI5 dossier.17

In the 1970s, Downing Street also lived in the shadow of Watergate, and was engulfed by growing paranoia about grand political conspiracy. Plots and bugging seemed to be almost normal. In March 1976, during the last days of the Wilson government, Tony Benn, then secretary of state for energy, attended a reception at the American embassy at which he chatted amiably to Cord Meyer, head of the CIA station in London. They reviewed the continuing fallout from Watergate, and Benn offered the opinion that Nixon was in fact quite charming, and that the media had been ‘unfair to him’. Meyer countered that Nixon was a ‘terrible man’, and had done a lot of damage to his service. But Benn took the wider view that given the catalogue of human error that went with the political experience, ‘bugging your opponent wasn’t so bad’.18 A year later, James Callaghan and the cabinet secretary were debating whether Benn himself could be trusted to see intercepts and ‘sigint’ material from GCHQ.19

Labour prime ministers also worried about plots. Ramsay MacDonald famously feared that MI5 was working against him. During the 1930s, Neville Chamberlain employed a former MI5 officer, Joseph Ball, to spy on both the Labour Party and his own rivals within the Conservative Party, using human agents and telephone taps. The Harold Wilson government rightly feared that a number of secret elements – both domestic and foreign – were seeking to destabilise his regime. For reasons yet unknown, Harold Macmillan had insisted that Downing Street be wired for sound to allow recording of conversations, much in the same style as the Kennedy White House. The extent to which this was used or abused during his time in office and subsequent administrations remains a mystery. The sensitivity of this subject was such that the Cabinet Office insisted that all references to it be cut out of what was otherwise a remarkably candid authorised history of MI5 published in 2009.

Harold Wilson was not the only senior figure who feared bugs. In 1973, in the final days of Edward Heath’s Conservative government, William Armstrong, the head of the home civil service, turned up in the Cabinet Office and demanded to speak to the cabinet secretary somewhere that was ‘not bugged’. Ushered into a suitably secure room, he took off his clothes, lay on the floor chain-smoking and talked ‘very wildly’ about the whole system collapsing and ‘the world coming to an end’. The next day he summoned a meeting of all Whitehall permanent secretaries and told them to prepare for ‘Armageddon’. ‘He was babbling incoherently’, and was ‘taken off to hospital for treatment’. He later spent a month recovering at Lord Rothschild’s private villa in Barbados. Although Wilson said almost nothing about the security services in his first volume of memoirs, penning a bizarre chapter of just two and a half pages on the subject, he later said much more to the press. Cabinet Office officials were by turns amused, embarrassed and then dismissive when these stories first appeared. But by 1979, evidence of interference by South African intelligence in London was mounting, and officials gradually came to accept that there was real substance to Wilson’s fears.20

Wilson’s public ramblings on intelligence required a new approach by Downing Street. Since the early 1960s, prime ministers had been forced to confront a new era of exposure. Now, a decade later, they had begun to manage their own gentle counter-offensive. Encouraged by the cabinet secretary, Burke Trend, with whom Wilson had enjoyed a good relationship, the intelligence agencies initiated a deliberate policy of emerging from the shadows. The Cabinet Office presided over the writing of the official history of wartime intelligence, and approved the release of the papers of Bletchley Park. This project was expressly about intelligence at the top, and traced the interaction between secrets and high strategy. Throughout the 1970s, British prime ministers were feeling their way towards a more public profile for the security agencies and their own engagement with them, pondering the possibility of public avowal of their existence and activities.

Margaret Thatcher hated this. Even as opposition leader in the late 1970s, she had repeatedly attempted to veto any public revelations about intelligence, however carefully controlled. Once she entered Downing Street, she was immediately confronted by the media frenzy surrounding the revelation that Anthony Blunt, Surveyor of the Queen’s Pictures, was the ‘Fourth Man’. Tasked with explaining the decision not to prosecute him, she went against her instincts and, rather than saying as little as possible, made a detailed statement to the House of Commons. She soon regretted it. Each morsel of detail was picked over, and seemed to draw out further press revelations. Now confirmed in her personal commitment to absolute secrecy, Thatcher’s later years were partially defined by the absurd battle with MI5’s supreme mole-hunter Peter Wright, whose Spycatcher memoir she sought to suppress. Her cabinet secretary endured public humiliation in the Australian courts in a futile attempt to hide what was already in the public domain.

By contrast, John Major found himself surrounded by modernisers. Not only were the intelligence chiefs of the 1990s happy to adopt an avowed legal identity, they were also keen to emerge from the shadows. In 1994 the new director-general of MI5, Stella Rimington, gave the BBC Dimbleby Lecture on ‘Security and Democracy’ – despite the intense unease of Home Office officials. MI6 opened an extraordinary new building at Vauxhall Cross, on the south bank of the Thames, that looked like something out of science fiction. Not to be outdone, GCHQ then commissioned its own new headquarters that looked like a flying saucer, and became the first British agency to have a presence on the internet. Yet the press still seemed determined to turn success into scandal. In 1999, when MI6 revealed the treasures of the Mitrokhin archive, compiled by a KGB archivist who had given his secrets to Britain, surely one of the most magnificent intelligence successes of the late Cold War, the press instead focused on the failure to prosecute some of the espionage ‘small fry’ this had revealed. Government spin doctors puzzled over how MI6 had managed to turn a golden story to ashes, and Duncan Wilson of the Cabinet Office agreed that the secret services now needed assistance from Downing Street in managing their public image.

Tony Blair exuded confidence in all areas of public affairs, including secret service. When asked about MI6 operations in Moscow he quipped to a waiting band of journalists, ‘We never comment on intelligence matters … except when we want to, obviously.’21 Even before 9/11, his public relations team were fascinated by the media operations of MI6 and the interface between openness and secrecy. After the attacks on the World Trade Center in New York, secret matters were on the front pages of the newspapers each week during a decade-long ‘War on Terror’. This raised important questions. To what extent should prime ministers place intelligence in the public domain to explain policy? And how should this be done? In two notorious Downing Street dossiers, Tony Blair and his press secretary Alastair Campbell did this badly, and paid a considerable price in terms of trust and public confidence. In the background, senior MI6 officers, some of them modernisers like Richard Dearlove and some old-school types like John Scarlett, argued over issues of public image and the proximity of their own service to Number 10. None of them had any idea how disastrously this episode would explode in their faces.

Yet David Cameron retraced Blair’s footsteps. In early 2013, faced with the issue of the use of chemical weapons against its own people by the government of Syria, Cameron released JIC material to the press, going even further than Blair in the public use of intelligence. He realised that prime ministers must balance intelligence’s inherent need for secrecy with their public duties leading the government. If this was an effort to reassert control and confidence, it surely failed. After Blair’s Iraq fiasco, the public were deeply sceptical about intelligence as a justification for pre-emption, and Parliament voted against military action in Syria. Moreover, Downing Street was about to discover that it did not have control over what became public in this realm. Even as it agonised over the release of a two-page letter from Jon Day, the chair of the JIC, to Cameron, an unknown figure called Edward Snowden was preparing to release a deluge of secret material that amounted to thousands of documents. Much of this was British, and each remarkable revelation was more eye-wateringly secret than the last.22

Cameron said almost nothing about Snowden. This was because much of the material shone a harsh light on the personal interaction between premiers and espionage. In a world in which diplomacy has become increasingly personal, characterised by G7 summits and mobile-phone calls, spies and statesmen rub shoulders with increasing frequency. For wealthy and powerful states, Snowden’s revelations were a story of hacked emails and rude words. GCHQ was specifically accused of hacking into the Belgian telecoms firm Belgacom, which includes EU institutions as clients. Cameron refused to answer questions about whether he had been able to reassure allies that British intelligence had not been involved in any bugging.23 In fact, this was hardly news. Generations of British cabinet ministers have been amazed by the scale of Britain’s surveillance of its European partners.24 But what Snowden, and previously Chelsea Manning, also revealed was the jostling of less powerful states, such as Chile and Mexico, which suffered not only stolen briefcases, but also bullying and bribery by the West. This rather raw business of personal espionage has now been exposed, and to the surprise of many, we have realised that James Bond often takes his orders in these delicate matters direct from the prime minister in Downing Street, rather than from ‘M’ in MI6 headquarters.

In the summer of 2007, the British embassy in Baghdad received five bloody index fingers through the post. They were claimed to be the body parts of five British citizens who had recently been kidnapped by a pro-Iranian offshoot of Hezbollah in Baghdad. On 29 May that year, Shia militants disguised as policemen had conducted a violent raid on the Iraqi Ministry of Finance. Their main target was Peter Moore, a British consultant who had been installing software designed to prevent fraud surrounding the billions of dollars of aid flowing into the new government’s accounts. Downing Street was determined to secure Moore’s release. SAS and MI6 representatives attended four separate cabinet-level security conferences to plan his rescue with ministers, and as a result special forces conducted more than two dozen house assaults seeking persons involved in the kidnap. Gordon Brown came within minutes of ordering a raid on the kidnappers, but the intelligence on their location was judged to be too thin. Eventually, in December 2009, after 946 days of captivity, Moore was swapped for Qais Khazali, a Shia militant leader, and his brother, who had been seized by the SAS on the streets of Basra at around the same time he was kidnapped. The negotiations were carried out by an MI6 officer operating in Baghdad. The other four Britons, who had served as his security guards, had all been executed the previous year.25

The dirtiest diplomacy is talking to terrorists, kidnappers and insurgents. This has to be deniable, since even if governments are willing to admit their part, terrorists frequently are not, and all parties ultimately fear being perceived as ‘weak’. This can place prime ministers in a difficult position, potentially forcing them to mislead the House of Commons. Harold Wilson kept his MI6 back channel with the IRA incredibly secret – many in his own cabinet did not know what was going on. Similarly, Margaret Thatcher publicly insisted that Britain did not talk to terrorists, and yet, as under her predecessors Wilson, Heath and Callaghan, various back channels remained open. In fact, Thatcher was personally involved in some exchanges. She made handwritten comments on one statement sent to the IRA as part of negotiations over the hunger strikers in 1981, and personally amended and approved Britain’s negotiating position.26 Her successor, John Major, faced a similar dilemma. Despite stating in the House of Commons that he did not negotiate with terrorists, he had approved precisely that, and his emissaries were secret service officers whom the IRA regarded as ‘untouchable’. But when the secret talks leaked, Major and the IRA descended into a welter of accusation and counter-accusation, with each side publicising its version of the hitherto secret communiqués.

Bizarrely, even as Thatcher and Major talked to terrorists, they were also their top targets. Both came within inches of being eliminated by the IRA, and one of the darker stories of intelligence, security and 10 Downing Street is the ever greater level of protection required to prevent assassination. Margaret Thatcher, characteristically, had just finished rehearsing a speech with her private secretary, Robin Butler, at ten to three in the morning when a bomb destroyed a large part of the Grand Hotel in Brighton during the 1984 Conservative Party Conference. One minister recalled the security shambles in the immediate aftermath, as injured politicians made their way out of the rubble with barely a policeman in sight. Two IRA gunmen could have ‘got the whole Government as they blearily emerged’, and made their getaway unimpeded. The IRA also purposely eliminated two of Thatcher’s closest advisers, with the assassinations of Airey Neave in 1979 and Ian Gow in 1990, bracketing her arrival and her departure from office.27

As a result, Downing Street was turned into a fortress. Large reinforced black gates prevented public access. Harold Wilson had resisted elaborate gates when they were suggested in 1974, but now they arrived, together with armed policemen.28 Several levels below Downing Street, technicians were putting the final touches to ‘Project Pindar’, a command bunker deep beneath the Ministry of Defence on Whitehall to which selected denizens of Downing Street could retreat in times of peril, at a cost of £126.3 million.29

By 2005, threats against the life of the prime minister had become an almost daily occurrence. The intelligence warnings of assassination attempts often landed on the desk of David Blunkett, the home secretary, who observed with delightful sangfroid that these plots did not worry him unduly – unless they related to an event at which he was likely to be sitting next to Tony Blair. Towards the end of Blair’s administration, simply moving outside the Whitehall government complex became a problem. Alastair Campbell gazed at the vast prime ministerial convoy of armoured vehicles proofed against nerve gas, the motorcycle outriders stopping the traffic amid a cacophony of wailing sirens, and wondered how many votes were lost every time this baroque parade of vehicles took the prime minister to the airport.

The personal threat to the prime minister is at its worst overseas. In October 2001, Blair set off on a trip to Russia, India, Pakistan, Saudi Arabia and finally Egypt. The BBC ignored security guidance and announced the prime minister’s destinations to the world, thereby increasing security concerns. As a result, Blair’s entourage had to be ‘surrounded by outriders with machine guns and tanks carrying anti-aircraft missiles’.30 On their return to the UK, Blair’s team told the BBC director of news that Number 10 was ‘exasperated by their continued reporting of his movements’.31 Blair remains a high-profile target. In 2014, MI5 planted a listening device in the car of one terrorism suspect and listened in on his conversations for two weeks after searching the vehicle and finding Blair’s home address on a scrap of paper folded inside a Versace glasses case.32 MI5 had previously established covert surveillance of Ishaq Kanmi, the self-proclaimed leader of al-Qaeda in Britain. His stated aims included ‘the elimination of political leaders and capitalists Blair and Brown’.33 In the twenty-first century, British prime ministers value their intelligence and security services partly because they keep them alive.

No less worrying are the threats to visiting heads of state. Each premier on a state visit to Downing Street, whether from Russia, Israel, Sri Lanka or Saudi Arabia, has brought in his wake an exotic trail of would-be assassins. The grim prelude to every visit is the backstairs diplomacy of death. As the leaders make their way around London, they are shadowed by ambulances carrying copious supplies of their blood group. The most vexed discussions concern the intelligence precautions involved in each visit, with every drain and culvert along the routes taken by visiting dignitaries being searched for explosives. Defensive weaponry is an issue, with each American president asking to bring with him an increasingly formidable array of automatic weapons and ground-to-air missiles. George W. Bush was denied permission to bring an SUV containing an M134D mini-gun in an armoured pop-up turret. MI5 routinely regard the US president’s secret service as a more dangerous threat to the British public than potential assassins.34

Death has hovered over Downing Street for more than a century. While attacks and plots have become steadily more serious in recent years, enterprising amateurs have always abounded, and earlier prime ministers were more accessible to the public. On 28 August 1939, panic swept through a crowd of protesters immediately outside the door of 10 Downing Street when a London clerk called Laurence Hislam opened a small suitcase and scattered its contents, which resembled the deadly devices beloved of bearded bomb-throwing anarchists. Mayhem ensued, until the crowd realised that the ‘bombs’ were actually large black rubber balls inscribed with the message ‘Peace Conference Now’. Hislam was in fact a pacifist protesting against the mounting international crisis and the spectre of another major war in Europe. The police led him away, and despite his defence that he acted ‘in the cause of peace’, the court sentenced him to six months’ hard labour.35