GCHQ

For one year, a large amount of very valuable documentary material concerning the work of the Americans on deciphering Soviet cyphers, intercepting and analysing open-radio correspondence of Soviet Institutions was received…On the basis of Weisband material, our state security organs carried out a number of defensive measures, resulting in the reduced efficiency of the American deciphering service. This has led to a considerable current reduction in the amount of deciphering and analysis by the Americans.29

In May 1950 Weisband was named by another agent who had been revealed by Venona and interrogated by the FBI. Although Weisband was questioned, there was insufficient evidence to charge him. There was also a fear that a court case would advertise the work of signals intelligence to other countries, which might then take steps to upgrade their communications. He was never prosecuted for espionage.30

Yet Weisband caused immense damage to Western code-breaking. On Friday, 29 October 1948 the Soviets implemented a massive change in all their communications security procedures. American code-breakers referred to this fateful event as ‘Black Friday’. Many Soviet radio nets moved over to one-time pads, which henceforth were not re-used. Much of the procedural material that had been sent ‘in clear’, or unencrypted, between operators running medium-grade Army, Navy, Air Force and Police systems, was now encrypted for the first time. Operator chatter was banned. In the space of twenty-four hours, most Soviet systems from which the West had been deriving intelligence were lost.31 This affected the ‘Poets Systems’ which the British and Americans had been reading successfully as a result of their raids into Germany in 1945.32 This was the most serious British intelligence loss of the early Cold War.

For the British, Venona was full of irony. As a joint programme with the Americans it symbolised the highest level of trust. However, its subsequent revelations damaged the most important parts of the transatlantic relationship, including agreements on code-breaking and atomic cooperation. This was because in early 1950 Venona uncovered Klaus Fuchs, who had come to Los Alamos as part of the British contribution to the Manhattan Programme, but was in fact an agent for the KGB. Venona also raised serious doubts about the possibility of Anglo–American–Commonwealth sigint and defence cooperation because of the number of KGB agents identified in Australia. Directly or indirectly, Venona also exposed four of the KGB’s top agents inside the British establishment: Kim Philby, Donald Maclean, Guy Burgess and John Cairncross. The main problem for the KGB was that it did not know how many of its previous messages had been broken by the Venona project, and which of its agents had been exposed. This made it hard for it to warn specific agents. Venona also contributed to Soviet paranoia about double agents who might be planting disinformation. The KGB’s strange tendency not to wholly trust even its best sources, including the SIS officer Kim Philby, was one manifestation of this.33

In August 1949 Philby returned from a posting in Istanbul to London. He was preparing to take over from Peter Dwyer as SIS liaison officer with the CIA in Washington, and was briefed by Sir Stewart Menzies, Chief of SIS, together with two of his senior officers, James Easton and Maurice Oldfield. Oldfield, whose responsibility was counter-intelligence, explained Venona to him in detail. Philby’s blood probably ran cold as Oldfield observed that they had broken about 10 per cent of the KGB’s Washington–Moscow telegrams and were now searching for a British diplomat working for the KGB and code-named ‘Homer’.34 Philby immediately requested a conference with his KGB controller, Yuri Modin. The KGB station in the Soviet Embassy in London reported the crisis that now confronted them:

Stanley [Philby] asked to communicate that the Americans and the British had constructed a deciphering machine which in one day does ‘the work of a thousand people in a thousand years’. Work on deciphering is facilitated by three factors: (1) A one-time pad used twice; (2) our cipher resembles the cipher of our trade organisation in the USA; (3) a half-burnt codebook has been found in Finland and passed to the British and used to decrypt our communications. They will succeed within the next twelve months. The Charles [Klaus Fuchs] case has shown the counter-intelligence service the importance of knowing the past of civil servants…Stanley, Paul [Guy Burgess], and Yan [Anthony Blunt] consider that the situation is serious35

Yuri Modin recalls that Venona ‘hung over us like the sword of Damocles’.36 Nevertheless, he and Philby agreed gloomily that in the short term there was nothing they could do, ‘only wait and behave with extreme care and caution’.37 Arriving in Washington in November 1949, Philby was offered a ringside seat on Venona. He was given Venona summaries by the GCHQ liaison officer in Washington, and was actually taken to Arlington and briefed on the project in detail several times.38 Incredibly, in July 1950 he put in a successful request for GCHQ to give him an extra copy of any Venona-related material it was sending to the Americans in Washington, so he could peruse it at leisure. In any other circumstances this would have been an espionage triumph, but it caused Philby no joy. The arrests at this time of Julius and Ethel Rosenberg, perhaps the most famous figures ever to be charged with espionage for the Soviet Union, cannot have calmed his nerves.39

Understandably, the Americans had initially refused to show the British the KGB Washington–Moscow traffic.40 This delayed the search for the Foreign Office spy code-named ‘Homer’, who eventually turned out to be Donald Maclean. In 1947, the earliest period of good code recovery, analysts knew that several messages from late March 1944 began with a stock preamble and greeting. Such standard openings were a gift for code-breakers. In this case it read: ‘To the 8th section. Material “G”.’ The Eighth section was thought to receive political intelligence, and short breaks in other KGB messages showed that the material concerned Britain’s Ambassador in Washington, Sir Archibald Clark Kerr. By December 1948, further work by Philip Howse revealed that it seemed to originate from telegrams sent by Churchill. In January 1949, after a month of frantic night-time digging in the registry of the Foreign Office (a daytime search would have alerted the regular diplomatic staff), the originals were found. The circle of suspects was gradually narrowing. The final breakthrough came in August 1950, when the Americans recovered two short stretches of material that referred to ‘Homer’ being entrusted with decyphering a telegram from ‘Boar’ [Churchill] to ‘Captain’ [Roosevelt]. This pointed directly to someone in Britain’s wartime Washington Embassy, and the finger of suspicion began to circle over the heads of a very few people. Further work on the messages suggested that ‘Homer’ was married. However, it was only on 30 March 1951 that the code-breakers were sure that ‘G’ and ‘Homer’ were the same. This information placed him in New York in June 1944.41

At this moment, Philby knew that Maclean had been identified. However, he also knew that MI5 would have to gather traditional evidence against him to support an arrest, so a window of opportunity existed. Philby’s friend and fellow Soviet agent, the diplomat Guy Burgess, was being sent home from Washington in disgrace after an especially embarrassing drunken episode, and Philby used him to pass a message to Yuri Modin, their KGB controller in London. On Friday, 25 May 1951, Burgess and Maclean fled from Britain on a ferry to St Malo. It was a narrow escape: MI5 had planned to confront Maclean when he turned up for work the following Monday. Once in France, a KGB contact handed them false papers which ensured that they could travel in relative safety across Europe towards Moscow. The false papers were essential, since by now every security service in Europe was looking for them. Inevitably, suspicion also fell on Philby, not least because Burgess had been lodging with him in Washington, but there was no hard evidence. Philby was recalled and forced into retirement, but no other action was taken against him.42

Venona also had ramifications in the British Commonwealth. In July 1947, Field Marshal Montgomery, now Chief of the Imperial General Staff, had held a meeting with Australia’s Minister for External Affairs, Dr H.V. Evatt, about joint weapons development in Australia. Montgomery noted that ‘good security precautions are very necessary’ because of the appearance in Australia of a spy who was connected to the Igor Gouzenkou case, in which a defecting KGB cypher clerk had revealed a major spy ring in Ottawa in 1946. But in November and December 1947 Venona revealed that despite enhanced security precautions, sensitive documents were regularly leaking from Canberra to the KGB.43 These revelations soon made their way to the highest level. On 27 January 1948, Admiral Roscoe Hillenkoeter, Director of the CIA, warned President Truman: ‘Indications have appeared that there is a leak in high government circles in Australia, to Russia.’ He explained that MI5 was engaged in expansive undercover investigations to determine just where the leakages were.44 Highly sensitive material had been passed to the KGB from the Department of External Affairs in Canberra. The Soviets considered it to be spectacular stuff, for it included copies of the ‘explosive’ future strategy papers drafted by the British Post Hostilities Planning Committee, or ‘PHP’. This was bare-faced anti-Soviet planning material, prepared with the encouragement of the British Chiefs of Staff, that had already resulted in rows in Whitehall. Anthony Eden, then Foreign Secretary, had banned its circulation abroad in late 1944, but by then it was too late, and the volatile PHP reports had already made their way via Canberra to Moscow.45 The KGB chief in Australia considered the PHP papers to be such an important coup that he asked Moscow for permission to send them by cypher rather than courier. This was a bad mistake, for the two lengthy papers, ‘Security in the Western Mediterranean and the Eastern Atlantic’ and ‘Security of India and the Indian Ocean’, provided the code-breakers with a vast word-for-word ‘crib’ to get into other Soviet traffic.46 Partly because it was relatively easy to identify which documents had been taken in Australia, the KGB Moscow–Canberra cables proved to be the most successful part of the Venona operation. Remarkably, by early 1948 so much progress had been made that GCHQ was virtually reading the messages in real time.47

London did not regard the Australians as competent enough to handle this security crisis. In February 1948 Sir Percy Sillitoe, the head of MI5, was despatched to Australia. With him came Roger Hollis, head of MI5’s C Division (later himself wrongly accused of working for the KGB), concerned with protective security and background checks, and another senior security officer, Roger Hemblys-Scales. With Courtney Young, MI5’s resident Security Liaison Officer in Australia, they persuaded the Prime Minister, Ben Chifley and Defence Minister, Frederick Shedden, to permit vigorous investigations. In July, following further discussions with British Prime Minister Clement Attlee, the Australians accepted British proposals for the creation of an Australian equivalent of MI5 later known as the Australian Security Intelligence Organisation (ASIO).48 Sillitoe returned to London, but Hollis and Hemblys-Scales remained in Australia to set up ASIO and work on the list of Venona suspects, which numbered twelve.

Thereafter, ASIO was almost entirely focused on what it called ‘The Case’. Tracing documents quoted in KGB traffic indicated likely suspects, including a typist, Frances Bernie, who helped to run a Communist youth league and who worked personally for Dr Evatt, the Minister for External Affairs. It also pointed to two Australian diplomats with Communist leanings, Ian Milner and Jim Hill. Hollis and Courtney Young did not tell the Australians that the names came from intercepts, but the nature of the material led some of the more experienced ASIO hands to suspect sigint as the key source. Some of the suspects were referred to by code names rather than real names, and their identities could only be deduced by careful circumstantial guesswork. Milner and Hill, who were identified positively, refused to ‘come over’. William Skardon, MI5’s most experienced interrogator, made a soft approach to Hill when he visited London in 1950, trying to persuade him to ‘be sensible’ and ‘make a clean breast of it’, but Hill denied everything.49

The ‘Venona Twelve’ kept ASIO’s staff of close to two hundred busy well into the 1950s. Each new suspect opened a world of further associates and contacts who required separate examination. The task was difficult, since the Communist Party of Australia had long expected to be banned, and had built up a substantial underground organisation. Not unlike the Communist Party of India, seasoned by years of security attention, it had also achieved some infiltration of the police. Even the infiltration of ASIO seemed a possibility. ASIO’s staff worked around the clock watching and bugging the flats of suspect Soviet diplomats in Canberra. Each visitor was tailed and investigated. ASIO’s staff were learning the hardest lesson of counter-espionage and counter-subversion: working security cases really diligently only manufactured more leads and opened more cases.50 Almost a quarter of the Venona messages relating to Canberra still remain classified, presumably because they relate to KGB agents not pursued or prosecuted.51

The British and Australians were not alone in suffering KGB penetration. Although headlines about Klaus Fuchs and Donald Maclean generated anxiety about Britain amongst the American elite, those on the inside knew Washington had its fair share of Soviet agents. Venona uncovered spies in the State Department, the Treasury, even in the White House. They included Harry Dexter White, a senior Treasury official, and Laughlin Currie, who had been a personal assistant to Franklin D. Roosevelt. This was not particularly surprising, since the vast influx of academics and scientists moving into government work during wartime had inevitably included some Communist Party sympathisers. The Office of Strategic Services, forerunner of the CIA, which had recruited heavily from the East Coast academic establishment, harboured perhaps a dozen people working for the Soviets.

Venona had profound implications for the development of the security state in America, Britain, Canada and Australia. Just at the moment when the public were anxious to throw off the claustrophobic constraints of wartime security, officials were confronted with irrefutable evidence of a massive programme of Soviet espionage. Selling strong security measures in the late 1940s was an uphill task. This was nowhere more true than in Australia. The creation of ASIO by a Labor Prime Minister, Ben Chifley, was a remarkable development. Like the British Labour Party, its Australian counterpart had historically been sceptical about surveillance, associating it with right-wing anti-union activities. In Britain too, Venona led indirectly to the introduction of detailed personal background checks, or ‘positive vetting’, for officials. British civil servants resisted the idea, but it was increasingly clear that without it, Anglo–American strategic cooperation on matters like atomic energy was likely to end.52

Venona represents a documentary source of high value, and has helped to resolve some of the most bitterly contested Cold War espionage cases. These include the famously controversial cases of the atomic scientist Julius Rosenberg and the diplomat Alger Hiss, who were both active espionage agents for the Soviets. In these important cases, Venona offers us what Nigel West has rightly called ‘a glimpse of the unvarnished truth’.53 At the same time, much of the Venona material is rather fragmentary, and in 1995 it was further obfuscated by the lamentable decision of the British and American governments to blank out some names on grounds of potential political embarrassment. Some KGB code names for individuals were re-used and given to more than one person. Moreover, it is possible that a minority of the people who appear in the Venona cables did not knowingly have a relationship with Soviet intelligence officers, or were identified as possible targets for future recruitment, but were never actually recruited. The tendency of some intelligence officers to exaggerate their triumphs has also to be borne in mind. In short, Venona has provided us with fabulous revelations, but the full story awaits the moment when historians access the files of the KGB and Soviet military intelligence, or GRU, in Moscow. That will not happen for a long time yet.

5 UKUSA – Creating the Global Sigint Alliance

Much discussion about 100 per cent cooperation with the USA about SIGINT. Decided that less than 100 per cent cooperation was not worth having.

Admiral Andrew Cunningham, Chief of the Naval Staff,

21 November 19451

One of the most important legacies of the Second World War was the creation of the vast global signals intelligence alliance known as ‘UKUSA’. The signing of the UKUSA intelligence treaty between Britain, the United States, Canada, Australia and New Zealand has long been regarded as marking the birth of a secretive leviathan, a global multilateral alliance that has grown to embrace numerous countries and to command almost unlimited intelligence power. Its origins are often traced to a single landmark treaty between Britain, the United States and the Commonwealth deemed to be concluded in 1948. Indeed, the highly classified UKUSA treaty is widely considered to be nothing less than the linchpin of the West’s post-war intelligence system. UKUSA supposedly created a cosy Anglo-Saxon club sharing everything in the super-secret realm of sigint.

Remarkably, there is in fact no singular UKUSA ‘treaty’ of 1948, and none of the above assertions is true. Instead, UKUSA is less an alliance than a complex network of different alliances built up from many different overlapping agreements. It is the sum of a curious agglomeration of many understandings that were mostly between two countries only, that accumulated over more than two decades.2 Britain and the United States concluded the main agreements in 1943 and 1946, together with a further convention in 1948. According to the historian Peter Hennessy they are still in force, and as recently as August 2006, some sixty years on, the authorities deemed them so sensitive that, after anxious deliberation, they announced that they could not be released.3 Further agreements were added – and continue to be added – creating a complex spider’s web of cooperation. However, each agreement has its limits, and all parties have withheld sigint material from each other. In short, there is no common pooling of material. Moreover, relations between the various parties have often been tense, and latterly Washington has threatened some adherents, including Britain, Australia and New Zealand, with suspension or exclusion. If UKUSA is an alliance, its members are only ‘allies of a kind’.4

It is also wrong to think of UKUSA as exclusively concerned with sigint. It is, rather, a sigint and security network. Security agreements on physical control of the sigint product and on protecting the security of communications were perhaps the most important aspects of the UKUSA network. Sigint reports on particular subjects were rigidly compartmentalised and given ‘Codeword’ status, ensuring that they could only be seen by people cleared to see that series, and making them effectively ‘above Top Secret’. Venona is the best-known example of such a Codeword. Much of this obsessive secrecy was codified in a biblical tome entitled ‘International Regulations on Sigint’, or ‘IRSIG’, which had reached its third edition by 1967.5 UKUSA was also about secretly undermining the communications security of other states, even neutrals and allies. Communications security, or ‘comsec’, is perhaps even more sensitive than sigint. The efforts of the UKUSA powers to control it have been among the darkest secrets of alliance politics in Western Europe. In short, the realm of sigint alliances is profoundly realist – at times even paranoid – with operators ‘taking what they can get’. While UKUSA might appear from the outside to represent a single powerful intelligence colossus, on the inside it was anything but unified.

The best example of allies spying on allies is provided by Finland. The end of the Second World War had not turned out well for the Finns, since their Russian enemy had returned to the Baltic in overwhelming strength. Anticipating the arrival of the Russians, the talented Finnish code-breakers decamped en masse to Sweden, complete with their relatives, equipment and support staff. There they began a veritable car-boot sale of their cryptographic wares, including the results of sixteen years of continuous work against Russian systems. The beauty of selling codes is that the same items can be sold many times over. Predictably, the Finns paid their ground rent by assisting the Swedish equivalent of Bletchley Park, the Förvarets Radioanstalt, or FRA. In the last days of the war they also sold complete Russian codebooks to the American wartime intelligence agency, the Office of Strategic Services, to Britain’s SIS and also to the Japanese. They also sold the Americans the details of the British codes they had broken, and work they had completed against some US State Department cyphers. The Americans were eager customers. This episode – known as the ‘Stella Polaris’ case because of its northern origins – underlines the duplicitous nature of friendships in the realm of code-breaking.6

In the autumn of 1945, even while the Stella Polaris case was ‘live’, President Roosevelt’s successor, Harry S. Truman, was engaged in the abolition of the Office of Strategic Services. Two years later its remnants would be revived to become the Central Intelligence Agency, but for now many of its intelligence officers were dispersed and its agents paid off. By contrast, Truman regarded sigint as indispensable, and secretly gave permission for the American code-breaking agencies to work on into the post-war period and ‘to continue collaboration in the field of communication intelligence between the United States Army and Navy and the British’.7 All major countries desired the maximum world coverage. On 19 November 1945, Admiral Andrew Cunningham, Britain’s senior naval commander, attended a critical meeting of the British Chiefs of Staff. There was ‘Much discussion about 100 per cent cooperation with the USA about Sigint,’ he recorded, adding that they ‘Decided that less than 100 per cent was not worth having.’ In Ottawa, George Glazebrook, a senior Canadian diplomat, recommended to the Canadian Joint Intelligence Committee that Canada enhance her independent sigint effort in order to stake a claim in this secretive emerging cooperative system. ‘It is paramount,’ he insisted, ‘that Canada should make an adequate contribution to the general pool.’8

Yet a ‘general pool’ was not what emerged. Moreover, the way ahead was strewn with obstacles and tortuous negotiations. The complex package of agreements, letters and memoranda of understanding was not completed until 1953. In this process, Britain derived considerable benefit from her dominance over her Commonwealth partners and her imperial bases. GCHQ’s approach was to align her Commonwealth affiliates to create a critical mass before entering negotiations with the Americans. The story of Britain’s sigint relations with Australia illustrates this well. In March 1945, with the end of the European war looming, Edward Travis set off from Bletchley Park on a veritable world sigint tour. The possibility of transforming wartime cooperative arrangements into a post-war sigint alliance was already in his mind.9 En route, he and his party visited major sigint centres at Heliopolis in Egypt and HMS Anderson in Ceylon. They arrived in Melbourne in early April, and spent time with the Australian code-breaking organisation there, called the Central Bureau. On 17 April they departed for New Zealand and then moved on to Hawaii, San Francisco and finally Washington. By the time they reached Hawaii they were running low on funds, and had to beg a cash advance from the Foreign Office before they could proceed further. At each stop, the possibility of continued post-war cooperation was gently raised.10