The British Secret Intelligence Service – popularly known as MI6 – is the oldest continuously surviving foreign intelligence-gathering organisation in the world. It was founded in October 1909 as the ‘Foreign Section’ of a new Secret Service Bureau, and over its first forty years grew from modest beginnings to a point in the early Cold War years when it had become a valued and permanent branch of the British state, established on a recognisably modern and professional basis. Although for most of this period SIS supervised British signals intelligence operations (most notably the Second World War triumphs at Bletchley Park over the German ‘Enigma’ cyphers), it is primarily a human intelligence agency. While this history traces the organisational development of SIS and its relations with government – essential aspects for an understanding of how and why it operated – its story is essentially one of people, from the brilliant and idiosyncratic first Chief, Mansfield Cumming, and his two successors, Hugh Sinclair and Stewart Menzies, to the staff of the organisation – men and women who served it across the world – and, not least, to its agents, at the sharp end of the work. It is impossible to generalise about this eclectic and cosmopolitan mix of many nationalities. They included aristocrats and factory workers, society ladies and bureaucrats, patriots and traitors. Among them were individuals of high courage, many of whom (especially during the two world wars) paid with their lives for the vital and hazardous intelligence work they did.

SIS did not emerge from a complete intelligence vacuum. For centuries British governments had covertly gathered information on an ad-hoc basis. In the seventeenth century successive English Secretaries of State assembled networks of spies when the country was particularly threatened, and from its establishment in 1782 the Foreign Office, using funding from what became known as the ‘Secret Service Vote’ or the ‘Secret Vote’, annually approved by parliament, employed a variety of clandestine means to acquire information and warning about Britain’s enemies. By the end of the nineteenth century the army and the navy, too, had intelligence-gathering branches, which processed much information acquired relatively openly by naval and military attachés posted to foreign countries. But, after the turn of the twentieth century, with foreign rivals (Germany in particular) posing a growing challenge to national interests, British policy-makers began to look beyond these unsystematic and uncoordinated methods, and, as the Foreign Office worried about the possibility of its diplomatic and consular representatives becoming caught up in (and inevitably embarrassed by) intelligence-gathering, the notion of establishing a dedicated, covert and, above all, deniable agency came to find favour.

The Secret Service Bureau, and the subsequent Secret Intelligence Service, remained publicly unacknowledged by the British government for over eighty years and was given a formal legal basis only by the Intelligence Services Act of 1994. Th e fact that a publicly available history of any sort has been commissioned, let alone one written by an independent professional historian, is an astounding development, bearing in mind the historic British legacy of secrecy and public silence about intelligence matters. It is also an extraordinary once-in-a-lifetime opportunity (and privilege) to be appointed to write this history, though I am well aware that the fact that I have been deemed suitable to undertake it may in some eyes precisely render me unsuitable to produce an independent account of SIS’s history. But of that the reader must judge.

Part of the agreement made on my appointment was that I should have utterly unrestricted access to the Service archives over its first forty years. I am absolutely confident that this has been the case and it has been an unparalleled treat to be let loose in the archive, which is an immensely rich (though in places patchy) treasure-trove of historical materials. In addition to this access, I have also been allowed to read some post-1949 materials bearing on the history of the Service. In general, the SIS attitude to archives was that they should be kept only if they served some clear operational purpose. Certainly, since no one envisaged that a professional history of any sort would be written, let alone one that might be published, there was no imperative to retain materials for historical reasons. When the Service did begin to think historically, which, from the evidence I have seen, was not much before the 1960s, a huge amount of material had already been lost.

Within SIS the practice appears to have been routinely to destroy documents once their immediate relevance or utility had passed. There is plenty of internal evidence indicating this, some of which has occasionally slipped out into the public domain. In a 1935 letter to Valentine Vivian, head of the counter-espionage Section V in SIS, Oswald ‘Jasper’ Harker of MI5 remarked, ‘An old report of yours regarding a Madame Stahl has just come to light – I enclose a copy as I believe your 1920 records have been destroyed.’ Reviewing the work of SIS in the early 1920s, one officer observed that the SIS headquarters ‘receives from its overseas branches over 13,000 different reports per annum, exclusive of correspondence about these reports and administrative matters’. He noted that ‘the mass of papers involved immediately becomes apparent’. In order to keep the volume of material under control, he added that ‘every effort is made to destroy all matter … not needed for reference’. The practice of clearing out old papers has also been powerfully stimulated by the fact that the organisation has moved house on some six occasions during the last century.

Over the years some documents were recognised as having real historical significance and were preserved. One such is the ‘Bethell letter’, from the Director of Naval Intelligence to Mansfield Cumming on 10 August 1909 inviting him to become (as it turned out) the first Chief of the Service. There has, nevertheless, been intermittent, methodical and substantial destruction of records which may, or may not, have been of historical value. But I have found no evidence that the destruction was carried out casually or maliciously, as some sort of cover-up to hide embarrassing facts about SIS’s past or whatever. The destruction has resulted more from a cultural attitude where the retention of documents in general was assessed in the light of their current (and certainly not historical) value to the Service, primarily in operational terms.

The corollary to unrestricted access to the archives has been an extremely painstaking and fastidious disclosure process. From the start (and for obvious reasons) it was laid down that the identity of any agent could not be revealed for the first time in this book. One result of this stipulation is the regrettable need (from the historian’s point of view) to omit some significant and important SIS stories, as it would not be possible to include them without providing at least circumstantial details which could potentially help identify agents. Exceptionally, however, some agents’ names do appear in the book, but each case has been subject to the most careful and rigorous disclosure criteria. Where agents have clearly named themselves (not uncommon for individuals who worked during the world wars), this has been relatively straightforward, but simply arguing that an agent’s name is ‘in the public domain’ is not in itself sufficient, as the ‘public domain’ constitutes a great range of contexts, from unsubstantiated assertions in sensationalist and evanescent publications (what might be called ‘sub-prime intelligence literature’) to serious and scholarly articles by professional historians.

What remains? Quite a lot, despite the fact that immense quantities of documents were destroyed, especially during the period covering the headquarters move from Broadway to Century House in the early 1960s.  The first thing to be said, however, is that (perhaps surprisingly) the archive contains comparatively little actual intelligence. Over the 1909–49 period with which I am concerned SIS was always primarily a collection agency, responding to specific or general requests for information from customer departments, principally its parent department, the Foreign Office, and the armed service ministries. The information requested (if available) was collected and passed on to the relevant department. Little or no analysis was applied to this material within SIS, apart from some outline indication about the reliability, or otherwise, of the source. Once the raw material was passed on to the user department, they processed it and normally destroyed the original documents. Intelligence assessments were the job of the particular desk in the Foreign Office, the Directorate of Military Intelligence and so on, not of SIS.

SIS’s deployment and work, therefore, was principally defined by the priorities and perceptions of external agencies. Between the wars Soviet Communism remained the chief target, and a particular concern with naval matters in the Mediterranean and Far East clearly reflected Admiralty perceptions and intelligence requirements. During the early and mid-1930s SIS resources, in any case constrained by an acute shortage of funding, were not focused on the developing challenge of Nazi Germany as much as (admittedly with the benefit of hindsight) they might have been. Although the Service was, nevertheless, quick off the mark to report German rearmament, there was evidently little demand in London for secret intelligence about internal German political developments. There is, for example, almost nothing in the SIS archives (both for this period and during the Second World War) about the persecution of Jews generally or the Final Solution. A report from Switzerland in January 1939 is a rare exception. An SIS representative had asked an Austrian- Jewish refugee if he could supply ‘any information about people in concentration camps’. The source said that he knew a man in Geneva who had spent nine months in Dachau, ‘but he doubted whether he could get this man to talk. He said German refugees were frightened of saying anything against Germany, because European countries were riddled with Nazi agents and they feared reprisals.’

One of the things I had hoped to do in this history was find instances when I could track the process from the acquisition of a specific piece of intelligence to its actual use, but in the absence of much of the raw material I have found this quite difficult (though in some cases not impossible) to achieve. I might remark that the situation is quite different with regard to signals intelligence where a considerable volume of the raw (or rawish) product survives and can readily be used, as in Sir Harry Hinsley’s magisterial volumes, to estimate ‘its influence on strategy and operations’, as his sub-title promises. During the First World War, nevertheless, I have for example been able to trace the use of human intelligence from the ‘Dame Blanche’ organisation in occupied Belgium, as well as the ready and informative response of the German naval spy TR/16 to requests for details of German losses in the Battle of Jutland in 1916. In the mid-1930s (though it was not always taken as seriously as it should have been) SIS reporting was used to inform British assessments of German rearmament. In the Second World War, specific SIS intelligence underpinned the important Bruneval raid in February 1942 and provided early indications of the German V-weapons development programme.

But, on the whole, the story of human intelligence is not generally one of fiendishly clever master-spies, or Mata-Hari-like seductresses (though in this volume the keen-eyed reader will find one or two possible examples of these types), achieving fantastic, war-winning intelligence coups. It is more like a pointillist painting, containing tiny fragments of information, gathered by many thousands of individual men and women in circumstances fraught with danger, which need to be collected together to provide the big picture. Watchers along the Norwegian coast in the Second World War, for example, provided precious information about enemy ship movements. These individuals had to get to what were inevitably exposed situations; once there they had not only to collect their intelligence unobserved, but also to communicate it quickly back to London; and at each stage of the process the penalty for discovery was almost certain death. In both world wars, ordinary men and women in enemy-occupied Europe ran similar risks, for example train-watching, carefully logging the movements of railway trains and their cargoes and endeavouring to identify the military units they carried. We ought not to pass over in silence the astonishingly brave actions of these numberless, and for the most part nameless, people, few of whom were the kind of spies so beloved of film and fiction, but many of whom contributed to the successes of British intelligence during the first half of the twentieth century.

The material which survives in the SIS archive is more abundant on the process and administration of acquiring intelligence than on the intelligence itself. ‘Sources and methods’, the most sensitive of all aspects of intelligence work, are embedded in this material: names of officers, agents, sources, helpers, organisations, commercial companies, operational techniques, various sorts of technical expertise and the rest. While some of these no longer pose any security risk – for example there seems little danger that national security may now be jeopardised by revealing 1940s wireless technology – documents relating to agents and their activities have the potential to jeopardise them and their families even long after they may have ceased working for SIS. A typical agent file, for instance, may, without giving very much detail, note that she (or he) produced ‘much valuable intelligence’. The  bulk of the documents may thereafter contain details for years afterwards of the agent’s address (say in some foreign city), pension payments and perhaps reports of visits by an SIS welfare officer, bearing a Christmas bottle of whisky or some other suitable gift. This is exactly the kind of material which the Service rightly believes can never be released.

This history, written as it were from headquarters, reflects the surviving SIS documentation upon which it is primarily based. This means that it has sometimes been difficult to recreate the personal relationships between case-officers and agents which lie at the heart of human intelligence work. Busy case-officers did not often have the time to write reflective notes on their agents’ personalities or motivations, though some hints of these fascinating matters have, happily, survived, and are included in my narrative. I have in general used memoir material very sparingly. Although often revealing on the personal side, the recollection of events and emotions, sometimes many years after, presents critical problems of interpretation and assessment for the historian, particularly in the matter of espionage and other covert activities, which are not infrequently cloaked about with a melodramatic air of secrecy, conspiracy, conjecture and invention. This is not to say that such things do not exist – indeed examples of each might be found in this book – and I have drawn on secondary sources in cases where they seem to be particularly illuminating. Nevertheless, my primary objective has been to base the narrative as closely as possible on the surviving contemporaneous documentary record. If this approach risks some loss of vividness, then it does so expressly for the purposes of historical accuracy.

As will be apparent from the reference notes, I have also had privileged access to relevant but closed documents held by other British government departments. These have been especially useful in helping place SIS in its wider bureaucratic context. With a very small number of exceptions, all other primary source materials (including some extremely valuable sources in foreign archives) are fully open to the public.

Quotations from documents in closed and open archives are reproduced exactly as originally written with the following exceptions: proper names rendered in most official papers in block capitals have been given in title case, with agent and operation code-names in quotation marks; numbering or lettering of individual paragraphs in cables and other documents has not been reproduced; in communications where names of people, places and organisations were given letter codes (‘A’, ‘B’, ‘C’ and so on), the key being transmitted separately, the correct name has been substituted for the code-letter. Queried words in deciphered messages are as in the original (for example, ‘reliable’). In a few cases punctuation has been silently adjusted for the sake of clarity. Since records from the SIS archive are not released into the public domain, no individual source references are provided to them. In this case I have followed the precedent set by past British official histories. Calculations of current value of historical sums of money are based on the Retail Price Index, as indicated in www.measuringworth.com, which has also been used for exchange-rate information.

This account of SIS’s history finishes in 1949, at a moment when the Service had moved from being a tiny, one-man outfit to a recognisably modern and professional organisation. After forty years’ existence, SIS was on the threshold of four decades when the Cold War challenge of Soviet Communism would dominate its activities. But these are matters which I leave to my successor, if there is one.