How Innovation Works

One of the most surprising stories of the twenty-first century has been the rise of natural gas, a fuel that just a decade ago was thought to be on the brink of running out and is now both cheap and plentiful. It is mainly the story of the innovation that led to the production of gas from shale. Right up till 2008 or so, it was conventional wisdom among energy experts that cheap natural-gas supplies would be exhausted to all practical extent fairly early in the twenty-first century. Oil and coal would last longer. This prediction had been made before, repeatedly. In 1922 the US Coal Commission, set up by President Warren Harding, interviewed 500 people in the energy industry over eleven months, and came to the conclusion that ‘already the output of gas has begun to wane’. In 1956 the oil expert M. King Hubbert predicted that natural-gas production in the United States would peak in 1970 at 38 billion cubic feet per day and decline. In fact it was 58 bcf a day then and still rising. Today it is over 80 bcf per day.

These predictions proved gloriously wrong, for two reasons. First, in America, strict price regulation of gas in the 1970s, based on the theory that it was scarce, effectively halted gas exploration in its tracks. Companies flared off or shut down gas as a nuisance, and pursued oil instead. This did indeed produce a peak in production which many mistook for the beginning of exhaustion of reserves. Incredibly, the US government passed several measures in the 1970s to forbid the generation of electricity by oil or gas in any utility that could get access to coal, and forbade the building of plants that could not use coal. Deregulation of the gas industry under President Reagan led to a surge in production.

The second reason for the gas glut of the second decade of the twenty-first century was innovation. Throughout the United States, gas and oil exploration companies set out to find ways to squeeze more out of each field, and to squeeze gas and oil out of ‘tight’ rocks, whence it did not flow naturally. This resulted in the serendipitous discovery of ‘slick-water’ hydraulic fracturing in the 1990s in Texas, which, combined with the new ability to drill round corners, and thus go horizontally within seams of rock for miles on end, made tight shales, where most hydrocarbons are stored, into huge sources of gas and oil. Add in offshore gas, plus the ability to liquefy gas for transport by sea, and it becomes clear why the world now has ample supplies of gas, the cleanest, lowest-carbon and safest of the fossil fuels.

The key location of the slick-water fracking breakthrough was the Barnett shale near Fort Worth, where an entrepreneur named George Mitchell, born to a Greek goatherd father, had grown rich supplying Chicago with gas. He had a good fixed-price contract. If he moved elsewhere he would have to drop his price. So he was desperate to squeeze more from the Barnett shale, where he had lots of drilling rights. By the late 1990s output was dropping, and so was Mitchell Energy’s share price, which was causing Mitchell personal difficulties, because of commitments he had made to philanthropy, backed by loans against his shares. His wife had Alzheimer’s and he had prostate problems. By rights the 78-year-old multimillionaire should have been reasonable, should have given up on America as the oil majors were already doing, and cut his losses. The future of gas lay offshore, or in Russia and Qatar. But Mitchell, like many innovators, was not reasonable, so he kept trying to get the gas to flow.

The Barnett shale was known to be rich in hydrocarbons, but they would not flow easily, so the rock needed to be cracked deep underground, and the microscopic cracks propped open. A technology to do this was well known, and relied on gels to prop open the cracks and let the gas out. It worked well in some rocks but not in shale. Mitchell sank $250m into trying to make it work in the Barnett field without success.

One day in 1996 a Mitchell employee named Nick Steinsberger noticed an odd result. He was employing contractors to pump a stiff gel with large amounts of sand in it down the well. But since gel and sand were expensive, he had been forcing the service companies to lower the amount of gel and chemicals in the mixture pumped down the hole in an attempt to lower costs and pump less of the viscous material into the shale. On this day, the gel was so dilute it would not ‘gel’ properly. Steinsberger pumped it down the hole anyway and noticed the well produced a decent surge of gas. He tried some more wells with similar results. Attending a baseball game with a friend from another company, Mike Mayerhofer, he heard a similar story – water with a little lubricant and much less sand was working well in a different kind of rock, in this case tight sandstone in east Texas.

So in 1997 Steinsberger then began deliberately using a more watery liquid, basically water mixed with less sand and a very small quantity of ordinary kitchen sink chemicals (bleach and soap, essentially), instead of gel. He tried this on three wells, but it did not work. ‘The pressure went up too high, forcing me to terminate the pump job, because the slickwater wouldn’t carry the sand in shale like it would in much more permeable tight sands.’ In early 1998, getting pretty desperate and with his bosses ready to give up on the Barnett shale, he convinced management to let him try three more wells. This time he pumped a lot more slick water but increased the sand from extremely low concentrations to higher over the course of the job. The first well, S. H. Griffin Estate 4, produced a surge of gas and kept on doing so for weeks and months. He realized he had stumbled on a formula that was not just half as expensive, but twice as productive. A flash in the pan? No, the other two wells had similar results.

Steinsberger’s breakthrough transformed the last years of George Mitchell’s life, turning him into a billionaire when he sold his company. It turned the Barnett shale into America’s largest gas producer. Copied elsewhere, and steadily improved by further innovation, it had the same effect in shale after shale, in Louisiana, Pennsylvania, Arkansas, North Dakota, Colorado, then Texas again. Soon the same technique was being adapted to get oil out as well. Today America is not only the world’s biggest producer of gas; it is also the world’s biggest producer of crude oil, thanks entirely to the shale-fracking revolution. The Permian basin in Texas alone now produces as much oil as the whole of the United States did in 2008, and more than any OPEC country except Iran and Saudi Arabia. America was building huge gas import terminals in the early 2000s; these have now been converted into export terminals. Cheap gas has displaced coal in the country’s electricity sector, reducing its emissions faster than any other country. It has undermined OPEC and Russia, leaving the latter frantically supporting anti-fracking activists to try to defend its markets – with much success in innovation-phobic Europe, where shale exploitation has been largely prevented.

A cheap-gas, cheap-oil glut brought on deliberately by OPEC in 2015 to try to bust the frackers had the opposite effect, killing weaker companies but forcing the survivors to work out how to remain competitive at sixty, fifty and forty dollars per barrel of oil. The availability of cheap hydrocarbons gave American manufacturing an edge, resulting in a rapid ‘reshoring’ of chemical industries to the United States and a surge of chemical companies leaving Europe. The energy policies of a dozen countries like Britain, predicated on ever-rising fossil-fuel energy prices to make wind and nuclear look less expensive, became expensive follies almost overnight.

Why did this revolution happen in America, an old, played-out and well-explored oil and gas region? The answer lies partly in property rights. Because of mineral rights belonging to local landowners, rather than the state, and because oil companies had never been nationalized, as they were in so many other countries, from Mexico to Iran, America had a competitive, pluralistic and entrepreneurial oil-drilling mindset, manifested in a ‘wild-cat’ industry, backed by deep pockets of risk capital – the early frackers spent vast sums of borrowed money before turning cash-positive. As one account of the story by the key innovators put it:

Small companies often have the upper hand in leasing mineral rights from landowners as their interaction with landowners is generally more personalized. Shale production was hotly pursued by many small companies resulting in a multitude of varied drilling and completion methods being implemented and tested across multiple basins. These ‘laboratories’ have resulted in continuous improvements and fostered economic success.

So trial and error was vital to innovation in fracking. Steinsberger made a series of lucky mistakes, failing many times along the way. And when he had found the formula, he did not know why it worked. A seismology expert, Chris Wright, soon explained it. Wright, an engineer whose company, Pinnacle, was using new tiltmeter devices to help track the progress of fractures underground for Mitchell, figured out that slick-water fracs created large networks of multiple fractures. He had developed a model of simultaneous growth of multiple fractures in the early 1990s ‘which was widely derided by all the old-timers in the frac world as they insisted multiple fracs would always rapidly coalesce into a single frac’. It turned out Wright was right. The pressurized water was creating cross-cutting fractures in the rocks, greatly increasing the surface area exposed to the sand. Fractures were propagating a mile or more in one direction, but spreading hundreds of metres either side of this axis too. In this case science came in behind the technology, rather than vice versa. Recent attempts to credit the federal government with starting this innovation mostly miss the point. Yes, lots of research was done at government laboratories, but much of it under contract to the gas industry, and largely because there were entrepreneurs like Mitchell and Wright (now one of the industry leaders) creating the demand for such research.

At first environmentalists welcomed the shale gas revolution. In 2011 Senator Tim Wirth and John Podesta welcomed gas as ‘the cleanest fossil fuel’, writing that fracking ‘creates an unprecedented opportunity to use gas as a bridge fuel to a 21st-century energy economy that relies on efficiency, renewable sources, and low-carbon fossil fuels such as natural gas’. Robert Kennedy, Jr, head of the Waterkeeper Alliance, wrote in the Financial Times that ‘In the short term, natural gas is an obvious bridge fuel to the “new” energy economy.’ But then it became clear that this cheap gas would mean the bridge was long, posing a threat to the viability of the renewable-energy industry. Self-interest demanded a retraction by Kennedy, which he duly provided, calling shale gas a ‘catastrophe’.

In the heartlands where fracking began, Texas, Louisiana, Arkansas and North Dakota, there was little opposition. A lot of empty land, a long tradition of oil drilling and a culture of can-do enterprise ensured that the shale revolution prospered unhindered by much if any local protest. But when it spread to the East Coast, to Pennsylvania and then New York, suddenly shale gas began to attract enemies, and environmentalists spotted an opportunity to fund-raise on the back of opposition. Recruiting some high-profile stars, including Hollywood actors such as Mark Ruffalo and Matt Damon, the bandwagon gathered pace. Accusations of poisoned water supplies, leaking pipes, contaminated waste water, radioactivity, earthquakes and extra traffic multiplied. Just as the early opponents of the railways accused trains of causing horses to abort their foals, so no charge was too absurd to level against the shale gas industry. As each scare was knocked on the head, a new one was raised. Yet despite millions of ‘frac jobs’ in thousands of wells, there were very few and minor environmental or health problems.

The reign of fire

One of the flaws in the way we recount stories of innovation is that we unfairly single out individuals, ignoring the contribution of lesser mortals. I have chosen to tell the stories of Newcomen, Watt, Edison, Swan, Parsons and Steinsberger, but they were all stones in an arch or links in a chain. And not all of them ended up wealthy, let alone their descendants. There is no foundation named after any of them today and funded by their wealth. It was the rest of us who reaped most of the benefit of their innovations.

Yet energy itself does deserve to be singled out. It is the root of all innovation if only because innovation is change and change requires energy. Energy transitions are crucial, difficult and slow. For the vast majority of history, argues John Constable, the supply of energy, from wheat and wind and water, was just too thin to generate complex structures on a sufficient scale to transform people’s lives. Along came the heat-to-work transition of 1700 and suddenly it became possible to create ever more improbable and complex material structures from the harnessing of fossil fuels with their huge energy yield on energy invested. The fossil-fuel dependence of the modern world is roughly the same today – at about 85 per cent of primary energy – as it was twenty years ago. The vast majority of society’s need for energy is supplied by heat. What will eventually depose the ‘impellent use of fire’, that strange link between heat and work that came into the lives of humanity around the year 1700 and is still vital to the world? Nobody yet knows.

2

Public health

An operation invented not by persons conversant in philosophy or skilled in physic, but by a vulgar, illiterate people; an operation in the highest degree beneficial to the human race.

GIACOMO PYLARINI on smallpox inoculation, 1701

Lady Mary’s dangerous obsession

In the same year that Thomas Newcomen was building his first steam engine, 1712, and not far away, a more romantic episode was in train, and one that would indirectly save even more lives. It was much higher up the social scale. Lady Mary Pierrepoint, a well-read, headstrong young woman of twenty-three, was preparing to elope in order to escape the prospect of a dull marriage. Her wealthy suitor, Edward Wortley Montagu, with whom she had carried on a voluminous correspondence characterized by furious disagreement as well as outrageous flirtation, had failed to agree a marriage settlement with her even wealthier father, the Earl (later Duke) of Kingston. But the prospect of being forced by her father to marry instead a pecunious dullard, the Honourable Clotworthy Skeffington, persuaded Mary to rekindle the romance with Wortley (as she called him). She proposed elopement, and he, despite thus missing out on her dowry, and in a fit of uncharacteristic impetuosity, agreed. The episode turned to farce: he was late, she set off for the rendezvous alone, he overtook her at an inn but did not realize she was there, but after further mishaps they found each other and married on 15 October 1712 in Salisbury.

After this romantic start the marriage was a disappointment, Wortley proving a cold and unimaginative husband. His bride – learned, eloquent and witty – cut a swathe through literary London, writing eclogues with Alexander Pope in the style of Virgil, and befriending the literary lions and social tigers of the day. Joseph Spence would later write: ‘Lady Mary is one of the most extraordinary shining characters in the world; but she shines like a comet; she is all irregular and always wandering. She is the most wise, most imprudent; loveliest, disagreeablest; best natured, cruellest woman in the world.’

Then smallpox marked her skin and made her reputation. This vicious virus, humankind’s greatest killer, was constantly a threat in early-eighteenth-century London. It had recently killed Queen Mary and her nephew, the young Duke of Gloucester, the last Stuart heir to the throne who was not Catholic; it had almost killed the Electress of Hanover, Sophia, and her son George, destined to be the next king of England instead. It killed Lady Mary’s brother in 1714 and very nearly killed her the next year, leaving her badly scarred and lacking in eyelashes, her beauty cruelly ravaged.

But it was smallpox that would bring her lasting fame, for she became one of the first, and certainly one of the most passionate, champions in the Western world of the innovative practice of inoculation. In 1716 her husband was sent as ambassador to Constantinople and Lady Mary accompanied him with her young son. She did not invent inoculation, she did not even bring the news of it for the first time, but being a woman she was able to witness in detail the practice among women cloistered in Ottoman society, and then to champion it back home among mothers terrified for their children, to the point where it caught on. She was an innovator, not an inventor.

Two reports had reached the Royal Society in London from Constantinople of the practice of ‘engrafting’ as a cure for smallpox. According to the correspondents, Emmanuel Timonius and Giacomo Pylarini, both physicians working in the Ottoman Empire, the pus from a smallpox survivor would be mixed with the blood in a scratch on the arm of a healthy person. The reports were published by the Royal Society but dismissed as dangerous superstition by all the experts in London. More likely to spark an epidemic than prevent it; an unconscionable risk to be running with people’s health; an old wives’ tale; witchcraft. Given the barbaric and unhelpful practices of doctors at the time, such as bloodletting, this was both ironic and perhaps understandable.

It seems the Royal Society had been told of the practice even earlier, in 1700, by two correspondents in China, Martin Lister and Clopton Havers. So there was nothing new about this news. But where these doctors failed to persuade the British, Lady Mary Wortley Montagu had better luck. On 1 April 1718 she wrote to her friend Sarah Chiswell from Turkey with a detailed account of inoculation:

The smallpox, so fatal and so general amongst us, is here entirely harmless by the invention of engrafting, which is the term they give it. There is a set of women who make it their business to perform the operation … When they are met (commonly fifteen or sixteen together) the old woman comes with a nutshell full of the matter of the best sort of smallpox, and asks what veins you please to have opened. She immediately rips open that you offer to her with a large needle (which gives you no more pain than a common scratch) and puts into the vein as much venom as can lie upon the head of her needle … There is no example of any one that has died in it, and you may believe I am well satisfied of the safety of the experiment, since I intend to try it on my dear little son. I am patriot enough to take pains to bring this useful invention into fashion in England.

Lady Mary did indeed engraft her son Edward, anxiously watching his skin erupt in self-inflicted pustules before subsiding into immunized health. It was a brave moment. On her return to London she inoculated her daughter as well, and became infamous for her championing of the somewhat reckless procedure – a sort of version of the trolley problem so beloved of moral philosophers: do you divert a runaway truck from a line where it will kill five people to another line where it will kill one? Do you deliberately take one risk to avoid a greater one? By then, some doctors had joined the cause, notably Charles Maitland. His inoculation of the children of the Prince of Wales in 1722 was a significant moment in the campaign. But even afterwards there was furious denunciation of the barbaric practice. Misogyny and prejudice lay behind some of it, as when Dr William Wagstaffe pronounced: ‘Posterity will scarcely be brought to believe that an experiment practised only by a few ignorant women amongst an illiterate and unthinking people should on a sudden – and upon a slender Experience – so far obtain in one of the politest nations in the world as to be received into the Royal Palace.’

In America, the practice of inoculation arrived around the same time, through the testimony of an African slave named Onesimus, who told the Boston preacher Cotton Mather about it, possibly as early as 1706, who in turn informed the physician Zabdiel Boylston. For trying inoculation on 300 people, Boylston was subject to fierce criticism and life-threatening violence, abetted by rival physicians, to the point where he had to hide for fourteen days in a secret closet lest the mob kill him. Innovation often requires courage.

In due course inoculation with smallpox itself – later known as variolation – was replaced by the safer but similar practice of vaccination, that is to say, using a related but less dangerous virus than smallpox, an innovation usually credited to Edward Jenner. In 1796 he deliberately infected an eight-year-old boy, James Phipps, with cowpox from blisters on the hands of a milkmaid called Sarah Nelmes, who had caught it from a cow called Blossom. He then tried to infect Phipps with smallpox itself and showed that he was immune to it. This demonstration proof, not the vaccination itself, was his real contribution and the reason he had such an impact. The idea of deliberately giving people cowpox to immunize them against smallpox was by then already thirty years old. It had been tried by a physician named John Fewster in 1768, and by several other doctors in Germany and England in the 1770s. It was already probably in use among farmers before that date.

So, yet again, innovation proves to be gradual and to begin with the unlettered and ordinary people, before the elite takes the credit. That is perhaps a little unfair on Jenner, who, like Lady Mary Wortley, deserves fame for persuading the world to adopt the practice. Napoleon, despite being at war with Britain, had his armies vaccinated, on the strength of Jenner’s advocacy, and awarded Jenner a medal, calling him ‘one of the greatest benefactors of mankind’.

Pasteur’s chickens

Vaccination conquered smallpox so comprehensively that by the 1970s the disease – once the greatest taker of human lives on the planet – had died out altogether. The last case of the more deadly strain, Variola major, was in Bangladesh in October 1975. Rahima Banu, then three years old, survived and is still alive. The last case of Variola minor was in October 1977 in Somalia. Ali Maow Maalin, who was an adult when he caught it, also survived, working for most of his life on the campaign against polio, and dying in 2013 of malaria.

Vaccination exemplifies a common feature of innovation: that use often precedes understanding. Throughout history, technologies and inventions have been deployed successfully without scientific understanding of why they work. To a rational person in the eighteenth century, Lady Mary’s idea that exposure to one strain of a fatal disease could protect against that disease must have seemed crazy. There was no rational basis to it. It was not until the late nineteenth century that Louis Pasteur began to explain how and why vaccination worked.

Pasteur proved that germs were microscopic organisms by boiling a fermented liquid and showing that it remained inert and could not generate further fermentation unless exposed to germs carried in on the breeze. His final blow was to leave the liquid open to the air but only through a narrow swan-necked vessel whose shape ensured that bacteria did not pass through. He boasted, in 1862: ‘Never will the doctrine of spontaneous generation recover from the mortal blow of this simple experiment.’

If contagious diseases were caused by microbes – the distinction between bacteria and far smaller viruses was still to be made – then could inoculation be explained by a change in the character of the microbe and a change in the human body’s vulnerability to it? Pasteur’s explanation came about as a result of a serendipitous accident. In the summer of 1879 he went on holiday, leaving his assistant, Charles Chamberland, to inoculate some chickens with cholera from an infected chicken broth, as part of a series of experiments to understand the nature of the cholera bacterium. Chamberland forgot and went on holiday himself. When they returned from holiday, and did the experiment, the stale broth proved capable of making the chickens ill, but not killing them.