The Creativity Code: How AI is learning to write, paint and think

Copyright

4th Estate

An imprint of HarperCollinsPublishers

1 London Bridge Street

London SE1 9GF

www.4thEstate.co.uk

First published in Great Britain by 4th Estate in 2019

Copyright © Marcus du Sautoy 2019

Marcus du Sautoy asserts the moral right to be identified as the author of this work.

All reasonable efforts have been made by the author and the publisher to trace the copyright holders of the images and material quoted in this book. In the event that the author or publisher are contacted by any of the untraceable copyright holders after the publication of this book, the author and the publisher will endeavour to rectify the position accordingly.

Diagrams redrawn by Martin Brown

Cover image © Hands of God and Adam, detail from The Creation of Adam, from the Sistine Ceiling, 1511 (fresco) (pre restoration), Buonarroti, Michelangelo (1475-1564) / Vatican Museums and Galleries, Vatican City / Bridgeman Images

Author photo © Oxford University Images/Joby Sessions

A catalogue record for this book is available from the British Library.

All rights reserved under International and Pan-American Copyright Conventions. By payment of the required fees, you have been granted the non-exclusive, non-transferable right to access and read the text of this e-book on-screen. No part of this text may be reproduced, transmitted, down-loaded, decompiled, reverse engineered, or stored in or introduced into any information storage and retrieval system, in any form or by any means, whether electronic or mechanical, now known or hereinafter invented, without the express written permission of HarperCollins.

Source ISBN: 9780008288150

Ebook Edition © March 2019 ISBN: 9780008288167

Version: 2019-01-28

Dedication

To Shani, for all her love and support, creativity and intelligence

CONTENTS

Cover

Title Page

Copyright

Dedication

1 The Lovelace Test

2 Creating Creativity

3 Ready, Steady, Go

4 Algorithms, the Secret to Modern Life

5 From Top Down to Bottom Up

6 Algorithmic Evolution

7 Painting by Numbers

8 Learning from the Masters

9 The Art of Mathematics

10 The Mathematician’s Telescope

11 Music: The Process of Sounding Mathematics

12 The Songwriting Formula

13 DeepMathematics

14 Language Games

15 Let AI Tell You a Story

16 Why We Create: A Meeting of Minds

Footnote

Illustrations

Further Reading

Index

Acknowledgements

By the same author

About the Publisher

1

THE LOVELACE TEST

Works of art make rules; rules do not make works of art.

Claude Debussy

The machine was a thing of beauty. Towers of gears with numbers on their teeth pinned to rods driven by a handle that you turned. The seventeen-year-old Ada Byron was transfixed as she cranked the handle of Charles Babbage’s machine to watch it crunch numbers, calculate squares and cubes and even square roots. Byron had always had a fascination with machines, fanned by the tutors her mother had been happy to provide.

Studying Babbage’s plans some years later for the Analytical Engine, it dawned on Ada, now married to the Earl of Lovelace, that this was more than just a number cruncher. She began to record what it might be capable of. ‘The Analytical Engine does not occupy common ground with mere “calculating machines.” It holds a position wholly its own, and the considerations it suggests are more interesting in their nature.’

Ada Lovelace’s notes are now recognised as the first inroads into the creation of code. That kernel of an idea has blossomed into the artificial intelligence revolution that is sweeping the world today, fuelled by the work of pioneers like Alan Turing, Marvin Minsky and Donald Michie. Yet Lovelace was cautious as to how much any machine could achieve: ‘It is desirable to guard against the possibility of exaggerated ideas that might arise as to the powers of the Analytical Engine. The Analytical Engine has no pretensions whatever to originate anything. It can do whatever we order it to perform.’ Ultimately, she believed, it was limited: you couldn’t get more out than you’d put in.

This idea was a mantra of computer science for many years. It is our shield against the fear that we will set in motion something we can’t control. Some have suggested that to program a machine to be artificially intelligent, you would first have to understand human intelligence.

What is going on inside our heads remains a mystery, but in the last few years a new way of thinking about code has emerged: a shift from a top-down attitude to programming to a bottom-up effort to get the computer to chart its own path. It turns out you don’t have to solve intelligence first. You can allow algorithms to roam the digital landscape and learn just as a child does. Today’s code created by machine learning is making surprisingly insightful moves, spotting previously undiscovered features in medical images, and investing in shrewd trades on the stock market. This generation of coders believes it can finally prove Ada Lovelace wrong: that you can get more out than you programmed in.

Yet there is still one realm of human endeavour that we believe the machines will never be able to touch, and that is creativity. We have this extraordinary ability to imagine and innovate and to create works of art that elevate, expand and transform what it means to be human. These are the outpourings of what I call the human code.

This is code that we believe depends on being human because it is a reflection of what it means to be human. Mozart’s requiem allows us to contemplate our own mortality. Witnessing a performance of Othello gives us the chance to navigate our emotional landscape of love and jealousy. A Rembrandt portrait seems to capture so much more than just what the sitter looks like. How can a machine ever hope to replace or even to compete with Mozart, Shakespeare or Rembrandt?

I should declare at the outset that my field of reference is dominated by the artistic output of the West. This is the art I know, this is the music I have been brought up on, the literature that dominates my reading. It would be fascinating to know if art from other cultures might be more amenable to being captured by the output of a machine, but my suspicion is that there is a universal challenge here that transcends cultural boundaries. And so although I make some apology for my Western-focused viewpoint, I think it will provide a suitable benchmark for the creativity of our digital rivals.

Of course, human creativity extends beyond the arts: the molecular gastronomy of the Michelin-star chef Heston Blumenthal; the football trickery of the Dutch striker Johan Cruyff; the curvaceous buildings of Zaha Hadid; the invention of the Rubik’s cube by the Hungarian Ernö Rubik. Even the creation of code to make a game like Minecraft should be regarded as part of some of the great acts of human creativity.

More unexpectedly creativity is an important part of my own world of mathematics. One of the things that drives me to spend hours at my desk conjuring up equations and penning proofs is the allure of creating something new. My greatest moment of creativity, one that I go back to again and again, is the time I conceived of a new symmetrical object. No one knew this object was possible. But after years of hard work and a momentary flash of white-hot inspiration I wrote on my yellow notepad the blueprint for this novel shape. That sheer buzz of excitement is the allure of creativity.

But what do we really mean by this shape-shifting term? Those who have tried to pin it down usually circle around three ideas: creativity is the drive to come up with something that is new and surprising and that has value.

It turns out it’s easy to make something new. I can get my computer to churn out endless proposals for new symmetrical objects. It’s the surprise and value that are more difficult to produce. In the case of my symmetrical creation, I was legitimately surprised by what I’d cooked up, and so were other mathematicians. No one was expecting the strange new connection I’d discovered between this symmetrical object and the unrelated subject of number theory. The fact that this object suggested a new way of understanding an area of mathematics that is full of unsolved problems is what gave it value.

We all get sucked into patterns of thought. We think we see how the story will evolve and then suddenly we are taken in a new direction. This element of surprise makes us take notice. It is probably why we get a rush when we encounter an act of creativity, either our own or someone else’s.

But what gives something value? Is it simply a question of price? Does it have to be recognised by others? I might value a poem or a painting I’ve created but my conception of its value is unlikely to be shared more widely. A surprising novel with lots of plot twists could be of relatively little value. But a new and surprising approach to storytelling or architecture or music that begins to be adopted by others and that changes the way we see or experience things will generally be recognised as having value. This is what Kant refers to as ‘exemplary originality’, an original act that becomes an inspiration for others. This form of creativity has long been thought to be uniquely human.

And yet all of these expressions of creativity are at some level the products of neuronal and chemical activity. This is the human code that millions of years of evolution has honed inside our brains. As you begin to unpick the creative outpourings of the human species you start to see that there are rules at the heart of the creative process. Could our creativity be more algorithmic and rule-based than we might want to acknowledge?

The challenge of this book is to push the new AI to its limits to see whether it can match or even surpass the marvels of our human code. Can a machine paint, compose music or write a novel? It may not be able to compete with Mozart, Shakespeare or Picasso, but could it be as creative as our children when they write a story or paint a scene? By interacting with the art that moves us and understanding what distinguishes it from the mundane and bland, could a machine learn to be creative? Not only that, could it extend our own creativity and help us see opportunities we are missing?

Creativity is a slippery word that can be understood in many different ways in different circumstances. I will mostly focus on the challenge of creativity in the arts, but that does not mean this is the only sort of creativity possible. My daughters are being creative when they build their castles in Lego. My son is heralded as a creative midfielder when he leads his football team to victory. We can solve everyday problems creatively, and run organisations creatively. And, as I shall illustrate, mathematics is a much more creative subject than many recognise, a creativity that actually shares much in common with the creative arts.

The creative impulse is a key part of what distinguishes humans from other animals and yet we often let it stagnate inside us, falling into the trap of becoming slaves to our formulaic lives, to routine. Being creative requires a jolt to take us out of the smooth paths we carve out each day. That is where a machine might help: perhaps it could give us that jolt, throw up a new suggestion, stop us from simply repeating the same algorithm each day. The machines might ultimately help us, as humans, to behave less like machines.

You may ask why a mathematician is offering to take you on this journey. The simple answer is that AI, machine learning, algorithms and code are all mathematical at heart. If you want to understand how and why the algorithms that control modern life are doing what they do, you need to understand the mathematical rules that underpin them. If you don’t, you will be pushed and pulled around by the machines.

AI is challenging us to the core as it reveals how many of the tasks humans engage in can be done equally well, if not better, by machines. But rather than focus on a future of driverless cars and computerised medicine, this book sets out to explore whether these algorithms can compete meaningfully with the power of the human code. Can computers be creative? What does it mean to be creative? How much of our emotional response to art is a product of our brains responding to pattern and structure? These are some of the things we will explore.

But this isn’t just an interesting intellectual challenge. Just as the artistic output of humans allows us to get some insight into the complex human code that runs our brains, we will see how the art generated by computers provides a surprisingly powerful way to understand how the code is working. One of the challenges of code emerging in this bottom-up fashion is that the coders often don’t really understand how the final code works. Why is it making that decision? The art it creates may provide a powerful lens through which to gain access to the subconscious decisions of the new code. And it may also reveal limitations and dangers that are inherent in creating code that we don’t fully understand.

There is another, more personal, reason for wanting to go on this journey. I am going through a very existential crisis. I have found myself wondering, with the onslaught of new developments in AI, if the job of mathematician will still be available to humans in decades to come. Mathematics is a subject of numbers and logic. Isn’t that what a computer does best?

Part of my defence against the computers knocking on the door of the department, wanting their place at the table, is that as much as mathematics is about numbers and logic, it is a highly creative subject, involving beauty and aesthetics. I want to argue in this book that the mathematics we share in our seminars and journals isn’t just the result of humans cranking a mechanical handle. Intuition and artistic sensitivity are important qualities for making a good mathematician. Surely these are traits that can never be programmed into a machine. Or can they?

This is why, as a mathematician, I am attentive to how successful the new AI is being in gaining entry to the world’s galleries, concert halls and publishing houses. The great German mathematician Karl Weierstrass once wrote: ‘a mathematician that is not something of a poet will never be a true mathematician.’ As Ada Lovelace perfectly encapsulates, you need a bit of Byron as much as Babbage. Although she thought machines were limited, Lovelace began to realise the potential of these machines of cogs and gears to express a more artistic side of its character:

It might act upon other things besides number … supposing, for instance, that the fundamental relations of pitched sounds in the science of harmony and of musical composition were susceptible of such expression and adaptations, the engine might compose elaborate and scientific pieces of music of any degree of complexity or extent.

Yet she believed that any act of creativity would lie with the coder, not the machine. Is it possible to shift the weight of responsibility more towards the code? The current generation of coders believes it is.

At the dawn of AI, Alan Turing famously proposed a test to measure intelligence in a computer. I would now like to propose a new test: the Lovelace Test. To pass the Lovelace Test, an algorithm must originate a creative work of art such that the process is repeatable (i.e. it isn’t the result of a hardware error) and yet the programmer is unable to explain how the algorithm produced its output. This is what we are challenging the machines to do: to come up with something new, surprising and of value. For a machine to be deemed truly creative requires one extra step: its contribution should be more than an expression of the coder’s creativity or that of the person who built the data set. That is the challenge Ada Lovelace believed was insurmountable.

2

CREATING CREATIVITY

The chief enemy of creativity is good sense.

Pablo Picasso

The value placed on creativity in modern times has led to a range of writers and thinkers trying to articulate what it is, how to stimulate it, and why it is important. It was while sitting on a committee at the Royal Society assessing what impact machine learning was likely to have on society in the coming decades that I first encountered the theories of the cognitive scientist Margaret Boden. Her ideas on creativity struck me as the most relevant when it came to addressing or evaluating creativity in machines.

Boden is an original thinker who over the decades has managed to fuse many different disciplines: philosopher, psychologist, physician, AI expert and cognitive scientist. In her eighties now, with white hair flying like sparks and an ever-active brain, she is enjoying engaging enthusiastically with the prospect of what these ‘tin cans’, as she likes to call computers, might be capable of. To this end, she has identified three different types of human creativity.

Exploratory creativity involves taking what is already there and exploring its outer edges, extending the limits of what is possible while remaining bound by the rules. Bach’s music is the culmination of a journey Baroque composers embarked on to explore tonality by weaving together different voices. His preludes and fugues push the boundaries of what is possible before breaking the genre open and entering the Classical era of Mozart and Beethoven. Renoir and Pissarro reconceived how we could visualise nature and the world around us, but it was Claude Monet who really pushed the boundaries, painting his water lilies over and over until his flecks of colour dissolved into a new form of abstraction.

Mathematics revels in this type of creativity. The classification of finite simple groups is a tour de force of exploratory creativity. Starting from the simple definition of a group of symmetries – a structure defined by four simple axioms – mathematicians spent 150 years producing a list of every conceivable element of symmetry, culminating in the discovery of the Monster Symmetry Group, which has more symmetries than there are atoms in the Earth and yet fits into no pattern of other groups. This form of mathematical creativity involves pushing the limits while adhering to the rules of the game. It is like the explorer who thrusts into the unknown but is still bound by the limits of our planet.

Boden believes that exploration accounts for 97 per cent of human creativity. This is the sort of creativity that computers excel at: pushing a pattern or set of rules to the extremes is perfect for a computational mechanism that can perform many more calculations than the human brain. But is it enough? When we think of truly original creative acts, we generally imagine something more utterly unexpected.

The second sort of creativity involves combination. Think of how an artist might take two completely different constructs and seek to combine them. Often the rules governing one world will suggest an interesting new framework for the other. Combination is a very powerful tool in the realm of mathematical creativity. The eventual solution of the Poincaré Conjecture, which describes the possible shapes of our universe, was arrived at by applying very different tools to understand flow over surfaces. It was the creative genius of Grigori Perelman which realised that the way a liquid flows over a surface could unexpectedly help to classify the possible surfaces that might exist.

My own research takes tools from number theory to understand primes and applies them to classify possible symmetries. The symmetries of geometric objects at first sight don’t look anything like numbers. But applying the language that has helped us to navigate the mysteries of the primes and replacing primes by symmetrical objects has revealed surprising new insights into the theory of symmetry.

The arts have also benefited greatly from this form of cross-fertilisation. Philip Glass took ideas he learned from working with Ravi Shankar and used them to create the additive process that is at the heart of his minimalist music. Zaha Hadid combined her knowledge of architecture with her love of the pure forms of the Russian painter Kasimir Malevich to create a unique style of curvaceous buildings. In cooking, too, creative master chefs have fused cuisines from opposite ends of the globe.

There are interesting hints that this sort of creativity might also be perfect for the world of AI. Take an algorithm that plays the blues and combine it with the music of Boulez and you will end up with a strange hybrid composition that might just create a new sound world. Of course, it could also be a dismal cacophony. The coder needs to find two genres that can be fused algorithmically in an interesting way.

It is Boden’s third form of creativity that is the more mysterious and elusive, and that is transformational creativity. This describes those rare moments that are complete game changers. Every art form has these gear shifts. Think of Picasso and Cubism, Schoenberg and atonality, Joyce and modernism. They are like phase changes, when water suddenly goes from a liquid to a gas. This was the image Goethe hit on when he sought to describe wrestling for two years with how to write The Sorrows of Young Werther, only for a chance event to act as a sudden catalyst: ‘At that instant, the plan of Werther was found; the whole shot together from all directions, and became a solid mass, as the water in a vase, which is just at the freezing point, is changed by the slightest concussion into ice.’

Quite often these transformational moments hinge on changing the rules of the game, or dropping an assumption that previous generations had been working under. The square of a number is always positive. All molecules come in long lines not chains. Music must be written inside a harmonic scale structure. Faces have eyes on either side of the nose. At first glance it would seem hard to program such a decisive break, and yet there is a meta-rule for this type of creativity. You start by dropping constraints and see what emerges. The art, the creative act, is to choose what to drop or what fresh constraint to introduce such that you end up with a new thing of value.

If I were asked to identify a transformational moment in mathematics, the creation of the square root of minus one in the mid-sixteenth century would be a good candidate. This was a number that many mathematicians believed did not exist. It was referred to as an imaginary number (a derogatory term Descartes came up with to indicate that of course there was no such thing). And yet its creation did not contradict previous mathematics. It turned out it had been our mistake to exclude it. How can a computer come up with the concept of the square root of minus one when the data it is fed will tell it that there is no number whose square can be negative? A truly creative act sometimes requires us to step outside the system and create a new reality. Can a complex algorithm do that?