The Music of the Primes: Why an unsolved problem in mathematics matters

The prime number staircase – the graph counts the cumulative number of primes up to 100.

On this small scale the result is a jumpy staircase, where it is hard to predict how long one has to wait for the next step to appear. We are still seeing at this scale the minutiae of the primes, the individual notes.

Now step back and look at the same function where N is taken over a much wider range of numbers, say counting the primes up to 100,000.

The prime number staircase counting primes up to 100,000.

The individual steps themselves become insignificant and we end up seeing the overarching trend of this function creeping up. This was the big theme that Gauss had heard and could imitate using the logarithm function.

The revelation that the graph appears to climb so smoothly, even though the primes themselves are so unpredictable, is one of the most miraculous in mathematics and represents one of the high points in the story of the primes. On the back page of his book of logarithms, Gauss recorded the discovery of his formula for the number of primes up to N in terms of the logarithm function. Yet despite the importance of the discovery, Gauss told no one what he had found. The most the world heard of his revelation were the cryptic words, ‘You have no idea how much poetry there is in a table of logarithms.’

Why Gauss was so reticent about something so momentous is a mystery. It is true that he had only found early evidence of some connection between primes and the logarithm function. He knew that he had absolutely no explanation or proof of why these two things should have anything to do with each other. It wasn’t clear that this pattern might not suddenly disappear as you counted higher. Gauss’s reluctance to announce unproved results marked a turning point in mathematical history. Although the Greeks had introduced the idea of proof as an important component of the mathematical process, mathematicians before Gauss’s time were much more interested in scientific speculation about mathematics. If the mathematics worked, they weren’t too concerned about a rigorous justification of why it worked. Mathematics was still the tool of the other sciences.

Gauss broke from the past by stressing the value of proof. For him, presenting proofs was the primary goal of the mathematician, an ethos which has remained fundamental to this day. Without a proof of the connection between logarithms and primes, Gauss’s discovery was worthless to him. The freedom that the patronage of the Duke of Brunswick permitted him meant he could be quite choosy, almost indulgent about the mathematics he produced. His prime motivation was not fame and recognition but a personal understanding of the subject he loved. His seal bore the motto Pauca sed matura (‘Few but ripe’). Unless a result had reached full maturity it remained an entry in his diary or a doodle at the back of his table of logarithms.

For Gauss, mathematics was a private pursuit. He even encrypted entries in his diary using his own secret language. Some of them are easy to unravel. For example, on July 10, 1796, Gauss wrote Archimedes’ famous declaration ‘Eureka!’ followed by the equation

num = Δ + Δ + Δ

which represents his discovery that every number can be written as the sum of three numbers from the list of triangular numbers, 1, 3, 6, 10, 15, 21, 28, …, those numbers for which Gauss had produced his schoolroom formula. For example, 50 = 1 + 21 + 28. But other entries remain a complete mystery. No one has been able to unravel what Gauss meant when he wrote on October 11, 1796, ‘Vicimus GEGAN’. Some have blamed Gauss’s failure to disseminate his discoveries for holding back the development of mathematics by half a century. If he had bothered to explain half of what he had discovered and not been so cryptic in the explanations he did offer, mathematics might have advanced at a quicker pace.

Some people believe that Gauss kept his results to himself because the Paris Academy had rejected his great treatise on number theory, Disquisitiones Arithmeticae, as obscure and dense. Having been stung by rejection, to protect himself from any further humiliation he insisted that every last piece of the mathematical jigsaw be in place before he would consider publishing anything. Disquisitiones Arithmeticae did not receive immediate acclaim partly because Gauss continued to be cryptic even in the work he did expose to public view. He always claimed that mathematics was like a piece of architecture. An architect never leaves the scaffolding for people to see how the building was constructed. This was not a philosophy that helped mathematicians to penetrate Gauss’s mathematics.

But there were other reasons why Paris was not as receptive to Gauss’s ideas as he had hoped. By the end of the eighteenth century, mathematics in Paris was becoming ever more dedicated to serving the demands of an increasingly industrialised state. The Revolution of 1789 had shown Napoleon the need for a more centralised teaching of military engineering, and he responded by founding the École Polytechnique to further his war aims. ‘The advancement and perfection of mathematics are intimately connected with the prosperity of the State,’ Napoleon declared. French mathematics was dedicated to solving problems of ballistics and hydraulics. But despite this emphasis on the practical needs of the state, Paris still boasted some of the leading pure mathematicians in Europe.

One of the great authorities in Paris was Adrien-Marie Legendre, who was born twenty-five years before Gauss. Portraits of Legendre depict a rather puffed-up gentleman with a round, chubby face. In contrast to Gauss, Legendre came from a wealthy family but he had lost his fortune during the Revolution and been forced to rely instead on his mathematical talents for his livelihood. He too was interested in the primes and number theory, and in 1798, six years after Gauss’s childhood calculations, he announced his discovery of the experimental connection between primes and logarithms.

Although it would later be proved that Gauss had indeed beaten Legendre to the discovery, Legendre did nonetheless improve on the estimate for the number of primes up to N. Gauss had guessed that there were roughly N/log(N) primes up to N. Although this was close, it was found to gradually drift away from the true number of primes as N got larger and larger. Here is a comparison of Gauss’s childhood guess, shown as the lower plot, with the true number of primes, the upper plot:

A comparison between Gauss’s guess and the true number of primes.

This graph reveals that although Gauss was on to something, there still seemed to be room for improvement.

Legendre’s improvement was to replace the approximation N/log(N) by

thus introducing a small correction which had the effect of shifting Gauss’s curve up towards the true number of primes. As far as the values of these functions that were then within computational reach, it was impossible to distinguish the two graphs of π(N) and Legendre’s estimate. Legendre, steeped in the prevailing preoccupation with the practical application of mathematics, was much less reluctant to stick his neck out and make some prediction about the connection between primes and logarithms. He was not a man scared to circulate unproven ideas, or even proofs with gaps in them. In 1808 he published his guess at the number of primes in a book about number theory entitled Théorie des Nombres.

The controversy over who first discovered the connection between primes and logarithms led to a bitter dispute between Legendre and Gauss. It was not limited to the argument about primes – Legendre even claimed that he had been the first to discover Gauss’s method for establishing the motion of Ceres. Time and again, Legendre’s assertion that he had uncovered some mathematical truth would be countered by an announcement by Gauss that he had already plundered that particular treasure. Gauss commented in a letter of July 30, 1806, written to an astronomical colleague named Schumacher, ‘It seems to be my fate to concur in nearly all my theoretical works with Legendre.’

In his lifetime, Gauss was too proud to get involved in open battles of priority. When Gauss’s papers and correspondence were examined after his death, it became clear that due credit invariably went to Gauss. It wasn’t until 1849 that the world learnt that Gauss had beaten Legendre to the connection between primes and logarithms, which Gauss disclosed in a letter to a fellow mathematician and astronomer, Johann Encke, written on Christmas Eve of that year.

Given the data available at the start of the nineteenth century, Legendre’s function was much better than Gauss’s formula as an approximation to the number of primes up to some number N. But the appearance of the rather ugly correction term 1.083 66 made mathematicians believe that something better and more natural must exist to capture the behaviour of the prime numbers.

Such ugly numbers may be commonplace in other sciences, but it is remarkable how often the mathematical world favours the most aesthetic possible construction. As we shall see, Riemann’s Hypothesis can be interpreted as an example of a general philosophy among mathematicians that, given a choice between an ugly world and an aesthetic one. Nature always chooses the latter. It is a constant source of amazement for most mathematicians that mathematics should be like this, and explains why they so often get wound up about the beauty of their subject.

It is perhaps not surprising that in later life Gauss further refined his guess and arrived at an even more accurate function, one which was also much more beautiful. In the same Christmas Eve letter that Gauss wrote to Encke, he explains how he subsequently discovered how to go one better than Legendre’s improvement. What Gauss did was to go back to his very first investigations as a child. He had calculated that amongst the first 100 numbers, 1 in 4 are prime. When he considered the first 1,000 numbers the chance that a number is prime went down to 1 in 6. Gauss realised that the higher you count, the smaller the chance that a number will be prime.

So Gauss formed a picture in his mind of how Nature might have decided which numbers were going to be prime and which were not. Since their distribution looked so random, might tossing a coin not be a good model for choosing primes? Did Nature toss a coin – heads it’s prime, tails it’s not? Now, thought Gauss, the coin could be weighted so that instead of landing heads half the time, it lands heads with probability 1/log(N). So the probability that the number 1,000,000 is prime should be interpreted as l/log(1,000,000), which is about 1/15. The chances that each number N is a prime gets smaller as N gets bigger because the probability 1/log(N) of coming up heads is getting smaller.

This is just a heuristic argument because 1,000,000 or any other particular number is either prime or it isn’t. No toss of a coin can alter that. Although Gauss’s mental model was useless at predicting whether a number is prime, he found it very powerful at making predictions about the less specific question of how many primes one might expect to encounter as one counted higher. He used it to estimate the number of primes you should get after tossing the prime number coin N times. With a normal coin which lands heads with probability ½, the number of heads should be ½N. But the probability with the prime number coin is getting smaller with each toss. In Gauss’s model the number of primes is predicted to be

Gauss actually went one step further to produce a function which he called the logarithmic integral, denoted by Li(N). The construction of this new function was based on a slight variation of the above sum of probabilities, and it turned out to be stunningly accurate.

By the time Gauss, in his seventies, wrote to Encke, he had constructed tables of primes up to 3,000,000. ‘I very often used an idle quarter of an hour to count through another chiliad [an interval of 1,000 numbers] here and there’ in his search for prime numbers. His estimate for primes less than 3,000,000 using his logarithmic integral is a mere seven hundredths of 1 per cent off the mark. Legendre had managed to massage his ugly formula to match π(N) for small N, so with the data available at the time it looked as if Legendre’s formula was superior. As more extensive tables began to be drawn up, they revealed that Legendre’s estimate grew far less accurate for primes beyond 10,000,000. A professor at the University of Prague, Jakub Kulik, spent twenty years single-handedly constructing tables of primes for numbers up to 100,000,000. The eight volumes of this gargantuan effort, completed in 1863, were never published but were deposited in the archives of the Academy of Sciences in Vienna. Although the second volume has gone astray, the tables already revealed that Gauss’s method, based on his Li(N) function, was once again outstripping Legendre’s. Modern tables show just how much better Gauss’s intuition was. For example, his estimate for the number of primes up to 1016 (i.e. 10,000,000,000,000,000) deviates from the correct value by just one ten-millionth of 1 per cent, whilst Legendre’s is now off by one-tenth of 1 per cent. Gauss’s theoretical analysis had triumphed over Legendre’s attempts to manipulate his formula to match the available data.

Gauss noticed a curious feature about his method. Based on what he knew about the primes up to 3,000,000, he could see that his formula Li(N always appeared to overestimate the number of primes. He conjectured that this would always be the case. And who wouldn’t back Gauss’s hunch, now that modern numerical evidence confirms Gauss’s conjecture up to 1016? Certainly any experiment that produced the same result 1016 times would be regarded as pretty convincing evidence in most laboratories – but not in the mathematician’s. For once, one of Gauss’s intuitive guesses turned out to be wrong. But although mathematicians have now proved that eventually π(N) must sometimes overtake Li(N), no one has ever seen it happen because we can’t count far enough yet.

A comparison of the graphs of π(N) and Li(N) shows such a good match that over a large range it is barely possible to distinguish the two. I should stress that a magnifying glass applied to any portion of such a picture will show that the functions are different. The graph of π(N) looks like a staircase, whilst Li(N) is a smooth graph with no sharp jumps.

Gauss had uncovered evidence of the coin that Nature had tossed to choose the primes. The coin was weighted so that a number N has a chance of 1 in log(N) of being prime. But he was still missing a method of predicting precisely the tosses of the coin. That would take the insight of a new generation.

By shifting his perspective, Gauss had perceived a pattern in the primes. His guess became known as the Prime Number Conjecture. To claim Gauss’s prize, mathematicians had to prove that the percentage error between Gauss’s logarithmic integral and the real number of primes gets smaller and smaller the further you count. Gauss had seen this far-off mountain peak, but it was left to future generations to provide a proof, to reveal the pathway to the summit, or to unmask the connection as an illusion.

Many blame the appearance of Ceres for distracting Gauss from proving the Prime Number Conjecture himself. The overnight fame he received at the age of twenty-four steered him towards astronomy, and mathematics no longer had pride of place. When his patron, Duke Ferdinand, was killed by Napoleon in 1806, Gauss was forced to find other employment to support his family. Despite overtures from the Academy in St Petersburg, which was seeking a successor to Euler, he chose instead to accept a position as director of the Observatory in Göttingen, a small university town in Lower Saxony. He spent his time tracking more asteroids through the night sky and completing surveys of the land for the Hanoverian and Danish governments. But he was always thinking about mathematics. Whilst charting the mountains of Hanover he would ponder Euclid’s axiom of parallel lines, and back in the observatory he would continue to expand his table of primes. Gauss had heard the first big theme in the music of the primes. But it was one of his few students, Riemann, who would truly unleash the full force of the hidden harmonies that lay behind the cacophony of the primes.

CHAPTER THREE

Riemann’s Imaginary Mathematical Looking-Glass

Do you not feel and hear it? Do I alone hear this melody so wondrously and gently sounding … Richard Wagner. Tristan und Isolde (Act III, Scene iii)

In 1809 Wilhelm von Humboldt became the education minister for the north German state of Prussia. In a letter to Goethe in 1816 he wrote, ‘I have busied myself here with science a great deal, but I have deeply felt the power antiquity has always wielded over me. The new disgusts me …’ Humboldt favoured a movement away from science as a means to an end, and a return to a more classical tradition of the pursuit of knowledge for its own sake. Previous education schemes had been geared to providing civil servants for the greater glory of Prussia. From now on, more emphasis was to be placed on education serving the needs of the individual rather than the state.

In his role as a thinker and civil servant, Humboldt enacted a revolution with far-reaching effects. New schools, called Gymnasiums, were created across Prussia and the neighbouring state of Hanover. Eventually the teachers in these schools were not to be members of the clergy, as in the old education system, but graduates of the new universities and polytechnics that were built during this period.

The jewel in the crown was Berlin University, founded in 1810 during the French occupation. Humboldt called it the ‘mother of all modern universities’. Housed in what had once been the palace of Prince Heinrich of Prussia on the grand boulevard Unter den Linden, the university would for the first time promote research alongside teaching. ‘University teaching not only enables an understanding of the unity of science but also its furtherance,’ Humboldt declared. Despite his passion for the Ancient World, under his guidance the university pioneered the introduction of new disciplines to sit beside the classical faculties of Law, Medicine, Philosophy and Theology.

For the first time, the study of mathematics was to form a major part of the curriculum in the new Gymnasiums and universities. Students were encouraged to study mathematics for its own sake and not simply as a servant of the other sciences. This contrasted starkly with Napoleon’s educational reforms, which saw mathematics harnessed to further French military aims. Carl Jacobi, one of the professors in Berlin, wrote to Legendre in Paris in 1830 about the French mathematician Joseph Fourier, who had reproached the German school of thought for ignoring more practical problems:

It is true that Fourier was of the opinion that the principal object of mathematics is public use and the explanation of natural phenomena; but a philosopher like him ought to have known that the sole object of the science is the honour of the human spirit, and that on this view a problem in the theory of numbers is worth as much as a problem of the system of the world.

For Napoleon, it was education that would finally destroy the arcane rules of the ancien régime. His recognition that education was the backbone for building his new France had led to the establishment in Paris of some of the institutes which are still famous today. Not only were the colleges meritocratic, allowing students from all backgrounds to attend, but also the educational philosophy put a greater emphasis on education and science serving society. One of the French Revolutionary regional officers wrote to a professor of mathematics in 1794, commending him on teaching a course in ‘Republican arithmetic’: ‘Citizen. The Revolution not only improves our morals and paves the way for our happiness and that of future generations, it even unlooses the shackles that hold back scientific progress.’

Humboldt’s approach to mathematics was very different from this utilitarian philosophy that prevailed across the border. The liberating effect of Germany’s educational revolution was to have a great impact on mathematicians’ understanding of many aspects of their field. It would allow them to establish a new, more abstract language of mathematics. In particular, it would revolutionise the study of prime numbers.

One town that benefited from Humboldt’s initiatives was Lüneberg, in Hanover. Lüneberg, once a thriving commercial centre, was now a town in decline. Its narrow streets paved with cobblestones were no longer buzzing with the business it had seen in previous centuries. But in 1829 a new building was erected amidst the tall towers of the three Gothic churches in Lüneberg: the Gymnasium Johanneum.

By the early 1840s the new school was flourishing. Its director, Schmalfuss, was a keen proponent of the neo-humanist ideals initiated by Humboldt. His library reflected his enlightened views: it featured not only the classics and the works of modern German writers, but also volumes from farther afield. In particular, Schmalfuss managed to get his hands on books coming out of Paris, the powerhouse of European intellectual activity during the first half of the century.

Schmalfuss had just accepted a new boy at the Gymnasium Johanneum, Bernhard Riemann. Riemann was very shy and found it difficult to make friends. He had been attending the Gymnasium in the town of Hanover, where he had been boarding with his grandmother, but when she died, in 1842, he was forced to move to Lüneberg where he could board with one of the teachers. Joining the school after all his contemporaries had established friendships did not make life easy for Riemann. He was desperately homesick and was teased by the other children. He would rather walk the long distance back to his father’s house in Quickborn than play with his contemporaries.

Riemann’s father, the pastor in Quickborn, had high expectations for his son. Although Bernhard was unhappy at school, he worked hard and conscientiously, determined not to disappoint his father. But he had to battle with an almost disabling streak of perfectionism. His teachers would often get frustrated at Riemann’s inability to submit his work. Unless it was perfect, the boy could not bear to suffer the indignity of anything less than full marks. His teachers began to doubt whether Riemann would ever be able to pass his final examinations.

It was Schmalfuss who saw a way to bring the young boy on and exploit his obsession with perfection. Early on, Schmalfuss had spotted Riemann’s special mathematical skills and was keen to stimulate the student’s abilities. He allowed Riemann the freedom of his library, with its fine collection of books on mathematics, where the boy could escape the social pressures of his classmates. The library opened up a whole new world for Riemann, a place where he felt at home and in control. Suddenly here he was in a perfect, idealised mathematical world where proof prevented any collapse of this new world around him, and numbers became his friends.