One afternoon in February 2016, I watched in rapt attention as a team of scientists unveiled the first definitive evidence for the existence of gravitational waves – ripples in space and time, produced by a pair of massive black holes crashing together 1.3 billion light-years away from Earth. These ripples had been predicted 100 years earlier by Albert Einstein, the inventor of the theory of general relativity, but remained impossible to detect throughout the 20th century. Their discovery was a feat of both technological prowess and human ingenuity, pitting the far-sighted physicists who first came up with the idea for a detector against the initial scepticism of the wider scientific community.
Just a few years before, in 2012, I’d witnessed an equally profound and long-awaited event – the finding of the Higgs boson. Since the mid-1960s, physicists had posited that a fundamental particle ought to exist that would complete the Standard Model of particle physics, the roadmap to the subatomic world. The Higgs would explain why leptons, another fundamental particle, have mass. A machine costing billions of dollars, the Large Hadron Collider (LHC), had been painstakingly assembled near Geneva in Switzerland to look for it. In that case, too, compelling evidence was found, producing an excitement that reverberated around the world.
These discoveries share a common theme. They suggest that fundamental physics is on a roll, and that the foundational theories cemented by the end of the 1970s are perfectly consistent with the data we’re now seeing. The measurements aren’t throwing up anything new or unexpected; all the pieces fit together, and more or less confirm our pre-existing hunches and hypotheses.
But I’m not being entirely honest when I say everything is working out so well. While physicists have been busily verifying ideas devised in the past century, we’ve made almost no progress in figuring out where to go in this one. In fact, we’re at a complete loss at how to explain some of the most fundamental but baffling observations of how our Universe behaves. There is a tremendous, even cosmic, chasm between the physics we know and love, and some of the phenomena that we observe, but simply can’t make head nor tail of. We have no idea how to bridge this chasm – yet we are proceeding, at pace, to construct ever more expensive experiments and observatories in the hope that we will.
I’m primarily a theorist, but I’ve had my own small brush with the buzz of discovery. Twenty years ago, I was moonlighting as a data analyst on two experiments that mapped the cosmic microwave background radiation left over from the (very hot) beginning of the Universe. This relic radiation was emitted when the Universe was 300,000 to 400,000 years old, a minute fraction of its current age, and so serves as an archaeological record of the distant past.
A clever and remarkably consistent model had been built up over the years of what this map of primordial light would look like; it’s mostly based on the established pillars of modern physics, combined with a dash of speculation about how the Universe began. Apart from the speculation, this model is remarkably conservative. For my part, I’d spent some time as a junior physicist in a somewhat maverick environment, working with a bunch of iconoclasts who’d developed an alternative model to what I thought was the boring, conservative model – one involving high-energy structures known as ‘cosmic strings’ that were meant to stir up the firmament and lead to very different patterns of light and dark in the cosmic sky. Even so, years later, on that dark November day that I was picking over data from the experiments, I saw the distinct patterns predicted by the boring model. Surprisingly, given how much time I’d invested in its rival, I was elated. I had the same emotions that would wash over me in 2012 and 2016 – a sense of triumph at the power of fundamental physics to predict things about the natural world with extraordinary accuracy.
These observations are useful in that they confirm the laws of physics that we know. The problem is that they don’t offer much guidance about how to approach data that simply don’t fit with what we already suspect to be the case. One such puzzle concerns the very early Universe – the speculative bit of the boring model I described above. With two more decades of exquisite measurements of the cosmic microwave background, we can now pinpoint the properties of the Universe in the first fractions of a second of its evolution. Aficionados of this period of cosmological history are overwhelmingly convinced that the Universe underwent a very rapid period of expansion, which imprinted what were once quantum ripples in space-time and matter onto the largest scales, and seeded the patterns that we can now see in the relic radiation. Dubbed the theory of inflation – because the Universe inflated uncontrollably during that period – it’s very elegant and mathematically simple, but also highly speculative. The reality is that we have no idea what actually happened at the beginning of the Universe; if it was inflation, what was driving it; how it came about; and how it ended.
We’ll have a much more precise statement about our ignorance but nothing more
If what happened at the beginning of the Universe is a mystery, what’s happening now is no less puzzling. We’ve come to realise that most of the matter in the Universe, the stuff that gravitates around galaxies, for example, is dark. In other words, it doesn’t emit or reflect light like the particles from the Standard Model, or the atoms and molecules that we’re familiar with from the lab. This dark matter outnumbers ordinary matter by a factor of six to one, and underpins any explanation of why the Universe looks the way it does on the largest scales. We know almost nothing about what it actually is, apart from the fact that it coalesces under the pull of gravity. Whether it’s one type of particle or an amalgam of different types of particles, whether it could be something more complex and substantial, like a sea of black holes, or some kind of quantum field, the explanation is completely up for grabs. And that’s been the situation since the idea of dark matter first took off more than 50 years ago.
Dark matter is only a small part of a much wider sea of dark stuff that permeates the Universe. Since the mid-1980s, we’ve had an inkling that the Universe isn’t expanding as we might expect. And, in recent years, we’ve found incontrovertible evidence that the Universe’s expansion is in fact speeding up – driven, it seems, by some form of energy that doesn’t clump under the pull of gravity like ‘ordinary’ energy. Rather, it’s repulsive, pushing space and time apart, and doesn’t emit or interact with light. One possibility is that the dark energy is simply a constant, undiluted by the expansion of the Universe. This cosmological constant was first invented by Einstein when he constructed his general theory of relativity, but was put aside for almost a century as superfluous (and, in his mind, pointless). While it’s favoured by many, it has such bizarre properties, that it jars with the rest of physics. And so, as with the dark matter, we really have no idea what this dark energy is.
These three puzzles – how the Universe began, what dark matter is, and what dark energy consists of – make a compelling science case for future particle accelerators, observatories and satellite missions. Indeed, this decade already promises to be one of plenty on the observational front. A number of telescopes and satellite missions will be assembled and launched with the stated goal of mapping the Universe; discussions are underway to build a more powerful particle accelerator than the LHC; and new gravitational-wave observatories are being designed as we speak. With all of these new instruments, there’ll be much more data than we ever imagined, which might help us understand what’s going on with the Universe.
Except there’s a real risk that, even after spending all this time and money, we won’t really understand more about the Universe than we already do. By going down the route of building bigger and better instruments, sure, our measurements will be much more precise. However, there remains a possibility that all we’ll attain is yet more confidence that there’s some form of dark matter, some form of dark energy, and that the beginning of the Universe did undergo inflation. But we still won’t know what the dark matter or dark energy is, or what drove inflation. We’ll have a much more precise statement about our ignorance but nothing more.
And so we come back to our seemingly unbridgeable, cosmic chasm – between tried and tested physics, and the exotic effects we see playing themselves out on cosmic scales. On the one hand, we have a set of laws of physics, a mathematical model that works perfectly well and that we’ve confirmed with exquisite prediction. Yet we’re at a loss about how to address the true nature of what dark matter, dark energy and inflation really are.
When you don’t have data to help you, the physicist Richard Feynman advised his colleagues to get creative. ‘[P]retend that there are experiments and calculate,’ he said in the 1950s, at a time when there were few observations to provide constraints on the theory of general relativity. ‘In this field we are not pushed by experiments but pulled by imagination.’ A cadre of theoretical physicists has been using this strategy ever since, relying on their powers of invention to bridge the cosmic chasm between confirmed laws and unexplained observations. These physicists don’t all agree on what’s the best way to do this – and, to be brutally honest, they haven’t made much progress in the past few decades.
Those physicists most removed from the data and observations are what we might call the hardcore theorists. What drives them is the grand architecture that underpins the physical laws, the mathematical principles and how they mesh together to give a robust and consistent picture of reality. They still want to make sure that what they work on has some semblance of reality, and agrees with the Universe we see, but they’re less concerned about making immediate predictions. Even so, they come up with possibilities – such as a huge family of new particles, or the notion that space-time should have more than four dimensions. Or even more exotic proposals – eg, that space-time doesn’t really exist as such, and instead emerges from the relationship between the quantum building blocks of the Universe.
While the hardcore theorists are driven by lofty ideals, the phenomenologists are more practical and crave a more immediate connection with the observed Universe. In principle, they take the architecture built by their more mathematically minded colleagues, and figure out what it might really predict and how one might go about testing these predictions. The most conservative phenomenologists restrict themselves to established theories and try to crack really hard technical problems within those bounds. For example, they try to figure out what protons do when they’re slammed against each other in the LHC; how gas orbits, accretes and falls into a black hole; or how the Milky Way is put together.
Just measure things better and hope something will turn up
The more speculative phenomenologists, though, act with more abandon. While they might pay attention to mathematical consistency, what really drives them is a form of instant gratification – the chance to model some observed but unexplained physical phenomenon. They’ll propose a new particle to be the dark matter, for example, or a new quantum field to be the dark energy or to drive inflation. The more adventurous play fast and loose with the laws of physics, arguing that, if their ideas pass the observational tests, then hardcore theorists should take heed.
So, for example, they might jettison part or all of Einstein’s theory of general relativity, and propose a completely different theory of gravity with completely new and baroque properties. The hope is that such a new theory will be able to replicate all the successes of Einstein’s – and, if it doesn’t, they mend it, adding on bits and pieces until it works. These new contraptions might do the job, but often lack the simplicity and elegance of general relativity, and make you wonder if it’s worth all that effort.
Such a maelstrom of approaches comes with its fair share of tension. The hardcore theorists feel that the speculative phenomenologists are too impressionistic, make up too much stuff and have gone too far. The more conservative phenomenologists also feel that too much speculation is a bad sign, but they also direct their opprobrium at the hardcore theorists, whom they consider to be too detached from the experimental and observational data. That leaves us with the more speculative phenomenologists – who, indeed, make up a lot of stuff and, sometimes, are right out there, on the edge of what’s acceptable. When you start dismantling theories that we know work, such as general relativity, only to replace them with untested, less elegant models, it makes you question whether you are, in fact, taking a step back and not forwards.
For decades, this family of unruly theorists has been unable to take any steps towards bridging the cosmic chasm. Yet we’re still somehow expecting them to leap over it and point us in the direction of the new physics. Some people believe that pure thought is the route out of this bind – that, if we figure out the correct mathematical structure of the Universe, the rest will fall into our lap. Such a new, mathematically more consistent theory of the Universe will have explanations for inflation, dark matter and dark energy even without us having to put them in by hand. But that’s a risky strategy, an almighty gamble that might or might not yield results.
There’s a limit to how much more we’ll ever be able to learn about the Universe
At the other extreme, instead of focusing on the overarching mathematics, there are those who think we should stick more closely to specific, smaller problems, and try to figure out the simplest models to resolve them. For example, by figuring out how to solve the problem of dark energy on its own, one might happen to stumble across some physical mechanism that was overlooked when the standard theories of particles and forces were constructed. Uncovering such a mechanism might lead us to go back, reassess and reconstruct some of the foundations. But that’s more of a scattergun approach that could fail to add up to a coherent picture or grand theory; and there’s no clear guide, apart from personal or aesthetic inclinations about how to choose the correct solution.
The only way to actually know if the theorists have finally bridged the gap is to go out and measure. The theorists can tell us where and how to look, while the experimentalists and astronomers build and use the instruments to do so. That’s been the strategy for many decades now. And as the processes we’re interested in are difficult to measure, we build bigger, more elaborate and more expensive instruments than ever. We further refine our measurements until we get to a point where we’ll start to see the details of why the Universe behaves the way it does. Just measure things better and hope something will turn up.
This process of building bigger and better infrastructure is already playing itself out. That means bigger and better maps of the Universe, more precise measurements of gravitational waves and particle collisions at higher energies. But this does force us to confront dramatically escalating costs. At some point, an incremental improvement in the instrumental or observational set-up involves an inordinate increase in the investment required to make it happen.
This can’t go on forever. For example, there’s a limit to how much of the whole sky we can observe, and how far we can look back in time. That means there’s a limit to how much more we’ll ever be able to learn about the Universe. There’s also a limit to how big a particle accelerator we can build. To reach higher energies, we need to construct bigger tunnels but, of course, we live on a finite planet, and so there’s a limit to how long these tunnels can be. And what if, even with all these souped-up observatories and instruments, we still don’t find out what the dark matter is, what dark energy is, or how the Universe began?
An optimist might say that we should carry on exploiting the same seam of ideas and methods that we’ve followed for years, and which have proven to be remarkably fruitful in the past. So, for example, we might make ever more precise measurements of how galaxies coalesce to form the large-scale tapestry of light in the Universe, known as the cosmic web. By monitoring how the cosmic web forms over time, we’re able to tease out how the Universe has been expanding and what role gravity has played in the process. While we’ve been making these measurements for decades now, future observations will be far more precise. And the hope is that with observations such as these we might discover the crucial piece of evidence that allows us to bridge the gap between the theories we know and the mysterious, unknown physics we seem to be detecting. But that’s only if we’re lucky. To my mind, relying on luck isn’t a wise way to solve problems, especially when we’re using a narrow set of approaches.
There’s an alternative point of view, which is to simply explore. We can step back and look at the diversity of views that theorists have advanced over the past few decades – propelled largely by imagination – and so try to unshackle ourselves from the few well-established approaches that have dominated our attempts to answer fundamental questions. In this way, we might unearth ideas and guesses for new directions that will make us look at the data in a different light.
It’s a tricky business and, if you’re not careful, you can end up investing time and energy in ideas that make no real sense from the outset – they’re no better than the crackpot emails that I and many colleagues receive on a regular basis. So a sound principle is to keep your eyes on the fundamentals, being careful not to jettison the things we know to be true. But you need to stray far enough that you don’t end up repeating the mistakes of those who have attempted before you. With luck, you might come up with something new that offers a fresh perspective on the data. And this is where the fun begins. Theorising can be useful and interesting, sure – but you also want to test whether it’s real, and whether it will help you search for new, overlooked, phenomena.
We’re no closer to answering the big questions about dark matter, dark energy and the origins of the Universe
There are a few ways in which we might then look. For a start, we already have some of the raw material: in the process of trying to create these vast data sets, we’re creating a repository of information that might just contain hints of a new physics that we’re overlooking. It’s then just a matter of looking at these data sets in different ways, mining the bits we’ve passed over while we were using the more well-established methods. So, for example, we might look at images of individual galaxies – and we have many millions of them to choose from – to see if we can find any sign of something odd, or new, like the effect of an overlooked type of force or particle, which might change a galaxy’s shape or motion. Or we might look in detail at what happens when black holes collide and merge, to see if there are any unaccounted for jolts and shudders in the gravitational-wave signal that had been discounted. By looking at the data in new ways that theorists have proposed, this approach just might lead to something. For sure, it’s still a risky business and we might end up empty-handed. But at least we don’t have to build a giant new machine to follow up these ideas.
It might also be time to think differently about which experiments we’re deploying in our search for the new physics. While the impetus has been for bigger and better, it makes sense to step back and consider alternatives. There’s a glorious history of research in fundamental physics driving technological change – forcing researchers to come up with ingenious new devices and experiments that allow them to measure elusive phenomena. Some countries are already investing millions of pounds and dollars in research in new quantum technologies for such purposes – peanuts compared with the really big experiments and new colliders under consideration. Efforts are underway, for example, to harness the quantum interference of atoms to open a new window on to gravitational waves. Or, on a different front, tabletop experiments are being devised to look for some of the more exotic forms of dark matter that have been proposed. Again, it’s an exploratory route, guided by controlled theoretical speculation, but the payoffs would be far-reaching.
I’ve spent most of my adult life staring at the cosmic chasm – the abyss between what we know and what we don’t. And while our knowledge of the Universe has improved dramatically in that time, our ignorance has become only more focused. We’re no closer to answering the big questions about dark matter, dark energy and the origins of the Universe than when I started out. This isn’t for lack of trying, and a titanic effort is now underway to try and figure out all these mysterious aspects of the Universe. But there’s no guarantee we’ll succeed, and we might end up never really grasping how the Universe works. That’s why we need to be creative and to explore. As Einstein once said: ‘Let the people know that a new type of thinking is essential if mankind is to survive and move toward higher levels.’ While bridging the cosmic chasm might not be a matter of survival, undoubtedly it’s one of the most pressing challenges of modern science.
This Essay was made possible through the support of a grant to Aeon from the John Templeton Foundation. The opinions expressed in this publication are those of the author and do not necessarily reflect the views of the Foundation. Funders to Aeon Magazine are not involved in editorial decision-making.
