This article is archived and some details may be out of date.

Infinite progress: Why ideas will continue to drive growth.

June 2022

Key Points

  • Scientific enquiry and its search for ‘good explanations’ has led to the longest period of growing wealth and wellbeing in human history
  • Innovators combine existing ideas to create new ones and then test them, creating an ever-expanding resource
  • Existing limits on progress will be overcome by advanced technologies, such as quantum computing

Please remember that the value of the investment can fall and you may not get back the amount invested. Past performance is not a guide to future returns.

Progress, as defined by growth in wealth and wellbeing, is not guaranteed. We only take it for granted because it has been a consistent feature in our lives.

You can’t say the same for our distant ancestors. For them, stasis was the norm. Indeed, the last several hundred years represent the only period that we sustained significant progress over many generations.

There were short, localised periods of prosperity in, for example, Athens and Venice. But these mini-enlightenments fizzled out quickly.

Before the Industrial Revolution, economies didn’t grow much. And when they did, increases in total output tended to be offset by population growth.

On the rare occasions when GDP per capita did grow, it was often associated with sudden population drops. For example, in 1300s England per capita incomes rose by 50 per cent because the Black Death wiped out half the population, meaning there was more to go around.

Fortunately, pandemics are no longer the most reliable way to create abundance.


The scientific method

The late 1600s marked a turning point. Ever since, progress has been spectacularly steady. The power of compounding has driven GDP per capita up 20-fold in the last 350 years. 

And it’s not just per capita wealth that grew. We’ve enjoyed improvements in scientific understanding, cultural richness and welfare.

So why the change? This inflection point coincided with:

  • the Scientific Revolution
  • the Age of Enlightenment
  • the subsequent Industrial Revolution

This was when the search for knowledge shifted away from customs and authority and towards conjecture and criticism. The scientific method was popularised.

People rejected mysticism in favour of good explanations and testable theories.


Good explanations 

As the physicist David Deutsch puts it: “All progress, both theoretical and practical, has resulted from a single human activity: the quest for what I call good explanations.”

In Deutsch’s terms, ‘good explanations’ are hard to vary because when you change the details you ruin the theory.

Vague theories along the lines of “such and such a supernatural being did it” fail the good explanations test. You can easily change the details without ruining the hypothesis because the protagonist’s name doesn’t matter.

On the contrary, our theory for why the Earth has seasons is hard to vary and is, therefore, a good explanation. For example, if you assume a change in the planet’s axis tilt, the expected outcomes alter and become at odds with the evidence.


Never-ending progress 

Deutsch suggests a three-stage knowledge generation process:

  1. we create theories through conjecture
  2. we test conjectures by subjecting them to criticism
  3. conjectures that survive the criticism become new knowledge

We create conjectures by rearranging or recombining existing ideas. Because of this, knowledge creation is a compounding process that feeds on itself. The more existing knowledge we have, the more new knowledge we can create.

It is also never-ending. Creating new knowledge helps solve existing problems but also creates new problems. This is a good thing because there is always room for further progress.

Deutsch argues that humans, as universal problem solvers, have an unlimited capacity for progress. So long as we continue creating knowledge via conjecture and criticism, the universe is infinite and therefore so is our capacity for progress.

This may seem like an extreme position. But it’s unwise to take the other side of the argument.

Clever people have long predicted the end of progress. The Reverend Thomas Malthus is perhaps the most famous example. He worried that population growth would eventually outstrip food supplies. But technological progress helped ensure that didn’t happen.

Deutsch also points to the physicist Albert Michelson, who claimed in 1899 that the most important fundamental laws of physics had been discovered and the chances of them being replaced were “exceedingly remote”.

Six years later, Einstein published his theory of Special Relativity. A decade later he wrote on General Relativity. Another 12 years on, Heisenberg’s Uncertainty Principle marked the start of modern quantum mechanics, which spawned a host of new problems, many of which still don’t have solutions.

Betting against progress is a bad idea.


Recombining ideas

If we accept that the capacity for progress is unbounded, important questions follow: What shape might it take? Will the rate of progress slow down over time? Or remain steady? Or even accelerate?

The economist Matt Clancy explores this in his blog, What’s New Under The Sun?

As Clancy explains, the late economist Martin Weitzman wrote about the trajectory of future progress in his 1998 paper Recombinant Growth, where he divided innovation into two phases:

  • the generation phase, where you combine two pre-existing ideas
  • a processing phase, where you test the new idea

These phases are analogous to Deutsch’s ‘conjecture’ and ‘criticism’ steps.

Weitzman used Thomas Edison’s search for a filament for his ‘electric candle’ as an example of this combinatorial process.

Edison tried 6,000 materials before settling on a carbonised strip of Japanese bamboo. The effort that went into this discovery chimes with Edison’s remark that genius is 1 per cent inspiration and 99 per cent perspiration.

Innovation requires significant effort.


Combinatorial growth

The wonderful thing about the combinatorial process is that new ideas fuel further ideas. Clancy notes how lightbulbs paved the way for spotlights and vehicle headlights, which in turn unlocked further innovations.

The maths behind this is staggering. We are all familiar with the power of exponential growth. Combinatorial growth makes that look tame.

Mathematically put:

The number of potential pairings between two objects = n*(n-1)/2
Where n is the number of objects

This equation starts slowly but becomes explosive, vastly outstripping an exponential curve:

  • for two ideas, there is one potential combination three ideas can be combined into pairs in three ways
  • for 5, it’s 10
  • for 10, it’s 45
  • for 100, it’s 4,950
  • and 1,000 ideas can be combined into pairs in almost half a million ways

And if we assume that new ideas can be created by combining more than two existing ideas, the maths becomes even more extreme.


Resource-limited growth?

While innovation is initially constrained by the number of ideas, Weitzman’s model suggests the processing of ideas eventually becomes the rate-limiting factor.

Ideas grow super-exponentially, but the economy grows exponentially. There are limited resources available to explore ideas. So rather than accelerating indefinitely, progress settles into an exponential pattern.

This is consistent with what we see in the GDP growth numbers. Indeed, Weitzman suggests this explains the historical pattern of progress.

“The model can thus be invoked,” he wrote, “to rationalise how a sequence of growth rates, which start by hovering near zero during all previous millennia of human history, suddenly take off at some stage like an S-shaped logistic trajectory, finally settling into the trendless high rates of the past century or so that characterise modern economic growth.”

From this, we can argue that the Industrial Revolution was not just a cultural phenomenon but reflected the crossing of a minimum threshold of ideas required for a rapid acceleration of the combinatorial process.


Progress over time

Weitzman’s prediction of steady exponential growth is exciting because it aligns with Deutsch’s infinite progress hypothesis. But Clancy scrutinises this further.

For exponential growth to be constant, we need to assume that in the processing step some constant proportion of tested ideas is viable. But that isn’t necessarily true.

Perhaps it gets harder to combine ideas as we exhaust the low-hanging fruit, leading to a drop in the proportion of viable ideas over time. In this scenario, growth would stall.

Or perhaps it gets easier to create new ideas, as the new ideas themselves spawn new technologies that make it easier to recombine old ideas into new ones. This assumption leads to a finite time singularity!

Clancy cites an alternative scenario, as first proposed by the economist Charles Jones.

Instead of making the binary assumption that new ideas either do or don’t work, Jones assumes that some innovations are more useful than others. He also assumes that the world is composed of lots of distinct activities.

So, a car and a horse address the same activity – getting from A to B. But the car is more useful than the horse in many ways: it is faster, cheaper, and more reliable.

If we define technological progress as the process of getting better at specific activities, then progress becomes more challenging over time. You’re not just trying to create something new and useful. You are trying to replace something that exists already with something better. And the more progress we make, the higher we set the bar.

For example, if we assume that the productivity of new ideas follows a normal distribution, then finding new ideas better than old ones starts easy and then explodes in difficulty.

However, Jones points out that if one assumes combinatorial growth in the number of attempts at generating new ideas, then the two equations net out at exponential growth. We are back to exponential growth again!


Sifting for the best

But Clancy identifies an obvious gap: the above fails to address the resource constraint to combinatorial growth that Weitzman noticed.

Jones gets around this by assuming that the initial sift is costless and that only the best ideas are taken forward to the research and development stage.

It is perfectly plausible that humans could do this, up to a certain level of complexity. Experts routinely sift through competing conjectures in their minds and pick out the most plausible ones without expending any resources other than thought.

Einstein’s great theories were seeded in his mind in the year he spent “loafing aimlessly”, to quote Carlo Rovelli. However, as great as Einstein’s mind was, all human brains have limits. Luckily, we can now supplement the human mind with computers.

This is an example of how technology might help fuel technology’s creation. When human brains reach the point that our cognitive resources can’t sift through the exploding number of ideas, AI could take over and keep us on an exponential track.


New technologies

This is where Clancy and I start to differ. He rightly points out that, with explosive combinatorial growth, even AI will eventually be overwhelmed: the number of potential combinations will exceed the number of atoms in the observable universe.

But Clancy might be making the same mistake as Malthus and Michelson by viewing challenges through the prism of existing technological paradigms.

Traditional computers cannot take us beyond a certain point because of their physical constraints. But new computers will likely come along with greater potential.

For example, quantum computers can theoretically explore problems with more possibilities than there are atoms in the observable universe.

© Andrew Brookes, National Physical Laboratory / Science Photo Library

That may sound like science fiction, but quantum computers are closer to reality than most appreciate:

  • Google announced that in 2019, its Sycamore quantum computer had completed a calculation in 200 seconds that would have taken a conventional computer 10,000 years
  • PsiQuantum, a Baillie Gifford US Growth Trust holding, hopes to have a commercially viable quantum computer working within five years

Progress is unpredictable. As David Deutsch says: “We have no option but to see the world through our best existing explanations – which include our existing misconceptions. And that biases our intuition. Among other things, it inhibits us from conceiving of significant changes.”


Cause for optimism

I’m confident the combined computational capacity of humans and machines will continue to grow over time.

If I’m correct, then the feared cognitive constraints to infinite growth may not materialise, and exponential growth will continue for a long time. Perhaps infinitely.

It is plausible that the process of conjecture and criticism that began with the Industrial Revolution marked the start of an infinite period of progress.

If we view the world through existing paradigms, we are bound to expect progress to end. But we may only be at the beginning. We may always be at the beginning.

There will always be new problems to solve. And with each solution, we take a step forward.

This is an exciting backdrop for investors, entrepreneurs and the wider world. We can all play a role.

In the context of infinite progress, optimism about the future isn’t only rational, it is self-fulfilling.

Find out more about the American Fund,
or for more detail view the American Fund Factsheet

Risks and Important Information

Investment markets can go down as well as up and market conditions can change rapidly. The value of an investment in the fund, and any income from it, can fall as well as rise and investors may not get back the amount invested. The American Fund’s exposure to a single market and currency may increase share price movements. The fund’s concentrated portfolio relative to similar funds may result in large movements in the share price in the short term. The fund’s share price can be volatile due to movements in the prices of the underlying holdings and the basis on which the fund is priced.

The views expressed in this article should not be considered as advice or a recommendation to buy, sell or hold a particular investment. The article contains information and opinion on investments that does not constitute independent investment research, and is therefore not subject to the protections afforded to independent research.

Some of the views expressed are not necessarily those of Baillie Gifford. Investment markets and conditions can change rapidly, therefore the views expressed should not be taken as statements of fact nor should reliance be placed on them when making investment decisions.

Baillie Gifford & Co Limited is wholly owned by Baillie Gifford & Co. Both companies are authorised and regulated by the Financial Conduct Authority and are based at: Calton Square, 1 Greenside Row, Edinburgh EH1 3AN. Baillie Gifford & Co Limited is the OEICs’ Authorised Corporate Director.

FP 22084 10011591

About the authors