Complexity, Prediction, and Optimisation
What can economists learn from Weather forecasting?
For most of human history, weather forecasting was not a science. It was folklore.
Farmers watched the clouds. Sailors studied the sky. Communities accumulated generations of local knowledge and observation. Some of these observations were genuinely useful. Yet the atmosphere itself remained overwhelmingly complex. Countless variables interacted simultaneously: temperature, pressure, humidity, wind systems, ocean currents, geography and seasonal cycles. The weather appeared far too complicated to model with any useful degree of accuracy.
Compounding the innate complexity of the system was the human mind’s propensity for confirmation bias. Thunder occurs after a disagreement in the tribe – an association is made between the disagreement and the anger of the Thunder God. After that first association, any re-occurrence is taken as confirmation. Any exception is interpreted as the capriciousness of the Gods. They work in mysterious ways, don’t you know.
Meteorologists could easily have responded by declaring the problem impossible. They could have doubled down on misguided intuition.
Instead, they measured more. Developed scientific method as an antidote to human cognitive biases.
As meteorology matured, researchers discovered that complexity does not necessarily imply randomness. Beneath the apparent chaos lay recurring patterns and regularities. Improvements in measurement, data collection, meteorological theories, barometric pressure, mathematics and computational power gradually transformed weather forecasting from an exercise in misguided intuition into a scientific discipline.
Weather forecasts remain incomplete. The atmosphere is still immensely complex. Yet nobody today seriously proposes choosing whether to carry an umbrella by consulting horoscopes, Gods, or competing ideological schools of meteorology. We simply ask:
What does the best available model predict?
What can Economists learn from Medicine?
The science of medicine illustrates a second phenomenon in the study of complex systems.
Medicine is concerned not merely with understanding a system, but with intervening in it.
Many medical debates that once appeared ideological eventually became optimisation problems. Exercise provides health benefits. Excessive exercise can cause strain and injury. Too little nutrition leads to disease. Too much of a good thing can lead to obesity, overdose, or disease.
Many substances are necessary for survival at appropriate doses, yet toxic at high doses. Salt, sunlight, potassium, oxygen, iron, even water.
The scientific question is therefore rarely:
Is salt good or bad?
Or:
Is medicine good or bad?
The scientific question is:
What dosage produces the best outcome?
Benefits, costs, side effects, diminishing returns and trade-offs are measured, quantified and analysed. There may be conflicting processes – some positive and some negative, which counteract each other resulting in an optimum balance. The objective of science is to identify the range that most effectively achieves the desired goal.
Extremes are often easier to argue for than optima are to discover. And human cognitive biases are predisposed to admitting absolute truths and then reinforcing them. Yet science progresses by searching for the optimum, not by defending the extreme.
Where is Economics as a science?
Meteorology taught us that seemingly intractable complexity can become scientifically manageable through better measurement, theoretical understanding, and modelling.
Medicine taught us that once a system is understood, interventions can be transformed into optimisation problems.
Economics, however, has only partially completed either transition.
Despite studying one of the most important systems in modern society, economics continues to lean heavily on simplified representations of reality, while many policy debates remain framed as contests between competing absolutes, rather than searches for optimal solutions.
Questions such as taxation, redistribution, inequality and government spending are still frequently discussed in binary terms:
- Is inequality good or bad?
- Should taxes be higher or lower?
- Is redistribution harmful or beneficial?
An economist arguing that inequality is either universally good or universally bad may have more in common with a medieval physician arguing that medicine itself is either good or bad.
The possibility that both benefits and costs exist simultaneously should not be surprising. The existence of trade-offs is the norm in complex systems, not the exception.
The interesting question is therefore not whether a policy is good or bad as an absolute article of faith. The interesting question is whether there exists an optimal setting. And if there is, how might we find it?
Politics Chooses the Destination. Economics Chooses the Route.
The previous section established that many economic debates contain competing mechanisms whose relative importance cannot be determined through ideology, intuition or logic alone. They are empirical questions.
But before we can optimise anything, we must answer a more fundamental question:
What are we trying to optimise?
This is where economics often becomes entangled with politics.
Questions such as economic growth, equality, poverty, social mobility, opportunity, security and prosperity involve value judgements. Different individuals and different societies may reasonably place different weights on these competing objectives. Science cannot determine those weights. Thus these questions fall outside the purview of a science of economics.
Science cannot tell us whether society ought to prioritise growth over equality, or equality over growth. It cannot tell us whether reducing poverty is more important than increasing average income. These are questions of values, ethics and democratic choice.
In other words, science cannot choose the destination. That responsibility belongs to society, through democratic institutions. Once a destination has been chosen, however, the character of the problem changes completely.
Suppose a society decides that its primary objective is long-run economic growth. Or suppose it decides that growth matters, but only alongside economic mobility and a minimum standard of living. Or perhaps it chooses some explicit combination of prosperity, opportunity and resilience. Once those objectives are specified, the question ceases to be political.
It becomes technical.
- Which policies produce the greatest growth?
- Which tax structure best promotes mobility?
- Which welfare system delivers the lowest poverty rate?
- Which combination of policies most effectively achieves the chosen objectives?
These are not questions of ideology. They are questions of cause and effect. Questions about how a complex system behaves. Questions that should be answered through evidence, modelling, and empirical investigation. The distinction is important because much of modern political debate conflates objectives with methods.
And much of economics conflates questions of what is, with questions of what ought to be. The whole of neoclassical economics is founded on an error of this type. Neoclassical economics models human decision making as economically rational, because that’s what people ought to do if they wish to optimise their economic outcomes. Theory ignores the ample evidence that people actually behave rather differently. The actuality is ignored, and the theory is founded on the ought. From a scientific perspective, this is completely backwards.
Compounding this problem, economists often defend particular policies as though they were ends in themselves.
Tools carry moral baggage in economic discussions. Taxation is neither good nor bad. Welfare redistribution is neither good nor bad in itself. Markets are a useful tool for achieving certain outcomes. Redistribution is not an end in itself, without a goal or standard. These are tools.
The scientific question is not whether a particular tool is morally virtuous or ideologically pure. The scientific question is whether it effectively contributes toward the objective society has chosen.
In this sense, economics should resemble engineering far more than politics.
An engineer does not argue endlessly about steel and concrete as purely good or bad. The engineer does not decide whether to solve for first order loads, or deflection. He begins with a design brief and asks which design best achieves it.
Economics should aspire to the same standard.
Politics decides where society wishes to go. Economics should determine the most effective path for getting there.
Why Economics Struggles to Optimise Public Policy
If transforming public policy into an optimisation problem sounds obvious, an immediate question arises:
Why hasn’t economics already done it?
After all, economists have spent decades discussing optimal taxation, optimal welfare systems, optimal monetary policy and optimal growth strategies.
The problem is not that economics failed to recognise the value of optimisation.
The problem is that optimisation can only be analysed in the context of an adopted model.
An engineer attempting to design the strongest possible bridge requires an accurate understanding of materials, loads, stresses and failure mechanisms. A weather forecaster requires a model that contains the major forces driving atmospheric behaviour. A physician seeking the optimal dosage of a medication must first understand how the human body responds to that medication, including both its beneficial and harmful effects.
Finding the optimum is only the final step.
The much harder challenge is building a model that adequately reflects the competing processes operating within the real-world system.
Unfortunately, this is where macroeconomics has often struggled.
The economy is an extraordinarily complex system consisting of billions of individuals, millions of firms, governments, financial institutions and countless interactions between them. Faced with this complexity, economists historically chose simplification as the path toward tractability. That decision was understandable, especially at the time it was first made.
The mathematics available a century ago could not easily accommodate millions of heterogeneous decision makers interacting dynamically through time. Simplifying assumptions were necessary. And theory needed to be constructed one step at a time.
Over time, however, some of those simplifying assumptions became so deeply embedded within the discipline that they ceased to be recognised as assumptions at all. The conclusions reached on the basis of simplifying assumptions became articles of faith, rather than provisional, conditional conclusions dependent on the assumptions that generated them.
Individuals became representative agents. Diverse households became average households. Heterogeneous behaviour became representative behaviour.
Rationality stopped being recognised as a convenient assumption and started being defended as a model of human cognition.
And increasingly, economies were analysed as though they tended towards a stable equilibrium populated by rational decision makers with similar objectives.
These assumptions undoubtedly produced elegant mathematics. They also made many models solvable.
But there is an obvious danger.
A model becomes easier to solve every time real complexity is removed from it.
At some point a discipline must ask whether the simplification is worth the realism sacrificed.
No meteorologist would attempt to forecast tomorrow’s weather by replacing the atmosphere with a representative homogeneous gas. No physician would attempt to understand human health by replacing billions of interacting cells with a representative cell.
Yet macroeconomics has often been comfortable replacing heterogeneous populations with representative agents whose behaviour is connected to reality only through averages.
The result is a discipline that has sometimes prioritised mathematical elegance over explanatory power.
The irony is that these simplifications often eliminate precisely the mechanisms we wish to study.
The Balance of Countervailing Forces
Consider the question of inequality and economic growth.
One commonly proposed argument is that inequality promotes growth by strengthening incentives, encouraging entrepreneurship, rewarding productive effort and increasing investment.
Perhaps it does.
That certainly sounds like one plausible causal mechanism.
But if we are interested in finding the growth-maximising level of inequality, we must also account for the possibility of competing effects.
For example, we know that households do not all behave identically.
Lower-income households typically spend a larger fraction of additional income.
Higher-income households typically save a larger fraction.
As income becomes increasingly concentrated among wealthier households, a growing share of national income is directed toward groups that spend less and save more.
This may increase savings and the funds available to be invested in growth.
But logically, it must also reduce the circulation of spending through the economy. Therefore it must weaken multiplier effects. It must reduce economic dynamism.
And if economic opportunities emerge from demand for goods and services, it may also reduce some forms of productive investment.
So now we have two plausible mechanisms operating simultaneously:
and
Which mechanism matters more?
Which is stronger?
Where does the balance lie?
Pure logic cannot tell us.
Both mechanisms are logically plausible.
Logic can tell us that the effects may exist. It cannot tell us their relative magnitude.
Those are empirical questions.
They are exactly the kinds of competing mechanisms that a scientific model ought to quantify.
Yet under many traditional macroeconomic assumptions, the second mechanism largely disappears before the analysis even begins. It is assumed away.
If households are assumed to behave similarly, then differences in consumption behaviour across the income distribution are averaged away.
The model effectively assumes that a dollar received by a poor household has the same economic consequences as a dollar received by a wealthy household.
The balance point is therefore not discovered. It is embedded within the premises. Assume identical behaviour. Derive identical behaviour. Conclude that distribution does not matter.
The optimisation problem collapses before it begins.
A model cannot optimise over mechanisms that it assumes away.
If we reverse some of those simplifying assumptions, we recover the ability to model the competing forces that produce the balance point. Once competing mechanisms are admitted into the analysis, many of the great ideological debates in economics suddenly change character.
The question is no longer: Is inequality good for growth?
Nor: Is inequality bad for growth?
Instead, the question becomes: What level of inequality maximises growth?
The same logic applies to taxation. Taxes may weaken some incentives. They may also fund infrastructure, education, healthcare, scientific research and institutions that support growth.
The question is not whether taxation is good or bad.
The question is: What tax structure produces the best outcome according to the objective we have chosen?
The same logic applies to redistribution. Redistribution may weaken some incentives. Redistribution may also strengthen demand, improve opportunity, increase human capital development and enhance economic mobility.
Again, the scientific question is not which political tribe is correct.
The scientific question is: How large are the competing effects?
And: What policy setting produces the best overall outcome?
Notice how different this is from conventional political debate. An ideological debate seeks to determine which side is right. A scientific investigation seeks to identify competing mechanisms, measure them, and determine where their interaction produces an optimum.
The objective is specified. The mechanisms are identified. The competing effects are modelled. The optimum is estimated empirically.
That is exactly how medicine approaches dosage. It is exactly how engineering approaches design. And it is how economics should approach public policy.
The challenge facing a more scientific economics is therefore not complexity for its own sake.
It is identifying the mechanisms that matter, measuring them, and building models rich enough to contain them.
Only then can we move from ideological debates about public policy toward something far more useful:
A genuine optimisation problem.
We need an economics that can find the balance of two competing forces:
What we currently have is an economics that assumes away the very question that creates the scientifically interesting and policy relevant question:
Many of the most profound economic prescriptions of the hegemonic neoclassical economic school are derived directly from assumptions, rather than from a balanced analysis:
- “Taxation is anti-growth” relies on the assumption of homogeneous agents
- “Inequality is necessary” relies on an assumption of rationality and homogeneity
- “Smaller Government is better” relies on similar assumptions
Demand-side economics makes the same mistake of assuming away half the problem, but with the opposite assumptions. It makes assumptions regarding aggregate demand, whilst assuming away the complex question of the economic motivation of producers.
In either case, the assumptions that drive many of the most important policy prescriptions were not made because they were accurate. They were made as simplifying assumptions to make the math easier, and to make questions tractable. They assumed away half of the competing forces that creates balance, and then found that the lone remaining force pushes all one way.
A scientific approach to economics does not attempt to identify a single “correct” mechanism. It does not reduce the question of complex economic systems to either supply-side or demand-side.
Analysis of macro-economics should not be a choice of either Neoclassical or New Keynesian. We need to find a way of modelling macroeconomic systems that include both insights from the demand-side and the supply-side. A form of economic system taht attempts to identify all material mechanisms, measure them, and understand how they interact. Only then can meaningful optimisation occur.
This is ultimately the promise of a more empirical and scientifically grounded economics.
Not that politics disappears. Not that governments become omniscient. Not that economic forecasting becomes perfect. But that public policy can gradually move away from arguments about what tribal economists believe should occur, and toward evidence about the economic forces that actually operate.
The challenge is no longer deciding which ideology to trust. The challenge is understanding the system well enough to find the optimum of competing processes, each of which may be championed by competing political ideologies, but none of them are likely to represent a complete understanding or balance of the economic impacts.
Conclusion: The Future of Economic Science
For centuries, debates about taxation, redistribution, inequality and government intervention have largely been conducted as contests between competing ideologies.
One side argues that incentives matter most. The other argues that demand, opportunity or fairness matter most.
Both sides point to examples. Both sides can construct plausible political narratives. Both sides can find evidence that appears to support their position.
But this is not how mature sciences approach complex systems.
Meteorology progressed by abandoning human tendency toward confirmation bias in favour of measurement and scientific discipline. Rather than attempting to intuit the cause of storms based on religious dogma, scientists sought to understand and model the physical forces that created them. Barometric pressure yields to measurement in a way that Thor never did.
Medicine progressed by recognising that interventions rarely produce exclusively benefits or pure harm. Everything from foods to minerals to medicines involve competing effects acting simultaneously. The scientific challenge is to understand those effects well enough to identify the dosage that produces the optimal outcome.
Economics should aspire to the same standard.
The goal is not to remove politics from public policy. Politics remains essential because society must decide what it wishes to achieve. Economic growth, reward for effort, prosperity, incentive, mobility, opportunity, resilience, poverty reduction, and fairness are ultimately questions of values. These are political questions.
Only once those goals have been chosen, the discussion can change to methods of achieving the chosen goals. The role of economics is not to defend political ideological positions, but to forecast the impact of policy action toward defined goals.
Its role should be to identify the mechanisms operating within the economy, measure their relative importance, build models capable of representing those mechanisms, and estimate the policies most likely to achieve society’s politically defined objectives.
The question should no longer be: Is inequality good or bad?
Nor: Is redistribution good or bad?
Nor: Is government intervention good or bad?
The scientific question is: What policy setting produces the best outcome according to the objectives society has chosen?
That is an optimisation problem.
To solve such problems, economics must move beyond models that assume away the very mechanisms that call for optimisation. It must become more empirical, more realistic, and more willing to allow evidence to determine which mechanisms matter.
The future of economic science may therefore lie in a deceptively simple change in perspective.
Rather than asking which ideology is correct, we should ask: What does the evidence suggest is optimal strategy?
That shift would not give us perfect forecasts. It would not eliminate uncertainty. It would not end political disagreement. But it would transform the debate.
The central challenge would no longer be choosing between competing articles of economic faith. It would be understanding the economy well enough to discover its optima.
And that is a challenge worthy of a scientific discipline.