Chaos & Climate – Part 1: Linearity

Guest Essay by Kip Hansen

“…we should recognise that we are dealing with a coupled nonlinear chaotic system, and therefore that the long-term prediction of future climate states is not possible.”

– IPCC AR4 WG1

Introduction:

The IPCC has long recognized that the climate system is 1) nonlinear and therefore, 2) chaotic. Unfortunately, few of those dealing in climate science – professional and citizen scientists alike – seem to grasp what this really means. I intend to write a short series of essays to clarify the situation regarding the relationship between Climate and Chaos. This will not be a highly technical discussion, but an even-handed basic introduction to the subject to shed some light on just what the IPCC means when it says “we are dealing with a coupled nonlinear chaotic system” and how that should change our understanding of the climate and climate science.

My only qualification for this task is that as a long-term science enthusiast, I have followed the development of Chaos Theory since the late 1960s and during the early 1980s often waited for hours, late into the night, as my Commodore 64 laboriously printed out images of strange attractors on the screen or my old Star 9-pin printer.

PART 1: Linearity

In order to discuss nonlinearity, it is best to start with linearity. We are talking about systems, so let’s look at a definition and a few examples.

Edward Lorenz, the father of Chaos Theory and a meteorologist, in his book “The Essence of Chaos” gives this:

Linear system: A system in which alterations of an initial state will result in proportional alterations in any subsequent state.

In mathematics there are lots of linear systems. The multiplication tables are a good example: x times 2 = y. 2 times 2 = 4. If we double the “x”, we get 4 times 2 = 8. 8 is the double of 4, an exactly proportional result.

When graphing a linear system as we have above, we are marking the whole infinity of results across the entire graphed range. Pick any point on the x-axis, it need not be a whole number, draw a vertically until it intersects the graphed line, the y-axis value at that exact point is the solution to the formula for the x-axis value. We know, and can see, that 2 * 2 = 4 by this method. If we want to know the answer for 2 * 10, we only need to draw a vertical line up from 10 on the x-axis and see that it intersects the line at y-axis value 20. 2 * 20? Up from 20 we see the intersection at 40, voila!

[Aside: It is this feature of linearity that is taught in the modern schools. School children are made to repeat this process of making a graph of a linear formula many times, over and over, and using it to find other values. This is a feature of linear systems, but becomes a bug in our thinking when we attempt to apply it to real world situations, primarily by encouraging this false idea: that linear trend lines predict future values. When we see a straight line, a “trend” line, drawn on a graph, our minds, remembering our school-days drilling with linear graphs, want to extend those lines beyond the data points and believe that they will tell us future, uncalculated, values. This idea is not true in general application, as you shall learn. ]

Not all linear systems are proportional in that way: the ratio between the radius of a circle and its circumference is linear. C =2πR, as we increase the radius, R, we get a proportional increase in Circumference, in a different ratio, due to the presence of the constants in the equation: 2 and π.

In the kitchen, one can have a recipe intended to serve four, and safely double it to create a recipe for 8. Recipes are [mostly] linear. [My wife, who has been a professional cook for a family of 6 and directed an institutional kitchen serving 4 meals a day to 350 people, tells me that a recipe for 4 multiplied by 100 simply creates a mess, not a meal. So recipes are not perfectly linear.]

An automobile accelerator pedal is linear (in theory) – the more you push down, the faster the car goes. It has limits and the proportions change as you change gears.

Because linear equations and relationships are proportional, they make a line when graphed.

A linear spring is one with a linear relationship between force and displacement, meaning the force and displacement are directly proportional to each other. A graph showing force vs. displacement for a linear spring will always be a straight line, with a constant slope.

In electronics, one can change voltage using a potentiometer – turning the knob – in a circuit like this:

In this example, we change the resistance by turning the knob of the potentiometer (an adjustable resistor). As we turn the knob, the voltage increases or decreases in a direct and predictable proportion, following Ohm’s Law, V = IR, where V is the voltage, R the resistance, and I the current flow.

Geometry is full of lovely linear equations – simple relationships that are proportional. Knowing enough side-lengths and angles, one can calculate the lengths of the remaining sides and angles. Because the formulas are linear, if we know the radius of a circle or a sphere, we can find the diameter (by definition), the area or surface area and the circumference.

Aren’t these linear graphs boring? They all have these nice straight lines on them

Richard Gaughan, the author of Accidental Genius: The World’s Greatest By-Chance Discoveries, quips: “One of the paradoxes is that just about every linear system is also a nonlinear system. Thinking you can make one giant cake by quadrupling a recipe will probably not work. …. So most linear systems have a ‘linear regime’ –- a region over which the linear rules apply–- and a ‘nonlinear regime’ –- where they don’t. As long as you’re in the linear regime, the linear equations hold true”.

Linear behavior, in real dynamic systems, is almost always only valid over a small operational range and some models, some dynamic systems, cannot be linearized at all.

How’s that? Well, many of the formulas we use for the processes, dynamical systems, that make civilization possible are ‘almost’ linear, or more accurately, we use the linear versions of them, because the nonlinear version are not easily solvable. For example, Ian Stewart, author of Does God Play Dice?, states:

“…linear equations are usually much easier to solve than nonlinear ones. Find one or two solutions, and you’ve got lots more for free. The equation for the simple harmonic oscillator is linear; the true equation for a pendulum is not. The classic procedure is to linearize the nonlinear by throwing away all the awkward terms in the equation.

….

In classical times, lacking techniques to face up to nonlinearities, the process of linearization was carried out to such extremes that it often occurred while the equations were being set up. Heat flow is a good example: the classical heat equation is linear, even before you try to solve it. But real heat flow isn’t, and according to one expert, Clifford Truesdell, whatever good the classical heat equation has done for mathematics, it did nothing but harm to the physics of heat.”

One homework help site explains this way: “The main idea is to approximate the nonlinear system by using a linear one, hoping that the results of the one will be the same as the other one. This is called linearization of nonlinear systems.” In reality, this is a false hope.

The really important thing to remember is that these linearized formulas of dynamical systems –that are in reality nonlinear – are analogies and, like all analogies, in which one might say “Life is like a game of baseball”, they are not perfect, they are approximations, useful in some cases, maybe helpful for teaching and back-of-an-envelope calculations – but – if your parameters wander out of the system’s ‘linear regime’ your results will not just be a little off, they risk being entirely wrong — entirely wrong because the nature and behavior of nonlinear systems is strikingly different than that of linear systems.

This point bears repeating: The linearized versions of the formulas for dynamic systems used in everyday science, climate science included, are simplified versions of the true phenomena they are meant to describe – simplified to remove the nonlinearities. In the real world, these phenomena, these dynamic systems, behave nonlinearly. Why then do we use these formulas if they do not accurately reflect the real world? Simply because the formulas that do accurately describe the real world are nonlinear and far too difficult to solve – and even when solvable, produce results that are, under many common circumstances, in a word, unpredictable.

Stewart goes on to say:

“Really the whole language in which the discussion is conducted is topsy-turvy. To call a general differential equation ‘nonlinear’ is rather like calling zoology ‘nonpachydermology’.”

Or, as James Gleick reports in CHAOS, Making of a New Science:

“The mathematician Stanislaw Ulam remarked that to call the study of chaos “nonlinear science” was like calling zoology “the study of non-elephant animals.”

Amongst the dynamical systems of nature, nonlinearity is the general rule, and linearity is the rare exception.

Nonlinear system: A system in which alterations of an initial state need not produce proportional alterations in any subsequent states, one that is not linear.

When using linear systems, we expect that the result will be proportional to the input. We turn up the gas on the stove (altering the initial state) and we expect the water to boil faster (increased heating in proportion to the increased heat). Wouldn’t we be surprised though, if one day we turned up the gas and instead of heating, the water froze solid! That’s nonlinearity! (Fortunately, my wife, the once-professional cook, could count on her stoves behaving linearly, and so can you.)

What kinds of real world dynamical systems are nonlinear? Nearly all of them!

Social systems, like economics and the stock market are highly nonlinear, often reacting non-intuitively, non-proportionally, to changes in input – such as news or economic indicators.

Population dynamics; the predator-prey model; voltage and power in a resistor: P = V²2R; the radiant energy emission of a hot object depending on its temperature: R = kT⁴; the intensity of light transmitted through a thickness of a translucent material; common electronic distortion (think electric guitar solos); amplitude modulation (think AM radios); this list is endless. Even the heating of water, as far as the water is concerned, on a stove has a linear regime and a nonlinear regime, which begins when the water boils instead of heating further. [The temperature at which the system goes nonlinear allowed Sir Richard Burton to determine altitude with a thermometer when searching for the source of the Nile River.] Name a dynamic system and the possibility of it being truly linear is vanishing small. Nonlinearity is the rule.

What does the graph of a nonlinear system look like? Like this:

Here, a simple little formula for Population Dynamics, where the resources limit the population to a certain carrying capacity such as the number of squirrels on an idealized May Island (named for Robert May, who originated this work): x_next = rx(1-x). Some will recognize this equation as the “logistic equation”. Here we have set the carrying capacity of the island as 1 (100%) and express the population – x – in a decimal percentage of that carrying capacity. Each new year we start with the ending population of the previous year as the input for the next. r is the growth rate. So the growth rate times the population times the bit (1-x), which is the amount of the carrying capacity unused. The graph shows the results over 30 years using several different growth rates.

We can see many real life population patterns here:

1) With the relatively low growth rate of 2.7 (blue) the population rises sharply to about 0.6 of the carrying capacity of the island and after a few years, settles down to a steady state at that level.

2) Increasing the growth rate to 3 (orange) creates a situation similar to the above, except the population settles into a saw-tooth pattern which is cyclical with a period of two.

3) At 3.5 (red) we see a more pronounced saw-tooth, with a period of 4.

4) However, at growth rate 4 (green), all bets are off and chaos ensues. The slams up and down finally hitting a [near] extinction in the year 14 – if the vanishing small population survived that at all, it would rapidly increase and start all over again.

5) I have thrown in the purple line which graphs a linear formula of simply adding a little each year to the previous year’s population – x_next = x(1+(0.0005*year)) — slow steady growth of a population maturing in its environment – to contrast the difference between a formula which represents the realities of populations dynamics and a simplified linear versions of them. (Not all linear formulas produce straight lines – some, like this one, are curved, and more difficult to solve.) None of the nonlinear results look anything like the linear one.

Anyone who deals with populations in the wild will be familiar with Robert May’s work on this, it is the classic formula, along with the predator/prey formula, of population dynamics. Dr. May eventually became Princeton University’s Dean for Research. In the next essay, we will get back to looking at this same equation in a different way.

In this example, we changed the growth element of the equation gradually upwards, from 2.7 to 4 and found chaos resulting. Let’s look at one more aspect before we move on.

This image shows the results of x_next = 4x(1-x), the green line in the original, extended out to 200 years. Suppose you were an ecologist who had come to May Island to investigate the squirrel population, and spent a decade there in the period circled in red, say year 65 to 75. You’d measure and record a fairly steady population of around 0.75 of the carrying capacity of the island, with one boom year and one bust year, but otherwise fairly stable. The paper you published based on your data would fly through peer review and be a triumph of ecological science. It would also be entirely wrong. Within ten years the squirrel population would begin to wildly boom-and-bust and possibly go functionally extinct in the 81^st or 82^nd year. Any “cause” assigned would be a priori wrong. The true cause is the existence of chaos in the real dynamic system of populations under high growth rates.

You may think this a trick of mathematics but I assure you it is not. Ask salmon fishermen in the American Northwest and the sardine fishermen of Steinbeck’s Cannery Row. Natural populations can be steady, they can ebb and flow, and they can be truly chaotic, with wild swings, booms and busts. The chaos is built-in and no external forces are needed. In our May Island example, chaos begins to set in when the squirrels become successful, their growth factor increases above a value of three and their population begins to fluctuate, up and down. When they become too successful, too many surviving squirrel pups each year, a growth factor of 4, disaster follows on the heels of success. For real world scientific confirmation, see this paper: Nonlinear Population Dynamics: Models, Experiments and Data by Cushing et. al. (1998)

Let’s see one more example of nonlinearity. In this one, instead of doing something as obvious as changing a multiplier, we’ll simply change the starting point of a very simple little equation:

At the left of the graph, the orange line overwrites the blue, as they are close to identical. The only thing changed between the blue and orange is that the last digit of the initial value 0.543215 has been rounded up to 2, 0.54322, a change of 1/10000^th, or rounded down to 0.54321, depending on the rounding rule, much as your computer, if set to use only 5 decimal places, would do, automatically, without your knowledge. In dynamical sciences, a lot of numbers are rounded up or down. All computers have a limited number of digits that they will carry in any calculation, and have their own built in rounding rules. In our example, the values begin to diverge at day 14, if these are daily results, and by day 19, even the sign of the result is different. Over the period of a month and a half, whole weeks of results are entirely different in numeric values, sign and behavior.

This is the phenomena that Edward Lorenz found in the 1960’s when he programmed the first computational models of the weather, and it shocked him to the core.

This is what I will discuss in the next essay in this series: the attributes and peculiarities of nonlinear systems.

Take Home Messages:

1. Linear systems are tame and predictable – changes in input produce proportional changes in results.

2. Nonlinear systems are not tame – changes in input do not necessarily produce proportional changes in results.

3. Nearly all real world dynamical systems are nonlinear, exceptions are vanishingly rare.

4. Linearized equations for systems that are, in fact, nonlinear, are only approximations and have limited usefulness. The results produced by these linearized equations may not even resemble the real world system results in many common circumstances.

5. Nonlinear systems can shift from orderly, predictable regimes to chaotic regimes under changing conditions.

6. In nonlinear systems, even infinitesimal changes in input can have unexpectedly large changes in the results – in numeric values, sign and behavior.

# # # # #

Author’s Comment Reply Policy:

This is a fascinating subject, with a lot of ground to cover. Let’s try to have comments about just the narrow part of the topic that is presented here in this one essay which tries to introduce readers to linearity and nonlinearity. (What this means to Climate and Climate Science will come in further essays in the series.)

I will try to answer your questions and make clarifications. If I have to repeat the same things too many times, I will post a reading list or give more precise references.

# # # # #