This is my 2nd post summarizing the key takeaways I got from reading the book Modeling Life.
I recommend reading my earlier post first to get a good grounding on the foundations covered in the book.
A system can exhibit three different types of behavior: equilibrium, oscillation, and chaos. The shark-tuna system we saw in the previous post is one example of oscillatory behavior. This post will cover equilibrium with a special focus on the role of sigmoid functions.
Consider a forest with only 10 rabbits. Assume the rabbits do not die and have an abundant supply of food, allowing them to reproduce freely. Suppose the per capita birth rate is 0.2, meaning each rabbit gives birth to 0.2 rabbits in a given time period.
The change equation for the rabbit population would be 0.2 * number of rabbits. This equation represents the rate of change in the rabbit population, where the change is directly proportional to the current number of rabbits.
Our naive model results in exponential growth. In the long run this unrealistic model predicts a ball of fifteen quintillion rabbits expanding outward at half the speed of light. This model reminds me of the saying, “All models are wrong, but some are useful.”
How do we fix this naive model? Rabbits need food to survive. No environment has infinite resources, and they all have a finite carrying capacity. Let’s assume the carrying capacity of this environment is K rabbits, and the current number of rabbits is X. Then we can model the change equation as 0.2 * X * (1 – X / K).
When X is less than the carrying capacity K, the ratio X/K is less than 1, and the change equation will be positive. This means the rabbits can continue to grow as long as their population is below the carrying capacity of the environment.
When X is equal to the carrying capacity K, the ratio X/K equals 1, and 1 minus 1 is 0, resulting in the change equation becoming 0. This indicates that when the rabbits reach the carrying capacity, their population stops growing.
When X is greater than K, the ratio X/K is greater than 1, resulting in the change equation being negative. This means the number of rabbits will start to decrease, as they have outgrown the environment’s carrying capacity.
After setting the environment’s carrying capacity to 1,000, the number of rabbits stops growing once it reaches 1,000. It doesn’t matter if the initial number of rabbits is below or above the carrying capacity, the population will stabilize at 1,000 as long as the initial number of rabbits is greater than zero.
The point at which the change equation equals zero is called the equilibrium point (EP). In this particular example, the two EPs are 0 and 1,000. To find the EPs, we set the change equation equal to zero and solve for X.
Setting the change equation 0.2 * X * (1 – X / K) equal to zero implies either the term 0.2 * X is zero or the term (1 – X / K) is equal to zero. The first term is zero when X equals zero. The second term is zero when X equals K.
By identifying the EPs, we can predict how the system will behave without having to try the model with an infinite number of rabbits. But how do we know there are exactly two EPs, and not more or less? The change equation 0.2 * X * (1 – X / K) is a quadratic function. According to the Fundamental Theorem of Algebra, a second-degree polynomial will have two roots.
Allee Effect
In some species, a minimum number of animals is necessary for the group’s survival. For example, African hunting dogs require the assistance of others to raise their young. As a result, the species itself can go extinct if its population declines below a certain threshold. This decline in per capita population growth rates at low population sizes is known as the Allee effect.
To model the Allee effect in the rabbit population change equation 0.2 * X * (1 – X / K), we can add the term (X / a – 1) as a multiplicative factor. The modified equation becomes 0.2 * X * (1 – X / K) * (X / a – 1) where:
- X is the current rabbit population
- K is the carrying capacity, set to 100 in this case
- a is the minimum number of rabbits needed, set to 5 in this case
When I ran this model with a carrying capacity (K) of 100 and a minimum population (a) of 5, I observed the following:
If the initial rabbit population is below the minimum viable population (a = 5), the population will decline and eventually go extinct. This is because when the population (X) is less than the minimum viable population (a), the fraction X / a will be less than 1, making the term (X / a – 1) negative. The red line in the graph above never rises, as the initial value of 3 rabbits is less than the minimum viable population of 5.
If the initial rabbit population is above 5, the population will grow towards the carrying capacity. This is because when the population (X) is greater than the minimum viable population (a), the fraction X / a will be greater than 1, making the term (X / a – 1) positive. The blue line and green line exhibit this behavior.
If the initial rabbit population equals the minimum viable population (a = 5), the population will not change. This is because the fraction X / a will be equal to 1, making the term (X / a – 1) equal to 0. The author’s analogy connecting the Allee effect to Elon Musk’s strategy of open-sourcing Tesla’s patents was thought-provoking.
As an example of the Allee effect, consider the strategy employed by Elon Musk, the developer of the Tesla electric car. Musk announced that he would give away, free of charge, all the patents that his company held on electric cars. These patents are valuable. Why would he give them away? Because he realized that for electric cars to succeed, they require substantial infrastructure: tax benefits, dedicated highway lanes, and public recharging networks. None of these would happen if there was only one electric car company. In other words, if there were only one electric car company, there would soon be no electric car companies. A critical mass is necessary. — Modeling Life
How many equilibrium points (EPs) does the Rabbit Allee Model have? The change equation for the model, 0.2 * X * (1 – X / K) * (X / a – 1), is a 3rd degree polynomial. According to the Fundamental Theorem of Algebra, a 3rd degree polynomial equation will have three roots, and each root corresponds to an EP.
To find the equilibrium points, we need to set the change equation equal to zero. This can happen in three ways. When the term 0.2 * X is zero, which occurs at X = 0. When the term (1 – X / K) is zero, which occurs at X = K. When the term (X / a – 1) is zero, which occurs at X = a. Therefore, the Rabbit Allee Model has three equilibrium points, located at X = 0, X = a, and X = K.
Stable and Unstable Equilibrium
After identifying the equilibrium points (EPs) of the model, the next step is to determine whether each EP is stable or unstable.
An equilibrium point is considered stable if the system remains at that EP permanently when subjected to a small perturbation or disturbance. Conversely, an equilibrium point is considered unstable if the system does not remain at that EP when a small perturbation is applied, and instead moves away from the unstable EP.
How do we find out if X = 0, X = a, and X = K in the Rabbit Allee Model has stable or unstable EP? Plot the rabbits in the x-axis and their corresponding change equation in the y-axis. What do you see from the image below?
The equilibrium points at X = 0 and X = K are stable, while the equilibrium point at X = a is unstable. Let me explain why the equilibrium point at X = K is stable. Consider two points, one to the left of K and the other to the right of K.
When the point is to the left of K, the change equation 0.2 * X * (1 – X / K) * (X / a – 1) is positive. This is indicated by the change vector pointing to the right on the left side of K. Conversely, when the point is to the right of K, the change equation is negative. This is indicated by the change vector pointing to the left on the right side of K.
The fact that the change vector points towards the equilibrium point at X = K, both from the left and the right side, indicates that this equilibrium point is stable. If the system is perturbed slightly away from X = K, it will tend to return back to K. You can use similar reasoning to identify the equilibrium points at X = 0 as stable and X = a as unstable.
There is another, more generic method to determine the stability of the equilibrium points, which requires some knowledge of calculus. If you need a quick refresher on the relevant calculus concepts, you can refer to my earlier post titled How a Machine Learns.
The stability of the equilibrium points (EPs) can be determined by the direction of the slope of the rabbit change equation with respect to the rabbit population at each EP. This method of analyzing the slope of the change equation can be generalized and applied to systems with single or multiple variables.
A negative slope at an EP indicates that the EP is stable. For example, the EPs at X = 0 and X = K rabbits have negative slopes, indicating they are stable EPs. Conversely, if the slope is positive at an EP, that EP is considered unstable. The EP at X = a rabbit has a positive slope, making it an unstable EP.
E.Coli Bacteria and Sigmoids
E. coli bacteria love lactose (a type of sugar) and use it as an energy source. They import the extracellular lactose into their cells using a transport protein called lactose permease. Producing this large, complex, and costly protein only makes economic sense for the bacteria when there is an abundance of extracellular lactose available.
Lactose activates the production of messenger RNA (mRNA), which then leads to the synthesis of more lactose permease. This creates a positive feedback loop, where the presence of lactose induces the production of the transport protein responsible for importing more lactose into the cell.
The state variable we’re interested in tracking is the intracellular lactose X. What causes X to go up? Lactose imported from the environment increases X. What causes X to go down? Lactose needed to run the cell, also known as metabolism, decreases X. So the change equation X’ is the difference between the rate of lactose import and the rate of lactose metabolism.
Lactose metabolism can be modeled using a simple linear function, such as kX, where k represents the degradation rate of lactose. In this case, we can choose k = 0.4.
As for the lactose import function, it should ramp up the production of the lactose transport protein only when there is an abundance of extracellular lactose available. Otherwise, the bacteria should keep the production of this protein low. The best analogy for this import function is that of a shrewd value investor, who buys heavily when the market crashes and sits on the sidelines during an exuberant bull market.
What mathematical function would best model the import function? It turns out that the sigmoid function is the ideal choice for this. I often wonder whether nature discovered mathematics to carry out its processes, or whether we humans invented mathematics to understand nature. My guess is that it’s both.
Rather than a logic gate, a neuron is more like a voltage-to-frequency converter. This curve, which looks like an elongated S, is variously known as the logistic, sigmoid, or S curve. Peruse it closely, because it’s the most important curve in the world. At first the output increases slowly with the input, so slowly it seems constant. Then it starts to change faster, then very fast, then slower and slower until it becomes almost constant again. The transfer curve of a transistor, which relates its input and output voltages, is also an S curve. So both computers and the brain are filled with S curves. But it doesn’t end there. The S curve is the shape of phase transitions of all kinds. The effect of financial gains and losses on your happiness follows an S curve, so don’t sweat the big stuff. Mood swings are phase transitions. So are birth, adolescence, falling in love, getting married, getting pregnant, getting a job, losing it, moving to a new town, getting promoted, retiring, and dying. The universe is a vast symphony of phase transitions, from the cosmic to the microscopic, from the mundane to the life changing. — The Master Algorithm
The denominator of the lactose import function will be high when the intracellular lactose X is low. This results in low lactose permease production and hence low lactose getting imported into the cell. As X goes beyond a critical threshold, the numerator becomes larger than the denominator, causing the production of lactose permease to increase. This results in more extracellular lactose into the cell, leading to a rise in X.
Even when the intracellular lactose concentration, X, is zero, the change function will produce a constant amount of lactose, denoted as ‘a’. This is because the cell needs to maintain a baseline level of lactose inside to function properly. The cell uses this baseline lactose to assess whether the intracellular lactose level is rising, so that it can then ramp up the production of the lactose permease protein accordingly.
Let’s investigate the EPs and its stability. When does the change equation become zero? It happens when the lactose import function equals lactose degradation function. You can visualize the EPs by plotting intracellular lactose (X) on the x-axis and the change in lactose (X’) in the y-axis.
Why do we have 3 EPs? The change equation is a 3rd degree polynomial, and according to the Fundamental Theorem of Algebra, a 3rd degree polynomial will have three roots. As the image above shows, the X value is divided into three regions: low, medium, and high.
Let’s focus on the point to the left of the “low” value. In this region, the change in X, denoted as X’, is positive because the sigmoid function (blue) is above the degradation function (red). This is why the arrow to the left of “low” is pointing to the right.
To the right of “low”, the sigmoid function is lower than the degradation function, resulting in a negative change. This is why the arrow to the right of “low” is pointing to the left. From this, we can conclude that the equilibrium point (EP) at “low” is a stable point.
Through similar reasoning, we can conclude that the EP marked as “medium” is unstable, while the EP marked as “high” is stable. This configuration resembles a biological switch, where a slight perturbation to the left of the “medium” region will push the system towards the “low” EP, and a slight disturbance to the right of “medium” will drive it towards the “high” EP.
Nature is indeed full of such biological switches. It’s fascinating to see how we have leveraged this idea in the design of neural networks, where we employ activation functions. Once the input to a neuron crosses a certain threshold, the neuron then emits an output. I found a reference on phase transitions to be particularly insightful in understanding this concept further.
Nature is replete with phase transitions. The transformation from frozen water to liquid water is one example; the change from liquid water to gaseous water (steam) is another. However, these transitions are not confined to chemical phenomena. They also occur in social systems, where millions of individual decisions or attitudes can interact to rapidly shift the entire system to a new equilibrium. Phase transitions are at play during events like speculative bubbles, stock market crashes, and spontaneous traffic jams. On a more positive note, they were also evident in the breakup of the Soviet Bloc and the exponential rise of the Internet. — The Tell-Tale Brain
I’ve only scratched the surface of what it means for a system to reach an equilibrium. The book covers multiple examples for equilibrium including one on stock market bubbles and crashes. It shows how to nudge the system toward a particular equilibrium point in order to prevent an outbreak. The reason why we’re modeling is to predict how the system will behave and to alter the outcome in our favor by nudging it.
In my next post, I’ll write about oscillatory behavior. After that, I will explore how linear algebra is used in dynamic modeling, and finally, I will conclude with a post on chaotic behavior.
Seeking Wisdom 