Adaptive Dynamics and Evolutionary Stability

J. Hofbauer and K. Sigmund

Institut für Mathematik, Universität Wien and IIASA, Laxenburg

Game theory has been used successfully to model the evolution of biological traits whose advantage depends on their frequency in the population (see Maynard Smith [7]). The basic notion of evolutionary stability singles out solutions which, while safe from future change, need not be likely outcomes of past adaptations. The first to point this out was Eshel ([3], see also [4]). Recently, Taylor [11] and Nowak [8] stressed the independence of (a) being able to resist any invasion and (b) being able to invade everything. In this note we propose a dynamics to model the effect of adaptation, relate it with the stability of equilibria, and discuss some examples, concerning in particular iterated interactions and gamete sizes.

The phenotypic traits we consider are determined by continuous variables (like size, sex ratio or the probability of some behaviour). We assume that the population is essentially monomorphic (all in some state x except for an occasional mutant in a near-by state y = x + h.) The fitness of such an individual is denoted by A(y, x), its relative advantage A(y, x) − A(x, x) by W (h, x). The state x̂ is said to be evolutionarily stable if A(y, x̂) ≤ A(x̂, x̂) for all y, with A(x̂, y) > A(y, y) if A(y, x̂) = A(x̂, x̂) for some y ≠ x̂. This implies that a population x̂ cannot be invaded by a small population y. But it offers no argument for the establishment of x̂. It can happen that an evolutionarily stable state x̂ is a ‘garden of Eden’ configuration, i.e. without a supplanted predecessor: near-by states evolve away from it. Since a slight perturbation of the environment can make x̂ lose its evolutionary stability, subsequent adaptations will lead it further astray.

The proposed adaptive dynamics (first version) is

ẋ_{i} = \frac{∂}{∂y_{i}} A(y, x)|_{y=x} i = 1, ..., n.  (1)

The rationale is that a few mutants y close to x test out alternatives, and that the whole population evolves in the direction of the most promising one. This is supposed to mimick an evolution favouring individual fitness. It can also be used for learning models, under the assumption that trials are myopic, i.e. explore only the vicinity of the current state. If A is frequency independent, i.e. depends only on y, we get the usual hill-climbing dynamics leading to local optimization.

In the one-dimensional case, (1) is

ẋ = \frac{∂W}{∂h} (0, x).  (2)

Following Taylor [11] , a state x̂ will be said to be h-stable if W (h, x̂) < 0 for all small h ≠ 0, and x-stable if W (h, x) has the sign of h(x̂ − x) for small h. Thus h-stability means that x̂ is a strict (local) Nash equilibrium, and hence locally evolutionarily stable, while x-stability means that if the population is in a state x close to x̂, then any mutations in the right direction (towards x̂) will succeed. In each case x̂ is an equilibrium of (2). Generically, an equilibrium is x-stable iff \frac{∂^{2}W}{∂x∂h} < 0 and h-stable iff \frac{∂^{2}W}{∂h^{2}} < 0.

For example, if x is the sex-ratio, then A(y, x) = \frac{y}{x} + \frac{1−y}{1−x} and hence (2) yields

ẋ = x(1 − x)(1 − 2x).

The equilibrium x̂ = \frac{1}{2} is both x-stable and h-stable. Eshel [3] gave some ingenious examples of equilibria which are h- but not x-stable. Taylor [11] states that he has yet to see biologically plausible equilibria which are x- but not h-stable. We believe that they can be found for iterated interactions.

The Iterated Prisoner’s Dilemma is a paradigm for the evolution of cooperation ([2]). In each round of the interaction, the choice is between E_{1} (cooperation) and E_{2} (defection) and the payoff is given by the 2 × 2 matrix a_{ij} satisfying

a_{21} > a_{11} > a_{22} > a_{12}  and  2a_{11} > a_{22} + a_{12}.  (3)

Tit For Tat (start with E_{1}, then do whatever the other did last) proved amazingly successful in computer tournaments. But in biological applications, there is always the probability of an error which turns a mistaken defection into the starting point of an expensive feud. In this context, it is interesting to study stochastic reactive strategies defined by the conditional probability x_{i} to play E_{1}, given that the other’s last move was E_{i}. For simplicity we consider the infinitely repeated case; the initial move, then, does not matter (but see [9] for the general situation). A simple computation yields

A(y, x) = ∑_{i,j=1}^{2} a_{ij}c_{i}d_{j}

with

c_{1} = \frac{y_{2} + (y_{1} − y_{2})x_{2}}{1 − (y_{1} − y_{2})(x_{1} − x_{2})} d_{1} = \frac{x_{2} + (x_{1} − x_{2})y_{2}}{1 − (y_{1} − y_{2})(x_{1} − x_{2})}

and c_{2} = 1 − c_{1}, d_{2} = 1 − d_{1}. Hence (1) yields

ẋ_{1} = \frac{x_{2}}{(1 − r)^{4}} F (x_{1}, x_{2})  ẋ_{2} = \frac{1 − x_{1}}{(1 − r)^{4}} F (x_{1}, x_{2})

with

F (x_{1}, x_{2}) = ax_{2}(1 − r) + ((a_{12} − a_{22}) + (a_{21} − a_{22})r) (1 − r)

where a = a_{11} − a_{21} + a_{22} − a_{12} and r = x_{1} − x_{2}. Now suppose there is no variation in x_{2}: it is fixed at some small value ε (the probability of a mistake). If a < 0, there is a unique x̂_{1} with F (x̂_{1}, ε) = 0. This x̂_{1} is h- but not x-stable (see Fig. 1). On the other hand, if x_{1} = 1 − ε is fixed at some value close to 1, there exists for a > 0 an x̂_{2} such that F (1 − ε, x̂_{2}) = 0, which is x- but not h-stable (see Fig. 2). Is this biologically plausible? The fixation of x_{1} = 1 − ε can be viewed as a consensus to cooperate as long as the other does. The evolutionary interesting variable is then the probability 1 − x_{2} to retaliate after a defection. It should offer no incentive for exploitation, but avoid needless recrimination after a mistake.

A similar example holds for the repeated game of Chicken (or Hawk-Dove): E_{2} means now escalation of the fight, E_{1} avoidance of it. Instead of (3) one has a_{21} > a_{11} > a_{12} > a_{22}. If x_{1} = ε is fixed at some small value (if the adversary chickened out last time, one gets bold), the question is to find the right probability to escalate following an escalation of the adversary. It is given by the value x̂_{2} of F (ε, x̂_{2}) = 0. Again, x̂_{2} is x-stable (the adaptive dynamics leads towards it), but not h-stable (see Fig. 3).

Figs. 1, 2, 3 (see text)

Let us consider now a sexually reproducing species with two mating types 1 and 2. A phenotype x corresponds to the choice of gamete weights x_{1} and x_{2} when of type 1 or 2. The number of gametes is inversely proportional to their weight. The survival probability V (m) of a zygote of mass m is increasing in m. Then, assuming sex ratio \frac{1}{2} and mass action, we get the payoff

A(y, x) = \frac{1}{2} (\frac{V (y_{1} + x_{2})}{y_{1}x_{2}} + \frac{V (y_{2} + x_{1})}{y_{2}x_{1}})

so that (1) becomes

ẋ_{i} = \frac{1}{x_{1}x_{2}} (V′(x_{1} + x_{2}) − \frac{V (x_{1} + x_{2})}{x_{i}}) i = 1, 2

Since

ẋ_{1} − ẋ_{2} = \frac{4V (x_{1} + x_{2})}{(x_{1}x_{2})^{2}}(x_{1} − x_{2})

we see that the flow points away from the (invariant) diagonal. The equilibria are on the diagonal (satisfying V (2x̂_{1}) = x̂_{1}V′(2x̂_{1})). They are evolutionarily stable (uninvadable if one of the gamete sizes is fixed) if V′′(2x̂_{1}) < 0: but it is highly unlikely that evolution will be trapped at such a point. The adaptive dynamics leads to a dimorphism of gamete sizes (one mating type follows the male strategy of producing many small gametes, the other mating type the female one).

So far we assumed that x varies in an open subset of R^{n} equipped with the Euclidean metric: fluctuations in every directions are equally likely. It may happen that another metric is more appropriate, e.g. if the genetic or developmental constraints render variations in one direction more likely than in another. It may also happen that x is restricted to some subset of R^{n}, for example to the simplex S_{n} if the x_{i} are probabilities of some strategies, summing up to 1.

Again, in an adaptive dynamics, the prevalent state x of the population will tend in the direction of the maximal local increase of the fitness A(y, x), i.e. the y-gradient of A(y, x) at y = x: ẋ will be proportional to the unit vector ξ maximizing A(x + εξ, x) − A(x, x), in the limit ε → 0. Obviously this notion of gradient, and therefore the adaptive dynamics associated with the fitness function A(y, x), depends on a Riemannian metric to be chosen on the state space.

Let G be a general Riemannian metric which associates (in a smooth way) to each x a symmetric positive definite matrix G(x) = (g_{ij}(x)), such that the inner product at x is given by

⟨ξ, η⟩_{x} = ξ^{T} G(x)η = ∑_{ij} g_{ij}(x)ξ_{i}η_{j}.  (4)

If the state space is (an open subset of) R^{n} then it is well-known that the G-gradient of a function V (x) is given by G(x)^{−1} grad V (x), where grad V (x) is the Euclidean gradient. From (4) and the definition of adaptive dynamics we have

⟨η, ẋ⟩_{x} = η^{T} G(x)ẋ = ∑ g_{ij}(x)η_{i}ẋ_{j} = D_{y}A(y, x)|_{y=x}(η)  (5)

for all η.

Let us now consider the most important special case where x ∈ S_{n} and A is linear in y, i.e.

A(y, x) = y^{T} f(x) = ∑_{i} y_{i}f_{i}(x).  (6)

Then (5) must hold for all η in the tangent space R_{0}^{n}, and we obtain

∑_{j} g_{ij}ẋ_{j} = f_{i}(x) + Φ(x)  (7)

for some function Φ which is determined by the constraint ∑_{i} ẋ_{i} = 0. If the matrix G is invertible, which can always be achieved by extending it in a positive definite way from R_{0}^{n} to R^{n}, we can transform (7), i. e. Gẋ = f + Φ1 1 1, into

ẋ = G^{−1}f(x) + gΦ(x),  (8)

with g = G^{−1}1 1 1, 1 1 1 denoting the n-vector of 1’s. Then 1 1 1^{T} ẋ = ∑ ẋ_{i} = 0 implies 1 1 1^{T} G^{−1}f + Φ1 1 1^{T} g = 0 or Φ = \frac{g^{T} f}{1 1 1^{T} g} , where 1 1 1^{T} g = 1 1 1^{T} G^{−1}1 1 1 > 0. Hence (7) is equivalent to the explicit form

ẋ_{i} = ∑_{j} c_{ij}(x)f_{j}(x)  (9)

with

C = G^{−1} − \frac{gg^{T}}{g^{T} 1 1 1} .  (10)

It is not hard to see that C restricted to R_{0}^{n} is positive definite. Conversely, if C is a symmetric n × n matrix, with C1 1 1 = 0 and ξ^{T} Cξ > 0 for ξ ∈ R_{0}^{n} \ {0}, then we can define G as a generalized inverse of C and thus obtain a Riemannian metric on S_{n}, such that (9) is the corresponding adaptive dynamics.

We illustrate this with two examples: a) Euclidean metric on S_{n}. Here g_{ij}(x) = δ_{ij}, hence g = G^{−1}1 1 1 = 1 1 1 and C = I − \frac{1}{n}1 1 1 1 1 1^{T}, i.e. c_{ij} = δ_{ij} − \frac{1}{n}, so that the adaptive dynamics reads

ẋ_{i} = f_{i}(x) − \frac{1}{n} ∑_{k=1}^{n} f_{k}(x).

b) The Shahshahani metric [1] on S_{n} is defined by g_{ij} = δ_{ij} \frac{1}{x_{i}} . Here g = G^{−1}1 1 1 = x, so by (10), c_{ij} = x_{i}δ_{ij} − x_{i}x_{j} and the adaptive dynamics reads

ẋ_{i} = x_{i}(f_{i}(x) − f̄(x)),  f̄(x) = ∑_{k} x_{k}f_{k}(x).

This is the replicator equation [6]. It is interesting to note that while the metric is defined only in int S_{n}, the dynamics extends continuously to the boundary, and leaves the boundary faces invariant. Hence this kind of adaptation does not lead to new strategies.

Let now x̂ ∈ int S_{n} be an ESS for the fitness function A(y, x) = y^{T} f(x), then by Pohley and Thomas [10]

x̂^{T} f(x) > x^{T} f(x) for x near x̂, x ≠ x̂.  (11)

We claim that x̂ is locally asymptotically stable for each adaptive dynamics (9). Let V (x) = ⟨x − x̂, x − x̂⟩x̂ = (x − x̂)^{T} G(x̂)(x − x̂), an approximation for the geodesic distance of x to x̂. Then by (7)

V̇ (x) = 2(x − x̂)^{T} G(x̂)ẋ ≈ 2(x − x̂)^{T} G(x)ẋ =
= 2(x − x̂)^{T} (f(x) + Φ1 1 1) = 2(x − x̂)^{T} f(x) < 0.

Hence V is a local Ljapunov function, and x̂ is asymptotically stable. Therefore, near an ESS, the geodesic distance between orbits decreases monotonically for each adaptive dynamics. In particular, for the replicator equation, the distance induced by the Shahshahani metric, i.e. the arccos distance [1, p. 39], tends monotonically to 0, near an ESS. Actually, at least for linear payoff functions f(x) = Ax, this gives a remarkable relation between the notion of ESS and the Shahshahani metric: x̂ is an ESS, if and only if the geodesic distance decreases near x̂ in the replicator equation. A related result of Hines [5] shows some other sort of converse: If x̂ is stable for each adaptive dynamics (7), i.e. for each choice of C or G, then x̂ must (essentially) be an ESS.

References

1. E. Akin, The Geometry of Population Genetics, Lecture Notes in Biomathematics 31, Springer, Berlin – Heidelberg – New York, 1979.

2. R. Axelrod, The Evolution of Cooperation, Basic Books, New York, 1984.

3. I. Eshel, Evolutionary and continuous stability, J. Theor. Biol. 103 (1983), 99–111.

4. I. Eshel and U. Motro, Kin selection and strong evolutionary stability of mutual help, Theor. Pop. Biol. 19 (1981), 420–433.

5. W.G.S. Hines, Three characterizations of strategy stability, J. Appl. Prob. 17 (1980), 333–340 21 (1984), 213–214.

6. J. Hofbauer and K. Sigmund, The Theory of Evolution and Dynamical Systems, Cambridge University Press, 1988.

7. J. Maynard Smith, Evolution and the Theory of Games, Cambridge University Press, 1982.

8. M. Nowak, An evolutionarily stable strategy may be inaccessible, J. Theor. Biol. (to appear).

9. M. Nowak and K.Sigmund, The evolution of reactive strategies in iterated games, Acta Applicandae Math. (submitted).

10. H.-J. Pohley and B. Thomas, Nonlinear ESS-models and frequency dependent selection, Biosystems 16 (1983), 87-100.

11. P.D. Taylor, Evolutionary stability in one-parameter models under weak selelction, Theor. Pop. Biol. 36 (1989), 125-143.