An Introduction to Stochastic Processes with Applications to Biology 2nd Edition by Linda Allen 9781439818824 1439818827

Chapter 1 Review of Probability Theory and an Introduction to Stochastic Processes

1.1 Introduction

1.2 Brief Review of Probability Theory

1.2.1 Basic Probability Concepts

FIGURE 1.1: Discrete uniform c.d.f.

FIGURE 1.2: Continuous uniform c.d.f.

1.2.2 Probability Distributions

FIGURE 1.3: Poisson mass function with parameter λ = 3 and the standard normal density.

1.2.3 Expectation

1.2.4 Multivariate Distributions

1.3 Generating Functions

1.4 Central Limit Theorem

FIGURE 1.4: Binomial mass function b(6, 1/3), and the standard normal density. Approximate probability histogram of W15 (defined in the Central Limit Theorem) is graphed against the standard normal density.

1.5 Introduction to Stochastic Processes

FIGURE 1.5: Three stochastic realizations of the simple birth process and corresponding deterministic exponential growth model y = et (dashed curve).

1.6 An Introductory Example: A Simple Birth Process

FIGURE 1.6: Graphs of the p.m.f. pn(t), n = 0, 1, 2,…, at t = 0, 1, 2, 3 when X0 = a = 1 and b = 1 equals the birth rate.

Table 1.1: Mean μt, variance σt2, standard deviation σt, and ODE solution n(t) for the simple birth process at t = 0, 1, 2, 3 when a = 1 and b = 1

FIGURE 1.7: Three stochastic realizations of the simple birth process when b = 1, a = 1, and X0 = 1. The deterministic exponential growth model, n(t) = et, is the dashed curve.

Table 1.2: For two stochastic realizations, the times at which a birth occurs for a simple birth process with b = 1 and a = 1

1.7 Exercises for Chapter 1

1.8 References for Chapter 1

1.9 Appendix for Chapter 1

1.9.1 Probability Distributions

1.9.2 MATLAB® and FORTRAN Programs

1.9.3 Interevent Time

Chapter 2 Discrete-Time Markov Chains

2.1 Introduction

2.2 Definitions and Notation

2.3 Classification of States

FIGURE 2.1: Directed graph with i→j(pji>0) and i→k(pki(2)>0), but it is not the case that k → i.

FIGURE 2.2: Directed graph for Example 2.2.

FIGURE 2.3: Probability of winning is p and losing is q. Boundaries, 0 and N, are absorbing, p00 = 1 = pNN

FIGURE 2.4: Unrestricted random walk with the probability of moving right equal to p and left equal to q.

FIGURE 2.5: Directed graph for Example 2.5.

FIGURE 2.6: Directed graph for Example 2.6.

FIGURE 2.7: Directed graph for Example 2.7.

2.4 First Passage Time

2.5 Basic Theorems for Markov Chains

2.6 Stationary Probability Distribution

2.7 Finite Markov Chains

2.7.1 Mean First Passage Time

2.8 An Example: Genetics Inbreeding Problem

2.9 Monte Carlo Simulation

2.10 Unrestricted Random Walk in Higher Dimensions

2.10.1 Two Dimensions

2.10.2 Three Dimensions

2.11 Exercises for Chapter 2

FIGURE 2.8: Directed graph of a stochastic matrix.

2.12 References for Chapter 2

2.13 Appendix for Chapter 2

2.13.1 Proofs of Theorems 2.5 and 2.6

2.13.2 Perron and Frobenius Theorems

2.13.3 The n-Step Transition Matrix

2.13.4 Genetics Inbreeding Problem

Chapter 3 Biological Applications of Discrete-Time Markov Chains

3.1 Introduction

3.2 Proliferating Epithelial Cells

FIGURE 3.1: Cell division results in two new vertices and three new sides per cell.

3.3 Restricted Random Walk Models

3.4 Random Walk with Absorbing Boundaries

3.4.1 Probability of Absorption

Table 3.1: Gambler’s ruin problem with a beginning capital of k = 50 and a total capital of N = 100

3.4.2 Expected Time until Absorption

FIGURE 3.2: Expected duration of the games, τk for k = 0, 1, 2, . . . , 100, when q = 0.55.

3.4.3 Probability Distribution for Absorption

FIGURE 3.3: Three sample paths for the gambler’s ruin problem when N = 100, k = 50, and q = 0.55.

3.5 Random Walk on a Semi-Infinite Domain

3.6 General Birth and Death Process

3.6.1 Expected Time to Extinction

FIGURE 3.4: Expected time until population extinction τ when the maximal population size is N = 20 and b = 0.02 0.02 = d. Three sample paths for the case b = 0.025 = d with X0 = 10.

3.7 Logistic Growth Process

FIGURE 3.5: Three sample paths of the logistic growth process compared to the solution of the deterministic logistic equation in case (b), where X0 = 5, bi = ri, di = ri2/K, r = 0.004, r˜ = 1, K = 50, Δt = 0.004, and N = 100. Time is measured in increments of Δt, 2000 time increments.

FIGURE 3.6: Expected time until population extinction when the birth and death rates satisfy (a) and (b) and the parameters are r = 0.015, K = 10, and N = 20.

FIGURE 3.7: Probability distribution for the stochastic logistic model in cases (a) and (b) when r = 0.015, K = 10, N = 20, and X0 = 1.

3.8 Quasistationary Probability Distribution

FIGURE 3.8: Quasistationary probability distribution, q* (solid curve), and the approximate quasistationary probability distribution, q∼* (diamond marks), when r = 0.015, K = 10, and N = 20 in cases (a) and (b). In (c), r = 0.015, K = 5, N = 10, where bi = ri and di = ri2/K.

FIGURE 3.9: Stochastic logistic probability distribution for n = 0, …, 2000 and the solution of the logistic differential equation are compared for parameter values and initial conditions: r = 0.004, r∼ = 1, Δt = 0.004, K = 50, N = 100, y(0) = 5 = X0(top two figures). The mean of the stochastic logistic equation (dashed and dotted curve) and the solution of the deterministic model (solid curve) are also compared (bottom figure).

3.9 SIS Epidemic Model

FIGURE 3.10: Compartmental diagram of the SIS epidemic model. A susceptible individual becomes infected with probability βI/N and an infected individual recovers with probability γ (solid lines). Birth and death probabilities equal b (dotted lines).

3.9.1 Deterministic Model

3.9.2 Stochastic Model

FIGURE 3.11: Probability distribution p(n) for n = 0, 1, …, 2000 when I0 = 1 and the expected duration of the epidemic τk as a function of the initial number of infected individuals k = I0. Parameters for the SIS epidemic model are N = 100, β = 0.01, b = 0.0025 = γ, and R0 = 2.

3.10 Chain Binomial Epidemic Models

3.10.1 Greenwood Model

Table 3.2: Sample paths, duration, and size for the Greenwood and Reed-Frost models when s0 = 3 and i0 = 1

FIGURE 3.12: Four sample paths for the Greenwood chain binomial model when s0 = 6 and i0 = 1, {6, 6}, {6, 5, 5}, {6, 4, 3, 2, 1, 1}, and {6, 2, 1, 0, 0}.

3.10.2 Reed-Frost Model

3.10.3 Duration and Size

FIGURE 3.13: Probability distributions for the size of the epidemic, W, and the duration of the epidemic, T, when p = 0.5, s0 = 3, and i0 = 1.

3.11 Exercises for Chapter 3

FIGURE 3.14: Probability of population extinction for the stochastic logistic model when bi =ri(l −i/(2K)) and di = ri2/(2K), fori = 0, 1, 2, …, 2K, r = 0.015 andK = 20.

Table 3.3: Estimates of the probability of rapid population extinction for the stochastic logistic model when N = 2K, bN = 0, r = 0.005, bi = ri, and di = ri2/K for i = 0, 1, 2, …, N

3.12 References for Chapter 3

3.13 Appendix for Chapter 3

3.13.1 MATLAB® Programs

3.13.2 Maple™ Program

Chapter 4 Discrete-Time Branching Processes

4.1 Introduction

4.2 Definitions and Notation

FIGURE 4.1: Sample path of a branching process { Xn }n=0∞. In the first generation, four individuals are born, X1=4. The four individuals in generation one give birth to three, zero, four, and one individuals, respectively, making a total of eight individuals in generation two, X2 = 8.

4.3 Probability Generating Function of Xn

FIGURE 4.2: Sample path of a branching process { Xn }n=0∞, where X0=3, X1=5, and X2=9.

4.4 Probability of Population Extinction

FIGURE 4.3: The p.g.f. y = f(t) intersects y = t in either one or two points on [0,1].

FIGURE 4.4: Probability of extinction in Lotka’s model in generation n, if X0=1(q≈0.83) and X0=5(q5≈0.40).

Table 4.1: Approximations to the probability that a mutant gene becomes extinct, qN. or becomes established, 1 – qN. with initally N mutant genes. Offspring distribution is Poisson with m = 1.01 = σ2

4.5 Mean and Variance of Xn

4.6 Environmental Variation

4.7 Multitype Branching Processes

4.7.1 An Example: Age-Structured Model

4.7.2 Environmental Variation

FIGURE 4.5: Solution of the deterministic model (solid curve), one sample path of the size-dependent stochastic Ricker model (dotted curve), and the the mean of 100 sample paths (dashed curve). In the figure on the left, the size-independent offspring distribution is pk = 1/6 for k = 1, 2, …,6 with r = 1.2528 2 and γ = 0.005; the deterministic solution and stochastic mean oscillate between two values.

4.8 Exercises for Chapter 4

4.9 References for Chapter 4

Chapter 5 Continuous-Time Markov Chains

5.1 Introduction

5.2 Definitions and Notation

FIGURE 5.1: Sample path of a CTMC, illustrating waiting times and interevent times.

FIGURE 5.2: Sample path of an explosive CTMC.

5.3 The Poisson Process

5.4 Generator Matrix Q

FIGURE 5.3: Sample path of a Poisson process with λ = 1.

5.5 Embedded Markov Chain and Classification of States

5.6 Kolmogorov Differential Equations

5.7 Stationary Probability Distribution

5.8 Finite Markov Chains

FIGURE 5.4: Directed graph of the embedded Markov chain {Yn}n=0∞.

5.9 Generating Function Technique

5.10 Interevent Time and Stochastic Realizations

FIGURE 5.5: Sample path X(t) of a CTMC, illustrating the jump times {Wi}i=0∞ and the interevent times {Ti}i=0∞,X(0)=2,X(W1)=3,X(W2)=4, and X(W3)=3..

FIGURE 5.6: Three sample paths of the simple birth process with X(0) = 1 and b = 1.

FIGURE 5.7: Three sample paths of the simple death process with X(0) = 100 and d = 0.5.

FIGURE 5.8: Three sample paths of the simple birth and death process with X(0) = 5, b = 1, and d = 0.5.

5.11 Review of Method of Characteristics

5.12 Exercises for Chapter 5

5.13 References for Chapter

5.14 Appendix for Chapter 5

5.14.1 Calculation of the Matrix Exponential

5.14.2 MATLAB® Programs

Chapter 6 Continuous-Time Birth and Death Chains

6.1 Introduction

6.2 General Birth and Death Process

FIGURE 6.1: Directed graph for the embedded Markov chain of the general birth and death process when λ0>0 and λi+μi>0 for i = 1, 2,….

6.3 Stationary Probability Distribution

6.4 Simple Birth and Death Processes

6.4.1 Simple Birth

FIGURE 6.2: Probability distributions of X(t) for the simple birth process for t = 1,2,3,4 and three sample paths when λ = 0.5 and X(0) = 5. The mean and variance are m(t)=5e0.5t,σ2(t)=5(et−e0.5t).

6.4.2 Simple Death

FIGURE 6.3: Probability distributions of X(t) for the simple death process for t = 1,2,3,4 and three sample paths when μ = 0.5 and X(0) = 20. The mean and variance are m(t)=20e−0.5t and σ2=20(e−0.5t−e−t).

6.4.3 Simple Birth and Death

Table 6.1: Mean, variance, and p.g.f. for the simple birth, simple death, and simple birth and death processes, where X(0) = N and ρ=e(λ−μ)t , λ≠μ

FIGURE 6.4: Three sample paths for the simple birth and death process when λ = 1 = μ and X(0) = 50.

6.4.4 Simple Birth and Death with Immigration

Table 6.2: Mean, variance, and p.g.f for the simple birth and death with immigration process, where X(0) = N and ρ=e(λ−μ)t, λ≠μ

FIGURE 6.5: Stationary probability distributions for the simple birth, death and immigration process with λ = 0.5, μ = 1, and v = 0.5, 1.0, 1.5, or 5.

FIGURE 6.6: Four sample paths corresponding to the birth, death, and immigration process when X(0) = 20, λ = 0.5, μ = 1, and v = 0.5, 1.0, 1.5, 5. The mean values of the respective stationary distributions are m = 1, 2, 3, and 10.

6.5 Queueing Process

FIGURE 6.7: A simple queueing system.

6.6 Population Extinction

6.7 First Passage Time

6.7.1 Definition and Computation

6.7.2 Summary of First Passage Time

6.8 Logistic Growth Process

FIGURE 6.8: Three sample paths of the stochastic logistic model for cases (a) and (b) with X(0) = 10.

FIGURE 6.9: Expected time until extinction in the stochastic logistic model with K = 5 and K = 10.

6.9 Quasistationary Probability Distribution

FIGURE 6.10: Approximate quasistationary probability distribution π˜ for cases (a) and (b). The mean and standard deviation for case (a) are m∼≈8.85, σ∼≈3.19 and for case (b), m∼≈9.44 and σ∼≈2.31.

6.10 An Explosive Birth Process

6.11 Nonhomogeneous Birth and Death Process

6.12 Exercises for Chapter 6

FIGURE 6.11: Three sample paths in case (ii) of Exercise 16.

6.13 References for Chapter 6

6.14 Appendix for Chapter 6

6.14.1 Generating Functions for the Simple Birth and Death Process

6.14.2 Proofs of Theorems 6.2 and 6.3

6.14.3 Comparison Theorem

Chapter 7 Biological Applications of Continuous-Time Markov Chains

7.1 Introduction

7.2 Continuous-Time Branching Processes

FIGURE 7.1: Sample path of a continuous-time branching process. At time t1, an individual gives birth to two individuals and these individuals give birth to three and two individuals at times t2 and t3, respectively.

7.3 SI and SIS Epidemic Processes

Table 7.1: Approximate duration (T) until infection in the deterministic SI epidemic model for population size N and contact rate β when I(0) = 1

7.3.1 Stochastic SI Model

Table 7.2: Approximate and exact mean durations E(TN, 1)  and exact variance var(TN, 1) for the time until absorption in the SI stochastic epidemic model when I(0)  =  1 and β  =  1

7.3.2 Stochastic SIS Model

FIGURE 7.2: Expected duration until absorption at I = 0 and the quasistationary distribution of the stochastic SIS model when β  =  2, b + γ = 1, and N = 100.

7.4 Multivariate Processes

7.5 Enzyme Kinetics

7.5.1 Deterministic Model

FIGURE 7.3: Concentration of N(t) and B(t) for t ∈ [0,100] for initial conditions N(0) = 5 × 10−7M and B(0) = 0.

FIGURE 7.4: Solutions N(t) and P(t) for Michaelis-Menten enzyme kinetics model (7.19) when N(0) =  p∞  and P(0) = 0.

7.5.2 Stochastic Model

FIGURE 7.5: Sample path for the stochastic enzyme kinetics model, N(t) and B(t), and an approximate probability histogram for the time T until the number of molecules N(T) + B(T) = 0 given N(0) =  p∼∞  and B(0) = 0.

FIGURE 7.6: Three sample paths for the stochastic Michaelis-Menten enzyme kinetics model N(t) and P(t), and a probability histogram that approximates the probability density for the time T until N(T)  =  0, given N(0)  =  p∼∞ .

FIGURE 7.7: Expected duration for the stochastic enzyme kinetics model.

7.6 SIR Epidemic Process

7.6.1 Deterministic Model

Table 7.3: Final size of an SIR epidemic, R(∞), when γ = 1, S(0) = N − 1, and I(0) = 1

7.6.2 Stochastic Model

7.6.3 Final Size

FIGURE 7.8: Directed graph of the embedded Markov chain of the SIR epidemic model when N = 4. The maximum path length beginning from state (3,1) is indicated by the thick arrows.

FIGURE 7.9: Probability distribution for the final size of a stochastic SIR epidemic model when I(0) = 1, S(0)  = N − 1, γ = 1, and β = 0.5, 2, or 5 (R0 = 0.5, 2, or 5). In (a), N = 20 and in (b), N = 100.

Table 7.4: Expected final size of a stochastic SIR epidemic when γ = 1, S(0) = N − 1, and I(0) = 1

7.6.4 Duration

FIGURE 7.10: Three sample paths of a stochastic SIR epidemic model with N = 100, β = 2, γ = 1, S(0) = 99, and I(0) = 1; R0 = 2.

FIGURE 7.11: Probability distribution for the duration of an SIR epidemic, N = 100, β = 2, and γ = 1 (estimated from 1000 sample paths). In (a), I(0) = 1 and S(0) = 99 and in (b), I(0) = 5 and S(0) = 95.

FIGURE 7.12: Mean and standard deviation, graphs (a) and (b), respectively, of the distribution for the duration of an SIR epidemic as a function of the initial number of infected individuals, I(0) = i and S(0) = N − i, i = 1, 2,…, 100, N = 100, when β = 2, and γ = 1 (estimated from 1000 sample paths).

7.7 Competition Process

7.7.1 Deterministic Model

7.7.2  Stochastic Model

FIGURE 7.13: Sample paths of the CTMC competition model graphed as a function of time and in the X1 – X2 phase plane (top two graphs). The dashed lines are the equilibrium values and the solid lines are the nullclines of the deterministic model. Approximate probability histograms for X1 and X2 at t =  5, based on 10,000 stochastic realizations (bottom two graphs). Birth and death rates are given in (7.26) with parameter values and initial conditions a10 = 2, a20 = 1.5, a11 = 0.03, a12 = 0.02, a21  =  0.01, a22 = 0.04, X1(0) = 50, and X2(0) = 25.

7.8 Predator-Prey Process

7.8.1 Deterministic Model

7.8.2 Stochastic Model

FIGURE 7.14: Sample path of the stochastic Lotka-Volterra predator-prey model with the solution of the deterministic model. Solutions are graphed over time and in the phase plane. The parameter values and initial conditions are a10  =  1, a20 = 1, a12 = 0.02, a21 = 0.01, X(0) = 120, and Y(0) = 40. Solutions with the smaller amplitude are the predator population.

7.9  Exercises for Chapter 7

Table 7.5: Probabilities associated with changes in the chemostat model

FIGURE 7.15: Sample path of the stochastic predator-prey model with the solution of the deterministic model. Parameter values and initial conditions are  r = 1,  K = 100, a =  1, d =  20, b =  0.02, c =  1, X(0) = 20, and Y(0) = 30. Solutions with the smaller amplitude are the predator population.

FIGURE 7.16: SEIR epidemic model with immigration of infectives; the system exhibits oscillations before convergence to an endemic equilibrium. Initial conditions are  S(0) = 249, 995, E(0) = 0, I(0) = 5, and R(0) = 0.

7.10 References for Chapter 7

7.11 Appendix for Chapter 7

7.11.1 MATLAB® P rograms

Chapter 8 Diffusion Processes and Stochastic Differential Equations

8.1  Introduction

8.2  Definitions and Notation

8.3  Random Walk and Brownian Motion

FIGURE 8.1: Solution of the diffusion equation with drift with c = 1 and D = 1 at t = 1, 3, 5.

8.4  Diffusion Process

8.5  Kolmogorov Differential Equations

8.6  Wiener Process

8.7  Itô Stochastic Integral

8.8  Itô Stochastic Differential Equation

FIGURE 8.2: Three sample paths of the stochastic exponential growth model (8.28) with the deterministic solution, X(0) exp(rt), where X(0) = 5. In (a), r = 1 and c = 0.5. In (b), r = 1 and c = 2.

8.9  First Passage Time

FIGURE 8.3: Mean time to reach either a population size of one or a population size of K = 1000 for the stochastic process in Example 8.14.

8.10  Numerical Methods for SDEs

FIGURE 8.4: Two sample paths W(t) of the Wiener process on [0, 1].

8.11  An Example: Drug Kinetics

FIGURE 8.5: Three sample paths of C(t) and the mean E(C(t)|C(0)) (smooth curve) for the drug kinetics example with parameter values and initial condition α = 3, σ = 1, Cs = 10, and C(0) = 5 for t ∈ [0, 1], 2α>σ2.

8.12  Exercises for Chapter 8

8.13  References for Chapter 8

8.14  Appendix for Chapter 8

8.14.1  Derivation of Kolmogorov Equations

8.14.2  MATLAB® PROGRAM

Chapter 9 Biological Applications of Stochastic Differential Equations

9.1 Introduction

9.2 Multivariate Processes

9.3 Derivation of Ito^ SDEs

Table 9.1: Probabilities associated with changes in the interacting population model, 0 ≤ Pi ≤ 1

9.4 Scalar Itô SDEs for Populations

9.4.1 Simple Birth and Death with Immigration

Table 9.2: Probabilities associated with changes in the birth, death, and immigration model.

9.4.2 Logistic Growth

Table 9.3: Probabilities associated with changes in the logistic growth model

FIGURE 9.1: Three stochastic realizations of the logistic model for case (a), equation (9.18), and case (b), equation (9.19), with r = 1, K = 10, and X(0) = 10.

9.4.3 Quasistationary Density Function

FIGURE 9.2: Approximate quasistationary p.d.f. for logistic growth in cases (a) and (b) when r=1 and K = 10. In case (a), the mean and standard deviation are m = 8.86 and σ=3.19 and in case (b), they are m = 9.43 and σ=2.32.

9.5 Enzyme Kinetics

Table 9.4: Probabilities associated with changes in the enzyme kinetics model

FIGURE 9.3: Sample path of the stochastic enzyme kinetics model, N(t) and B(t), and the average of 10,000 sample paths when N(0)=p∼∞ and B(0) = 0.

9.6 SIR Epidemic Process

Table 9.5: Probabilities associated with changes in the SIR model

9.7 Competition Process

Table 9.6: Probabilities associated with changes in the competition model

FIGURE 9.4: Sample paths of the SDE competition model as a function of time and in the X1-X2 phase plane(top two graphs). The dashed lines are the equilibrium values and the solid lines are the nullclines of the deterministic model. Approximate probability histograms for  X1 and X2 for  t = 5, based on 10,000 stochastic realizations (bottom two figures). The parameter values and initial conditions are a10=2,a20=1.5,a11=0.03,a12=0.02,a21=0.01,a22=0.04,X1(0)=50 and X2(0)=25

9.8 Predator-Prey Process

Table 9.7: Probabilities associated with changes in the predator-prey model

FIGURE 9.5: Sample path of the stochastic Lotka-Volterra predator-prey model with the solution of the deterministic model. Solutions are graphed over time as well as in the phase plane. The parameter values and initial conditions are a10=1, a20=1, a12=0.02, a21=0.01, X(0)=120, and Y(0)=40. Solutions with the smaller amplitude are the predator population.

9.9 Population Genetics Process

FIGURE 9.6: Three sample paths for the population genetics random drift model with X(0)=1/2 and  N = 100. The approximate probability histograms of X(t) at  t = 10, 50, 200.

9.10 Exercises for Chapter 9

Table 9.8: Probabilities associated with changes in the chemostat model

Table 9.9: Probabilities associated with changes in the viral kinetics model

9.11 References for Chapter 9

9.12 Appendix for Chapter 9

9.12.1 MATLAB® Programs