## Derivation of the inbreeding effective population size Ne

The inbreeding coefficient F in any closed, finite population will increase over time, as an increasing proportion of individuals will necessarily come to have related parents. The effective population size Ne is defined by observing how the observed inbreeding coefficient changes from one generation to the next, and defining Ne as the size of the ideal population with the same change as the real population under consideration.

In any finite population, the inbreeding coefficient F in the current generation t changes from the previous generation t-1 as a recursion equation:

${\displaystyle F_{t}={\frac {1}{N}}\left({\frac {1+F_{t-2}}{2}}\right)+\left(1-{\frac {1}{N}}\right)F_{t-1}.}$

The equation has two terms, (1/N) and its complement (1 - 1/N).

(1) For a draw & replace experiment in a population size N in the previous generation t-1, for any individual drawn at random, the expectation of drawing the same individual a second time is 1/N, and if so the expectation of drawing the same allele is 1/2. Therefore, the expectation of drawing the same allele twice so as to obtain an individual with two alleles that are identical by descent (I by D) is 1/2N. This calculation is weighted by the expectation Ft-2 that any pair of alleles were already I by D in the next previous generation.

(2) The expectation of not drawing the same individual twice is  (1 - 1/N), weighted by the expectation Ft-1 that any random pair of alleles is already I by D in the previous generation.

To extend this recursion calculation from two generations to t generations, consider the complement of the inbreeding coefficient F as the Panmixia Index P = (1 − F), which is the expectation that any two individuals in a finite population size N do not share a common ancestor.  For an initial P0, the expectation that any two alleles drawn in the subsequent generation are not I by D is (1 - 1/2N). The same is true in each subsequent generation, so that if population size N is constant, the joint probability that this is never so over all generations i = 1, 2, 3, ..., t is simply the product of all those "not" terms,

Text material © 2020 by Steven M. Carr