2 Distributions and Generative Models for Biological Data
Chapter Aim
Chapter 02 asks a basic modeling question: what kind of random mechanism could have generated the entries, rows, or columns of a high-throughput biological data matrix?
This chapter reviews probability distributions as generative models for biological data. The aim is to understand how the support, parameters, expectation, variance, and dependence structure of a distribution encode assumptions about the data. For high-throughput biological data, the most important distinction is between continuous measurements, binary or categorical measurements, count measurements, and discrete compositions.
Notation follows the shared convention in the notation chapter. Vectors are columns by default, row vectors are written with transposes, and dimensions should be checked whenever matrices are multiplied, transformed, modeled, or decomposed.
The main reference is MSMB Generative Models, especially the discussion of Bernoulli, binomial, multinomial, and Poisson models. For likelihood and maximum likelihood, see MSMB Statistical Modeling. For overdispersed count models in RNA-seq, see MSMB High-Throughput Count Data. For mixture models, including Gamma-Poisson ideas, see MSMB Mixture Models.
The corresponding computations are developed in Exercise Sheet 01, which starts from sequence summaries and then works through multinomial likelihoods, Poisson count models, Poisson mixtures, expectation-maximization, and Gaussian mixture models (Exercise Sheet 01 2026).
Learning Goals
After this chapter, students should be able to:
- Identify whether a biological measurement is continuous, binary, categorical, count-valued, or compositional.
- Choose a simple probability distribution whose support matches the data type.
- Explain what it means to use a probability distribution as a generative model.
- Distinguish entry-wise, row-wise, and column-wise models for \(C\) and \(X\).
- Define overdispersion and recognize why Poisson and multinomial models are often too restrictive for biological count data.
- Explain how a Gamma-Poisson mixture leads to a negative binomial distribution.
- Explain how a Dirichlet-multinomial model extends the multinomial model for overdispersed discrete compositions.
- Describe Aitchison’s logistic-normal model as a generative model for continuous compositions.
- Connect generative assumptions to likelihood, estimation, model checking, and later normalization choices.
2.1 From A Data Matrix To A Random Variable
A generative model treats an observed entry \(c_{ij}\), a row vector \(c_i\), or a column vector \(c_{\cdot j}\) as a realization of a random object. This changes the question from “what number did we observe?” to “what random mechanism could plausibly produce numbers like this?”
For an entry-wise model we might write
\[\begin{equation} C_{ij} \sim P_{\theta_{ij}}, \label{eq:ch02-entry-wise-model} \end{equation}\]
where \(P_{\theta_{ij}}\) is a probability distribution with parameter \(\theta_{ij}\). For a row-wise compositional model we might write
\[\begin{equation} C_i \sim P_{\theta_i}, \label{eq:ch02-row-wise-model} \end{equation}\]
where \(C_i = (C_{i1},\ldots,C_{ip})^\top\) is the random count vector for observation \(i\). For a feature-wise model we might isolate one column and write
\[\begin{equation} Y_j = C_{\cdot j} \in \mathbb{N}_0^n. \label{eq:ch02-feature-column} \end{equation}\]
The same data matrix can therefore support several distinct generative views. Entry-wise models focus on single observations of single features. Row-wise models focus on the composition of one observation across many features. Column-wise models focus on how one feature varies across observations.
2.2 What Makes A Distribution Appropriate?
The first requirement is support. A probability model should assign probability to values that the measurement can actually take. A binary mutation indicator needs a distribution on \(\{0,1\}\). A genotype count may need a distribution on \(\{0,1,2\}\). A sequencing read count needs a distribution on nonnegative integers. A normalized protein abundance may need a continuous distribution on \(\mathbb{R}\) or \(\mathbb{R}_{>0}\). A relative abundance vector must live on a simplex.
The second requirement is moment structure. Many distributions imply a relationship between the mean and the variance, and this relationship is often decisive for biological data. A Poisson distribution has variance equal to its mean. A binomial distribution has variance smaller than or equal to a particular function of its mean. Biological count data often vary more than these simple models allow.
The third requirement is dependence structure. A model for a single entry does not automatically model dependencies between features. A multinomial model creates negative dependence among counts because the row total is fixed. A Dirichlet-multinomial model adds extra row-to-row variation in the underlying composition. Later network chapters will ask a different kind of dependence question: whether features are associated or conditionally dependent across observations.
2.3 A Coherent Distribution Map
The distributions in this chapter are best organized by the type of data they generate.
| Data type | Support | Typical biological example | Basic distribution | Overdispersed or flexible extension |
|---|---|---|---|---|
| continuous real-valued | \(\mathbb{R}\) | transformed expression, regression residual | normal | mixture of normals, Laplace |
| continuous positive | \(\mathbb{R}_{>0}\) | rates, intensities, waiting times, positive abundances | gamma | gamma mixtures, log-normal |
| binary | \(\{0,1\}\) | mutation present/absent, responder/non-responder | Bernoulli | beta-Bernoulli |
| bounded counts | \(\{0,\ldots,m\}\) | number of successes among \(m\) trials | binomial | beta-binomial |
| categorical | \(\{1,\ldots,K\}\) | class label, genotype category, nucleotide identity | categorical | mixture of categoricals |
| unbounded counts | \(\mathbb{N}_0\) | read counts, mutation counts, binding-site counts | Poisson | negative binomial, zero-inflated models |
| discrete composition | \(\{c \in \mathbb{N}_0^p: \sum_j c_j=N\}\) | amplicon or taxon count vector | multinomial | Dirichlet-multinomial |
| continuous composition | \(\Delta^{p-1}\) | relative abundance vector | Dirichlet | logistic-normal |
This table is not a decision tree. It is a map of assumptions. The model should be chosen because its support and variability match the scientific object, not because it is mathematically convenient.
2.4 Continuous Distributions
Continuous distributions are useful when measurements can be treated as real-valued after preprocessing. This is often appropriate for transformed gene expression, protein intensities, metabolite intensities, image-derived features, and residuals in regression models.
The normal distribution is the central reference model for real-valued data:
\[\begin{equation} X \sim \mathcal{N}(\mu,\sigma^2), \qquad \mathbb{E}(X)=\mu, \qquad \operatorname{Var}(X)=\sigma^2. \label{eq:ch02-normal} \end{equation}\]
Its density is
\[\begin{equation} f(x\mid \mu,\sigma^2) = \frac{1}{\sqrt{2\pi\sigma^2}} \exp\left\{-\frac{(x-\mu)^2}{2\sigma^2}\right\}, \qquad x\in\mathbb{R}, \qquad \sigma^2>0. \label{eq:ch02-normal-density} \end{equation}\]
The parameter \(\mu\in\mathbb{R}\) is the mean and location parameter. The parameter \(\sigma^2>0\) is the variance, and \(\sigma\) is the standard deviation. The normal distribution is symmetric and fully determined by these two parameters. It is often a useful model for transformed data or for residual variation, but it is not a good generative model for raw sequencing counts because counts are nonnegative integers and often show mean-variance relationships that are incompatible with constant variance.
For positive continuous quantities, the gamma distribution is useful. In this chapter it plays a special role because it will become a distribution over unobserved Poisson rates. We use the shape-rate parameterization:
\[\begin{equation} \Lambda \sim \operatorname{Gamma}(\alpha,\beta), \qquad \mathbb{E}(\Lambda)=\frac{\alpha}{\beta}, \qquad \operatorname{Var}(\Lambda)=\frac{\alpha}{\beta^2}. \label{eq:ch02-gamma} \end{equation}\]
Its density is
\[\begin{equation} f(\lambda\mid \alpha,\beta) = \frac{\beta^\alpha}{\Gamma(\alpha)} \lambda^{\alpha-1}\exp(-\beta\lambda), \qquad \lambda>0, \qquad \alpha>0,\ \beta>0. \label{eq:ch02-gamma-density} \end{equation}\]
Here \(\alpha\) is the shape parameter and \(\beta\) is the rate parameter. Increasing \(\alpha\) changes the shape and, together with \(\beta\), controls the mean. Increasing \(\beta\) shrinks the distribution toward smaller values because this is a rate, not a scale. The gamma distribution has support on \(\mathbb{R}_{>0}\). It is therefore natural for rates, intensities, and positive latent quantities. In a Gamma-Poisson model, it represents biological or technical variation in the expected count before a Poisson sampling step occurs.
The Laplace distribution is another continuous distribution that is useful in mixture examples. It is symmetric like the normal distribution, but more sharply peaked at its center and heavier-tailed:
\[\begin{equation} f(x \mid \mu,b) = \frac{1}{2b}\exp\left(-\frac{|x-\mu|}{b}\right), \qquad x\in\mathbb{R}, \qquad b>0. \label{eq:ch02-laplace} \end{equation}\]
For this parameterization,
\[\begin{equation} \mathbb{E}(X)=\mu, \qquad \operatorname{Var}(X)=2b^2. \label{eq:ch02-laplace-moments} \end{equation}\]
The parameter \(\mu\in\mathbb{R}\) is the location parameter and is also the mean and median. The parameter \(b>0\) is a scale parameter; larger \(b\) gives a wider distribution and heavier tails. For this course, the Laplace distribution is less important as a standalone biological model than as a reminder that the shape of a distribution matters. Two distributions can have similar centers but very different tails, and these differences affect estimation, robustness, and model checking.
2.5 Binary And Categorical Distributions
Binary data arise when each observation has only two possible outcomes. Examples include mutation present versus absent, responder versus non-responder, or a quality-control indicator. A Bernoulli random variable is written as
\[\begin{equation} X \sim \operatorname{Bernoulli}(\pi), \qquad \Pr(X=1)=\pi, \qquad \Pr(X=0)=1-\pi. \label{eq:ch02-bernoulli} \end{equation}\]
Equivalently, its probability mass function is
\[\begin{equation} \Pr(X=x\mid \pi) = \pi^x(1-\pi)^{1-x}, \qquad x\in\{0,1\}, \qquad 0\leq \pi\leq 1. \label{eq:ch02-bernoulli-pmf} \end{equation}\]
The expectation and variance are
\[\begin{equation} \mathbb{E}(X)=\pi, \qquad \operatorname{Var}(X)=\pi(1-\pi). \label{eq:ch02-bernoulli-moments} \end{equation}\]
The parameter \(\pi\) is the success probability and is also the mean. It is not a feature index; throughout the script the feature dimension is denoted by \(p\), while probabilities for binary, categorical, and compositional models are denoted by \(\pi\).
If \(m\) independent Bernoulli trials have the same success probability \(\pi\), their sum has a binomial distribution:
\[\begin{equation} X \sim \operatorname{Binomial}(m,\pi), \qquad \Pr(X=x\mid m,\pi)=\binom{m}{x}\pi^x(1-\pi)^{m-x}, \qquad x=0,\ldots,m. \label{eq:ch02-binomial} \end{equation}\]
The binomial moments are
\[\begin{equation} \mathbb{E}(X)=m\pi, \qquad \operatorname{Var}(X)=m\pi(1-\pi). \label{eq:ch02-binomial-moments} \end{equation}\]
Here \(m\in\mathbb{N}\) is the fixed number of trials and \(\pi\) is the success probability in each trial. The random variable \(X\) counts successes, so its support is finite: \(x=0,\ldots,m\). In biological data, \(m\) might be a known number of cells, reads, alleles, or trials, depending on the experiment.
The beta distribution is the two-category analogue of the Dirichlet distribution and is useful when the success probability itself varies across observations. We write
\[\begin{equation} \Pi \sim \operatorname{Beta}(a,b), \qquad \mathbb{E}(\Pi)=\frac{a}{a+b}, \qquad \operatorname{Var}(\Pi)=\frac{ab}{(a+b)^2(a+b+1)}. \label{eq:ch02-beta} \end{equation}\]
Its density is
\[\begin{equation} f(\pi\mid a,b) = \frac{\Gamma(a+b)}{\Gamma(a)\Gamma(b)} \pi^{a-1}(1-\pi)^{b-1}, \qquad 0<\pi<1, \qquad a>0,\ b>0. \label{eq:ch02-beta-density} \end{equation}\]
The parameters \(a\) and \(b\) are positive concentration parameters. Their ratio controls the mean success probability, while their sum controls how tightly \(\Pi\) concentrates around that mean. If
\[\begin{equation} \Pi_i \sim \operatorname{Beta}(a,b), \qquad X_i\mid \Pi_i \sim \operatorname{Binomial}(m,\Pi_i), \label{eq:ch02-beta-binomial-hierarchy} \end{equation}\]
then the marginal distribution of \(X_i\) is beta-binomial, with probability mass function
\[\begin{equation} \Pr(X_i=x\mid m,a,b) = \binom{m}{x} \frac{\Gamma(x+a)\Gamma(m-x+b)\Gamma(a+b)} {\Gamma(a)\Gamma(b)\Gamma(m+a+b)}, \qquad x=0,\ldots,m. \label{eq:ch02-beta-binomial-pmf} \end{equation}\]
This beta-binomial construction is the binary or bounded-count version of the same overdispersion idea used later for Gamma-Poisson and Dirichlet-multinomial models.
The categorical distribution generalizes Bernoulli data from two categories to \(K\) categories. If \(Y\) is a categorical random variable, then
\[\begin{equation} \Pr(Y=k)=\pi_k, \qquad k=1,\ldots,K, \qquad \sum_{k=1}^K \pi_k = 1. \label{eq:ch02-categorical} \end{equation}\]
The probability mass function can be written compactly as
\[\begin{equation} \Pr(Y=y\mid \pi_1,\ldots,\pi_K) = \prod_{k=1}^K \pi_k^{\mathbb{1}\{y=k\}}, \qquad y\in\{1,\ldots,K\}. \label{eq:ch02-categorical-pmf} \end{equation}\]
The parameter vector \(\pi=(\pi_1,\ldots,\pi_K)^\top\) contains category probabilities. Each \(\pi_k\) is nonnegative, and the probabilities sum to one. This distribution is useful for labels, classes, nucleotide identities, and other one-of-\(K\) outcomes. If we count the number of observations in each category across \(N\) independent trials, we arrive at the multinomial distribution.
2.6 Count Distributions: Poisson And Negative Binomial
The Poisson distribution is the simplest model for nonnegative unbounded counts. It is appropriate when events occur independently and at a constant rate over a fixed exposure. For virus mutation data, the observed quantity could be a mutation count along a sequence or over a time interval. In sequencing data, the observed quantity may be a read count assigned to a feature.
The Poisson model is
\[\begin{equation} X \sim \operatorname{Poisson}(\lambda), \qquad \Pr(X=x\mid \lambda)=\frac{e^{-\lambda}\lambda^x}{x!}, \qquad x \in \mathbb{N}_0. \label{eq:ch02-poisson} \end{equation}\]
The parameter \(\lambda>0\) is both the rate and the mean number of events in the relevant exposure unit. The exposure unit must be part of the scientific definition: counts per gene, per genomic window, per cell, per sample, or per unit time are different random variables.
Its defining moment relationship is
\[\begin{equation} \mathbb{E}(X)=\lambda, \qquad \operatorname{Var}(X)=\lambda. \label{eq:ch02-poisson-moments} \end{equation}\]
This equality between mean and variance is both elegant and restrictive. For biological count data, it is often too restrictive. Replicate counts with similar means may show much larger variability than the Poisson model allows. This phenomenon is called overdispersion.
Overdispersion means that the observed or modeled variance exceeds the variance implied by a simple reference model. For a Poisson reference model, overdispersion can be stated as
\[\begin{equation} \operatorname{Var}(X) > \mathbb{E}(X). \label{eq:ch02-overdispersion-poisson} \end{equation}\]
One generative explanation is that the Poisson rate itself varies across observations. Instead of assuming a fixed rate \(\lambda\), we introduce a latent random rate \(\Lambda\). To keep the notation consistent with feature-wise sequencing models, we use the negative-binomial mean-dispersion parameters \(\mu>0\) and \(\phi\geq 0\). For \(\phi>0\), the Gamma-Poisson hierarchy is
\[\begin{equation} X \mid \Lambda \sim \operatorname{Poisson}(\Lambda), \qquad \Lambda \sim \operatorname{Gamma}\left(\phi^{-1},(\phi\mu)^{-1}\right), \label{eq:ch02-gamma-poisson-hierarchy} \end{equation}\]
where the gamma distribution uses the shape-rate parameterization from Equation \(\ref{eq:ch02-gamma}\). Under this parameterization,
\[\begin{equation} \mathbb{E}(\Lambda)=\mu, \qquad \operatorname{Var}(\Lambda)=\phi\mu^2. \label{eq:ch02-gamma-poisson-latent-rate-moments} \end{equation}\]
This is the Gamma-Poisson model. It is generative in two stages: first draw an observation-specific rate, then draw a count conditional on that rate. Marginally, after integrating over \(\Lambda\), the count follows a negative binomial distribution in mean-dispersion form:
\[\begin{equation} X \sim \operatorname{NB}(\mu,\phi), \qquad \mathbb{E}(X)=\mu, \qquad \operatorname{Var}(X)=\mu+\phi\mu^2. \label{eq:ch02-negative-binomial-mean-dispersion} \end{equation}\]
For \(\phi>0\), the corresponding probability mass function is
\[\begin{equation} \Pr(X=x\mid \mu,\phi) = \frac{\Gamma(x+\phi^{-1})}{\Gamma(\phi^{-1})x!} \left(\frac{1}{1+\phi\mu}\right)^{\phi^{-1}} \left(\frac{\phi\mu}{1+\phi\mu}\right)^x, \qquad x\in\mathbb{N}_0. \label{eq:ch02-negative-binomial-pmf-mean-dispersion} \end{equation}\]
The parameter \(\mu>0\) is the mean count. The parameter \(\phi\geq 0\) is the dispersion parameter. If \(\phi=0\), the variance reduces to the Poisson variance \(\mu\) and the gamma mixing distribution has collapsed to a fixed rate; the probability mass function in Equation \(\ref{eq:ch02-negative-binomial-pmf-mean-dispersion}\) is then understood through its Poisson limit. As \(\phi\) increases, the latent rates vary more strongly across observations, and the counts become more variable than the Poisson model permits. Some books and software describe the same negative-binomial family with size-probability parameters. In this script, however, \(p\) is reserved for the number of features, so the canonical notation is always \(\operatorname{NB}(\mu,\phi)\). This mean-variance relationship is central in RNA-seq and other sequencing count models; see MSMB High-Throughput Count Data and reviews of count distributions for biological data (Townes 2020).
For feature-wise sequencing models, the expected count often includes an observation-specific size factor \(s_i\):
\[\begin{equation} C_{ij} \sim \operatorname{NB}(\mu_{ij},\phi_j), \qquad \mu_{ij}=s_i q_{ij}. \label{eq:ch02-feature-wise-nb-size-factor} \end{equation}\]
Here \(q_{ij}\) is an underlying abundance-like quantity and \(s_i\) accounts for sequencing depth or exposure. The dispersion \(\phi_j\) may be feature-specific. This equation already points toward Chapter 03, where normalization and variance stabilization become necessary.
2.7 Discrete Compositions: Multinomial And Dirichlet-Multinomial
Sequencing data often have another structure beyond being counts: a row of counts is constrained by a total. For observation \(i\), define the library size
\[\begin{equation} N_i = \sum_{j=1}^p C_{ij}. \label{eq:ch02-library-size} \end{equation}\]
Conditional on this total, a simple model for the full count vector is multinomial:
\[\begin{equation} C_i \mid N_i,\pi_i \sim \operatorname{Multinomial}(N_i,\pi_i), \qquad \pi_i \in \Delta^{p-1}. \label{eq:ch02-multinomial-row-model} \end{equation}\]
Here \(N_i\) is the fixed total count for observation \(i\), and \(\pi_i=(\pi_{i1},\ldots,\pi_{ip})^\top\) is the vector of feature probabilities for that observation. The entry \(\pi_{ij}\) is the probability that one draw from observation \(i\) is assigned to feature \(j\).
The simplex is
\[\begin{equation} \Delta^{p-1} = \left\{\pi \in \mathbb{R}_{\ge 0}^p: \sum_{j=1}^p \pi_j = 1 \right\}. \label{eq:ch02-simplex} \end{equation}\]
The multinomial probability mass function is
\[\begin{equation} \Pr(C_i=c_i \mid N_i,\pi_i) = \frac{N_i!}{\prod_{j=1}^p c_{ij}!} \prod_{j=1}^p \pi_{ij}^{c_{ij}}, \qquad \sum_{j=1}^p c_{ij}=N_i. \label{eq:ch02-multinomial-pmf} \end{equation}\]
Under this model, the marginal expectation and variance of component \(j\) are
\[\begin{equation} \mathbb{E}(C_{ij}\mid N_i,\pi_i)=N_i\pi_{ij}, \qquad \operatorname{Var}(C_{ij}\mid N_i,\pi_i)=N_i\pi_{ij}(1-\pi_{ij}). \label{eq:ch02-multinomial-moments} \end{equation}\]
The covariance between two different components is negative:
\[\begin{equation} \operatorname{Cov}(C_{ij},C_{i\ell}\mid N_i,\pi_i) = -N_i\pi_{ij}\pi_{i\ell}, \qquad j \ne \ell. \label{eq:ch02-multinomial-covariance} \end{equation}\]
This negative covariance is not necessarily biological competition. It is partly mathematical: when the total \(N_i\) is fixed, allocating more counts to one feature leaves fewer counts for the others.
The multinomial model is often too narrow for microbiome and other ecological count data because it assumes that the composition \(\pi_i\) is fixed. In practice, the composition itself varies across observations because of biological heterogeneity, sampling, environment, subject-to-subject variation, and technical effects. A generative extension is to make \(\pi_i\) random:
\[\begin{equation} \pi_i \sim \operatorname{Dirichlet}(\alpha), \qquad C_i \mid \pi_i,N_i \sim \operatorname{Multinomial}(N_i,\pi_i). \label{eq:ch02-dirichlet-multinomial-hierarchy} \end{equation}\]
The Dirichlet distribution is a distribution on the simplex. Its density is
\[\begin{equation} f(\pi_i\mid \alpha) = \frac{\Gamma(\alpha_0)}{\prod_{j=1}^p \Gamma(\alpha_j)} \prod_{j=1}^p \pi_{ij}^{\alpha_j-1}, \qquad \pi_i\in\Delta^{p-1}, \qquad \alpha_j>0. \label{eq:ch02-dirichlet-density} \end{equation}\]
The parameter vector \(\alpha=(\alpha_1,\ldots,\alpha_p)^\top\) contains positive concentration parameters, and
\[\begin{equation} \alpha_0=\sum_{j=1}^p \alpha_j \label{eq:ch02-dirichlet-total-concentration} \end{equation}\]
is the total concentration. The relative sizes of the \(\alpha_j\) determine the mean composition. The magnitude of \(\alpha_0\) controls how strongly compositions concentrate around that mean: large \(\alpha_0\) produces less row-to-row variation in \(\pi_i\), while small \(\alpha_0\) produces more variable and often more extreme compositions.
After integrating out \(\pi_i\), the count vector has a Dirichlet-multinomial distribution:
\[\begin{equation} C_i \sim \operatorname{DirichletMultinomial}(N_i,\alpha). \label{eq:ch02-dirichlet-multinomial} \end{equation}\]
Its probability mass function is
\[\begin{equation} \Pr(C_i=c_i\mid N_i,\alpha) = \frac{N_i!}{\prod_{j=1}^p c_{ij}!} \frac{\Gamma(\alpha_0)}{\Gamma(N_i+\alpha_0)} \prod_{j=1}^p \frac{\Gamma(c_{ij}+\alpha_j)}{\Gamma(\alpha_j)}, \qquad \sum_{j=1}^p c_{ij}=N_i. \label{eq:ch02-dirichlet-multinomial-pmf} \end{equation}\]
The mean composition is controlled by
\[\begin{equation} \pi_{0j}=\frac{\alpha_j}{\alpha_0}, \label{eq:ch02-dirichlet-mean-composition} \end{equation}\]
Thus \(\pi_{0j}\) is the expected relative abundance of feature \(j\) under the random composition \(\pi_i\).
The Dirichlet variance and covariance are
\[\begin{equation} \operatorname{Var}(\pi_{ij}) = \frac{\pi_{0j}(1-\pi_{0j})}{\alpha_0+1}, \qquad \operatorname{Cov}(\pi_{ij},\pi_{i\ell}) = -\frac{\pi_{0j}\pi_{0\ell}}{\alpha_0+1}, \qquad j\ne \ell. \label{eq:ch02-dirichlet-composition-moments} \end{equation}\]
This equation makes the role of \(\alpha_0\) visible. Holding the mean composition \(\pi_0\) fixed, increasing \(\alpha_0\) reduces variation in the latent composition \(\pi_i\).
The variance of component \(j\) is inflated relative to the multinomial variance by a factor depending on \(\alpha_0\):
\[\begin{equation} \operatorname{Var}(C_{ij}) = N_i\pi_{0j}(1-\pi_{0j}) \left(\frac{N_i+\alpha_0}{1+\alpha_0}\right). \label{eq:ch02-dirichlet-multinomial-variance} \end{equation}\]
This is the compositional analogue of the Gamma-Poisson idea. The multinomial assumes a fixed composition; the Dirichlet-multinomial assumes a random composition. Randomness in \(\pi_i\) creates extra variation in the observed counts. Dirichlet-multinomial models are therefore natural candidates for overdispersed ecological and microbiome count data (Harrison et al. 2020; Townes 2020).
2.8 Continuous Compositions: Log-Ratios And The Logistic-Normal Model
The Dirichlet and Dirichlet-multinomial models are natural first models for compositions, but they are not the only generative route. Aitchison’s logistic-normal model treats a composition as the inverse image of an unconstrained Gaussian random vector under a log-ratio transformation (Aitchison and Shen 1980). This idea is especially important because many normalization and distance choices in Chapter 03 and Chapter 04 are built around log-ratio geometry rather than ordinary Euclidean geometry on raw proportions.
Let \(\pi_i \in \Delta^{p-1}\) be the latent composition for observation \(i\). The centered log-ratio transform compares each component with the geometric mean of all components:
\[\begin{equation} \Delta_+^{p-1} = \left\{\pi \in \mathbb{R}_{>0}^p: \sum_{j=1}^p \pi_j = 1 \right\} \label{eq:ch02-positive-simplex} \end{equation}\]
is the positive interior of the simplex. Log-ratio transformations are defined on \(\Delta_+^{p-1}\), not on compositions with exact zeros.
\[\begin{equation} g(\pi_i) = \left(\prod_{j=1}^p \pi_{ij}\right)^{1/p}. \label{eq:ch02-geometric-mean-composition} \end{equation}\]
\[\begin{equation} \operatorname{clr}(\pi_i)_j = \log \frac{\pi_{ij}}{g(\pi_i)}, \qquad j=1,\ldots,p. \label{eq:ch02-clr-transform} \end{equation}\]
The vector \(\operatorname{clr}(\pi_i)\) has length \(p\), but its components sum to zero. It therefore lives in a \((p-1)\)-dimensional subspace of \(\mathbb{R}^p\). This is already a warning that compositions should not be treated as ordinary unconstrained vectors: only relative information is identifiable.
For a generative model it is convenient to use a log-ratio coordinate system with \(p-1\) unconstrained coordinates. One simple choice is the additive log-ratio transform with the last component as reference:
\[\begin{equation} \begin{aligned} U_i &= \operatorname{alr}(\pi_i) = \left( \log\frac{\pi_{i1}}{\pi_{ip}}, \ldots, \log\frac{\pi_{i,p-1}}{\pi_{ip}} \right)^\top, \\ U_i &\in \mathbb{R}^{p-1}. \end{aligned} \label{eq:ch02-alr-transform} \end{equation}\]
The inverse transform maps \(U_i=(u_{i1},\ldots,u_{i,p-1})^\top\) back to the simplex:
\[\begin{equation} \begin{aligned} \pi_{ij} &= \frac{\exp(u_{ij})}{1+\sum_{\ell=1}^{p-1}\exp(u_{i\ell})}, \qquad j=1,\ldots,p-1, \\ \pi_{ip} &= \frac{1}{1+\sum_{\ell=1}^{p-1}\exp(u_{i\ell})}. \end{aligned} \label{eq:ch02-alr-inverse} \end{equation}\]
A logistic-normal model is then defined by placing a multivariate normal distribution on these log-ratio coordinates:
\[\begin{equation} \begin{aligned} U_i &\sim \mathcal{N}_{p-1}(m,\Sigma), \\ \pi_i &= \operatorname{alr}^{-1}(U_i). \end{aligned} \label{eq:ch02-logistic-normal-composition} \end{equation}\]
Here \(m\in\mathbb{R}^{p-1}\) is the mean vector in additive log-ratio coordinates, and \(\Sigma\in\mathbb{R}^{(p-1)\times(p-1)}\) is a symmetric positive-definite covariance matrix. The density of \(U_i\) is
\[\begin{equation} \begin{aligned} q_U(u) &= (u-m)^\top\Sigma^{-1}(u-m), \\ f_U(u\mid m,\Sigma) &= \frac{1}{(2\pi)^{(p-1)/2}|\Sigma|^{1/2}} \exp\left\{-\frac{1}{2}q_U(u)\right\}, \\ &\hspace{2.5em} u\in\mathbb{R}^{p-1}. \end{aligned} \label{eq:ch02-logistic-normal-latent-density} \end{equation}\]
The induced logistic-normal density on the simplex, written in the same additive log-ratio coordinates, is
\[\begin{equation} \begin{aligned} q(\pi) &= \left(\operatorname{alr}(\pi)-m\right)^\top \Sigma^{-1} \left(\operatorname{alr}(\pi)-m\right), \\ f_\Pi(\pi\mid m,\Sigma) &= \frac{1}{(2\pi)^{(p-1)/2}|\Sigma|^{1/2}} \exp\left\{-\frac{1}{2}q(\pi)\right\} \\ &\quad{}\times{} \prod_{j=1}^p \pi_j^{-1}, \qquad \pi\in\Delta_+^{p-1}. \end{aligned} \label{eq:ch02-logistic-normal-density} \end{equation}\]
For sequencing counts, the composition can be coupled to a multinomial sampling layer:
\[\begin{equation} \begin{aligned} C_i \mid N_i,\pi_i &\sim \operatorname{Multinomial}(N_i,\pi_i), \\ \pi_i &= \operatorname{alr}^{-1}(U_i), \\ U_i &\sim \mathcal{N}_{p-1}(m,\Sigma). \end{aligned} \label{eq:ch02-logistic-normal-multinomial} \end{equation}\]
This model is generative in the same sense as the earlier hierarchical models. First draw an unconstrained Gaussian vector \(U_i\) in log-ratio space, then map it to a composition \(\pi_i\), then draw counts conditional on the library size \(N_i\). Compared with the Dirichlet, the logistic-normal can represent richer covariance structure among log-ratios. The price is that interpretation depends on the chosen log-ratio coordinates and zeros require special care before taking logarithms. Here we only need the basic model; Chapter 03 will return to log-ratio transformations, pseudocounts, and the distinction between count data and continuous compositional data. For a compositional-data overview, see the course notes by Egozcue and Tolosana-Delgado (Egozcue and Tolosana-Delgado 2007).
2.9 The Overdispersion Pattern
The two main overdispersion constructions in this chapter have the same mathematical logic. A simple sampling distribution is made more flexible by placing a distribution on one of its parameters.
| Simple model | Latent parameter model | Marginal model | What becomes random? |
|---|---|---|---|
| Poisson | Gamma-Poisson | negative binomial | count rate \(\Lambda\) |
| multinomial | Dirichlet-multinomial | Dirichlet-multinomial | composition \(\pi_i\) |
| Bernoulli/binomial | beta-Bernoulli/beta-binomial | beta-binomial | success probability \(\pi\) |
The generative principle can be written abstractly as
\[\begin{equation} \theta_i \sim \mathcal{G}, \qquad Y_i \mid \theta_i \sim P_{\theta_i}. \label{eq:ch02-hierarchical-generative-template} \end{equation}\]
The conditional distribution \(P_{\theta_i}\) describes sampling variation after the latent parameter is known. The mixing distribution \(\mathcal{G}\) describes variation in the latent parameter across observations. Marginally, the data are more variable than under a model with fixed \(\theta\).
This idea is one of the most important pieces of statistical intuition in the course. Overdispersion is not merely a nuisance parameter. It is often the signature that biological observations differ in ways the simplest sampling model does not represent.
2.10 Mixtures And Zero Inflation
Mixture models generalize the same idea. Instead of assuming that every observation comes from one homogeneous distribution, a mixture model assumes that each observation first chooses a latent component and then draws a value from that component. For a finite mixture with \(K\) components:
\[\begin{equation} L_i \sim \operatorname{Categorical}(\omega_1,\ldots,\omega_K), \qquad Y_i \mid L_i=k \sim F_k. \label{eq:ch02-finite-mixture-generative} \end{equation}\]
The marginal density or mass function is
\[\begin{equation} f(y)=\sum_{k=1}^K \omega_k f_k(y), \qquad \sum_{k=1}^K \omega_k=1. \label{eq:ch02-finite-mixture-density} \end{equation}\]
The parameters \(\omega_1,\ldots,\omega_K\) are mixture weights. Each \(\omega_k\geq 0\), and the weights sum to one. The component distribution \(F_k\) has density or mass function \(f_k\), and may have its own parameters, such as a component-specific mean or covariance matrix. Mixtures can describe heterogeneous populations, latent classes, heavy tails, and distributions with unusual shapes. A zero-inflated model is a particularly common count-data mixture:
\[\begin{equation} Y_i \sim \begin{cases} 0, & \text{with probability } \omega,\\ F, & \text{with probability } 1-\omega, \end{cases} \label{eq:ch02-zero-inflated-mixture} \end{equation}\]
where \(F\) may be a Poisson, negative binomial, or another count distribution. If \(F\) has probability mass function \(f\), the zero-inflated probability mass function is
\[\begin{equation} \Pr(Y_i=y) = \omega\,\mathbb{1}\{y=0\} + (1-\omega)f(y), \qquad 0\leq \omega\leq 1. \label{eq:ch02-zero-inflated-pmf} \end{equation}\]
The parameter \(\omega\) is the probability of the extra-zero component. Such models are useful when zeros arise from more than one mechanism, for example true absence and limited detection. In biological applications, however, zero inflation should be treated as a modeling hypothesis, not as an automatic conclusion.
2.11 Likelihood And Model Fitting
A generative model becomes a statistical model once its unknown parameters are estimated from data. Suppose \(Y_1,\ldots,Y_n\) are independent observations from a distribution with parameter \(\theta\). The likelihood is
\[\begin{equation} L(\theta; y_1,\ldots,y_n) = \prod_{i=1}^n f(y_i\mid \theta). \label{eq:ch02-likelihood} \end{equation}\]
The log-likelihood is
\[\begin{equation} \ell(\theta) = \sum_{i=1}^n \log f(y_i\mid \theta). \label{eq:ch02-log-likelihood} \end{equation}\]
Maximum likelihood estimation chooses the parameter value that makes the observed data most probable under the model:
\[\begin{equation} \hat{\theta}_{\mathrm{MLE}} = \operatorname*{arg\,max}_{\theta} \ell(\theta). \label{eq:ch02-mle} \end{equation}\]
Simple discrete-data examples show why this is a natural estimation principle (see MSMB Generative Models and MSMB Statistical Modeling). In Exercise Sheet 01, the same principle appears in maximum-likelihood calculations for nucleotide and count models and in the latent-component updates of mixture models (Exercise Sheet 01 2026). The key point for this course is that likelihood always depends on the chosen distribution. Estimating a Poisson rate, a negative binomial mean and dispersion, or a Dirichlet-multinomial concentration parameter are different statistical problems because they assume different generative mechanisms.
2.12 Model Checking
A distributional assumption should be checked against the data. The first check is support: does the model generate values of the right type? The second check is scale: are the model-implied means and variances plausible? The third check is shape: does the model capture skewness, tails, zeros, and heterogeneity?
For counts, a simple diagnostic is the relation between empirical means and variances across features. If the empirical variance is systematically larger than the empirical mean, a Poisson model is too restrictive. For compositions, the multinomial can be checked by asking whether replicate count vectors vary more than expected under a fixed composition. For continuous data, histograms, density estimates, Q-Q plots, and residual plots help assess symmetry, tail behavior, and transformations.
Model checking is not separate from generative thinking. A generative model makes predictions about what repeated data sets should look like. If simulated data from the fitted model do not resemble the observed data in the aspects that matter, then the model is not adequate for the analysis goal.
Summary
Chapter 02 provides the probability language used throughout the rest of the course. Continuous data suggest normal, gamma, Laplace, or mixture models depending on support and shape. Binary and categorical data suggest Bernoulli, binomial, categorical, and multinomial models. Sequencing counts often begin with Poisson reasoning but usually require negative binomial or related overdispersed models. Discrete compositions begin with the multinomial model but often require Dirichlet-multinomial or logistic-normal extensions. The logistic-normal model is the bridge from this chapter to log-ratio transformations in Chapter 03.
The most important conceptual move is generative. We do not only ask which formula fits a histogram. We ask what data-generating mechanism the formula represents. A Poisson model assumes a fixed rate. A negative binomial model can be understood as a Poisson model with a Gamma-distributed latent rate. A multinomial model assumes a fixed composition. A Dirichlet-multinomial model assumes that the composition itself varies across observations. These hierarchical constructions explain overdispersion and prepare the ground for normalization, variance stabilization, testing, and regression.