12  Notation and Dimensions

12.1 Core Convention

The working agreement for notation is stored in assets/references/notation-working-agreement.md. Treat that file as the source of truth when expanding chapters.

The central object in the lecture is the high-throughput sequencing data matrix:

\[\begin{equation} X \in \mathbb{R}^{n \times p}. \label{eq:notation-analysis-matrix} \end{equation}\]

where rows are observational units and columns are measured features.

  • \(n\): number of observations, such as samples, patients, cells, images, or experimental units
  • \(p\): number of features, such as genes, proteins, metabolites, genomic loci, peaks, or extracted image features
  • \(x_i^\top \in \mathbb{R}^{1 \times p}\): row vector for observation \(i\)
  • \(x_{\cdot j} \in \mathbb{R}^{n}\): column vector for feature \(j\)
  • \(x_{ij}\): value of feature \(j\) in observation \(i\)

Vectors are column vectors by default. A row vector is written with a transpose, for example \(x_i^\top\).

For sequencing data, the entries of \(X\) may represent raw counts, normalized abundances, transformed counts, or scaled values. The lecture must always state which version is being used. When the distinction matters, reserve \(C\) for raw counts and use \(X\) for the analysis matrix after the relevant preprocessing step.

12.2 Data Matrices

Symbol Dimension Meaning
\(X\) \(n \times p\) high-throughput sequencing data matrix used for analysis
\(C\) \(n \times p\) raw sequencing count matrix
\(Y\) \(n \times q\) response matrix or second data table
\(y\) \(n\) response vector
\(Z\) \(n \times r\) covariate, design, or metadata matrix
\(W\) \(p \times k\) feature loadings, weights, or decoder matrix, depending on context
\(H\) \(n \times k\) latent scores or low-dimensional representation
\(D\) \(n \times n\) distance or dissimilarity matrix between observations
\(A\) \(p \times p\) or \(n \times n\) adjacency matrix for a feature network or observation network

The meaning of \(A\) must always be stated because biological networks can connect genes, proteins, cells, samples, pathways, or phenotypes.

12.3 Indices

Index Range Meaning
\(i\) \(1,\ldots,n\) observation index
\(j\) \(1,\ldots,p\) feature index
\(g\) \(1,\ldots,G\) group, class, or cluster index
\(k\) \(1,\ldots,K\) latent dimension, component, or factor index
\(b\) \(1,\ldots,B\) batch, block, bootstrap draw, or resample index

12.4 Probability Operators

Use a single TeX style for probability operators throughout the script. Expectations are written as \(\mathbb{E}(Y)\) for simple arguments and as \(\mathbb{E}\left[g(Y)\right]\) for longer expressions. Probabilities are written with \(\Pr\), variances with \(\operatorname{Var}\), covariances with \(\operatorname{Cov}\), and correlations with \(\operatorname{Cor}\). Thus the preferred forms are

\[\begin{equation} \mathbb{E}(Y), \qquad \Pr(Y=y\mid \theta), \qquad \operatorname{Var}(Y), \qquad \operatorname{Cov}(Y_1,Y_2). \label{eq:notation-probability-operator-style} \end{equation}\]

Avoid switching to alternative spellings such as \(\operatorname{E}\), \(\mathrm{E}\), \(\mathbb{P}\), \(\mathrm{Var}\), or \(\mathrm{Cov}\) in chapter text.

12.5 Counts and Normalized Data

For count-based omics, use:

  • \(C \in \mathbb{N}_0^{n \times p}\): raw count matrix
  • \(c_{ij}\): raw count for feature \(j\) in observation \(i\)
  • \(s_i > 0\): size factor, library size factor, or exposure for observation \(i\)
  • \(X \in \mathbb{R}^{n \times p}\): analysis matrix after normalization, transformation, filtering, or scaling, depending on context
  • \(\tilde{X}\): explicitly normalized or transformed intermediate data matrix, when needed
  • \(\log_2(\tilde{x}_{ij} + a)\): log-transformed value with pseudocount \(a\), when needed

Do not use \(X\) for both raw counts and transformed values in the same section. State whether \(X\) is raw, normalized, scaled, or embedded.

12.6 Distributional View Of \(X\)

Many analyses model the entries, rows, or columns of \(X\) using distributional assumptions. The most common starting point in this course is that high-throughput sequencing data are count data or discrete compositions.

12.6.1 Entry-wise View

An entry-wise model treats each \(c_{ij}\) as a count:

\[\begin{equation} c_{ij} \sim \text{Poisson}(\lambda_{ij}). \label{eq:notation-poisson-entry-model} \end{equation}\]

or, when overdispersion matters,

\[\begin{equation} c_{ij} \sim \text{Negative Binomial}(\mu_{ij}, \phi_j). \label{eq:notation-negative-binomial-entry-model} \end{equation}\]

Here \(\lambda_{ij}\) or \(\mu_{ij}\) is the expected count, and \(\phi_j\) is a feature-specific dispersion parameter.

12.6.2 Row-wise Compositional View

Sequencing often measures relative abundance under a finite library size. For observation \(i\), the row vector of counts is:

\[\begin{equation} c_i^\top = (c_{i1}, \ldots, c_{ip}), \qquad N_i = \sum_{j=1}^p c_{ij}. \label{eq:notation-row-count-vector} \end{equation}\]

A simple row-wise compositional model is:

\[\begin{equation} c_i \sim \text{Multinomial}(N_i, \pi_i). \label{eq:notation-multinomial-row-model} \end{equation}\]

where \(\pi_i \in \Delta^{p-1}\) is a vector of feature probabilities that sums to one. The simplex notation means:

\[\begin{equation} \Delta^{p-1} = \left\{\pi \in \mathbb{R}_{\ge 0}^{p}: \sum_{j=1}^p \pi_j = 1\right\}. \label{eq:notation-simplex} \end{equation}\]

This emphasizes that sequencing data often carry relative information: increasing one component’s share can decrease another component’s share even if the underlying biology is unchanged.

12.6.3 Column-wise Feature View

For feature-wise testing or regression, we often isolate one column:

\[\begin{equation} y_j = c_{\cdot j} \in \mathbb{N}_0^n. \label{eq:notation-feature-response} \end{equation}\]

and model it across observations using covariates:

\[\begin{equation} y_j \sim \text{Negative Binomial}(\mu_j, \phi_j), \qquad \log \mu_j = \log s + Z\beta_j. \label{eq:notation-feature-wise-count-regression} \end{equation}\]

Here \(s \in \mathbb{R}_{>0}^n\) is a vector of size factors or exposures, \(Z \in \mathbb{R}^{n \times r}\) is the design matrix, and \(\beta_j \in \mathbb{R}^{r}\) contains feature-specific regression coefficients.

12.6.4 Practical Rule

When introducing a method, state which object is being modeled:

  • entries \(c_{ij}\)
  • rows \(c_i\)
  • columns \(c_{\cdot j}\)
  • transformed values \(x_{ij}\)
  • distances between rows or columns
  • latent representations derived from \(X\)

12.7 Metadata and Design Matrices

Use:

  • \(m_i\): metadata for observation \(i\), usually a row in a metadata table
  • \(Z \in \mathbb{R}^{n \times r}\): design or covariate matrix
  • \(\beta \in \mathbb{R}^{r}\): regression coefficients for one response
  • \(B \in \mathbb{R}^{r \times p}\): coefficient matrix when modeling many features

For feature-wise regression:

\[\begin{equation} y_j = Z\beta_j + \varepsilon_j. \label{eq:notation-feature-wise-regression} \end{equation}\]

where \(y_j = x_{\cdot j} \in \mathbb{R}^{n}\), \(Z \in \mathbb{R}^{n \times r}\), \(\beta_j \in \mathbb{R}^{r}\), and \(\varepsilon_j \in \mathbb{R}^{n}\).

12.8 Dimension Reduction

For PCA-like decompositions, use centered or scaled data \(X_c\):

\[\begin{equation} X_c \approx T_k P_k^\top. \label{eq:notation-pca-factorization} \end{equation}\]

with:

  • \(X_c \in \mathbb{R}^{n \times p}\)
  • \(T_k \in \mathbb{R}^{n \times k}\): scores or coordinates of observations
  • \(P_k \in \mathbb{R}^{p \times k}\): loadings or feature directions
  • \(k \ll \min(n,p)\)

For embeddings:

\[\begin{equation} U = f(X), \qquad U \in \mathbb{R}^{n \times d}. \label{eq:notation-embedding} \end{equation}\]

where \(d\) is usually 2 or 3 for visualization, or larger for learned representations.

12.9 Clustering

Use:

  • \(z_i \in \{1,\ldots,K\}\): cluster label for observation \(i\)
  • \(z \in \{1,\ldots,K\}^{n}\): vector of cluster assignments
  • \(\mu_k \in \mathbb{R}^{p}\): centroid for cluster \(k\)
  • \(D \in \mathbb{R}^{n \times n}\): distance matrix used by the clustering algorithm

If clustering features instead of observations, state that explicitly and use a feature distance matrix \(D_f \in \mathbb{R}^{p \times p}\).

12.10 Testing

Use:

  • \(H_{0j}\): null hypothesis for feature \(j\)
  • \(H_{1j}\): alternative hypothesis for feature \(j\)
  • \(p_j\): p-value for feature \(j\)
  • \(q_j\): adjusted p-value or false discovery rate estimate for feature \(j\)
  • \(\alpha\): chosen significance level
  • \(m\): number of tested hypotheses

Often \(m=p\), but not always. For pathway, interaction, or network tests, \(m\) can be different from the number of measured features.

12.11 Networks

Use:

  • \(G = (V,E)\): graph with nodes \(V\) and edges \(E\)
  • \(|V| = v\): number of nodes
  • \(|E|\): number of edges
  • \(A \in \mathbb{R}^{v \times v}\): adjacency matrix
  • \(L\): graph Laplacian, if used

Always state what nodes represent. A gene network, sample-similarity network, cell-cell graph, and pathway graph are different objects even if all are written as \(G\).

12.12 Deep Learning and Latent Models

Use:

  • \(h_i \in \mathbb{R}^{k}\): latent representation for observation \(i\)
  • \(H \in \mathbb{R}^{n \times k}\): latent representation matrix
  • \(f_\theta\): encoder, predictor, or learned function with parameters \(\theta\)
  • \(g_\phi\): decoder or generative function with parameters \(\phi\)
  • \(\hat{X}\): reconstruction of \(X\)

For an autoencoder:

\[\begin{equation} H = f_\theta(X), \qquad \hat{X} = g_\phi(H). \label{eq:notation-autoencoder} \end{equation}\]

In implementations, rows are typically processed in mini-batches even when the function is written mathematically as acting on the full matrix.

12.13 Dimension Checklist

Before finalizing any formula, check:

  1. What are the rows?
  2. What are the columns?
  3. Is the object raw, normalized, transformed, scaled, or embedded?
  4. Are vectors columns or rows?
  5. Does every matrix multiplication have matching inner dimensions?
  6. Does the result have the dimension claimed in the text?
  7. Are we clustering observations, features, or another object?
  8. Does a network connect features, observations, or external biological entities?