Analysis of High-Dimensional Biological Data

Lecture Script, Version 0.6

Author

Christian L. Müller

Published

July 23, 2026

Analysis of High-Dimensional Biological Data

Lecture Overview

High-dimensional biological data arise when modern measurement technologies record many molecular, cellular, or spatial variables for each biological observation. RNA sequencing, single-cell sequencing, proteomics, metabolomics, methylation assays, microbiome profiling, and spatial omics can produce thousands to millions of measurements per experiment. The promise is systems-level resolution; the difficulty is that the number of measured features is often much larger than the number of independent observations.

These lecture notes are the mathematical companion to the highly visual lecture slides and the course book. The slides provide intuition, examples, and visual anchors. The notes keep notation, dimensions, modeling assumptions, optimization problems, and interpretation explicit. Modern Statistics for Modern Biology serves as the main external online reference for fuller treatment, examples, R code, and exercises.

NoteCourse materials

The notes should be read together with the visual lecture slides and the course book. The main external online reference is Modern Statistics for Modern Biology by Susan Holmes and Wolfgang Huber, especially the chapters on data visualization, multivariate analysis, high-throughput count data, clustering, testing, and supervised learning.

TipNotation convention

Notation follows the shared convention in the notation chapter. Vectors are columns by default, row vectors are written with transposes, and dimensions should be checked whenever matrices are multiplied, transformed, modeled, or decomposed.

How To Use These Notes

The notes are not meant to replace the slides or the book. They provide the mathematical spine of the course: notation, assumptions, objectives, models, and interpretation. Use the linked book sections for deeper derivations and code, use the exercises for computation, and use the slides for the visual intuition that is intentionally kept lighter here.

The recurring workflow is simple but demanding: formulate a biological question, identify the observation and feature units, decide whether the measurements are counts, continuous values, binary variables, or compositions, choose a preprocessing and modeling scale, and validate whether the resulting pattern is stable and biologically plausible.

Course Roadmap

Chapter 01 introduces the biological context, matrix notation, and the object-oriented statistical workflow. Chapter 02 reviews probability distributions as generative models for continuous, binary, count, and compositional data. Chapter 03 connects normalization, variance stabilization, residuals, and log-ratio transformations to the models that motivate them. Chapters 04 and 05 turn transformed data into geometry: distances, embeddings, clustering, and cluster validation.

The second half of the course moves from exploratory structure to formal inference and modern high-dimensional modeling. Chapter 06 covers hypothesis testing and false discovery control. Chapter 07 develops regression, classification, regularization, sparse logistic regression, and log-contrast regression. Chapters 08 and 09 separate network learning from network analysis. Chapters 10 and 11 connect matrix factorization, autoencoders, deep learning, and biological foundation models.

Recurring Principles

Dimensions first. Every analysis begins by defining the rows, columns, and entry type of the data object. The most common analysis matrix is \(X\in\mathbb{R}^{n\times p}\), but raw sequencing data are usually better represented as a count matrix \(C\in\mathbb{N}_0^{n\times p}\).

Models and transformations belong together. Log transforms, variance-stabilizing transforms, Pearson residuals, and log-ratio coordinates are not arbitrary preprocessing tricks. They are tied to assumptions about how the data were generated and what variation should be treated as noise.

Exploration is not confirmation. PCA plots, UMAP embeddings, heatmaps, clusters, and networks can reveal structure, but they do not by themselves prove biological mechanisms. Testing, prediction, model selection, and validation each require their own assumptions.

High dimensionality changes evidence. When \(p\) is large relative to \(n\), random correlations, overfitting, batch effects, and multiple testing become central rather than peripheral. A good analysis therefore reports not only a result, but also the data scale, preprocessing, model, tuning choices, and validation strategy.