Analysis of high-dimensional biological data

Exercise sheet 4
Author
Published

June 15, 2024

Install packages required for this exercise sheet

Code
packages <- c("pheatmap", "RColorBrewer", "MicrobeDS", "phyloseq", "vegan", 
              "ggplot2", "factoextra", "cluster", "SpiecEasi", "dada2")

# Find uninstalled packages
not_installed <- packages[!(packages %in% installed.packages()[,"Package"])]
# Install the missing packages
if (length(not_installed) > 0) {
  # We use the BiocManager here, which can also install CRAN packages
  BiocManager::install(not_installed)
}
  • R version: 4.5.1 (2025-06-13)
  • BioC version: 3.22

Exercise 1 (Hierarchical clustering)

With the availability of whole-genome sequences for an increasing number of species, the challenge now is to decode the information contained within these DNA sequences. Previous comparative DNA sequence studies have revealed that genes similar to each other at the nucleotide level can be related to one another by different evolutionary histories.

Comparative analysis of DNA sequences from multiple species can tell whether two species have a recent common ancestor. In other words, for two species with a small distance between them (measured by differences in their DNA sequences), have a recent common ancestor and for two species with a large distance between them, their common ancestor is in the remote past. Genes derived from a common ancestral gene are homologs, and the level of similarity in their sequences often reflects the time since they diverged.

Given a distance matrix between species, your task is to use hierarchical clustering to represent the species. You can use R for this task.

Code
dist.mat <- data.frame("DM" = c(0,662,877, 255,412,996),
                       "DP" = c(662, 0, 295, 468,268,400),
                       "CE" = c(877,295,0,754,564,138),
                       "CB" = c(255,468,754,0,219,896),
                       "EC" = c(412,268,564,219,0,669),
                       "Sm" = c(996,400,138,869,669,0))

row.names(dist.mat) <- colnames(dist.mat)

dist.mat
    DM  DP  CE  CB  EC  Sm
DM   0 662 877 255 412 996
DP 662   0 295 468 268 400
CE 877 295   0 754 564 138
CB 255 468 754   0 219 869
EC 412 268 564 219   0 669
Sm 996 400 138 896 669   0

a) Hierarchical clustering and dendrograms

Perform hierarchical clustering with three different linkage methods: single, average, and complete linkage. Plot the three corresponding dendrograms. How do the plots differ?

Source: Lecture 7

Source: Lecture 7

We can use the hclust() function (stats package) for which we need to create a distance object with as.dist().

Code
sing.linkage <- hclust(as.dist(dist.mat), method = "single")
comp.linkage <- hclust(as.dist(dist.mat), method = "complete")
ave.linkage <- hclust(as.dist(dist.mat), method = "average")
Code
plot(sing.linkage, main = "Single linkage", xlab = "", sub = "")

Code
plot(comp.linkage, main = "Complete linkage", xlab = "", sub = "")

Code
plot(ave.linkage, main = "Average linkage", xlab = "", sub = "")

A striking difference is that with complete and average linkage, DP is assigned to the “CE-Sm” cluster on the left, while with single linkage, DP is assigned to the cluster on the right.

Let’s take a look at the state of the distance matrix after the CE-Sm and CB-EC clusters have been formed. In the lower triangle are the distances for single linkage and in the upper triangle those for complete linkage:

DM DP CE-Sm CB-EC
DM 0 662 996 412
DP 662 0 400 468
CE-Sm 877 295 0 896
CB-EC 255 268 564 0

The minimum value in the lower triangle (single linkage) is 255, which is why DM is assigned to the CB-EC cluster. In the upper triangle, the minimum distance is 400 so that DP gets assigned to the CB-EC cluster.

Characteristics of the three methods:

Single linkage:

  • Tends to produce elongated, chain-like clusters.
  • Sensitive to noise and outliers, as single outlier points can significantly affect the cluster formation.

Complete linkage:

  • Tends to produce more compact and spherical clusters.
  • Less sensitive to noise and outliers compared to single linkage.
  • Can lead to tighter and more evenly sized clusters.

Average linkage:

  • Produces clusters that are generally more balanced and less sensitive to outliers compared to single linkage.
  • Tends to avoid the chaining effect seen in single linkage.
  • Produces clusters that are more compact than those from single linkage but less so than those from complete linkage.
  • Can handle clusters of different shapes and sizes better than complete linkage.

b) Heatmap

Plot the corresponding heatmaps using the pheatmap function from pheatmap R package. What would you conclude from the plots?

Code
library(pheatmap)
library(RColorBrewer)
Code
pheatmap(dist.mat, cluster_rows = sing.linkage, cluster_cols = sing.linkage,
color = brewer.pal(9, "YlOrRd"), main = "Single linkage")

Code
pheatmap(dist.mat, cluster_rows = comp.linkage, cluster_cols = comp.linkage,
color = brewer.pal(9, "YlOrRd"), main = "Complete linkage")

Code
pheatmap(dist.mat, cluster_rows = ave.linkage, cluster_cols = ave.linkage,
color = brewer.pal(9, "YlOrRd"), main = "Average linkage")

The heatmaps indicate that DP, CE, and Sm may have a recent common ancestor, as may DM, CB, and EC.

Code
# Clear environment
rm(list = ls())

Exercise 2 (Hierarchical clustering, HMP data)

In this exercise, we will use data from the Human Microbiome Project (HMP) to find out if samples from the same body sites have a similar microbial composition.

The R package MicrobeDS contains a variety of data sets provided by large-scale microbiome studies. We use the “HMPv35” data set, which contains 16S rRNA sequencing data (V3-V5 region) from multiple body sites across hundreds of subjects.

To reduce the number of samples, we use only samples from the study centre “BCM”.

Read in the data and perform the following pre-processing steps:

  • Subset the samples to keep only samples with “sequencecenter = BCM”.
  • Agglomerate the data to family level using the tax_glom function from the phyloseq package.
  • Filter the samples so that only samples with a total read count > 100 are kept.
Code
# install.packages("remotes")
# remotes::install_github("twbattaglia/MicrobeDS")

library(MicrobeDS)
library(phyloseq)
Code
# Load the HMP data
data("HMPv35")

# Extract sample data
sample_data_df <- sample_data(HMPv35)
table(sample_data_df$sequencecenter)

  BCM    BI  JCVI WUGSC 
  792  1732  1146  2330 
Code
# Subset the phyloseq object for sequencecenter "BCM"
hmp_bcm <- subset_samples(HMPv35, sequencecenter == "BCM")
hmp_bcm
phyloseq-class experiment-level object
otu_table()   OTU Table:         [ 10730 taxa and 792 samples ]
sample_data() Sample Data:       [ 792 samples by 33 sample variables ]
tax_table()   Taxonomy Table:    [ 10730 taxa by 7 taxonomic ranks ]
phy_tree()    Phylogenetic Tree: [ 10730 tips and 10729 internal nodes ]
Code
# Agglomerate to family rank
hmp_bcm_family <- tax_glom(hmp_bcm, taxrank = "Family", NArm = TRUE)
Code
hmp_bcm_family
phyloseq-class experiment-level object
otu_table()   OTU Table:         [ 242 taxa and 792 samples ]
sample_data() Sample Data:       [ 792 samples by 33 sample variables ]
tax_table()   Taxonomy Table:    [ 242 taxa by 7 taxonomic ranks ]
phy_tree()    Phylogenetic Tree: [ 242 tips and 241 internal nodes ]
Code
# Filter out low sequenced samples (with overall count < 100)
hmp_bcm_family <- prune_samples(sample_sums(hmp_bcm_family) >= 100, hmp_bcm_family)
hmp_bcm_family
phyloseq-class experiment-level object
otu_table()   OTU Table:         [ 242 taxa and 777 samples ]
sample_data() Sample Data:       [ 777 samples by 33 sample variables ]
tax_table()   Taxonomy Table:    [ 242 taxa by 7 taxonomic ranks ]
phy_tree()    Phylogenetic Tree: [ 242 tips and 241 internal nodes ]

a) Hierarchical clustering and heatmaps

Compute the distance matrix using the Robust Aitchison distance (implemented in vegdist() from vegan package). For each of the three linkage methods single, average, and complete linkage, plot a heatmap using the pheatmap() function. Do you notice any structures in the plots?

Code
library(vegan)
library(pheatmap)
Code
# Extract OTU counts
otu_counts <- as.data.frame(t(otu_table(hmp_bcm_family)))

# Calculate distance matrix using robust Aitchison distance
dist_mat <- as.matrix(vegdist(otu_counts, method = "robust.aitchison"))
Code
# Plot heatmap
pheatmap(dist_mat, 
         clustering_method = "single",
         show_rownames = FALSE,
         show_colnames = FALSE)

Code
pheatmap(dist_mat, 
         clustering_method = "complete",
         show_rownames = FALSE,
         show_colnames = FALSE)

Code
pheatmap(dist_mat, 
         clustering_method = "average",
         show_rownames = FALSE,
         show_colnames = FALSE)

There is one sample with a high distance to all other samples. Let’s remove this oulier.

Code
torm <- names(which.max(rowSums(dist_mat)))
hmp_bcm_family <- prune_samples(sample_names(hmp_bcm_family) != torm, hmp_bcm_family)
Code
# Recompute distance matrix
otu_counts <- as.data.frame(t(otu_table(hmp_bcm_family)))
dist_mat <- as.matrix(vegdist(otu_counts, method = "robust.aitchison"))

b) Annotation

Find out if one of the following sample characteristics match the clusters: environment (“env”), body site (“body_site”). Here you can stick to average linkage.

Code
sample_info <- sample_data(hmp_bcm_family)
sampcol <- data.frame(environment = sample_info$env,
                      bodysite = sample_info$body_site)
rownames(sampcol) <- rownames(otu_counts)
head(sampcol)
                                   environment             bodysite
1928.SRS049355.SRX020682.SRR048316        Skin UBERON:skin of elbow
1928.SRS023023.SRX020679.SRR048066        Skin  UBERON:skin of nose
1928.SRS047619.SRX020681.SRR048244        Skin  UBERON:skin of nose
1928.SRS023019.SRX020682.SRR048301        Skin UBERON:skin of elbow
1928.SRS023021.SRX020682.SRR048302        Skin UBERON:skin of elbow
1928.SRS024545.SRX020682.SRR048323        Skin UBERON:skin of elbow
Code
# Combine the heatmap and the annotation
pheatmap(dist_mat, name = "counts_norm",
         annotation_col = sampcol,
         clustering_method = "average",
         show_rownames = FALSE,
         show_colnames = FALSE
)

The clusters found seem to fit well with the different environments. The body sites are less well represented by the clusters.

c) Optimal number of clusters

Find out the optimal number of clusters using the within-group sum-of-squares (WSS) or “elbow” method and the gap statistic.

Hint: The fviz_nbclust function from factoextra package and the clusGap function from cluster package may be useful. Since we might run into problems with the Robust Aitchison distance from vegan package, we compute the Aitchison distance “by hand” and write our own function for hierarchical clustering:

Code
# Extract OTU counts
otu_counts <- as.data.frame(t(otu_table(hmp_bcm_family)))

# Handle zero values (add a small constant)
otu_counts[otu_counts == 0] <- 1e-8

# Perform CLR transformation (using the function from SpiecEasi package)
clr_counts <- SpiecEasi::clr(otu_counts)

# Define a custom hierarchical clustering function
hierarchical_clustering <- function(x, k, method = "average") {
  # Compute the distance matrix
  dist_mat <- dist(x, method = "euclidean")
  # Perform hierarchical clustering
  hc <- hclust(dist_mat, method = method)
  # Cut the tree to get 'k' clusters
  cluster <- cutree(hc, k = k)
  # Return a list with the cluster assignments
  list(cluster = cluster)
}
Code
library(ggplot2)
library(factoextra)
library(cluster)

i) WSS

We use the “wss” (within-group sum of square) method and look for an elbow in the plot.

Code
factoextra::fviz_nbclust(clr_counts, hierarchical_clustering, method = "wss",
                         k.max = 10) +
  labs(subtitle = "Elbow method")

Code
# Optimal number of clusters (determined from the elbow plot)
nclust <- 3

# Plot heatmap with clustered tree
pheatmap(dist_mat, 
         clustering_method = "average",
         cutree_rows = nclust,
         cutree_cols = nclust,
         annotation_col = sampcol,
         show_rownames = FALSE,
         show_colnames = FALSE,
         main = paste("Heatmap with", nclust, "Clusters"))

ii) The gap statistic

Code
# Compute the GAP statistic
set.seed(123)  # For reproducibility
gap_stat <- clusGap(clr_counts, FUN = hierarchical_clustering, K.max = 10, B = 50)
saveRDS(gap_stat, file = "./data/HMPv35_GAPstat.rds")
Code
gap_stat <- readRDS("./data/HMPv35_GAPstat.rds")

# Visualize the GAP statistic
fviz_gap_stat(gap_stat, maxSE = list(method = "Tibs2001SEmax", SE.factor = 1))

The Gap statistic indicates that 1 is the optimal number of clusters. This can happen if the data does not exhibit clear clustering structure or if the clusters are not well-separated. In such cases, the within-cluster dispersion does not significantly decrease as more clusters are added, leading the GAP statistic to suggest that the best representation of the data is with a single cluster.

Bonus (Hierarchical clustering, American Gut Project)

Here are the solutions for the old version of this exercise with the American Gut data.

In this exercise, we will continue working with the 16S rRNA sequencing data from the American Gut Project used in Exercise sheet 03.

Read in the data (provided by the SpiecEasi R package) and perform sample filtering so that only samples with a total read count > 100 are kept (as in Exercise sheet 03). We want to cluster the samples using hierarchical clustering.

Code
library(SpiecEasi)
library(phyloseq)
Code
data(amgut2.filt.phy)

# Filter out low sequenced samples (with overall count < 100)
amgut_filt <- prune_samples(sample_sums(amgut2.filt.phy) >= 100, amgut2.filt.phy)

# Count matrix and sample information
otu_counts <- as.data.frame(t(otu_table(amgut_filt)))
sample_info <- as.data.frame(sample_data(amgut_filt))

a) Hierarchical clustering and heatmaps

Compute the distance matrix using the Robust Aitchison distance (implemented in vegdist() from vegan package). For each of the three linkage methods single, average, and complete linkage, plot a heatmap using the pheatmap() function. Do you notice any structures in the plots?

Code
library(vegan)
Code
dist.mat<- as.matrix(vegdist(otu_counts, method="robust.aitchison"))
dim(dist.mat)
[1] 261 261
Code
dist.mat[1:5, 1:5]
                  000001879.1076223 000002437.1076448 000003369.1076434
000001879.1076223          0.000000          9.727762         11.945309
000002437.1076448          9.727762          0.000000          5.937235
000003369.1076434         11.945309          5.937235          0.000000
000005954.1076427         10.980695         13.485434         18.800477
000004652.1076357          9.152187          3.928068          9.042431
                  000005954.1076427 000004652.1076357
000001879.1076223          10.98070          9.152187
000002437.1076448          13.48543          3.928068
000003369.1076434          18.80048          9.042431
000005954.1076427           0.00000         11.558671
000004652.1076357          11.55867          0.000000
Code
pheatmap(dist.mat, 
         clustering_method = "single",
         show_rownames = FALSE,
         show_colnames = FALSE)

Code
pheatmap(dist.mat, 
         clustering_method = "complete",
         show_rownames = FALSE,
         show_colnames = FALSE)

Code
pheatmap(dist.mat, 
         clustering_method = "average",
         show_rownames = FALSE,
         show_colnames = FALSE)

b) Annotation

Find out if one of the following subject characteristics match the clusters: antibiotic use (“ANTIBIOTIC_SELECT”), alcohol consumption (“ALCOHOL_FREQUENCY”), diet type (“DIET_TYPE”). Here you can stick to complete linkage.

Code
sampcol <- data.frame(antibio = sample_info$ANTIBIOTIC_SELECT,
                       alcohol = sample_info$ALCOHOL_FREQUENCY,
                       diet = sample_info$DIET_TYPE)
rownames(sampcol) <- rownames(otu_counts)
head(sampcol)
                               antibio                       alcohol
000001879.1076223     In the past year      Rarely (few times/month)
000002437.1076448 In the past 6 months      Rarely (few times/month)
000003369.1076434 Not in the last year                         Daily
000005954.1076427 In the past 6 months                         Daily
000004652.1076357 Not in the last year Occasionally (1-2 times/week)
000001827.1076437 Not in the last year    Regularly (3-5 times/week)
                                      diet
000001879.1076223                 Omnivore
000002437.1076448                 Omnivore
000003369.1076434                 Omnivore
000005954.1076427 Omnivore but no red meat
000004652.1076357                 Omnivore
000001827.1076437                 Omnivore
Code
# Combine the heatmap and the annotation
pheatmap(dist.mat, name = "counts_norm",
         annotation_col = sampcol,
         clustering_method = "complete",
         show_rownames = FALSE,
         show_colnames = FALSE,
         #cellwidth = 1.2, cellheight = 1.2,
         #width = 10, height = 50
)

There is no clear relationship between the considered sample information and the detected clusters.

Code
# Clear environment
rm(list = ls())

Exercise 3 (Divisive Amplicon Denoising Algorithm)

The DADA method (Divisive Amplicon Denoising Algorithm, Rosen et al. (2012)) uses a parameterized model of substitution errors that distinguishes sequencing errors from real biological variation.

The data for this exercise are the highly-overlapping Illumina Miseq 2 × 250 amplicon sequences from the V4 region of the 16S gene. These 360 fecal samples were collected from 12 mice longitudinally over the first year of life, to investigate the development and stabilization of the murine microbiome.

https://www.illumina.com/content/dam/illumina-marketing/documents/products/illumina_sequencing_introduction.pdf

Source: https://microbeonline.com/next-generation-sequencing/

Source: Wensel et el. (2022). “Next-generation sequencing: insights to advance clinical investigations of the microbiome”

https://joshuapenalba.com/illumina-sequencing/

Source: https://joshuapenalba.com/illumina-sequencing/

YouTube: “Next Generation Sequencing - A Step-By-Step Guide to DNA Sequencing”

Explained in more detail: YouTube: “Next Generation Sequencing 1: Overview - Eric Chow (UCSF)”

a) DADA method

Get yourself familiarized with the DADA method. The paper is available on the Moodle page.

b) Load the data

Follow Exercise 5.5. in MSMB book. Set the directories and load the downloaded data.

Code
#
BiocManager::install("dada2")
Warning: package(s) not installed when version(s) same as or greater than current; use
  `force = TRUE` to re-install: 'dada2'
Code
library(dada2)
Warning: package 'dada2' was built under R version 4.5.2
Warning: package 'Rcpp' was built under R version 4.5.2
Code
library(ggplot2)
Code
# Set directory
base_dir = "./data"
miseq_path = file.path(base_dir, "MiSeq_SOP")
filt_path = file.path(miseq_path, "filtered")

# Filenames of fastq files
fnFs = sort(list.files(miseq_path, pattern="_R1_001.fastq"))
fnRs = sort(list.files(miseq_path, pattern="_R2_001.fastq"))

sampleNames = sapply(strsplit(fnFs, "_"), `[`, 1)

if (!file_test("-d", filt_path)) dir.create(filt_path)

filtFs = file.path(filt_path, paste0(sampleNames, "_F_filt.fastq.gz"))
filtRs = file.path(filt_path, paste0(sampleNames, "_R_filt.fastq.gz"))

fnFs = file.path(miseq_path, fnFs)
fnRs = file.path(miseq_path, fnRs)

print(length(fnFs))
[1] 20

20 samples have been selected to illustrate how to preprocess the data.

Source: “Unraveling genomic variation from next generation sequencing data”, Pavlopoulos et al., BioData Mining, 2013

Quality scores:

Source: https://drive5.com/usearch/manual/quality_score.html

c) Quality profile

Generate similar plots as in Exercise 5.5 in MSMB for three randomly selected sets of forward and reverse reads.

Compare forward and reverse read qualities. What do you notice?

How would you trim and filter data?

Code
set.seed(42)
forwardReads.sampled <- sample(1:length(fnFs), 3)

# The function expects the file path as input
plotQualityProfile(fnFs[forwardReads.sampled]) + ggtitle("Forward")

Code
set.seed(42)
reverseReads.sampled <- sample(1:length(fnRs), 3)
plotQualityProfile(fnRs[reverseReads.sampled]) + ggtitle("Reverse")

Lines in the quality plots: green is the mean, orange is the median, and the dashed orange lines are the 25th and 75th quantiles.

Note that we also see the background distribution of quality scores at each position as a grey-scale heat map. The dark colors correspond to higher frequency.

The forward reads maintain high quality throughout, while the quality of the reverse reads drops significantly at about position 160. Therefore, we truncate the forward reads at position 240, and trimm the first 10 nucleotides as these positions are of lower quality. The reverse reads are trimmed at position 160.

Most Illumina sequencing data shows a trend of decreasing average quality towards the end of sequencing reads. Why is this? As we sequence more and more, there is more signal interference from previously sequenced nucleotides.

d) Trim and filter

Trim and filter the data according to the conclusions drawn from the previous point. Combine trimming parameters with standard filtering parameters, enforce a maximum of 2 expected errors per-read.

Code
out <- filterAndTrim(fnFs, filtFs, fnRs, filtRs, 
                     truncLen = c(240, 160), # Truncate reads (shorter reads are discarded)
                     maxN = 0,
                     maxEE = 2, 
                     truncQ = 2,
                     rm.phix = TRUE, 
                     trimLeft = 10,
                     compress = TRUE, # output fastq file(s) are gzipped
                     multithread = TRUE)
  • maxN = 0 omits all reads with ambiguous nucleotides.
  • maxEE = 2 excludes reads with more than 2 expected errors.
  • truncQ = 2 means sequences will be truncated at the first base where the quality score drops below or equals 2.
  • rm.phix is a logical flag that indicates whether to remove reads that match the phiX174 genome, which is often used as a control in sequencing runs.
  • trimLeft = 10 means that the first 10 nucleotides are removed from the start of each read.
Code
head(out)
                              reads.in reads.out
F3D0_S188_L001_R1_001.fastq       7793      7139
F3D1_S189_L001_R1_001.fastq       5869      5314
F3D141_S207_L001_R1_001.fastq     5958      5478
F3D142_S208_L001_R1_001.fastq     3183      2926
F3D143_S209_L001_R1_001.fastq     3178      2955
F3D144_S210_L001_R1_001.fastq     4827      4323

e) Infer RSVs

The DADA method uses a parameterized model of substitution errors that distinguishes sequencing errors from real biological variation.

The model assumes that the probabilities of base substitutions (such as seeing an A instead of a C) are independent of the position along the sequence. Because error rates vary substantially between sequencing runs and PCR protocols, the model parameters are estimated from the data themselves using an EM-type approach. A read is classified as noisy or exact given the current parameters, and the noise model parameters are updated accordingly. We want to use the DADA method to infer ribosomal sequence variants (RSVs) exactly, without imposing any arbitrary threshold, and thereby resolving variants that differ by as little as one nucleotide.

Follow item 5.8.3 in MSMB book to get the necessary data.

f) Run the DADA method

The DADA method relies on parameters that can be discovered from the data itself using a form of unsupervised learning in which sample inference is alternated with parameter estimation until both are jointly consistent.

Run the DADA method and plot the errors. What do you notice? When should we use ”pooled inference” and when not?

Code
# The dereplicated sequences are read in and then divisive denoising and 
# estimation is run with the dada function
derepFs = readRDS(file="./data/derepFs.rds")
derepRs = readRDS(file="./data/derepRs.rds")

ddF = dada(derepFs, err = NULL, selfConsist = TRUE, pool = TRUE)
ddR = dada(derepRs, err = NULL, selfConsist = TRUE, pool = TRUE)

The sequence inference function can run in two different modes:

Independent inference by sample (pool = FALSE), and pooled inference from the sequencing reads combined from all samples.

Independent inference has two advantages: as a functions of the number of samples, computation time is linear and memory requirements are constant.

Pooled inference is more computationally taxing, however it can improve the detection of rare variants that occur just once or twice in an individual sample but more often across all samples. As this dataset is not particularly large, we performed pooled inference.

Axes in the following error plots:

  • X-axis (Quality Score): Represents the quality score assigned to each base by the sequencer. Quality scores are typically on a scale (e.g., Phred scores) where higher scores indicate higher confidence in the base call.
  • Y-axis (Error Rate): Represents the error rate, which is the probability of an incorrect base call.

Interpretation (example):

  • A2A: Probability that a true base A was correctly read as base A.
  • A2C: Probability that a true base A was incorrectly read as base C (\(\rightarrow\) error).
Code
plotErrors(ddF, nominalQ = TRUE)

Code
plotErrors(ddR, nominalQ = TRUE)

Transition error rates as provided by plotErrors. This shows the frequencies of each type of nucleotide transition (A to C, A to G, . . . ) as a function of quality. Points are the observed error rates for each consensus quality score. The black line shows the estimated error rates after convergence of the algorithm.

g) Merging

Merge the inferred forward and reverse sequences, while removing paired sequences that do not perfectly overlap as a final control against residual errors and produce a contingency table of counts of ASVs.

Once the errors have been estimated, the algorithm is rerun on the data to find the sequence variants:

As this dataset is not particularly large, we perform pooled inference. As of version 1.2, multithreading can be activated with the arguments multithread = TRUE, which can substantially speed this step

Code
dadaFs <- dada(derepFs, err=ddF[[1]]$err_out, pool=TRUE)

## 20 samples were pooled: 140125 reads in 21932 unique sequences.
dadaRs <- dada(derepRs, err=ddR[[1]]$err_out, pool=TRUE)

## 20 samples were pooled: 140125 reads in 19665 unique sequences.
mergers <- mergePairs(dadaFs, derepFs, dadaRs, derepRs)

Now we produce a contingency table of counts of ASVs:

Code
seqtab.all = makeSequenceTable(mergers[!grepl("Mock",names(mergers))])

head(seqtab.all[,1:5])
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGCAGGCGGAAGATCAAGTCAGCGGTAAAATTGAGAGGCTCAACCTCTTCGAGCCGTTGAAACTGGTTTTCTTGAGTGAGCGAGAAGTATGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCATACCGGCGCTCAACTGACGCTCATGCACGAAAGTGTGGGT
F3D0                                                                                                                                                                                                                                        542
F3D1                                                                                                                                                                                                                                        396
F3D141                                                                                                                                                                                                                                      427
F3D142                                                                                                                                                                                                                                      261
F3D143                                                                                                                                                                                                                                      209
F3D144                                                                                                                                                                                                                                      390
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGCAGGCGGACTCTCAAGTCAGCGGTCAAATCGCGGGGCTCAACCCCGTTCCGCCGTTGAAACTGGGAGCCTTGAGTGCGCGAGAAGTAGGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCCTACCGGCGCGCAACTGACGCTCATGCACGAAAGCGTGGGT
F3D0                                                                                                                                                                                                                                        328
F3D1                                                                                                                                                                                                                                        341
F3D141                                                                                                                                                                                                                                      353
F3D142                                                                                                                                                                                                                                      288
F3D143                                                                                                                                                                                                                                      164
F3D144                                                                                                                                                                                                                                      265
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGTAGGCGGGCTGTTAAGTCAGCGGTCAAATGTCGGGGCTCAACCCCGGCCTGCCGTTGAAACTGGCGGCCTCGAGTGGGCGAGAAGTATGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCATACCGGCGCCCGACTGACGCTGAGGCACGAAAGCGTGGGT
F3D0                                                                                                                                                                                                                                        428
F3D1                                                                                                                                                                                                                                        218
F3D141                                                                                                                                                                                                                                      316
F3D142                                                                                                                                                                                                                                      150
F3D143                                                                                                                                                                                                                                      188
F3D144                                                                                                                                                                                                                                      267
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGTAGGCGGGCTTTTAAGTCAGCGGTAAAAATTCGGGGCTCAACCCCGTCCGGCCGTTGAAACTGGGGGCCTTGAGTGGGCGAGAAGAAGGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACCCCGATTGCGAAGGCAGCCTTCCGGCGCCCTACTGACGCTGAGGCACGAAAGTGCGGGG
F3D0                                                                                                                                                                                                                                        423
F3D1                                                                                                                                                                                                                                         66
F3D141                                                                                                                                                                                                                                      490
F3D142                                                                                                                                                                                                                                      155
F3D143                                                                                                                                                                                                                                      215
F3D144                                                                                                                                                                                                                                      338
       CCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGAGCGTAGGTGGATTGTTAAGTCAGTTGTGAAAGTTTGCGGCTCAACCGTAAAATTGCAGTTGAAACTGGCAGTCTTGAGTACAGTAGAGGTGGGCGGAATTCGTGGTGTAGCGGTGAAATGCTTAGATATCACGAAGAACTCCGATTGCGAAGGCAGCTCACTGGACTGCAACTGACACTGATGCTCGAAAGTGTGGGT
F3D0                                                                                                                                                                                                                                         147
F3D1                                                                                                                                                                                                                                         130
F3D141                                                                                                                                                                                                                                       186
F3D142                                                                                                                                                                                                                                       176
F3D143                                                                                                                                                                                                                                       106
F3D144                                                                                                                                                                                                                                       104

h) Remove chimeras

Explore the components of the objects dadaRs and mergers. What are Chimera sequences?

Chimera are sequences that are artificially created during PCR amplification by the melding of two (in rare cases, more) of the original sequences.

The error model in the sequence inference algorithm does not include a chimera component, and therefore we expect this sequence table to include many chimeric sequences. We now remove chimeric sequences by comparing each inferred sequence to the others in the table, and removing those that can be reproduced by stitching together two more abundant sequences.

To complete our denoising workflow, we remove them with a call to the function removeBimeraDenovo, leaving us with a clean contingency table we will use later on.

Code
seqtab <- removeBimeraDenovo(seqtab.all)

head(seqtab[,1:5])
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGCAGGCGGAAGATCAAGTCAGCGGTAAAATTGAGAGGCTCAACCTCTTCGAGCCGTTGAAACTGGTTTTCTTGAGTGAGCGAGAAGTATGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCATACCGGCGCTCAACTGACGCTCATGCACGAAAGTGTGGGT
F3D0                                                                                                                                                                                                                                        542
F3D1                                                                                                                                                                                                                                        396
F3D141                                                                                                                                                                                                                                      427
F3D142                                                                                                                                                                                                                                      261
F3D143                                                                                                                                                                                                                                      209
F3D144                                                                                                                                                                                                                                      390
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGCAGGCGGACTCTCAAGTCAGCGGTCAAATCGCGGGGCTCAACCCCGTTCCGCCGTTGAAACTGGGAGCCTTGAGTGCGCGAGAAGTAGGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCCTACCGGCGCGCAACTGACGCTCATGCACGAAAGCGTGGGT
F3D0                                                                                                                                                                                                                                        328
F3D1                                                                                                                                                                                                                                        341
F3D141                                                                                                                                                                                                                                      353
F3D142                                                                                                                                                                                                                                      288
F3D143                                                                                                                                                                                                                                      164
F3D144                                                                                                                                                                                                                                      265
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGTAGGCGGGCTGTTAAGTCAGCGGTCAAATGTCGGGGCTCAACCCCGGCCTGCCGTTGAAACTGGCGGCCTCGAGTGGGCGAGAAGTATGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACTCCGATTGCGAAGGCAGCATACCGGCGCCCGACTGACGCTGAGGCACGAAAGCGTGGGT
F3D0                                                                                                                                                                                                                                        428
F3D1                                                                                                                                                                                                                                        218
F3D141                                                                                                                                                                                                                                      316
F3D142                                                                                                                                                                                                                                      150
F3D143                                                                                                                                                                                                                                      188
F3D144                                                                                                                                                                                                                                      267
       GCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGTGCGTAGGCGGGCTTTTAAGTCAGCGGTAAAAATTCGGGGCTCAACCCCGTCCGGCCGTTGAAACTGGGGGCCTTGAGTGGGCGAGAAGAAGGCGGAATGCGTGGTGTAGCGGTGAAATGCATAGATATCACGCAGAACCCCGATTGCGAAGGCAGCCTTCCGGCGCCCTACTGACGCTGAGGCACGAAAGTGCGGGG
F3D0                                                                                                                                                                                                                                        423
F3D1                                                                                                                                                                                                                                         66
F3D141                                                                                                                                                                                                                                      490
F3D142                                                                                                                                                                                                                                      155
F3D143                                                                                                                                                                                                                                      215
F3D144                                                                                                                                                                                                                                      338
       CCGAGCGTTATCCGGATTTATTGGGTTTAAAGGGAGCGTAGGTGGATTGTTAAGTCAGTTGTGAAAGTTTGCGGCTCAACCGTAAAATTGCAGTTGAAACTGGCAGTCTTGAGTACAGTAGAGGTGGGCGGAATTCGTGGTGTAGCGGTGAAATGCTTAGATATCACGAAGAACTCCGATTGCGAAGGCAGCTCACTGGACTGCAACTGACACTGATGCTCGAAAGTGTGGGT
F3D0                                                                                                                                                                                                                                         147
F3D1                                                                                                                                                                                                                                         130
F3D141                                                                                                                                                                                                                                       186
F3D142                                                                                                                                                                                                                                       176
F3D143                                                                                                                                                                                                                                       106
F3D144                                                                                                                                                                                                                                       104

Highly recommended YouTube workshop, where the DADA2 pipeline is explained in detail: “Microbiome/Metagenome Analysis Workshop: DADA2”

Exercise 04 (Graph-Based Clustering of scRNA-seq Data with Seurat)

In this exercise, we perform a complete clustering analysis of single-cell RNA-seq data using the Seurat framework on the PBMC3k dataset (∼2,700 peripheral blood mononuclear cells from 10X Genomics). The dataset was already introduced in Exercise 2 of Exercise sheet 2, where we studied normalization and variance-stabilizing transformations. Here we focus on the downstream analysis steps: quality control, dimensionality reduction, and graph-based clustering.

NoteReference

This exercise follows the official Seurat Guided Clustering Tutorial (Satija Lab). Parts (a)–(b) and (c-ii) reproduce the tutorial directly. Parts (c-i) (SNN graph visualization via igraph), (c-iii) (resolution sensitivity scan), and (d-i) (t-SNE vs. UMAP comparison) go beyond the tutorial and are exercise-specific extensions. We intentionally stop before the tutorial’s biomarker discovery step (FindAllMarkers) and cell-type assignment.

Code
library(dplyr)
library(Seurat)
library(SeuratData)
library(patchwork)
library(ggplot2)
library(tidyr)
library(igraph)
library(scales)

# The Seurat tutorial loads from raw 10X output via
#   pbmc.data <- Read10X(data.dir = ".../filtered_gene_bc_matrices/hg19/")
#   pbmc     <- CreateSeuratObject(counts = pbmc.data, project = "pbmc3k",
#                                  min.cells = 3, min.features = 200)
# For consistency with Exercise sheet 2, we instead use the pre-built
# SeuratData package, which ships the same dataset.
InstallData("pbmc3k")
data("pbmc3k")
pbmc <- UpdateSeuratObject(pbmc3k)
pbmc
An object of class Seurat 
13714 features across 2700 samples within 1 assay 
Active assay: RNA (13714 features, 0 variable features)
 2 layers present: counts, data

a) Preprocessing

Compute the mitochondrial percentage using PercentageFeatureSet() with the pattern "^MT-". Visualize all three QC metrics using VlnPlot() and create scatter plots of nCount_RNA vs. percent.mt and nCount_RNA vs. nFeature_RNA using FeatureScatter(). Based on these plots, choose appropriate filtering thresholds and remove low-quality cells using subset(). Then apply the standard Seurat preprocessing steps as discussed on Exercise sheet 2: NormalizeData(), FindVariableFeatures(selection.method = "vst"), and ScaleData().

The Seurat tutorial documents three QC metrics that have become community standard (Ilicic et al., 2016):

  • nFeature_RNA (unique genes per cell): unusually low counts indicate empty droplets or dead cells; unusually high counts may indicate doublets — two cells captured in a single droplet.
  • nCount_RNA (total UMI count per cell): correlates strongly with nFeature_RNA; extremely high values are a doublet signal.
  • percent.mt (fraction of mitochondrial reads): in dying or damaged cells, cytoplasmic mRNA leaks through the compromised plasma membrane while mitochondrial transcripts — protected by the inner mitochondrial membrane — are retained. A high mitochondrial fraction is therefore a hallmark of poor-quality or apoptotic cells.

PercentageFeatureSet() flexibly computes the fraction of counts originating from any pattern-matched gene set; here we use pattern = "^MT-" to capture all genes whose symbol starts with MT-.

Code
pbmc[["percent.mt"]] <- PercentageFeatureSet(pbmc, pattern = "^MT-")
head(pbmc@meta.data, 5)
               orig.ident nCount_RNA nFeature_RNA seurat_annotations percent.mt
AAACATACAACCAC     pbmc3k       2419          779       Memory CD4 T  3.0177759
AAACATTGAGCTAC     pbmc3k       4903         1352                  B  3.7935958
AAACATTGATCAGC     pbmc3k       3147         1129       Memory CD4 T  0.8897363
AAACCGTGCTTCCG     pbmc3k       2639          960         CD14+ Mono  1.7430845
AAACCGTGTATGCG     pbmc3k        980          521                 NK  1.2244898
Code
VlnPlot(pbmc, features = c("nFeature_RNA", "nCount_RNA", "percent.mt"),
        ncol = 3, pt.size = 0.1)

Code
plot1 <- FeatureScatter(pbmc, feature1 = "nCount_RNA", feature2 = "percent.mt")
plot2 <- FeatureScatter(pbmc, feature1 = "nCount_RNA", feature2 = "nFeature_RNA")
plot1 + plot2

Observations:

  • The bulk of cells have between roughly 500 and 2500 detected genes; a thin tail extends well beyond 3000 — these are candidate doublets.
  • A small subpopulation shows percent.mt > 5 % combined with relatively few detected genes — the signature of dying cells.
  • nCount_RNA and nFeature_RNA correlate very strongly and roughly linearly (Pearson ≈ 0.95), exactly as expected for healthy droplets.

Following the tutorial, we filter cells with 200 < nFeature_RNA < 2500 and percent.mt < 5.

Code
pbmc <- subset(pbmc, subset = nFeature_RNA > 200 &
                              nFeature_RNA < 2500 &
                              percent.mt   < 5)
pbmc
An object of class Seurat 
13714 features across 2638 samples within 1 assay 
Active assay: RNA (13714 features, 0 variable features)
 2 layers present: counts, data

Filtering removes a few dozen low-quality cells; 2,638 cells remain. We then apply the standard preprocessing pipeline as in the Seurat tutorial and Exercise sheet 2 — log-normalization with a scale factor of 10,000, selection of the 2,000 most variable features using the vst method, and z-score scaling on the variable features.

Code
pbmc <- NormalizeData(pbmc, normalization.method = "LogNormalize",
                            scale.factor = 10000, verbose = FALSE) |>
  FindVariableFeatures(selection.method = "vst",
                       nfeatures = 2000, verbose = FALSE) |>
  ScaleData(verbose = FALSE)

b) PCA and dimensionality selection

Perform PCA on the variable features using RunPCA().

Code
pbmc <- RunPCA(pbmc, features = VariableFeatures(pbmc), verbose = FALSE)

i) Top loadings of PC1

Inspect the top 5 principal components using print(pbmc[["pca"]], dims = 1:5, nfeatures = 5). What do the genes with high positive and negative loadings on PC1 tell you about the major axis of variation in the data?

Code
print(pbmc[["pca"]], dims = 1:5, nfeatures = 5)
PC_ 1 
Positive:  CST3, TYROBP, LST1, AIF1, FTL 
Negative:  MALAT1, LTB, IL32, IL7R, CD2 
PC_ 2 
Positive:  CD79A, MS4A1, TCL1A, HLA-DQA1, HLA-DQB1 
Negative:  NKG7, PRF1, CST7, GZMB, GZMA 
PC_ 3 
Positive:  HLA-DQA1, CD79A, CD79B, HLA-DQB1, HLA-DPB1 
Negative:  PPBP, PF4, SDPR, SPARC, GNG11 
PC_ 4 
Positive:  HLA-DQA1, CD79B, CD79A, MS4A1, HLA-DQB1 
Negative:  VIM, IL7R, S100A6, IL32, S100A8 
PC_ 5 
Positive:  GZMB, NKG7, S100A8, FGFBP2, GNLY 
Negative:  LTB, IL7R, CKB, VIM, MS4A7 

PC1 cleanly separates two major hematopoietic lineages:

  • Positive loadings: CST3, TYROBP, LST1, AIF1, FTL — canonical myeloid / monocyte markers.
  • Negative loadings: MALAT1, LTB, IL32, IL7R, CD2 — lymphoid / T-cell markers.

The first principal component therefore captures the most prominent biological axis of variation in PBMC: the myeloid vs. lymphoid divide — exactly as documented in the Seurat tutorial. PCA on a noisy single-cell dataset has, without any supervision, already recovered a meaningful biological partition along its first principal axis.

PC2 follows the same logic for a finer subdivision: positive loadings are dominated by B-cell markers (CD79A, MS4A1, TCL1A, MHC class II), negative loadings by cytotoxic markers (NKG7, PRF1, CST7, GZMB, GZMA) — i.e. PC2 separates B cells from NK / cytotoxic T cells within the broader populations defined by PC1.

ii) Loadings and PCA embedding

Visualize the PCA loadings for PC1 and PC2 using VizDimLoadings() and the cells in PCA space using DimPlot().

Code
VizDimLoadings(pbmc, dims = 1:2, reduction = "pca")

Code
DimPlot(pbmc, reduction = "pca") + NoLegend()

The 2D PCA scatter shows a clear separation along PC1 between two main populations and finer structure spreading along PC2, but the populations are not cleanly separated in just two PCs — a strong motivation for using more PCs as input to graph construction and non-linear embedding.

iii) Heatmaps of the first 15 PCs

Use DimHeatmap() to plot heatmaps for the first 15 PCs. Which PCs appear to capture meaningful biological signal and which seem to represent noise?

For a given PC, DimHeatmap orders both cells and genes by their PC score and plots the gene-by-cell expression matrix as a heatmap. Setting cells = 500 displays only the 500 most extreme cells on each end, which speeds up plotting dramatically. A PC that captures real biological signal will produce a clean two-block pattern: a set of genes is consistently high in one extreme of cells and low in the other. A noisy PC produces a speckled heatmap without coherent gene-block structure.

Code
DimHeatmap(pbmc, dims = 1:15, cells = 500, balanced = TRUE)

The first ~9–10 PCs show clean, structured patterns: each row (gene) clearly separates the 500 displayed cells into a dark and a bright block, indicating that the PC corresponds to a coherent gene expression program shared by a subset of cells. Beyond roughly PC 10–12 the heatmaps lose this block structure and become noisy — rows look more random, suggesting these later components capture residual technical or stochastic variance rather than biological signal.

iv) Elbow plot

Use the ElbowPlot() function to visualize the percentage of variance explained by each PC. At which PC do you observe an “elbow”? How many PCs would you choose for downstream analysis?

Code
ElbowPlot(pbmc)

The elbow lies around PC 9–10: the standard deviation drops steeply for the first ~9 components and flattens noticeably afterwards. Combined with the heatmap inspection from (iii), choosing 10 PCs for downstream analysis is a defensible and conservative choice — and it matches the recommendation given in the Seurat tutorial, which notes that “anything between PC 7–12” would be justified for this dataset, with a slight preference for erring on the higher side.

c) Graph-based clustering

Seurat implements a graph-based clustering approach inspired by PhenoGraph (Levine et al., Cell, 2015) and SNN-Cliq (Xu and Su, Bioinformatics, 2015).

The Seurat tutorial describes the procedure as follows. First, a K-nearest neighbour (KNN) graph is built in PCA space (Euclidean distance), connecting each cell to its k closest neighbours. KNN edges alone are noisy: two cells may be mutual nearest neighbours but still differ in their wider neighbourhoods.

The shared nearest neighbour (SNN) graph refines this by weighting each edge by the Jaccard overlap of the two cells’ neighbour sets. Cells with many common neighbours are connected by strong edges; cells that happen to be close but live in different neighbourhoods get weak or no edges. The result is a graph in which dense subgraphs naturally correspond to homogeneous cell populations.

The Louvain algorithm (default in Seurat) then partitions this graph by iteratively moving nodes between communities to maximize modularity — a measure of how much more densely connected a partition is compared to a random graph with the same degree distribution.

i) SNN graph construction and visualization

Construct a KNN graph based on Euclidean distance in PCA space using FindNeighbors(), taking the first 10 PCs as input. Visualize the resulting SNN graph: extract it from the Seurat object (pbmc[["RNA_snn"]]), convert it to an igraph object using graph_from_adjacency_matrix(), and plot it with a force-directed layout (e.g., Fruchterman-Reingold). Color the nodes by their cluster identity. What do you observe about the connectivity within and between clusters?

Code
pbmc <- FindNeighbors(pbmc, dims = 1:10, verbose = FALSE)
Warning: package 'future' was built under R version 4.5.2

The Seurat tutorial uses the SNN graph implicitly as input to FindClusters but never visualizes the graph itself. Here we go a step further and inspect the graph topology directly — independently of any 2D embedding — by extracting the SNN adjacency matrix, building an igraph object, and laying it out with the Fruchterman-Reingold force-directed algorithm. This algorithm treats edges as springs and unconnected nodes as repelling charges; the resulting equilibrium layout reveals the graph’s intrinsic community structure without relying on UMAP or t-SNE coordinates.

Code
snn_mat <- pbmc[["RNA_snn"]]

snn_graph <- graph_from_adjacency_matrix(
  snn_mat, mode = "undirected", weighted = TRUE, diag = FALSE
)

# Declutter: drop very weak edges before layout
snn_graph <- delete_edges(snn_graph, E(snn_graph)[weight < 0.1])

We need cluster identities to colour the nodes; these come from FindClusters() in step (ii), which we anticipate here so the visualization is informative.

Code
pbmc <- FindClusters(pbmc, resolution = 0.5, verbose = TRUE)
Modularity Optimizer version 1.3.0 by Ludo Waltman and Nees Jan van Eck

Number of nodes: 2638
Number of edges: 95927

Running Louvain algorithm...
Maximum modularity in 10 random starts: 0.8728
Number of communities: 9
Elapsed time: 0 seconds
Code
set.seed(42)
layout_fr <- layout_with_fr(snn_graph, weights = E(snn_graph)$weight)
Code
clust_cols <- hue_pal()(length(levels(Idents(pbmc))))
node_cols  <- clust_cols[as.integer(Idents(pbmc))]

par(mar = c(0, 0, 1, 0))
plot(snn_graph,
     layout       = layout_fr,
     vertex.size  = 2,
     vertex.label = NA,
     vertex.color = node_cols,
     edge.color   = adjustcolor("grey70", alpha.f = 0.3),
     edge.width   = 0.3,
     main         = "SNN graph — Fruchterman-Reingold layout")
legend("topright", legend = levels(Idents(pbmc)),
       col = clust_cols, pch = 19, cex = 0.7,
       bty = "n", title = "Cluster")

Observations:

  • The force-directed layout produces a clear separation into dense blobs of nodes that match the clusters identified by Louvain — a strong indication that the cluster assignments reflect real topological communities in the SNN graph, not an artefact of any particular 2D embedding method.
  • Within-cluster edges are dense and bundled into compact regions: cells assigned to the same cluster share many neighbours.
  • Between-cluster edges are visibly sparser. Some clusters sit close to each other connected by a few “bridge” edges — typically the lymphoid sub-populations (T-cell subtypes, NK) that share a lineage — while clusters like B cells or platelets are placed further away with very few connecting edges.

ii) Louvain clustering at resolution 0.5

Apply the Louvain algorithm for community detection using FindClusters() with resolution = 0.5. How many clusters are found? Inspect the cluster assignments of the first few cells using Idents().

Code
n_clusters <- length(levels(Idents(pbmc)))
cat("Number of clusters at resolution 0.5:", n_clusters, "\n")
Number of clusters at resolution 0.5: 9 
Code
head(Idents(pbmc), 5)
AAACATACAACCAC AAACATTGAGCTAC AAACATTGATCAGC AAACCGTGCTTCCG AAACCGTGTATGCG 
             2              3              2              1              6 
Levels: 0 1 2 3 4 5 6 7 8

At resolution = 0.5 the Louvain algorithm identifies 9 clusters (labels 0–8), with a maximum modularity of ~0.87 across 10 random starts. This reproduces the result from the Seurat tutorial exactly. The tutorial subsequently identifies these 9 clusters as: naive CD4 T (0), CD14⁺ Mono (1), memory CD4 T (2), B (3), CD8 T (4), FCGR3A⁺ Mono (5), NK (6), DC (7), and platelet (8) — annotations we will not reproduce here, as biomarker discovery is beyond the scope of this exercise.

iii) Effect of the resolution parameter

Re-run FindClusters() with resolutions 0.1, 0.3, 0.5, 0.8, and 1.2. How does the number of clusters change? For each resolution, visualize the result on a UMAP plot (see part d). At what resolution do biologically meaningful cell populations start to merge, and at what resolution do they begin to over-fragment?

In modularity-based community detection, the resolution parameter γ rescales the null model that defines how much “denser than random” a community must be:

\[Q_\gamma = \frac{1}{2m} \sum_{i,j}\left[A_{ij} - \gamma \frac{k_i k_j}{2m}\right]\delta(c_i, c_j)\]

  • Small γ → weak random baseline → even loosely connected groups exceed it → fewer, larger communities.
  • Large γ → stringent baseline → only tightly knit groups survive → more, smaller communities.

The Seurat tutorial notes that values between 0.4 and 1.2 typically work well for datasets of ~3K cells, but emphasizes that there is no single “correct” resolution: the choice must be guided by the biological question and validated against marker genes.

The tutorial fixes the resolution at 0.5; we systematically vary it here to study the effect. We need a UMAP embedding to visualize the clustering results, so we anticipate part (d).

Code
pbmc <- RunUMAP(pbmc, dims = 1:10, verbose = FALSE)
Code
resolutions <- c(0.1, 0.3, 0.5, 0.8, 1.2)

cluster_results <- lapply(resolutions, function(res) {
  pbmc_tmp <- FindClusters(pbmc, resolution = res, verbose = FALSE)
  data.frame(
    UMAP1      = Embeddings(pbmc_tmp, "umap")[, 1],
    UMAP2      = Embeddings(pbmc_tmp, "umap")[, 2],
    cluster    = factor(Idents(pbmc_tmp)),
    resolution = paste0("res = ", res, "  (",
                        length(levels(Idents(pbmc_tmp))), " clusters)")
  )
})

umap_res_df <- bind_rows(cluster_results) |>
  mutate(resolution = factor(resolution, levels = unique(resolution)))
Code
ggplot(umap_res_df, aes(x = UMAP1, y = UMAP2, color = cluster)) +
  geom_point(size = 0.3, alpha = 0.7) +
  facet_wrap(~resolution, nrow = 2) +
  theme_bw() +
  guides(color = guide_legend(override.aes = list(size = 3))) +
  labs(title = "Louvain clustering of PBMC3k at varying resolutions",
       color = "Cluster")

Observations:

  • res = 0.1 — only 4–5 large clusters. The two monocyte subsets (CD14⁺ and FCGR3A⁺) are merged into one block, and the T-cell populations collapse into a single super-cluster. The major lymphoid/myeloid divide is preserved, but biologically meaningful subtypes are lost. This resolution under-clusters.
  • res = 0.3 – 0.5 — 7–9 clusters that match the canonical PBMC cell types from the tutorial. This is the biologically sensible range for this dataset.
  • res = 0.8 – 1.2 — the number of clusters keeps increasing without any obvious new structure appearing on the UMAP. Some clusters fragment into spatially mixed sub-pieces that no longer correspond to clean cell populations — this is over-fragmentation, driven by transcriptional noise within a homogeneous cell type rather than by additional biology.

For PBMC3k, the tutorial’s choice of resolution = 0.5 sits at the sweet spot of this range; in practice one should pick the resolution that best matches the biological question at hand and validate clusters against known marker genes.

d) Non-linear dimensionality reduction (UMAP and t-SNE)

Run UMAP using RunUMAP() with dims = 1:10 and visualize the clusters using DimPlot().

The Seurat tutorial emphasizes that 2D embeddings like UMAP and t-SNE are visualization tools — not replacements for the underlying clustering. Cells co-localize in these plots because they were already grouped together by the graph-based clustering; the embedding just helps us see the result. The tutorial warns explicitly: “avoid drawing biological conclusions solely on the basis of visualization techniques.”

We already computed the UMAP embedding in (c-iii). Here we restore the canonical clustering at resolution 0.5 and plot the resulting embedding.

Code
pbmc <- FindClusters(pbmc, resolution = 0.5, verbose = FALSE)
Code
DimPlot(pbmc, reduction = "umap", label = TRUE) + NoLegend()

i) UMAP vs. t-SNE

Additionally, compute a t-SNE embedding using RunTSNE() with the same PCs and compare the two embeddings side by side (use the patchwork package). What are the main visual differences?

The Seurat tutorial only computes the UMAP embedding; we add a t-SNE embedding here for comparison.

Code
pbmc <- RunTSNE(pbmc, dims = 1:10, verbose = FALSE)
Code
p_umap <- DimPlot(pbmc, reduction = "umap", label = TRUE) +
  NoLegend() + ggtitle("UMAP")
p_tsne <- DimPlot(pbmc, reduction = "tsne", label = TRUE) +
  NoLegend() + ggtitle("t-SNE")

p_umap + p_tsne

Observations:

  • Cluster separation. Both embeddings recover the same cluster structure. t-SNE tends to produce slightly more compact, well-separated clusters with more whitespace between them; UMAP retains a bit more scatter at cluster boundaries.
  • Global structure. UMAP places lineage-related populations close to each other — the two monocyte subsets sit next to each other and the T-cell clusters form a connected region. t-SNE spreads these same populations further apart, with less obvious global organization. This reflects the methodological difference: UMAP aims to preserve global topology in addition to local neighbourhoods, while t-SNE optimizes purely for local neighbourhood preservation and treats long distances as essentially interchangeable.
  • Overlap of related T-cell subsets. Neither method cleanly separates the naive CD4 T cluster (0) from the memory CD4 T cluster (2) — these populations are genuinely close in transcriptomic space, and no 2D embedding can manufacture a separation that the data does not contain.

For PBMC, with its well-separated populations, the two methods are essentially interchangeable — consistent with the Seurat tutorial’s choice to demonstrate only UMAP. The differences between UMAP and t-SNE become consequential mainly for datasets with continuous differentiation trajectories or subtle population boundaries, where the better global-structure preservation of UMAP is genuinely useful.