Analysis of high-dimensional biological data

Exercise sheet 1
Author
Published

May 8, 2026

Install packages required for this exercise sheet

Code
packages <- c("tidyverse", "BiocManager", "reshape2", "fitdistrplus", "LaplacesDemon", "mixtools", "plotmm", "AdaptGauss", "nor1mix", "mclust")
# Find uninstalled packages
not_installed <- packages[!(packages %in% installed.packages()[,"Package"])]
# Install the missing packages
if (length(not_installed) > 0) {
  install.packages(not_installed, repos = "https://cloud.r-project.org")
}
  • R version: 4.5.1 (2025-06-13)
  • BioC version: 3.22

Exercise 1 (Generative models in R)

In MSMB Chapter 1, you have familiarized yourself with (or recapitulated) the concept of generative models for discrete data in R. In particular, we learned about the Poisson, the Binomial, and the Multinomial distribution.

a) Types of biological data

Check again the lecture slides and recapitulate what types of biological data you have encountered so far.

Give two biological data examples and a corresponding scientific question where you think that any of the above probability distributions is useful.

Number of successes in a fixed number \(n\) of independent Bernoulli trials with a constant probability of success \(p\) in each trial.

Example: Mendelian inheritance

  • Consider a single gene with two allees (A and a)
  • Equal probability of being passed to the offspring (1/2 for A and 1/2 for a)
  • Probability of an offspring receiving a certain number of A alleles can be modeled as a Binomial distribution

  • Heterozygous parents (Aa x Aa)
  • \(p=1/2\) to receive A from each parent
  • \(n=2\) (two alleles)
  • \(P(X=k)\) is the probability of offspring having \(k\) alleles A (\(k\) can be 0, 1, or 2)
  • Probability of an event is small and the number of trials large
  • \(\rightarrow\) Binomial can be approximated by the simpler Poisson distribution
  • Approximation more accurate with increasing \(n\) or decreasing \(p\)

Models the probability of a given number of events in a fixed interval (e.g., time or space).

\(P(X=k) = \frac{\lambda^ke^{-\lambda}}{k!}\)

  • \(\lambda\) is the rate of occurence
  • \(\lambda\) is also the mean, and the variance of the distribution
  • \(k\) is the number of occurences

Example: Gene mutations

  • \(\lambda = 2\) mutations per month
  • \(n=30000\) nucleotides
  • \(p=\frac{\lambda}{n}\) = 6.6667e-05
  • \(P(X=k)\) (probability of observing \(k\) mutations)?

Other examples:

  • HIV mutation rate (Lecture 3 slide 63, MSMB 1.3.3)
  • Occurence of rare diseases

The multinomial distribution is a generalization of the Binomial distribution.

Multinomial distribution is used when conducting multiple independent trials of an experiment and each trial can have any number of possible outcomes (instead of just two as in the Binomial distribution).

Parameters:

  • Number of trials \(n\)
  • Number of possible outcomes \(k\) (Binomial if \(k=2\))
  • Probabilities of each outcome (\(p_1, p_2, ..., p_k)\)

Example: Nucleotide frequencies

  • Each nucleotide A, C, T, G has a certain probability of occuring in a gene/sequence: \(p_1, p_2, p_3, p_4\)
  • \(\rightarrow k=4\)
  • The probability of observing each nucleotide a certain number of times \(P(X_1=x_1, X_2=x_2, X_3=x_3, X_4=x_4)\) can be modeled by a Multinomial distribution.

It allows for more flexibility. Unfortunately, the multinomial distribution does not provide a good fit to 16S rRNA count data, as seeing in the lecture (please refer to Negative Binomial for the 16S rRNA count day).

Another example is the Hardy-Weinberg case (Lecture 3 page 71, MSBM 2.7.2), where the three possible genotypes can be modeled by a Multinomial distribution:

  • Homozygous AA (if both parents inherited A)
  • Heterozygous Aa (if one parent inherited A and the other a)
  • Homozygous aa (if both parents inherited a)

b) MSMB Exercise 1.1

Do MSMB Exercise 1.1 (below the text for completeness).

R can generate numbers from all known distributions. We now know how to generate random discrete data using the specialized R functions tailored for each type of distribution. We use the functions that start with an r as in rXXXX, where XXXX could be pois, binom, multinom.

If we need a theoretical computation of a probability under one of these models, we use the functions dXXXX, such as dbinom, which computes the probabilities of events in the discrete binomial distribution, and dnorm, which computes the probability density function for the continuous normal distribution. When computing tail probabilities such as P(X > a), it is convenient to use the cumulative distribution functions, which are called pXXXX. Find two other discrete distributions that could replace the XXXX above.

Use the function rXXXX for random numbers, dXXXX for the probabilities (density), and pXXXX for the cumulative distribution function.

Find two other discrete distributions that could replace the XXXX above.

Code
# Specification of a seed is important in a sense that guarantees the same 
# random number generator, and thus the same sampling result.
set.seed(100)

# Binomial
dbinom(x, size, prob, log = FALSE)
pbinom(q, size, prob, lower.tail = TRUE, log.p = FALSE)
rbinom(n, size, prob)

# Bernoulli (Number of trials set to 1)
rbinom(n, size = 1,prob)

# Multinomial
dmnom(x, size, prob, log = FALSE)
rmnom(n, size, prob)
pmultinom(lower=ls, upper=us, size=n, probs=ps, method="exact")

# Poisson
dpois(x, lambda, log = FALSE)
ppois(q, lambda, lower.tail = TRUE, log.p = FALSE)
rpois(n, lambda)

# Negative Binomial
dnbinom(x, size, prob, mu, log = FALSE)
pnbinom(q, size, prob, mu, lower.tail = TRUE, log.p = FALSE)
rnbinom(n, size, prob, mu)

# Gamma
dgamma(x, shape, rate = 1, scale = 1/rate, log = FALSE)
pgamma(q, shape, rate = 1, scale = 1/rate, lower.tail = TRUE,
log.p = FALSE)
rgamma(n, shape, rate = 1, scale = 1/rate)

Exercise 2 (C. elegans nucleotide frequencies)

In MSMB Chapter 1, we covered the multinomial distribution.

We are now using the distribution to model nucleotide frequencies in a real genome, the mitochondrial sequence of the worm C. elegans.

This exercise is adapted from MSMB Exercise 1.8.

This is our opportunity to use Bioconductor for the first time. Since Bioconductor’s package management is more tightly controlled than CRAN’s, we need to use a special install function (from the BiocManager package) to install Bioconductor packages:

BiocManager::install(c("Biostrings", "BSgenome.Celegans.UCSC.ce2"))

After that, we can load the genome sequence package as we load any other R packages.

The question we want to ask is whether the mitochondrial sequence of C. elegans is consistent with a model of equally likely nucleotides?

C. elegans (source: Wikipedia)

a) Nucleotide frequencies

Explore the nucleotide frequencies of chromosome M by using a dedicated function in the Biostrings package from Bioconductor.

Solution:

Let’s start by importing the sequence

library(BSgenome.Celegans.UCSC.ce2)

C. elegans is an eukaryotic organism, that has a genome consisting of several chromosomes.

An important feature of eukaryotes, is that their cells contain organelles known as mitochondria.

https://www.visiblebody.com/learn/biology/cells/eukaryotic-cells

Briefly, mitochondria are in charge of producing ATP molecules, which are used as a source of chemical energy to carry out various chemical reactions within the cell. Interestingly, mitochondria have their own genome, separate from the genomic material found in the nucleus. Why?

According to the endosymbiotic theory, proposed by Dr. Lynn Margulis, mitochondria were originally an independent prokaryotic organism (bacteria-like). At some point, a long time ago, these prokaryotic organisms found their way inside ancestral eukaryotic cells and established a symbiotic relationship (a relationship where both parties benefit from one another). This led to the prokaryotic organism to gradually lose its independence, eventually become a permanent part of the eukaryotic cell. One big piece of evidence is the fact that they still have their own genome!

By looking at the sequences available in the C. elegans genome, we can see that there are 6 nuclear chromosomes (chrI-V and chrX), and one mitochondrial chromosome (chrM).

Code
seqnames(Celegans)
[1] "chrI"   "chrII"  "chrIII" "chrIV"  "chrV"   "chrX"   "chrM"  

We can get a quick glance of the mitochondrial genome.

Code
Ce.chrM <- Celegans$chrM

Ce.chrM
13794-letter DNAString object
seq: CAGTAAATAGTTTAATAAAAATATAGCATTTGGGTT...TATTTATAGATATATACTTTGTATATATCTATATTA

We can then check the frequency of each nucleotide and represent them as a proportion of the total genome length:

Code
letterFrequency(Ce.chrM, c("A", "T", "C", "G"))
   A    T    C    G 
4335 6179 1225 2055 
Code
letterFrequency(Ce.chrM, c("A", "T", "C", "G")) / length(Ce.chrM)
         A          T          C          G 
0.31426707 0.44794838 0.08880673 0.14897782 

b) Uniform model

Test whether the C. elegans data is consistent with the uniform model (all nucleotide frequencies the same) using a simulation.

Solution:

We can start by gathering the expected value of each nucleotide under the uniform model.

Code
pvec <- rep(0.25, 4)

expected0 <- pvec * length(Ce.chrM)
expected0
[1] 3448.5 3448.5 3448.5 3448.5

Each nucleotide has an expected value of 3448.5. Now we can turn our attention toward the simulation.

Code
set.seed(42) # Always set your seed

observed0 <- rmultinom(n = 1000, # Lets do 1000 trials, for fun
                       size = length(Ce.chrM), # Number of nucleotides in the genome
                       prob = pvec) # The frequency of each nucleotide

Each column in the matrix observed0 is a simulated genome nucleotide frequency.

We plot the relative frequencies of the nucleotides for a subset of 10 repetitions.

Code
library(ggplot2)
Warning: package 'ggplot2' was built under R version 4.5.2
Code
library(reshape2)

# Convert to data frame
rownames(observed0) <- c("A", "T", "C", "G")
colnames(observed0) <- paste0("Rep", 1:ncol(observed0))
df <- as.data.frame(observed0[, 1:10])

# Transpose the data frame
data_t <- t(df)

# Convert the transposed data frame to a data frame with columns Sample, 
# Nucleotide, and Abundance
data_long <- melt(data_t, varnames = c("Repetition", "Nucleotide"), 
                  value.name = "Abundance")

# Add a relative abundance column by dividing each row by the sum of its sample
data_long$RelativeAbundance <- data_long$Abundance / sum(data_long$Abundance)

# Create a stacked bar plot with ggplot
ggplot(data_long, aes(x = Repetition, y = RelativeAbundance, fill = Nucleotide)) +
  geom_bar(stat = "identity") +
  labs(x = "Samples", y = "Relative Abundance") +
  scale_y_continuous(labels = scales::percent) +
  theme_minimal()

The values computed above represent the observed values of each nucleotide under a fair model. This gives us a chance to know how much variability can be expected under this process, and how much variability is too much.

To do this we can use the following variability measurement, where we compute the sum of the squares of the differences between the observed values and the expected values relative to the expected values.

Code
var.stat <- function (obs.data, expected.data=expected0) {
  sum((obs.data - expected.data)^2 / expected.data)
}

S0 <- apply(observed0, 2, var.stat)

hist(S0, 50)

In biology, we like to consider the 95th percentile as the critical value for rejecting the null hypothesis.

Code
crit.val <- quantile(S0, 0.95)
crit.val
     95% 
7.815311 

So, we will use 7.815311 as the cutoff for determining whether our observed values actually come from a fair process. Exciting!!

Code
test.var <- var.stat(letterFrequency(Ce.chrM, c("A", "T", "C", "G")))

test.var
[1] 4386.634
Code
crit.val
     95% 
7.815311 
Code
as.logical(test.var > crit.val)
[1] TRUE

It seems that the variation from the expected values is more than what would be expected if all nucleotides had the same frequency.

Therefore, we can reject the null hypothesis and say that the frequency of nucleotides is not equal. This is actually a true phenomenon!

Different organisms have different biases toward different nucleotides.

Exercise 3 (Displaying GC content in S. aureus)

In the lecture, we learned about nucleotide frequencies in genomes.

Now let’s look at the GC content (guanine-cytosine content) in a real bacterial genome in MSMB Exercise 2.4.

Display GC content in a sliding window along the sequence of Staphylococcus Aureus. Read in a fasta file sequence from a file. This file is available in the data folder associated with the MSMB book.

library("Biostrings")

staph <- readDNAStringSet("./data/staphsequence.ffn.txt", "fasta")

a) Sequences

Look at the complete staph object and then display the first three sequences in the set.

Code
#Checking the number of sequences in the data:
length(staph)
[1] 2650
Code
#Displaying the first 3 sequences:
staph[1:3]
DNAStringSet object of length 3:
    width seq                                               names               
[1]  1362 ATGTCGGAAAAAGAAATTTGGGA...AAAAAGAAATAAGAAATGTATAA lcl|NC_002952.2_c...
[2]  1134 ATGATGGAATTCACTATTAAAAG...TTTTACCAATCAGAACTTACTAA lcl|NC_002952.2_c...
[3]   246 GTGATTATTTTGGTTCAAGAAGT...TCATTCATCAAGGTGAACAATGA lcl|NC_002952.2_c...

b) Sequence window

Find the GC content in the sequence windows of width 100.

Solution:

Here we can use a special function that will count the frequency of each nucleotide in each sequence:

  • A = Adenine
  • C = Cytosine
  • G = Guanine
  • T = Thymine

You might find that there are other symbols besides A, T, C and G that denote ambiguous bases or combinations:

  • M = A or C
  • R = A or G
  • W = A or T
  • S = C or G
  • Y = C or T
  • K = G or T
  • V = A, C, or G
  • H = A, C, or T
  • D = A, G, or T
  • B = C, G, or T
  • N = any base (A, C, G, or T)
Code
head(alphabetFrequency(staph), 20)
        A   C   G   T M R W S Y K V H D B N - + .
 [1,] 522 219 229 392 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [2,] 413 176 193 352 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [3,]  85  31  56  74 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [4,] 411 168 207 327 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [5,] 685 293 423 531 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [6,] 887 395 586 793 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [7,] 275 137 169 250 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [8,] 510 244 316 445 0 0 0 0 0 0 0 0 0 0 0 0 0 0
 [9,] 487 180 263 357 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[10,] 191 111 142 252 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[11,]  93  61  53 123 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[12,] 309 151 199 310 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[13,] 320 152 169 346 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[14,] 684 309 394 581 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[15,] 199  59  83 106 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[16,] 458 229 305 409 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[17,] 442 201 266 375 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[18,] 246 106 152 204 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[19,] 656 277 360 534 0 0 0 0 0 0 0 0 0 0 0 0 0 0
[20,] 478 216 254 387 0 0 0 0 0 0 0 0 0 0 0 0 0 0

In our case, we are only interested in G and C. Therefore:

Code
window <- 100

# Select C and G
GC_content <- alphabetFrequency(staph)[, c("C", "G")]
head(GC_content)
       C   G
[1,] 219 229
[2,] 176 193
[3,]  31  56
[4,] 168 207
[5,] 293 423
[6,] 395 586
Code
# Relative abundances
GC_content_rel <- GC_content / width(staph)
head(GC_content_rel)
             C         G
[1,] 0.1607930 0.1681351
[2,] 0.1552028 0.1701940
[3,] 0.1260163 0.2276423
[4,] 0.1509434 0.1859838
[5,] 0.1516563 0.2189441
[6,] 0.1484404 0.2202180
Code
# Abundance within window of width 100
GC_content.100 <- rowSums(GC_content_rel) * window
head(GC_content.100)
[1] 32.89280 32.53968 35.36585 33.69272 37.06004 36.86584

c) Sliding window

Display the GC content in a sliding window as a fraction.

Solution:

Again, we can make use of a very useful function

Code
# Compute the GC content in a sliding window (as a fraction) for sequence no. 364
# (the same from the Figure 2.28 in the book was chosen)
gc <- rowSums(letterFrequencyInSlidingView(staph[[364]], 
                                           window, c("G", "C")))/window

plot(gc, type = 'l')

Length of the sequence 364:

Code
length(staph[[364]])
[1] 402

There are 303 sliding windows: * 1-100 * … * 303-402

d) Overall trend

How could we visualize the overall trends of these proportions along the sequence?

Solution:

Code
plot(1:length(gc), gc, type = "l")

lines(lowess(x = 1:length(gc), y= gc, f = 0.10), col = 12, lwd = 2)

The function lowess() is used for scatter plot smoothing.

  • GC content can vary significantly across a genome and influence its structural and functional properties
  • Detecting Functional Regions: Sliding window analysis can help identify regions with specific characteristics. For example, regions with high GC content may correspond to genes or regulatory elements, as these are often more stable and less prone to mutation.
  • Pattern Recognition: Sliding window analysis can help identify repetitive elements or other sequence motifs. For example, regions with alternating A/T or G/C bases might suggest the presence of certain types of repetitive DNA.

Exercise 4 (Maximum Likelihood)

Humans, and many other organisms, are also classified as diploid organisms This means that our cells contain two copies of genetic material (i.e. two copies of each chromosome). One copy comes from the mother, and the other copy comes from the father This means that we also have two copies of each gene.

Genes can have different versions. These variant forms of genes are also known as alleles In classical genetics, alleles are often represented as capital and lowercase letters. For example, imagine a gene that has two alleles. One allele could be represented as ’A’ and the other as ’a’ Remember that we have two copies of each gene This means that we can have different types of allele combinations

In a diploid organism, there are three possible genotypes for any gene with two alleles:

  • Homozygous for allele A (AA)
  • Homozygous for allele a (aa)
  • Heterozygous (Aa)

Source: https://www.ancestry.com/c/dna-learning-hub/genotype

Suppose that a particular gene occurs as one of the two alleles (A and a), where allele A has frequency θ in the population. In other words, a random copy of the gene is A with probability θ and a with probability 1 − θ

a) Genotype probabilities

Consider a diploid organism with a specific gene that has two alleles (e.g., A and a).

Define the probability of each genotype.

Hint: Think about the Hardy-Weinberg principle and how genes are inherited.

Nicely explained in: YouTube: “Hardy-Weinberg Equilibrium” by Amoeba Sisters

Assumptions: The Hardy-Weinberg principle assumes several key conditions:

  • No natural selection
  • No mutation
  • No migration (no individuals enter or leave the population)
  • Population must be very large (to avoid genetic drift)
  • Random mating must occur (no mating preferences).

Source: www.amoebasisters.com

Principle:

Given a gene with two alleles, A and a, the principle predicts that the frequencies of these alleles will stabilize at equilibrium after one generation of random mating if the assumptions are met.

Source: study.com

b) Multinomial distribution

Suppose we test a random sample of \(k\) people, where \(k_i\) identifies the group of people with a particular genotype. Find the MLE of θ using the probabilities obtained in (a).

Hint: think of a multinomial distribution.

  • \(k>0\) trials (here: people)
  • \(m>0\) mutually exclusive events (here: genotypes AA, Aa, and aa)
  • \(p_1, ..., p_m\) event probabilities with \(\Sigma p_i = 1\) (here: \(p_1,p_2, p_3\))
  • Support: \(k_i \in \{0, ..., k\}, i \in \{1, ..., m\}\) with \(\Sigma k_i = k\)

\[ p(\textbf{k}|\textbf{p}, k) = P(K_1=k_1,K_2=k_2,K_3=k_3) = \frac{k!}{k_1! k_2! k_3!} p_1^{k_1}p_2^{k_2}p_3^{k_3} \]

Note: \(\frac{k!}{k_1! k_2! k_3!} = {k \choose k_1, k_2, k_3}\) is the multinomial coefficient, which counts the number of ways to distribute \(k\) items into three different groups.

If the \(p_i\) were not specified:

\[ L(p_1,p_2,p_3) = \frac{k!}{k_1! k_2! k_3!} p_1^{k_1}p_2^{k_2}p_3^{k_3} \]

with constraint \(\sum_{i=1}^{3} p_i = 1\).

log-Likelihood:

\[ \text{log } L(\textbf{p}) = l(\textbf{p}) = \text{log}(\frac{k!}{k_1! k_2! k_3!}) + \sum_{i=1}^{3} k_i \log(p_i) \]

We would use the method of Lagrange multipliers for optimization.

The Lagrange multiplier \(\lambda\) incorporates the constraint into the optimization process.

The Lagrangian optimization problem would then be:

\[ \mathcal{L}(\textbf{p}, \lambda) = l(\textbf{p}) + \lambda (1-\sum_{i=1}^{3}p_i) \]

However, in our example the \(p_i\) are not free to be chosen, but are defined by \(\theta\).

  • \(p_1=\theta^2\)
  • \(p_2=2\theta(1-\theta)\)
  • \(p_3=(1-\theta)^2\)

\[ L(\theta) = \frac{k!}{k_1! k_2! k_3!} (\theta)^{2k_1} (2\theta(1-\theta))^{k_2} (1-\theta)^{2k_3} \]

\(\rightarrow\) Maximize!

\[\begin{align} l(\theta) = &\log(k!) - \sum_{i=1}^{3} \log(k_i!) +\\ &2k_1 \log(\theta) + k_2(\log(2) + \log(\theta) + \log(1-\theta)) + 2k_3 \log(1-\theta)\\ = &(2k_1 + k_2) \log (\theta) + (2k_3 + k_2) \log(1-\theta) + C \end{align}\]

\[ \frac{\partial l}{\partial \theta} = \frac{2k_1+k_2}{\theta} - \frac{2k_3 + k_2}{(1- \theta)} \]

\[ \frac{2k_1+k_2}{\theta} - \frac{2k_3 + k_2}{(1- \theta)} = 0 \]

\[ \Rightarrow \text{MLE } \theta= \frac{2k_1 + k_2}{2(k_1+k_2+k_3)} = \frac{k_1 + \frac{1}{2}k_2}{k} \]

Let’s look at a small example with \(k_1 = 6\), \(k_2 = 3\), \(k_3 = 1\).

\(\rightarrow k = 10\)

Code
# Multinomial coefficient function
multicoef <- function(k1, k2, k3){
  k <- sum(k1, k2, k3)
  factorial(k) / (factorial(k1) * factorial(k2) * factorial(k3))
}

# Likelihood function
L_theta <- function(k1, k2, k3, theta) {
  multicoef(k1, k2, k3) * theta^(2*k1) * (2*theta*(1-theta))^k2 * (1-theta)^(2*k3)
}

k1 <- 6
k2 <- 3
k3 <- 1

# Multinomial coefficient
multicoef(k1, k2, k3)
[1] 840
Code
# Sequence of possible thetas
thetaseq <- seq(0, 1, 0.01)

plot(thetaseq, L_theta(k1, k2, k3, theta = thetaseq), 
     ylab = expression(L(theta)), xlab = "theta")

# Maximum likelihood estimator
(mle <- (k1 + 0.5*k2) / (k1+k2+k3))
[1] 0.75
Code
abline(v = mle, col = "blue", lwd = 2)

The \(p_i\) are then:

Code
(p1 = mle^2)
[1] 0.5625
Code
(p2 = 2 * mle * (1 - mle))
[1] 0.375
Code
(p3 = (1 - mle)^2)
[1] 0.0625
Code
p1 + p2 + p3
[1] 1

Exercise 5 (EM Algorithm and Mixture Models)

The following table provides the number of patients (K ∈ {0, 1, 2, 3, 4, …}) in a hospital that tested positive for a certain bacterium in a given day. The frequency (\(n_k\)) represents the number of times that exactly \(k\) patients tested positive. Assume that the frequency is constant over time and the infections are independent and uncorrelated.

Reminder:
E-step: create a function for the expectation of the log-likelihood, evaluated using the current estimate for the parameters.
M-step: compute parameters maximizing the expected log-likelihood found on the E-step.

Infected patients data
Number of patients (k) 0 1 2 3 4 5 6 7 8 9
Frequency (\(n_k\)) 162 267 271 185 111 61 27 8 3 1

a) Poisson distribution

Let \(Y_i\) be the number of infected patients on the \(i\)th day.

Fit a Poisson distribution to the data.

Let \(Y_i \overset{iid}{\sim} Poisson (\lambda)\) with \(i=1,...,n\).

Parameters:

  • \(y_i\): Infected patients on a certain day \(i\)
  • \(\lambda\): Infection rate

PMF:

\[ p(y_i|\lambda) = \frac{\lambda^{y_i}}{y_i!} exp(-\lambda) \]

The probability of having observed the whole data vector is then

\[ p(\textbf{y}|\lambda) = \prod_{i=1}^n p(y_i|\lambda) \]

For which rate \(\lambda\) are the observed data most likely?

\[ L(\lambda) = p(\textbf{y}|\lambda) = \prod_{i=1}^n \frac{\lambda^{y_i}}{y_i!} exp(-\lambda) \]

\[ l(\lambda) = -n\lambda + \log(\lambda) \sum_{i=1}^n y_i - \sum_{i=1}^n \log(y_i!) \]

\[\begin{align*} \frac{\partial l}{\partial \lambda} l(\lambda)= & -n + \frac{1}{\lambda} \sum_{i=1}^n y_i\\ \\ \Rightarrow \hat{\lambda}_{MLE} = &\frac{1}{n} \sum_{i=1}^n y_i = \overline{y} \end{align*}\]

\[ \hat{\lambda}_{MLE} = \frac{1}{1096} \sum_{i=1}^{1096} y_i = \frac{1}{1096} \sum_{k=0}^{9} kn_k = 2.157 \]

library(ggplot2)
library(dplyr)
infection_info <- data.frame(npatients = 0:9,
                             freq = c(162,267,271, 185,111,61,27,8,3,1)) %>%
  mutate("data.source"="reported")

infection_info
   npatients freq data.source
1          0  162    reported
2          1  267    reported
3          2  271    reported
4          3  185    reported
5          4  111    reported
6          5   61    reported
7          6   27    reported
8          7    8    reported
9          8    3    reported
10         9    1    reported
ggplot() +
  geom_bar(data = infection_info,
           aes(x = npatients, y = freq),
           stat = "identity", fill = alpha("grey", 0.5), color = "steelblue") +
  labs(x = "Number of infected patients",
       y = "Frequency")

Maximum likelihood estimator:

Code
numerator <- sum(rep(infection_info$npatients, times=infection_info$freq))

denominator <- sum(infection_info$freq)

lambda_mle <- numerator / denominator

lambda_mle
[1] 2.156934

### Goodness-of-fit Does the proposed distribution fit well the data? Evaluate the goodness of fit.

We compare the observed values to the probability density values of a Poisson distribution with our computed \(\hat{\lambda}_{MLE}\).

We multiply the probabilities by \(n\) to get absolute frequencies.

Code
# Poisson probability density values
fitted_probs <- dpois(0:9, lambda_mle)
n <- sum(infection_info$freq)

fitted_df <- data.frame(npatients = 0:9,
                     freq = as.integer(n * fitted_probs),
                     data.source = "MLE")

comparison_df <- bind_rows(infection_info, fitted_df) %>%
  mutate("npatients_fctr" = factor(npatients, levels = 0:max(npatients)))

comparison_df
   npatients freq data.source npatients_fctr
1          0  162    reported              0
2          1  267    reported              1
3          2  271    reported              2
4          3  185    reported              3
5          4  111    reported              4
6          5   61    reported              5
7          6   27    reported              6
8          7    8    reported              7
9          8    3    reported              8
10         9    1    reported              9
11         0  126         MLE              0
12         1  273         MLE              1
13         2  294         MLE              2
14         3  212         MLE              3
15         4  114         MLE              4
16         5   49         MLE              5
17         6   17         MLE              6
18         7    5         MLE              7
19         8    1         MLE              8
20         9    0         MLE              9
Code
ggplot() +
  geom_bar(data=subset(comparison_df, data.source=="reported"),
           aes(x=npatients_fctr, y=freq),
           stat="identity", fill=alpha("grey", 0.5), color="steelblue") +
  geom_line(data = comparison_df %>% filter(data.source=="MLE"),
            aes(x=npatients_fctr, y=freq, group=1, color=data.source)) +
  geom_point(data = comparison_df %>% filter(data.source=="MLE"),
             aes(x=npatients_fctr, y=freq, group=1, shape=data.source)) +
  labs(x="Number of infectied patients",
       y="Frequency",
       color="",
       shape="") +
  scale_color_discrete(labels=c("MLE lambda")) +
  scale_shape_discrete(labels=c("MLE lambda"))

We can evaluate the goodness of fit with a chi squared critical value.

\[ \chi^2 = \sum{\frac{(O_i-E_i)^2}{E_i}} \]

For this, we need the probability density values from before.

Since the probabilities must add up to one, we add a 10th category.

Code
infection_new <- data.frame(k = 0:10,
                            freq = c(infection_info$freq, 0),
                            fitted_prob = c(fitted_probs, 1-sum(fitted_probs)))

chisq.test(infection_new$freq, p = infection_new$fitted_prob)

    Chi-squared test for given probabilities

data:  infection_new$freq
X-squared = 27.069, df = 10, p-value = 0.00254

\(H_0: Y_i \sim Poisson (\hat{\lambda}_{MLE})\)

\(\rightarrow\) Null-hypothesis must be rejected for common significance levels.

b) Poisson mixture model

Assume that there is evidence for seasonal variation in the incidence of infection. There are likely to be different patterns in winter and summer, for example.

Assuming a mixture of two Poisson distributions, write down the likelihood for the observed data \(L(\theta|Y_{obs})\)

PMF of a mixture of two Poisson distributions with rates \(\lambda_1\) and \(\lambda_2\), and weight \(p\) of the first component:

\[ p(y_i|\theta) = p \frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1) + (1 - p) \frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2) \] for \(\theta = (p, \lambda_1, \lambda_2)\).

The probability of observing the whole \(\textbf{y}=y_1, ..., y_n\) vector is:

\[ p(\textbf{y}|\theta) = \prod_{i=1}^n p(y_i|\theta) \]

The probability of observing a certain \(n_k\) is:

\[ p(n_k|\theta) = \left(p \frac{\lambda_1^k}{k!} exp (-\lambda_1) + (1 - p) \frac{\lambda_2^k}{k!} exp (-\lambda_2)\right)^{n_k} \]

Reminder: \(n_k\) is the number of days where exactly \(k\) patients tested positive.

The likelihood is thus, given by:

\[ L(\theta) = \prod_{i=1}^{1096}p(y_i|\theta) = \prod_{k=0}^{9} p(n_k|\theta) \]

c) Weather indicator

Let \(Z_i \sim Bin(1, p)\) be a group (weather/season) indicator:

\[ Z_i = \begin{cases} &1, Y_i \sim Poisson(\lambda_1)\\ &0, Y_i \sim Poisson(\lambda_2) \end{cases} \]

Compute the likelihood and the log-likelihood for the unobserved data \(L(\theta|Y, Z)\). Denote by \(m_k\) the number of days where exactly \(k\) patients tested positive on summer and \(n_k - m_k\) on winter.

From \(Z_i \sim Bin(1, p)\) follows:

\[ p(z_i) = p^{z_i} (1-p)^{1-zi} \]

\[ p(y_i|z_i) = \left( \frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1)\right)^{z_i} \left( \frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2)\right)^{1-z_i} \] i.e.:

\[\begin{align*} p(y_i|z_i=1) = &\left( \frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1)\right)^{1} \left( \frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2)\right)^{0}=\frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1)\\ p(y_i|z_i=0) = &\left( \frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1)\right)^{0} \left( \frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2)\right)^{1}=\frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2) \end{align*}\]

According to Bayes, the joint distribution \(p(y_i, z_i)\) is then \(p(z_i)p(y_i|z_i)\) and the

marginal distribution of \(y_i\) is the sum over all possible \(z\) values:

\[ p(y_i) = \sum_z p(y_i,z) = \sum_z p(z)p(y_i|z) \]

\[ log(p(\textbf{y})) = \sum_{i=1}^n log\left(p \frac{\lambda_1^{y_i}}{y_i!} exp (-\lambda_1) + (1 - p) \frac{\lambda_2^{y_i}}{y_i!} exp (-\lambda_2)\right) \] Unfortunately, the log-Likelihood would be difficult to maximize because there is a sum inside the log.

We therefore assume that the \(z_i\) were given and consider the joint probability of \(y_i\) and \(z_i\):

\[ p(y_i,z_i) = p(z_i)p(y_i|z_i) = p^{z_i} (1-p)^{1 - z_i} \Bigg(\frac{\lambda_1^{y_i}}{y_i!} exp(-\lambda_1) \Bigg)^{z_i} \Bigg(\frac{\lambda_2^{y_i}}{y_i!} exp(-\lambda_2) \Bigg)^{1 - z_i} \]

\[ L(\theta) = \prod_{i=1}^{n} p^{z_i} (1-p)^{1 - z_i} \Bigg(\frac{\lambda_1^{y_i}}{y_i!} exp(-\lambda_1) \Bigg)^{z_i} \Bigg(\frac{\lambda_2^{y_i}}{y_i!} exp(-\lambda_2) \Bigg)^{1 - z_i} \]

Since \(m_k\) is the number of days where exactly \(k\) patients tested positive: \[ L(\theta) = \prod_{k=0}^{\infty} p^{m_k} (1-p)^{n_k - m_k} \Bigg(\frac{\lambda_1^{k}}{k!} exp(-\lambda_1) \Bigg)^{m_k} \Bigg(\frac{\lambda_2^{k}}{k!} exp(-\lambda_2) \Bigg)^{n_k - m_k} \]

\[\begin{align*} l(\theta) &= \sum_{k} (m_k\log p + (n_k - m_k) \log (1-p)) \\ & + \sum_{k} m_k(k \log \lambda_1 - \lambda_1) + \sum_{k}(n_k - m_k)(k \log \lambda_2 - \lambda_2) \\ & - \sum_{k} \log(k!) \end{align*}\]

Problem: \(m_k\) is unknown because Z is a latent variable.

d) E-step

Step \(l\) (\(l\)th iteration): Given \(Z_i \sim Bin(1, {p_k}^{(l)})\). Define \({p_k}^{(l)}\).

Note: \({p_k}^{(l)}\) is also used in the computation of \({m_k}^{(l)} = \mathbb{E}(m_k|Y, \theta^{(l)})\) in the E-step.

Reminder:
E-step: create a function for the expectation of the log-likelihood, evaluated using the current estimate for the parameters.
M-step: compute parameters maximizing the expected log-likelihood found on the E-step.

Source: https://www.youtube.com/watch?v=REypj2sy_5U (Victor Lavrenko)

Also recommended: YouTube: “EM.3: Visualizing the EM algorithm” by Victor Lavrenko

In the E-step of the \(l\)th iteration, we assume that \(\theta^{(l)} = (p^{(l)}, {\lambda_1}^{(l)}, {\lambda_2}^{(l)})\) is given. For \(l=1\) we use an initial estimate for \(\theta\).

Given that \(\theta\) is known, we evaluate the membership weights in the E-step.

The posterior probability that a day with exactly \(k\) infections belongs to the summer season, given the current parameters:

\[ p_k^{(l)} = \frac{p^{(l)} \frac{(\lambda_1^{(l)})^k}{k} exp(-\lambda_1^{(l)})} {p^{(l)} \frac{(\lambda_1^{(l)})^k}{k!} exp(-\lambda_1^{(l)}) + (1-p^{(l)}) \frac{(\lambda_2^{(l)})^k}{k!} exp(-\lambda_2^{(l)})} \]

  • \(p^{(l)}\) is the probability that a day belongs to the summer season
  • \(\frac{(\lambda_1^{(l)})^k}{k} exp(-\lambda_1^{(l)})\) is the Poisson probability of observing \(k\) infections given the summer season infection rate \(\lambda_1^{(l)}\)
  • The numerator computes the joint probability of observing \(k\) infections and being a summer day.
  • The denominator sums the joint probabilities for both the summer and winter.

\(\Rightarrow\) Expected count of days with \(k\) infections attributed to the summer:

\[ m_k^{(l)} = n_kp_k^{(l)} \]

e) M-step

Given \({m_k}^{(l)} = \mathbb{E}(m_k|Y, \theta^{(l)}) = n_k {p_k}^{(l)}\), obtain \(p^{(l+1)}\), \({\lambda_1}^{(l+1)}\) and \({\lambda_2}^{(l+1)}\).

Hint\(_2\): MSMB Question 4.19

Substituting \(m_k^{(l)}\) for \(m_k\) in \(l (\theta|Y, Z)\), we will get estimates for the parameters \(p\), \(\lambda_1\), and \(\lambda_2\) (maximize by setting the derivative to zero).

\[\begin{align*} &p^{(l+1)} = \frac{\sum_{k} n_k p_k^{(l)}}{n}\\ &\lambda_1^{(l+1)} = \frac{\sum_{k} kn_k p_k^{(l)}}{\sum_{k}n_kp_k^{(l)}}\\ &\lambda_2^{(l+1)} = \frac{\sum_{k} kn_k (1 - p_k^{(l)})}{\sum_{k}n_k (1 - p_k^{(l)})}\\ \end{align*}\]

f) Numerical estimates

Get the numerical estimates for \(\theta\) obtained in the item above and evaluate the goodness of fit for the current model.

Code
pk_update <- function(K, l1, l2, p){
  # Summer marginal
  summer_b <- p * dpois(K, lambda = l1)
  
  # Winter marginal
  winter_b <- (1-p) * dpois(K, lambda = l2)
  
  # Posterior pk
  new_pk <- summer_b / (summer_b + winter_b)
  
  return(new_pk)
}

# K = the number of infected patients

emstep <- function(Nk_vector, K_vector, lambda_1, lambda_2, p){
  # Update posteriors (E)
  p_k <- pk_update(K_vector, lambda_1, lambda_2, p)
  m_k <- Nk_vector * p_k
  
  # Update parameters (M)
  p <- sum(m_k) / sum(Nk_vector)
  lambda_1 <- sum(K_vector * m_k) / sum(m_k)
  lambda_2 <- sum(K_vector * (1-p_k) * Nk_vector) / sum(Nk_vector * (1-p_k))
  
  # Check for recursion condition
  parameters <- list("lambda_1"=lambda_1,
                     "lambda_2"=lambda_2,
                     "p" = p)
  
  return(parameters)
}

# Now let's do this 5,000 times
em_param <- list("lambda_1" = 1,
                 "lambda_2" = 2,
                 "p" = 0.5)

for (i in 1:5000){
  if (i %% 1000 == 0){
    print(paste0("On iteration ", i))
  }
  
  em_param <- emstep(Nk_vector = infection_info$freq,
                     K_vector = infection_info$npatients,
                     lambda_1 = em_param[["lambda_1"]],
                     lambda_2 = em_param[["lambda_2"]],
                     p = em_param[["p"]])
}
[1] "On iteration 1000"
[1] "On iteration 2000"
[1] "On iteration 3000"
[1] "On iteration 4000"
[1] "On iteration 5000"
Code
em_param
$lambda_1
[1] 1.256095

$lambda_2
[1] 2.663404

$p
[1] 0.3598854

### Goodness-of-fit test

When we want to perform a chi-square test, we need expected values to compare our results with. Therefore, we simulate data from the Poisson mixture model with the estimated parameters.

There used to be a package “MixtureInf” for this task, which is not available on CRAN anymore. We therefore write our own function.

Code
rmix.pois <- function(n, probs, theta) {
  # n: number of samples
  # probs: vector of mixture probabilities
  # theta: vector of Poisson parameters (lambdas)
  
  # Determine which component each sample comes from
  components <- sample(1:length(probs), n, replace = TRUE, prob = probs)
  
  # Generate samples from Poisson distributions based on selected components
  sapply(components, function(x) rpois(1, lambda = theta[x]))
}
Code
nsamp <- sum(infection_info$freq)

# Simulate 100 data sets with the Poisson mixture model
# The columns contain the days from 0 to 15
mix_tab_mat <- matrix(0, nrow = 100, ncol = 16)
colnames(mix_tab_mat) <- 0:15

set.seed(18)

for (i in 1:100) {
  mix_samp <- rmix.pois(n = nsamp,
                   probs = c(em_param$p, 1-em_param$p),
                   theta = c(em_param$lambda_1, em_param$lambda_2))
  
  mix_tab <- table(mix_samp)
  
  mix_tab_mat[i, 1:length(mix_tab)] <- mix_tab
}

mix_tab_mat[1:10, ]
        0   1   2   3   4  5  6  7 8 9 10 11 12 13 14 15
 [1,] 180 272 233 191 130 66 13  8 3 0  0  0  0  0  0  0
 [2,] 164 302 281 174  92 52 23  3 3 2  0  0  0  0  0  0
 [3,] 161 269 273 192 104 55 33  6 2 1  0  0  0  0  0  0
 [4,] 156 259 279 200  97 63 28  6 6 2  0  0  0  0  0  0
 [5,] 174 267 249 186 130 46 20 13 8 2  1  0  0  0  0  0
 [6,] 160 260 244 199 132 61 25 10 1 4  0  0  0  0  0  0
 [7,] 161 278 257 164 122 68 29 14 3 0  0  0  0  0  0  0
 [8,] 159 262 263 205 108 52 27 12 8 0  0  0  0  0  0  0
 [9,] 164 281 267 196 104 51 21  9 2 1  0  0  0  0  0  0
[10,] 147 259 256 201 121 77 22  6 4 2  1  0  0  0  0  0
Code
mix_tab <- round(apply(mix_tab_mat, 2, mean))

mixed.df <- data.frame(npatients = 0:15, freq = mix_tab)

mixed.df <- subset(mixed.df, freq != 0)

mixed.df$npatients <- as.integer(as.character(mixed.df$npatients))

mixed.df$data.source = "mix_simulated"

comparison_df.m <- bind_rows(comparison_df, mixed.df) %>%
  mutate("npatients_fctr" = factor(npatients, levels = 0:max(npatients)))

comparison_df.m
      npatients freq   data.source npatients_fctr
...1          0  162      reported              0
...2          1  267      reported              1
...3          2  271      reported              2
...4          3  185      reported              3
...5          4  111      reported              4
...6          5   61      reported              5
...7          6   27      reported              6
...8          7    8      reported              7
...9          8    3      reported              8
...10         9    1      reported              9
...11         0  126           MLE              0
...12         1  273           MLE              1
...13         2  294           MLE              2
...14         3  212           MLE              3
...15         4  114           MLE              4
...16         5   49           MLE              5
...17         6   17           MLE              6
...18         7    5           MLE              7
...19         8    1           MLE              8
...20         9    0           MLE              9
0             0  161 mix_simulated              0
1             1  271 mix_simulated              1
2             2  261 mix_simulated              2
3             3  190 mix_simulated              3
4             4  116 mix_simulated              4
5             5   57 mix_simulated              5
6             6   25 mix_simulated              6
7             7    9 mix_simulated              7
8             8    3 mix_simulated              8
9             9    1 mix_simulated              9
Code
ggplot() +
  geom_bar(data=subset(comparison_df.m, data.source=="reported"),
           aes(x=npatients_fctr, y=freq),
           stat="identity", fill=alpha("grey", 0.5), color="steelblue") +
  geom_line(data = comparison_df.m %>% filter(data.source %in% c("MLE", "mix_simulated")),
            aes(x=npatients_fctr, y=freq, group=data.source, color=data.source)) +
  geom_point(data = comparison_df.m %>% filter(data.source %in% c("MLE", "mix_simulated")),
             aes(x=npatients_fctr, y=freq, group=data.source, shape=data.source)) +
  labs(x="Number of infectied patients",
       y="Frequency of value",
       color="Simulated Values",
       shape="Simulated Values") +
  scale_color_discrete(labels=c("Poisson Mixture", "MLE lambda")) +
  scale_shape_discrete(labels=c("Poisson Mixture", "MLE lambda"))

Code
expected_prop <- mixed.df$freq / sum(mixed.df$freq)

chisq.test(infection_info$freq, p=expected_prop, correct = T)

    Chi-squared test for given probabilities

data:  infection_info$freq
X-squared = 1.3412, df = 9, p-value = 0.9982

Exercise 6 (High quality graphics and mixture models)

In this exercise, we will follow the MSMB book to generate some high quality graphs using the ggplot2 package. Finally, we will fit a Gaussian Mixture Model to a sample of a real gene expression data set.

a) DNase data set

We start with a low dimensional data set to make familiar with the ggplot2 package. This exercise follows MSMB chapter 3.2.

Familiarize yourself with the DNase data set that comes with the R base package.

It contains data from an enzyme-linked immunosorbent assay (ELISA), a laboratory technique used for detecting and quantifying substances such as peptides, proteins, antibodies, and hormones.

The assay was used to quantify the activity of the enzyme deoxyribonuclease (DNase), which degrades DNA. The object DNase is a dataframe whose columns are

  • Run, the assay run
  • conc, the protein concentration that was used
  • density, the measured optical density

Use the R base plot function plot() to generate the following plots:

  • Scatterplot of concentration vs. density
  • Histogram of the density
  • Boxplot of the density values stratified by the assay run

Then produce the same plot types using the ggplot2 package.

Code
data("DNase")
#?DNase
head(DNase)
  Run       conc density
1   1 0.04882812   0.017
2   1 0.04882812   0.018
3   1 0.19531250   0.121
4   1 0.19531250   0.124
5   1 0.39062500   0.206
6   1 0.39062500   0.215
Code
# Scatterplot of concentration vs. density
plot(density ~ conc, data = DNase, xlab = "Protein concentration")

Code
# Histogram of the density
hist(DNase$density, breaks=25, main = "")

Code
# Boxplot of the densities by Run
# (Sort the run variable first)
DNase$Run <- factor(DNase$Run, levels = sort(as.numeric(levels(DNase$Run))))
boxplot(density ~ Run, data = DNase)

Code
library("ggplot2")
Code
ggp <- ggplot(DNase, aes(x = conc, y = density)) 

ggp

Code
ggp + geom_point()

Code
ggplot(DNase, aes(x = density)) + geom_histogram()
`stat_bin()` using `bins = 30`. Pick better value `binwidth`.

Code
ggplot(DNase, aes(x = Run, y = density)) + geom_boxplot()

b) Hiiragi data set - Descriptive analysis

We continue with the high dimensional data set “Hiiragi2013” that is used in MSMB chapter 3.3.

A description of the data set given in the book:

We’ll use a gene expression microarray dataset that reports the transcriptomes of around 100 individual cells from mouse embryos at different time points in early development. The mammalian embryo starts out as a single cell, the fertilized egg. Through synchronized waves of cell divisions, the egg multiplies into a clump of cells that at first show no discernible differences between them. At some point, though, cells choose different lineages. By further and further specification, the different cell types and tissues arise that are needed for a full organism. The aim of the experiment, explained by Ohnishi et al. (2014), was to investigate the gene expression changes associated with the first symmetry breaking event in the embryo. We’ll further explain the data as we go. More details can be found in the paper and in the documentation of the Bioconductor data package Hiiragi2013.

We first make familiar with the data:

Code
# BiocManager::install("Hiiragi2013")

library("Hiiragi2013")
data("x")
x
ExpressionSet (storageMode: lockedEnvironment)
assayData: 45101 features, 101 samples 
  element names: exprs 
protocolData
  sampleNames: 1 E3.25 2 E3.25 ... 101 E4.5 (FGF4-KO) (101 total)
  varLabels: ScanDate
  varMetadata: labelDescription
phenoData
  sampleNames: 1 E3.25 2 E3.25 ... 101 E4.5 (FGF4-KO) (101 total)
  varLabels: File.name Embryonic.day ... sampleColour (8 total)
  varMetadata: labelDescription
featureData
  featureNames: 1415670_at 1415671_at ... AFFX-TrpnX-M_at (45101 total)
  fvarLabels: symbol genename ensembl
  fvarMetadata: labelDescription
experimentData: use 'experimentData(object)'
Annotation: mouse4302 
Code
dim(x)
Features  Samples 
   45101      101 
  • 101 columns: samples (each of these is a single cell)
  • 45101 rows: genes probed by the array
Code
# phenoData
dim(pData(x))
[1] 101   8
Code
head(pData(x), n = 3)
        File.name Embryonic.day Total.number.of.cells lineage genotype
1 E3.25  1_C32_IN         E3.25                    32               WT
2 E3.25  2_C32_IN         E3.25                    32               WT
3 E3.25  3_C32_IN         E3.25                    32               WT
          ScanDate sampleGroup sampleColour
1 E3.25 2011-03-16       E3.25      #CAB2D6
2 E3.25 2011-03-16       E3.25      #CAB2D6
3 E3.25 2011-03-16       E3.25      #CAB2D6

The information provided is a mix of information about the cells (i.e., age, size and genotype of the embryo from which they were obtained) and technical information (scan date, raw data file name). By convention, time in the development of the mouse embryo is measured in days, and reported as, for instance, E3.5.

Follow MSMB chapter 3.4 to reproduce Figure 3.8, which displays the number of samples for each of the 8 groups.

Code
library("dplyr")
groups = group_by(pData(x), sampleGroup) |>
  summarise(n = n(), color = unique(sampleColour))
groups
# A tibble: 8 × 3
  sampleGroup         n color  
  <chr>           <int> <chr>  
1 E3.25              36 #CAB2D6
2 E3.25 (FGF4-KO)    17 #FDBF6F
3 E3.5 (EPI)         11 #A6CEE3
4 E3.5 (FGF4-KO)      8 #FF7F00
5 E3.5 (PE)          11 #B2DF8A
6 E4.5 (EPI)          4 #1F78B4
7 E4.5 (FGF4-KO)     10 #E31A1C
8 E4.5 (PE)           4 #33A02C
  • E3.25 refers to the embryonic stage at approximately 3.25 days post-coitum
  • FGF4-KO stands for “Fibroblast Growth Factor 4 Knock-Out.”
    • FGF4 is a gene known to play critical roles in cell signaling, embryogenesis, and pattern formation.
    • A “knock-out” (KO) indicates that in these embryos, the FGF4 gene has been genetically deactivated or removed.
    • This group would be used to study the impact of the loss of FGF4 on early development.
  • EPI stands for “Epiblast,”
    • This layer of cells in the blastocyst that will eventually give rise to the embryo proper.
    • The label likely indicates that the gene expression data pertains to cells specifically from the epiblast layer at this stage.
  • PE stands for “Primitive Endoderm.”
    • The primitive endoderm is another important cell layer in the developing blastocyst.
    • It forms during the early stages of development and will eventually contribute to the formation of the yolk sac, which provides nutrients to the developing embryo and forms blood vessels.

Source: https://www.sciencedirect.com/science/article/pii/S1084952115001238

Bar chart

Code
ggplot(groups, aes(x = sampleGroup, y = n)) +
  geom_bar(stat = "identity")

Code
groupColor = setNames(groups$color, groups$sampleGroup)

Colored bar chart

Code
pb = ggplot(groups, aes(x = sampleGroup, y = n))

pb = pb + geom_bar(stat = "identity")
pb = pb + aes(fill = sampleGroup)
pb = pb + theme(axis.text.x = element_text(angle = 90, hjust = 1))
pb = pb + scale_fill_manual(values = groupColor, name = "Groups")
pb

Polar chart

Code
pb.polar = pb + coord_polar() +
  theme(axis.text.x = element_text(angle = 0, hjust = 1),
        axis.text.y = element_blank(),
        axis.ticks = element_blank()) +
  xlab("") + ylab("")
pb.polar

c) Scatterplot

Reproduce MSMB Figure 3.9 displaying a scatterplot of the genes “X1426642“ vs. “X1418765“. Using ggplot2, you can easily add a smooth regression line and a confidence band to the scatterplot.

Code
# data frame with expression levels
dftx = data.frame(t(Biobase::exprs(x)), pData(x))
dftx[1:6, 1:4]
        X1415670_at X1415671_at X1415672_at X1415673_at
1 E3.25    4.910459    9.768979   10.411893    5.618108
2 E3.25    7.526672    9.144228   10.918942    6.439416
3 E3.25    6.956328    9.295010    9.495738    6.730465
4 E3.25    6.424048   11.059831   10.317996    4.914527
5 E3.25    5.105808    9.376749   11.143684    5.619778
6 E3.25    5.856685    9.681017   10.234943    7.188673
Code
ggplot( dftx, aes( x = X1426642_at, y = X1418765_at )) +
  geom_point( shape = 1 ) +
  geom_smooth( method = "loess" )
`geom_smooth()` using formula = 'y ~ x'

d) Probe intensities

The following exercises focus on cell number 20 from day E3.25 (“20 E3.25”), which can be accessed by:

Code
cell20 <- exprs(x)[, "20 E3.25"]

Generate a random sub-sample of 5000 genes from the cell20 vector to reduce execution time in the following tasks.

Code
set.seed(20325)
cell20samp <- sample(cell20, size = 5000, replace = FALSE)

Plot a histogram to visualize the probe intensities for the subsample.

Code
dfx <- data.frame(cell20samp = cell20samp)

hist(dfx$cell20samp, breaks = 50, xlab = "cell 20 E3.25")

Code
ggplot(dfx, aes(x = cell20samp)) + geom_histogram(binwidth = 0.2)

e) Common distributions

Use the descdist() function from fitdistrplus package to find out if the data follow a common distribution.

Code
library(fitdistrplus)

descdist(cell20samp, discrete=FALSE)

summary statistics
------
min:  1.360463   max:  14.46438 
median:  3.989822 
mean:  4.741477 
estimated sd:  2.475685 
estimated skewness:  1.165853 
estimated kurtosis:  3.737019 

As the plot shows, none of the common distributions are good approximations of our data. We could fit a beta distribution to the data, but it would not take into account the bimodality of our data.

f) Gaussian mixture model

Find out if the data can be approximated by a Gaussian Mixture Model (GMM):

Fit a Gaussian mixture model to the data. Decide which number of components could be suitable.

Hint: The normalmixEM() function from mixtools package could be useful.

We first check if the distribution is unimodal or bimodal.

Code
library(LaplacesDemon)
Code
is.unimodal(cell20samp)
[1] FALSE
Code
is.bimodal(cell20samp)
[1] TRUE
Code
Modes(cell20samp)$modes
[1] 3.335109 8.076428
Code
library(mixtools)
Code
set.seed(12345)

# EM algorithm for normal mixture models
mixmod <- normalmixEM(cell20samp, 
                      k=2, # 2 components
                      maxit=100, # max number of iterations
                      epsilon=0.01) # convergence criterion
number of iterations= 23 
Code
summary(mixmod)
summary of normalmixEM object:
         comp 1   comp 2
lambda 0.666015 0.333985
mu     3.390941 7.431864
sigma  0.997651 2.340436
loglik at estimate:  -10553.74 

Plot the results.

Code
# Method 1
plot(mixmod, which = 2) 

Code
# Method 2
# plotmm: Tidy Tools for Visualizing Mixture Models
library(plotmm)

plot_gmm(mixmod, 2)

Code
# Method 3

# With ggplot2 (https://dozenoaks.twelvetreeslab.co.uk/2019/06/mixture-models/)

n <- length(dfx$cell20samp)
binwidth <- 0.2
lambda <- mixmod[["lambda"]]

# Gaussian density must be scaled to the count numbers:
ggplot(dfx, aes(x = cell20samp)) + 
  geom_histogram(binwidth = 0.2) +
  mapply(
    function(mean, sd, lambda, n, binwidth) {
      stat_function(
        fun = function(x) {
          (dnorm(x, mean = mean, sd = sd)) * n * binwidth * lambda
        },
        color = "red", 
        linewidth = 1.2
      )
    },
    mean = mixmod[["mu"]], #mean
    sd = mixmod[["sigma"]], #standard deviation
    lambda = mixmod[["lambda"]], #amplitude
    n = length(dfx$cell20samp), #sample size
    binwidth = 0.2 #binwidth used for histogram,
)

Perform a goodness-of-fit test to find out if the data follow a Gaussian mixture model.

What is the null hypothesis of this test and what does it mean if \(H_0\) is rejected?

Hint: The AdaptGausspackage provides the Chi2testMixtures() function, which could be helpful.*

Pearson’s chi-squared goodness of fit test

H0: The data distribution follows GMM (Gaussian mixture model)

Code
library(AdaptGauss)
Code
chi2test <- Chi2testMixtures(Data = cell20samp, 
                             Means = mixmod$mu, 
                             SDs = mixmod$sigma, 
                             Weights = mixmod$lambda,
                             PlotIt = TRUE)

Code
chi2test$Pvalue
[1] 0

Q-Q plot

Code
library(nor1mix)
Code
nm <- norMix(mu    = mixmod$mu,
             sigma = mixmod$sigma,
             w     = mixmod$lambda)

distinct_samp <- sort(unique(cell20samp))

theo_cdf <- pnorMix(distinct_samp, nm)

emp_cdf <- ecdf(cell20samp)(distinct_samp)

plot(theo_cdf, emp_cdf, col = rgb(0,0,0,0.1))
abline(0, 1, col = "blue")

The two-component Gaussian mixture model does not seem to fit the data well. Let’s see if a mixture model with more components leads to a better fit.

We use the Mclust package to fit a multimodal Gaussian mixture. The optimal number of components is determined by the function.

Code
library(mclust)
Code
mixmod_multi <- Mclust(cell20samp)

summary(mixmod_multi)
---------------------------------------------------- 
Gaussian finite mixture model fitted by EM algorithm 
---------------------------------------------------- 

Mclust V (univariate, unequal variance) model with 4 components: 

 log-likelihood    n df       BIC       ICL
      -10408.62 5000 11 -20910.93 -23454.08

Clustering table:
   1    2    3    4 
1168 1358 1184 1290 
Code
params <- mixmod_multi$parameters
Code
chi2test <- Chi2testMixtures(Data = cell20samp, 
                             Means = params$mean, 
                             SDs = sqrt(params$variance$sigmasq), 
                             Weights = params$pro,
                             PlotIt = TRUE)
Warning in regularize.values(x, y, ties, missing(ties), na.rm = na.rm):
collapsing to unique 'x' values

Code
chi2test$Pvalue
[1] 0.20937