Research Radar — 2026-11-14

Methodology Review: Expectation–maximization: keep uncertainty in the assignments while learning the parameters

English · About 10 minutes

Original toy calculations are illustrative. RSEM evidence is historical full text; Wu abstract, introduction and section 2 were inspected, but the full convergence proof was not independently reproduced. No new dataset analysis or biological validation was performed.

The intuition

Imagine sorting observations into two sources when each observation carries only an imperfect clue about its origin. If source labels were known, estimating their proportions would be easy. But the proportions also affect how plausible each uncertain label is. Expectation–maximization, or EM, alternates between these two tasks. It assigns probabilities to the hidden labels using the current model, then fits parameters using the resulting fractional contributions. It does not turn its favorite label into a newly observed fact. The primer provides an accessible introduction to this distinction between soft assignments and hard guesses. [R4]

The incomplete part need not be a blank cell in a spreadsheet. It can be a latent origin introduced by the statistical model. Before applying the method, write down what was observed, what remains hidden, and which parameters would be easy to estimate if the hidden quantities were supplied. That short inventory is a useful diagnostic: if the complete-data problem is no easier, the proposed augmentation may not offer a computational advantage.

A small worked example

Here is an original toy experiment, not a transcript quantification benchmark. Each of ten independent tokens comes from source A or source B. A token has one of three observable colors: red, blue, or gray. Source A emits red and gray with equal probability and never emits blue. Source B emits blue and gray with equal probability and never emits red. The only unknown parameter is the fraction from A. We observe four red tokens, two blue tokens, and four gray tokens. The red and blue origins are certain in this model; the gray origins are not.

Start with the A fraction at one half. Because gray is equally probable under either source, each gray token contributes one half of an expected A token. The expected A count is therefore four plus two, or six. Dividing by ten gives the next A fraction, 0.6. Repeat: the four gray tokens now contribute 2.4 expected A tokens. The next fraction is 0.64. One more iteration gives 0.656. Notice what changed: the observed colors stayed fixed while the fractional interpretation of the gray tokens changed. No additional data arrived during these iterations.

For this toy, let \(a_t\) denote the A fraction at iteration \(t\). The update is \(a_{t+1}=(4+4a_t)/10\). Its fixed point is \(a_*=2/3\), because \(10a_*=4+4a_*\). Subtracting the fixed point gives \(a_{t+1}-a_*=0.4(a_t-a_*)\). Thus the error is multiplied by 0.4 each time. This is a direct calculation for this particular model, not a universal EM convergence rate. Starting at 0.9 instead gives 0.76, then 0.704, moving toward the same answer.

Check the answer without EM. The model probabilities of red, blue, and gray are respectively \(a/2\), \((1-a)/2\), and \(1/2\), where \(a\) is a proposed A fraction. Ignoring factors independent of that fraction, the likelihood is proportional to \(a^4(1-a)^2\). Differentiating its log gives \(4/a-2/(1-a)\); setting this to zero yields two thirds. Gray tokens provide no direct information about the fraction in this deliberately symmetric construction, although EM temporarily assigns them expected labels. This cross-check separates a computational device from actual information in the observations.

For a useful variation, replace all ten observations by gray tokens. Every starting fraction then remains unchanged: the update returns its input. An apparently stable algorithm would not have learned the source mixture at all. Ask a colleague to solve that variation before showing the answer. It illustrates why a stopping criterion alone cannot establish that a biological quantity is identifiable. Conversely, adding more red and blue observations changes the information available, rather than merely increasing the number of optimization iterations.

The necessary mathematics

Write \(y\) for observed data, \(z\) for a discrete hidden configuration, \(\theta\) for model parameters, and \(p_\theta(y,z)\) for their joint probability. At iteration \(t\), form the conditional weights \(q_t(z)=p_{\theta_t}(z\mid y)\). The E-step constructs \(Q(\theta;\theta_t)=\sum_z q_t(z)\log p_\theta(y,z)\). The M-step maximizes this function over the permitted parameters. The sum averages over possible hidden configurations; it does not replace the hidden variable by its mean inside an arbitrary nonlinear function. For continuous hidden variables the corresponding operation is an integral. [R1]

The observed log likelihood is \(\ell(\theta)=\log p_\theta(y)\). Exact EM uses a lower bound that touches this objective at the current parameters. Raising that bound cannot decrease the observed log likelihood, under the usual support and well-defined-expectation conditions. This is an ascent statement, not a global-optimality certificate. Approximate expectations and incomplete maximizations require their own checks rather than inheriting exact EM guarantees automatically. [R1] [R4]

Assumptions and failure modes

Wu separates questions often compressed into the word convergence: whether limit points are stationary, whether the parameter sequence converges, and when a unique maximum is reached. These conclusions need conditions. A monotone sequence of likelihood values alone does not guarantee a unique parameter estimate or the global optimum. The primary paper’s abstract, introduction and section 2 support this distinction; this review does not claim to have independently reproduced the full convergence proof. [R2]

In practice, keep a likelihood trace, inspect multiple initializations, and check that expected counts obey the model constraints. These are suggested diagnostics, not a theorem that every fitted model is trustworthy. In the toy, the expected counts must sum to ten at every step. If a program reports eleven expected tokens, improving likelihood values would not excuse the bookkeeping error. Also distinguish model fit from model adequacy: an optimizer cannot repair a missing source category simply by running longer.

Verified applications

Li and Dewey use EM in RSEM to estimate gene and isoform abundances from RNA-seq reads whose transcript origins can be ambiguous. Their primary paper reports a generative model, maximum-likelihood abundance estimation, and evaluation on simulated and real data. Posterior mean estimates and credibility intervals are separate outputs, not uncertainty intervals produced merely by the EM iterations. The toy above omits transcript lengths, alignment probabilities, fragment distributions and sequencing errors; it is not an implementation of RSEM. These are verified historical applications, not a comparison against contemporary quantifiers. [R3]

Possible transfers — speculative

A speculative teaching or method-development exercise would simulate two overlapping assay sources with a known mixing fraction, hide their labels, and compare soft allocation with discarding ambiguous observations. Vary the amount of source-specific evidence and record both estimation error and sensitivity to initialization. Introduce a third, unmodeled source as a deliberate stress test. This proposal is not validated by the RSEM paper. Any use for a new spatial or perturbation assay would need a justified observation model and independent validation, rather than assuming that fractional assignment automatically preserves the biology.

What to try and what to read next

Begin with the ten-token example and reproduce the three updates on paper. Then verify the fixed point by direct likelihood optimization. Keep the all-gray counterexample beside the successful calculation: it makes clear that numerical stability and scientific information are different questions. Read the short primer for intuition, the original paper for the construction, Wu for convergence qualifications, and RSEM for a concrete implementation context. The practical question is not simply whether EM runs, but whether the hidden-data model makes the observed experiment more intelligible.

Reading list

• [R1] Primary reading: Maximum Likelihood from Incomplete Data Via the EM Algorithm. A. P. Dempster, N. M. Laird and D. B. Rubin (1977). Read sections 1–3 for the general incomplete-data construction.

• [R2] Related reading: On the Convergence Properties of the EM Algorithm. C. F. Jeff Wu (1983). Read for the distinction between monotone likelihood and convergence to a unique maximum.

• [R3] Related reading: RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome. Bo Li and Colin N. Dewey (2011). Read as a concrete omics application, not as a claim about current best-performing software.

• [R4] Related reading: What is the expectation maximization algorithm?. Chuong B. Do and Serafim Batzoglou (2008). Read first for a compact introduction, then use the original mathematical paper.