Appendix B — Quick reference
This appendix collects lookup tables for your own analysis: first, typical problems and where to find help in this book; then which model, model structure, residual check and test fits which situation, together with the R syntax used in the book. Each row links to the section where the topic is explained. For the order in which to do things, see the workflow chapter (Chapter 12).
B.1 Where to find what
| My problem | Where to look |
|---|---|
| Which model fits my type of response? | Table B.4 |
| How do I interpret the regression table? | Section 2.2 |
| How do I check the residuals of my model? | Section 2.4, Section 9.3.1 |
| My residual checks show a problem | Table B.10 |
| The relationship is not linear | Section 2.5, Section 2.5.3, Chapter 5 |
| I have a categorical predictor: how to read the coefficients? | Section 2.6.1 |
| Which groups differ from each other? | Section 2.6.3, Note 3.1 |
| I have an interaction | Section 2.10, Section 9.4 |
| How do I report a curved effect or an effect in a GLM (ratio, predictions, data-averaged effect)? | Section 2.11.2, Section 9.4.1 |
| My predictors are correlated | Section 2.8 |
| Should I center or scale my predictors? | Section 2.9.2 |
| A reviewer asks for type I, II or III ANOVA | Section 3.2.3.2, Section 3.2.4.3 |
| My observations are grouped or repeated | Chapter 4, Table B.5 |
| My mixed model does not converge or is singular | Section 4.4.1 |
| How do I get p-values for a mixed model? | Table B.12, Section 4.5.1, Section 4.5.3 |
| Do I need a random slope? | Section 4.2.2, Section 4.4.5, Section 4.5.3.3 |
| How much variance is explained by the random effects? | Section 4.5.5 |
| How much variance does my mixed model explain (R²)? | Section 4.5.4 |
| Log-transform the response or use a GLM (e.g. Gamma)? | Section 9.2.3, Section 10.1.1 |
| Which package: lme4 or glmmTMB? | Section 4.2.3 |
| I have missing values | Chapter 6 |
| Which variables should I include to estimate a causal effect? | Section 7.3 |
| Which predictor is most important? | Section 2.9.2, Section 7.4.1 |
| How do I select between models? | Section 8.2, Section 8.3 |
| I have many predictors and want to predict | Section 8.2.3, Section 8.4 |
| My counts or k/n data are overdispersed | Section 9.3.2 |
| I have many zeros | Section 9.3.3 |
| My counts were sampled with different effort or area | Section 9.2.1.1 |
| The spread of the residuals is not constant | Chapter 10 |
| I have outliers | Section 10.3 |
| My residuals are correlated in time, space or phylogeny | Chapter 11 |
| How do I make predictions with confidence intervals? | Section 2.11, Section 4.5.2 |
| Are the estimates, p-values and CIs of my analysis correct for my design? | Section 12.6 |
| How large must my study be? | Section 12.6.1 |
| What should I report? | Section 12.5 |
| Can I use Bayesian methods instead? | Section 12.7 |
B.2 Formula syntax
| Formula | Interpretation | Section |
|---|---|---|
y ~ x |
intercept + slope | Section 2.1 |
y ~ 1 |
intercept only | Section 2.5.2 |
y ~ x - 1 or y ~ x + 0 |
no intercept (numeric x: line through origin; factor: one mean per level) | Section 2.5.2 |
log(y) ~ sqrt(x) |
predictor and response variable can be transformed | Section 2.5 |
y ~ x1 + x2 |
multiple predictors | Section 2.7 |
y ~ x1 * x2 |
interaction and main effects | Section 2.10.1 |
y ~ x1:x2 |
only the interaction | Section 2.10.1 |
y ~ (x1 + x2)^2 |
all 2nd order interactions; works also with 3rd order (^3) etc. |
Section 2.10.1 |
y ~ x + I(x^2) |
I() forces the mathematical interpretation of whatever is in the parentheses (here a quadratic effect, with directly interpretable coefficients; also for other transformations such as I(x^0.5)) |
Section 2.5.2, Caution 2.1 |
y ~ poly(x, 2) |
quadratic polynomial with uncorrelated (orthogonal) terms: same model as x + I(x^2), but numerically more stable; coefficients are not directly interpretable (raw = TRUE gives the plain powers) |
Caution 2.1 |
y ~ s(x) |
spline (mgcv only) | Section 2.5.3 |
y ~ te(x1, x2) |
tensor spline (mgcv only) | Section 2.10.4 |
y ~ . |
all variables in the data as main effects | Section 2.7 |
y ~ .^2 |
all variables as main effects and their 2nd order interactions | Section 2.10.1 |
y ~ x + offset(log(effort)) |
offset, a term with a fixed coefficient of 1 (e.g. for sampling effort or area in log-link GLMs) | Section 9.2.1.1 |
B.3 Preparing predictors
| What | Syntax | Section |
|---|---|---|
| Center / scale a predictor (keeps the transformation as attributes) | data$sx = scale(data$x); center only: scale(data$x, scale = FALSE) |
Section 2.9.2 |
| Set the reference level of a factor | data$f = relevel(factor(data$f), ref = "control") |
Section 2.6.1 |
| Sum-to-zero contrasts (e.g. for type III ANOVA) | lm(y ~ f * x, data = data, contrasts = list(f = contr.sum)) |
Section 3.2.4 |
| Check collinearity | cor(data[, c("x1", "x2")]), car::vif(fit) |
Section 2.8 |
B.4 Model families
| Response | Examples | Distribution (link) | R code | Section |
|---|---|---|---|---|
| Continuous, roughly symmetric residuals | height, temperature | normal (identity) | lm(y ~ x), lmer(y ~ x + (1|group)), glmmTMB(y ~ x), gls(y ~ x) |
Chapter 2, Chapter 4 |
| Continuous, strictly positive, variance increasing with the mean | biomass, concentration | log-transformed response with a normal model, or Gamma (log) | lm(log(y) ~ x), glm(y ~ x, family = Gamma(link = "log")) |
Section 9.2.3 |
| Counts (0, 1, 2, …) | number of individuals | Poisson (log) | glm(y ~ x, family = poisson) |
Section 9.2.1 |
| Overdispersed counts | negative binomial (log) | MASS::glm.nb(y ~ x), glmmTMB(y ~ x, family = nbinom2) (or nbinom1) |
Section 9.3.2 | |
| Counts with more zeros than expected | zero-inflated Poisson or negative binomial | glmmTMB(y ~ x, ziformula = ~ 1, family = nbinom2) |
Section 9.3.3 | |
| Binary (0/1) | presence / absence, survived yes / no | binomial (logit; cloglog with an offset for exposure time) | glm(y ~ x, family = binomial) |
Section 9.2.2 |
| k out of n | infected out of tested individuals | binomial; if overdispersed, beta-binomial | glm(cbind(k, n - k) ~ x, family = binomial), glmmTMB(..., family = betabinomial) |
Section 9.2.2, Section 9.3.2 |
| Continuous proportions | cover, fraction of time | beta (logit), for values strictly between 0 and 1 (exact 0s and 1s: see the section) | glmmTMB(y ~ x, family = beta_family) |
Section 9.2.4 |
| Ordered categories | damage classes | cumulative link model (only briefly covered) | ordinal::clm(y ~ x), with random effects ordinal::clmm() |
Section 9.2.2.1 |
| Unordered categories | food type chosen | multinomial regression (only briefly covered) | nnet::multinom(y ~ x) |
Section 9.2.2.1 |
| Several responses per unit | abundances of several species | multivariate GLM | glmmTMB |
Section 11.6 |
| Mechanistic nonlinear relationship | enzyme kinetics, growth curves | nonlinear regression | nls(y ~ a * x / (b + x), start = list(a = 1, b = 1)) |
Chapter 5 |
Start with the basic family for your type of response (e.g. Poisson for counts); change to a family for overdispersed or zero-inflated data only if the residual checks show the problem (Table B.10). All families can be combined with random effects (glmer(..., family = ...), glmmTMB(..., family = ...), Chapter 4, Section 9.5) and with smooth terms (mgcv::gam(y ~ s(x), family = ...), Section 2.5.3).
B.5 From design to model structure
Which random effects and correlation structures a design calls for; for the syntax, see the sections on random effects (Section B.6) and correlation structures (Section B.7). For how to derive the random effects from the study design, see Tip 4.1.
| Data structure | Model component | Section |
|---|---|---|
| Several observations per site, plot or individual | random intercept for site | Section 4.2.1 |
| The effect of a predictor varies between groups | random slope of x for site | Section 4.2.2 |
| Groups within groups (plots within sites) | nested random effects (plot within site) | Tip 4.2 |
| Two independent grouping factors (sites and observers) | crossed random effects (site and observer) | Section 4.2.3.1 |
| A grouping factor with fewer than about 5-10 levels (e.g. 4 years) | rule of thumb: fixed effect | Tip 4.1 |
| … and you are interested in the population-level effect of a predictor | a random effect can also work; after a singular fit, first remove the correlation (||), then switch to a fixed effect |
Section 4.3.3 |
| Unequal effort, area or duration (counts) | offset, offset(log(effort)) |
Section 9.2.1.1 |
| Repeated measurements over time | trend in time, then possibly a temporal correlation structure | Section 11.2 |
| Spatial coordinates | spatial trend, then possibly a spatial correlation structure | Section 11.3 |
| Species related by a phylogeny | phylogenetic correlation structure | Section 11.4 |
B.6 Random effect syntax
The following formulas are used by many R packages (e.g. glmmTMB and lme4), explained in the chapter on mixed models (Section 4.2.3.1, Tip 4.1). For a detailed overview, see Ben Bolker’s glmmFAQ.
| Formula | Interpretation | Section |
|---|---|---|
(1|group) |
Random intercept within group | Section 4.2.1 |
(x|group) |
Random intercept within group and random slope for x within group; correlation between random intercept and random slope | Section 4.2.2 |
(0+x|group) or (x-1|group) |
Random slope for x within group without random intercept for group | Section 4.2.2 |
(0+x|group) + (1|group) or (x||group) |
Random intercept and slope but uncorrelated | Section 4.2.2 |
(1|group/site) |
Same as (1|group) + (1|group:site): nested random effect (sites within group) |
Tip 4.2 |
(1|group:site) |
Only the group-by-site intercepts, without a group-level intercept | Tip 4.2 |
(1|group) + (1|site) |
Random intercept within groups and within sites (crossed random effects) | Section 4.2.3.1 |
B.7 Correlation structures
Correlation structures are explained in Chapter 11 (temporal: Section 11.2, spatial: Section 11.3, phylogenetic: Section 11.4).
B.7.1 glmmTMB
glmmTMB supports several correlation structures; the following table shows two common ones. See also the glmmTMB vignette about correlation structures.
| Type | Code | Interpretation | Section |
|---|---|---|---|
| Spatial | data$coords = numFactor(data$lon, data$lat); data$group = factor(1); glmmTMB(y ~ x + exp(0 + coords | group), data = data) |
random intercepts per location with exponentially decaying covariance (geostatistical); group = factor(1) if there is one spatial field |
Section 11.3 |
| Temporal | data$yearF = factor(data$year); glmmTMB(y ~ x + ar1(0 + yearF | group), data = data) |
random intercepts per time point (time must be a factor) with AR(1) correlation within each group; group is required, use data$group = factor(1) for a single time series |
Section 11.2 |
B.7.2 lme4
lme4 has no correlation structures for residuals (use glmmTMB, or nlme for linear models).
B.7.3 nlme::gls
nlme::gls.
| Type | Formula | Interpretation | Section |
|---|---|---|---|
| Spatial | gls(y~.., correlation = corExp(form = ~ Lat + Long|group)) |
Spatial correlation structure (no random effects) between sites within group (group is optional) | Section 11.3 |
| Temporal | gls(y~.., correlation = corAR1(form = ~ years | group)) |
Temporal correlation structure (no random effects) between years within group (group is optional) | Section 11.2 |
| Phylogeny | library(ape); gls(y~.., correlation = corBrownian(phy = phyTree, form = ~ species | group)) |
Phylogenetic correlation structure (no random effects) between species within group (group is optional) | Section 11.4 |
B.8 Dispersion, zero-inflation and offsets
| What | Syntax | Section |
|---|---|---|
| Offset for effort or area (log link) | glm(y ~ x + offset(log(effort)), family = poisson) |
Section 9.2.1.1 |
| Exposure time for binary data | glm(y ~ x + offset(log(time)), family = binomial(link = "cloglog")) |
Section 9.2.2.1 |
| Observation-level random effect (overdispersion) | data$obs = factor(1:nrow(data)), then glmer(y ~ x + (1|obs), family = poisson, data = data) |
Section 9.3.2 |
| Zero-inflation | glmmTMB(y ~ x, ziformula = ~ 1, family = poisson) (or ziformula = ~ x) |
Section 9.3.3 |
| Dispersion depending on predictors (glmmTMB) | glmmTMB(y ~ x, dispformula = ~ group) |
Section 10.1.2, Section 10.2 |
| Variance depending on predictors (gls) | gls(y ~ x, weights = varIdent(form = ~ 1 | group)) |
Section 10.1.2 |
B.9 Residual checks with DHARMa
If the residual checks show a problem, work through the following table roughly in this order: a wrong functional form or distribution can create apparent dispersion or correlation problems (see Section 12.3).
| What you see | Likely cause | What to do | Section |
|---|---|---|---|
| Residuals against a predictor show a pattern | wrong functional form | quadratic terms, interactions or splines (GAM) | Section 2.5, Section 2.5.3 |
| Dispersion test significant, ratio > 1 (counts, k/n data) | overdispersion | first check for model misfit and excess zeros; then a negative binomial (counts) or beta-binomial (k/n) family, if needed with a dispersion model; alternative: observation-level random effect; quasi families are discouraged | Section 9.3.2 |
| Spread of residuals changes with a predictor or the fitted values | heteroskedasticity | model the variance (gls weights, glmmTMB dispformula) |
Chapter 10 |
| More zeros than expected under the fitted model | zero-inflation | zero-inflation or hurdle model, compared with the model without via AIC or LRT | Section 9.3.3 |
| Residuals correlated in time, space or phylogeny | misfit trend or real autocorrelation | first remove the trend, then add a correlation structure if autocorrelation remains | Section 11.1.1 |
| Random effect estimates not normal, or correlated with predictors | misfit at the group level | check per group, add predictors or random slopes | Section 4.4.3, Section 4.4.4 |
| Single extreme observations | errors or outliers | check the data; robust regression | Section 10.3 |
| Convergence warnings, singular fit | model too complex for the data | if the warning mentions scales: center and scale predictors; otherwise change the optimizer or simplify the random effects | Section 4.4.1 |
The DHARMa functions for these checks:
| What | Syntax | Section |
|---|---|---|
| Simulate residuals and plot them | res = simulateResiduals(fit, plot = TRUE) |
Section 2.4.2, Section 9.3.1 |
| Residuals against a predictor | plotResiduals(res, form = data$x) |
Section 9.3.1 |
| Dispersion | testDispersion(res) |
Section 9.3.2.1 |
| Zero-inflation | testZeroInflation(res) |
Section 9.3.3.1 |
| Residuals aggregated per group (e.g. per year or site) | res2 = recalculateResiduals(res, group = data$year) |
Section 11.2, Section 11.2.1 |
| Temporal autocorrelation | testTemporalAutocorrelation(res2, time = unique(data$year)) |
Section 11.2 |
| Spatial autocorrelation | testSpatialAutocorrelation(res, x = data$x, y = data$y) |
Section 11.3 |
| Re-simulate the random effects (default: conditional on the fitted ones) | simulateResiduals(fit, simulateREs = "unconditional") |
Tip 4.4, Section 11.2 |
| Simulated likelihood ratio test | simulateLRT(m0, m1) |
Section 4.5.3.3 |
B.10 Tests, effects and predictions
| Model | Tests for single terms | Confidence intervals | Section |
|---|---|---|---|
lm, gls |
t-tests (summary); lm: F-tests (car::Anova; anova is sequential); gls: Wald \(\chi^2\) tests (car::Anova) or sequential F-tests (anova) |
confint() |
Section 2.2, Chapter 3 |
glm |
z-tests (summary; t-tests if the dispersion is estimated, e.g. Gamma), likelihood-ratio tests (car::Anova; anova(..., test = "Chisq") is sequential) |
confint() (profile) |
Section 9.2 (caveats: Caution 9.1) |
lmer |
t- and F-tests with approximate degrees of freedom (lmerTest), likelihood-ratio tests |
confint() (profile), parametric bootstrap (bootMer) |
Section 4.5.1, Section 4.5.3 |
glmer, glmmTMB |
z-tests, Wald tests (car::Anova), likelihood-ratio tests; for random effects simulated LRTs |
confint() (glmer: profile; glmmTMB: Wald by default) |
Section 4.5.3, Section 4.5.3.3, Section 9.5.2 |
| Comparisons between groups | emmeans with pairs() or contrast() (Tukey or Dunnett adjustment) |
emmeans |
Section 2.6.3, Note 3.1 |
Functions for these tests, effects and predictions:
| What | Syntax | Section |
|---|---|---|
| ANOVA table (type II / III) | car::Anova(fit), car::Anova(fit, type = "III") |
Section 3.2.3, Section 3.2.4 |
| Likelihood ratio test of nested models | anova(m0, m1) |
Section 8.2.1 |
p-values for lmer models |
load lmerTest before fitting |
Section 4.5.1 |
| Confidence intervals of parameters | confint(fit) |
Table B.12 |
| Group means and comparisons | emm = emmeans(fit, ~ treatment), then pairs(emm) or contrast(emm, "trt.vs.ctrl") |
Section 2.6.3, Note 3.1 |
| Means on the response scale (GLMs) | emmeans(fit, ~ treatment, type = "response") |
Caution 9.1 |
| Predictions with standard errors | predict(fit, newdata = newData, se.fit = TRUE) |
Section 2.11 |
| Conditional / unconditional predictions of mixed models | predict(fit, re.form = NULL) / predict(fit, re.form = NA) |
Tip 4.3, Section 4.5.2 |
| Confidence intervals for predictions of mixed models | bootMer(fit, FUN = function(m) predict(m, re.form = NA), nsim = 100) |
Section 4.5.2 |
| AIC comparison | AIC(m1, m2) |
Section 8.2.2 |
| Random effect variances and variance partitioning | VarCorr(fit) |
Section 4.5.5 |
| Marginal and conditional R² of a mixed model | MuMIn::r.squaredGLMM(fit) |
Section 4.5.4 |