One output of the summary() function that we didn’t look at in detail so far is the missing observations. Let’s take this regression for example:
fit =lm(Ozone ~ Wind + Temp, data = airquality)summary(fit)
Call:
lm(formula = Ozone ~ Wind + Temp, data = airquality)
Residuals:
Min 1Q Median 3Q Max
-41.251 -13.695 -2.856 11.390 100.367
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -71.0332 23.5780 -3.013 0.0032 **
Wind -3.0555 0.6633 -4.607 1.08e-05 ***
Temp 1.8402 0.2500 7.362 3.15e-11 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 21.85 on 113 degrees of freedom
(37 observations deleted due to missingness)
Multiple R-squared: 0.5687, Adjusted R-squared: 0.5611
F-statistic: 74.5 on 2 and 113 DF, p-value: < 2.2e-16
The summary() alerts us that “37 observations deleted due to missingness”. So, R has apparently removed observations from the data, but why?
Note
Missing values in R are coded with the special type NA. You can check whether a single variable entry is NA via is.na(). More convenient, however, is to use the summary() function to get an overview about the NAs in your dataset.
The answer is that a standard regression model cannot deal with a missing value in either the response or the predictors. Missing values are very common, however, and to avoid that regressions stop with an error all the time, there is the global R option na.action, which we can see via
options("na.action")
$na.action
[1] "na.omit"
We see that the default in R is “na.omit”, which means that if there is an missing value (NA) in either the response or one of the predictors, the entire observation will simply be omitted. You can change way NAs are handled per model
fit =lm(Ozone ~ Wind + Temp, data = airquality, na.action ="na.fail")
or also globally (careful)
options(na.action ="na.fail")
Setting na.fail means that you will get an error if there are NAs in the data used for your regression. Why would we want this, if the lm conveniently removes all NA lines? Because na.omit is not always a good idea, or at least you want to think about whether it’s a good idea in your case. Possible reasons are:
If you run a model selection, na.omit could result in different models having different number of observations (as the inclusion of a variable with NAs would reduce the number of observations)
If you run a model with many variables, each of them with a number of NAs, you might lose a lot of observations and thus power. It may be better to perform an imputation (see below).
If your NAs are not missing at random, you might actually have a bias in your model if you simply omit
For all this reason, you should get a good overview about your NAs, and consider what to do with them.
6.0.2 Summarizing NAs
As mentioned, a quick way to get a summary about the NAs in your data is to run the summary() function, which will return the number of NAs per predictor. You can check this out via running:
summary(airquality)
The problem with the summary is that in regression models, an observation will be removed as soon as one of the predictors has an NA. It is therefore crucial in which combinations NAs occur - we need complete rows. Let’s visualize the position of NAs in the data
image(is.na(t(airquality)), axes = F)axis(3, at =seq(0,1, len =6), labels =colnames(airquality))
We see that NAs are in Ozone and Solar.R, and sometimes they’re together, sometimes they are separate. To check if you have a complete row, we can use the function complete.cases(), which returns a vector with T/F if a row is complete or not. Here, I check how many complete observations we have
sum(complete.cases(airquality))
To create a dataset with all observations that contain NAs removed, use
airquality[complete.cases(airquality), ]
Note that in general, you should only do this for the variables that you actually use in your regression, else you might remove more observations than needed. Either you select the columns by hand, or a small hack is to use the model.matrix, which allows you to copy / past the regression formula syntax
After having an overview about the NAs in your data, as second step in the analysis (or ideally already during the design of the experiment) is to ask yourself under which process the missingness was created. We distinguish three crucial classes of processes:
MCAR = missing completely at random
MAR = missing at random
MNAR = missing not at random
The easiest cases is arguably MCAR = missing completely at random - in this case, the occurrences of NAs in variable X are completely random, and do not correlate with either X or other variables. This occurs when the causes of NAs are completely unrelated to the variables in the data. The opposite case is MNAR = missing not at random - in this case, the occurrences of NAs in variable X depend on the (true) value of X and possibly from other values in the dataset. A mix is MAR = missing at random - in this case, the occurrences of NAs in variable X do not depend on X, but possibly on other variables. See also (Bhaskaran and Smeeth 2014).
6.0.4 Imputation
The main alternative to throwing out observations with NAs is filling them up. This process is known as imputation. Multiple imputation refers to the insight that imputing several times often improves the result. There are a number of packages that do imputations, for example missRanger
fit =lm(Ozone ~ Wind + Temp, data = airqualityImp)summary(fit)
Call:
lm(formula = Ozone ~ Wind + Temp, data = airqualityImp)
Residuals:
Min 1Q Median 3Q Max
-41.251 -13.695 -2.856 11.390 100.367
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -71.0332 23.5780 -3.013 0.0032 **
Wind -3.0555 0.6633 -4.607 1.08e-05 ***
Temp 1.8402 0.2500 7.362 3.15e-11 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 21.85 on 113 degrees of freedom
Multiple R-squared: 0.5687, Adjusted R-squared: 0.5611
F-statistic: 74.5 on 2 and 113 DF, p-value: < 2.2e-16
The p-values are smaller now, as one would expect, having more data. The obvious problem with this is that there is uncertainty with the imputed values, which is currently not accounted for. One option to do this is running a multiple imputation.
# run 20 imputationsairqualityMImp <-replicate(20, missRanger(airquality), simplify =FALSE)
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
iter 4
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
iter 4
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
Variables to impute: Solar.R
Variables used to impute: Ozone, Solar.R, Wind, Temp, Month, Day
iter 1
|
| | 0%
|
|======================================================================| 100%
iter 2
|
| | 0%
|
|======================================================================| 100%
iter 3
|
| | 0%
|
|======================================================================| 100%
# fit 20 modelsmodels <-lapply(airqualityMImp, function(x) lm(Ozone ~ Wind + Temp, x))# use mice package to compute corrected p-valuesrequire(mice)summary(pooled_fit <-pool(models))
Bhaskaran, Krishnan, and Liam Smeeth. 2014. “What Is the Difference Between Missing Completely at Random and Missing at Random?”International Journal of Epidemiology 43 (4): 1336–39.
