miceFast package for fast multiple imputations.

Description

Fast imputations under the object-oriented programming paradigm. There was used quantitative models with a closed-form solution. Thus package is based on linear algebra operations. The biggest improvement in time performance could be achieve for a calculation where a grouping variable have to be used. A single evaluation of a quantitative model for the multiple imputations is another major enhancement. Moreover there are offered a few functions built to work with popular R packages such as 'data.table'.

Details

Please read the vignette for additional information

Author(s)

Maciej Nasinski

References

https://github.com/Polkas/miceFast

Class "Rcpp_corrData"

Description

This C++ class could be used to build a corrData object by invoking new(corrData,...) function.

Methods

initialize(...):

~~

finalize():

~~

fill(...):

generating data

Note

This is only frame for building C++ object which could be used to implement certain methods. Check the vignette for more details of implementing methods.

Vigniette: https://CRAN.R-project.org/package=miceFast

References

See the documentation for Rcpp modules for more details of how this class was built. vignette("Rcpp-modules", package = "Rcpp")

Examples

#showClass("Rcpp_corrData")
show(corrData)

Class "Rcpp_miceFast"

Description

This C++ class could be used to build a miceFast objects by invoking new(miceFast) function.

Methods

set_data(...):

providing data by a reference - a numeric matrix

set_g(...):

providing a grouping variable by a reference - a numeric vector WITOUT NA values - positive values

set_w(...):

providing a weightinh variable by a reference - a numeric vector WITOUT NA values - positive values

set_ridge(...):

providing a ridge i.e. the disturbance to diag of XX, default 1e-6

get_data(...):

retrieving the data

get_w(...):

retrieving the weighting variable

get_g(...):

retireiving the grouping variable

get_ridge(...):

retireiving the ridge disturbance

get_index(...):

getting the index

impute(...):

impute data under characterstics from the object like a optional grouping or weighting variable

impute_N(...):

multiple imputations - impute data under characterstics from the object like a optional grouping or weighting variable

update_var(...):

permanently update the variable at the object and data. Use it only if you are sure about model parameters

get_models(...):

get possible quantitative models for a certain type of dependent variable

get_model(...):

get a recommended quantitative model for a certain type of dependent variable

which_updated(...):

which variables at the object was modified by update_var

sort_byg(...):

sort data by the grouping variable

is_sorted_byg(...):

check if data is sorted by the grouping variable

vifs(...):

Variance inflation factors (VIF) - helps to check when the predictor variables are not linearly related

initialize(...):

...

finalize():

...

Note

This is only frame for building C++ object which could be used to implement certain methods. Check the vignette for more details of implementing these methods.

Vigniette: https://CRAN.R-project.org/package=miceFast

References

See the documentation for Rcpp modules for more details of how this class was built. vignette("Rcpp-modules", package = "Rcpp")

Examples

#showClass("Rcpp_miceFast")
show(miceFast)
new(miceFast)

VIF function for assessing VIF.

Description

VIF measure how much the variance of the estimated regression coefficients are inflated. It helps to identify when the predictor variables are linearly related. You have to decide which variable should be delete. Usually values higher than 10 (around), mean a collinearity problem.

Usage

VIF(x, posit_y, posit_x, correct = FALSE)

## S3 method for class 'data.frame'
VIF(x, posit_y, posit_x, correct = FALSE)

## S3 method for class 'data.table'
VIF(x, posit_y, posit_x, correct = FALSE)

## S3 method for class 'matrix'
VIF(x, posit_y, posit_x, correct = FALSE)

Arguments

Value

load a numeric vector with VIF for all variables provided by posit_x

Methods (by class)

• VIF(data.frame):

• VIF(data.table):

• VIF(matrix):

Note

The corrected VIF is obtained by raising the basic VIF to the power of one divided by two times the degrees of freedom.

See Also

fill_NA fill_NA_N

Examples

## Not run: 
library(miceFast)
library(data.table)

airquality2 <- airquality
airquality2$Temp2 <- airquality2$Temp**2
airquality2$Month <- factor(airquality2$Month)
data_DT <- data.table(airquality2)
data_DT[, .(vifs = VIF(
  x = .SD,
  posit_y = "Ozone",
  posit_x = c("Solar.R", "Wind", "Temp", "Month", "Day", "Temp2"),
  correct = FALSE
))][["vifs.V1"]]

data_DT[, .(vifs = VIF(
  x = .SD,
  posit_y = 1,
  posit_x = c(2, 3, 4, 5, 6, 7),
  correct = TRUE
))][["vifs.V1"]]

## End(Not run)

airquality dataset with additional variables

Description

airquality dataset with additional variables

Usage

air_miss

Format

A data frame and data table with 154 observations on 11 variables.

Ozone

numeric Ozone (ppb) - Mean ozone in parts per billion from 1300 to 1500 hours at Roosevelt Island

Solar.R

numeric Solar R (lang) - Solar radiation in Langleys in the frequency band 4000–7700 Angstroms from 0800 to 1200 hours at Central Park

Wind

numeric Wind (mph) - Average wind speed in miles per hour at 0700 and 1000 hours at LaGuardia Airport

Temp

numeric Temperature (degrees F) - Maximum daily temperature in degrees Fahrenheit at La Guardia Airport.

Day

numeric Day of month (1–31)

Intercept

numeric a constant

index

numeric id

weights

numeric positive values weights

groups

factor Month (1–12)

x_character

character discrete version of Solar.R (5-levels)

Ozone_chac

character discrete version of Ozone (7-levels)

Ozone_f

factor discrete version of Ozone (7-levels)

Ozone_high

logical Ozone higher than its mean

Details

Daily readings of the following air quality values for May 1, 1973 (a Tuesday) to September 30, 1973.

Source

The data were obtained from the New York State Department of Conservation (ozone data) and the National Weather Service (meteorological data).

References

Chambers, J. M., Cleveland, W. S., Kleiner, B. and Tukey, P. A. (1983) Graphical Methods for Data Analysis. Belmont, CA: Wadsworth.

Examples

## Not run: 
library(data.table)
data(airquality)
data <- cbind(as.matrix(airquality[, -5]),
  Intercept = 1, index = 1:nrow(airquality),
  # a numeric vector - positive values
  weights = rnorm(nrow(airquality), 1, 0.01),
  # months as groups
  groups = airquality[, 5]
)

# data.table
air_miss <- data.table(data)
air_miss$groups <- factor(air_miss$groups)

# Distribution of Ozone - close to log-normal
# hist(air_miss$Ozone)

# Additional vars
# Make a character variable to show package capabilities
air_miss$x_character <- as.character(cut(air_miss$Solar.R, seq(0, 350, 70)))
# Discrete version of dependent variable
air_miss$Ozone_chac <- as.character(cut(air_miss$Ozone, seq(0, 160, 20)))
air_miss$Ozone_f <- cut(air_miss$Ozone, seq(0, 160, 20))
air_miss$Ozone_high <- air_miss$Ozone > mean(air_miss$Ozone, na.rm = T)

## End(Not run)

Comparing imputations and original data distributions

Description

ggplot2 visualization to support which imputation method to choose

Usage

compare_imp(df, origin, target)

Arguments

Value

ggplot2 object

Examples

library(miceFast)
library(ggplot2)
data(air_miss)
air_miss$Ozone_imp <- fill_NA(
  x = air_miss,
  model = "lm_bayes",
  posit_y = 1,
  posit_x = c(4, 6),
  logreg = TRUE
)
air_miss$Ozone_imp2 <- fill_NA_N(
  x = air_miss,
  model = "pmm",
  posit_y = 1,
  posit_x = c(4, 6),
  logreg = TRUE
)

compare_imp(air_miss, origin = "Ozone", "Ozone_imp")
compare_imp(air_miss, origin = "Ozone", c("Ozone_imp", "Ozone_imp2"))

fill_NA function for the imputations purpose.

Description

Regular imputations to fill the missing data. Non missing independent variables are used to approximate a missing observations for a dependent variable. Quantitative models were built under Rcpp packages and the C++ library Armadillo.

Usage

fill_NA(x, model, posit_y, posit_x, w = NULL, logreg = FALSE, ridge = 1e-06)

## S3 method for class 'data.frame'
fill_NA(x, model, posit_y, posit_x, w = NULL, logreg = FALSE, ridge = 1e-06)

## S3 method for class 'data.table'
fill_NA(x, model, posit_y, posit_x, w = NULL, logreg = FALSE, ridge = 1e-06)

## S3 method for class 'matrix'
fill_NA(x, model, posit_y, posit_x, w = NULL, logreg = FALSE, ridge = 1e-06)

Arguments

Value

load imputations in a numeric/logical/character/factor (similar to the input type) vector format

Methods (by class)

• fill_NA(data.frame): S3 method for data.frame

• fill_NA(data.table): s3 method for data.table

• fill_NA(matrix): S3 method for matrix

Note

There is assumed that users add the intercept by their own. The miceFast module provides the most efficient environment, the second recommended option is to use data.table and the numeric matrix data type. The lda model is assessed only if there are more than 15 complete observations and for the lms models if number of independent variables is smaller than number of observations.

See Also

fill_NA_N VIF vignette("miceFast-intro", package = "miceFast")

Examples

library(miceFast)
library(dplyr)
library(data.table)

data(air_miss)

# dplyr: continuous variable with Bayesian linear model
air_miss %>%
  mutate(Ozone_imp = fill_NA(
    x = ., model = "lm_bayes",
    posit_y = "Ozone", posit_x = c("Solar.R", "Wind", "Temp")
  ))

# dplyr: categorical variable with LDA
air_miss %>%
  mutate(x_char_imp = fill_NA(
    x = ., model = "lda",
    posit_y = "x_character", posit_x = c("Wind", "Temp")
  ))

# dplyr: grouped imputation with weights
air_miss %>%
  group_by(groups) %>%
  do(mutate(., Solar_R_imp = fill_NA(
    x = ., model = "lm_pred",
    posit_y = "Solar.R",
    posit_x = c("Wind", "Temp", "Intercept"),
    w = .[["weights"]]
  ))) %>%
  ungroup()

# data.table
data(air_miss)
setDT(air_miss)
air_miss[, Ozone_imp := fill_NA(
  x = .SD, model = "lm_bayes",
  posit_y = "Ozone", posit_x = c("Solar.R", "Wind", "Temp")
)]

# data.table: grouped
air_miss[, Solar_R_imp := fill_NA(
  x = .SD, model = "lm_pred",
  posit_y = "Solar.R",
  posit_x = c("Wind", "Temp", "Intercept"),
  w = .SD[["weights"]]
), by = .(groups)]

# See the vignette for full examples:
# vignette("miceFast-intro", package = "miceFast")

fill_NA_N function for the multiple imputations purpose

Description

Multiple imputations to fill the missing data. Non missing independent variables are used to approximate a missing observations for a dependent variable. Quantitative models were built under Rcpp packages and the C++ library Armadillo.

Usage

fill_NA_N(
  x,
  model,
  posit_y,
  posit_x,
  w = NULL,
  logreg = FALSE,
  k = 10,
  ridge = 1e-06
)

## S3 method for class 'data.frame'
fill_NA_N(
  x,
  model,
  posit_y,
  posit_x,
  w = NULL,
  logreg = FALSE,
  k = 10,
  ridge = 1e-06
)

## S3 method for class 'data.table'
fill_NA_N(
  x,
  model,
  posit_y,
  posit_x,
  w = NULL,
  logreg = FALSE,
  k = 10,
  ridge = 1e-06
)

## S3 method for class 'matrix'
fill_NA_N(
  x,
  model,
  posit_y,
  posit_x,
  w = NULL,
  logreg = FALSE,
  k = 10,
  ridge = 1e-06
)

Arguments

Value

load imputations in a numeric/character/factor (similar to the input type) vector format

Methods (by class)

• fill_NA_N(data.frame): s3 method for data.frame

• fill_NA_N(data.table): S3 method for data.table

• fill_NA_N(matrix): S3 method for matrix

Note

It is assumed that users add the intercept column themselves. The miceFast module provides the most efficient environment; the second recommended option is data.table with a numeric matrix. Only "lm_bayes", "lm_noise", and "pmm" models are supported. The model is fitted only when the number of complete observations exceeds the number of independent variables.

See Also

fill_NA VIF vignette("miceFast-intro", package = "miceFast")

Examples

library(miceFast)
library(dplyr)
library(data.table)

data(air_miss)

# dplyr: PMM with 20 draws
air_miss %>%
  mutate(Ozone_pmm = fill_NA_N(
    x = ., model = "pmm",
    posit_y = "Ozone", posit_x = c("Solar.R", "Wind", "Temp"),
    k = 20
  ))

# dplyr: lm_noise with weights
air_miss %>%
  mutate(Ozone_imp = fill_NA_N(
    x = ., model = "lm_noise",
    posit_y = "Ozone",
    posit_x = c("Solar.R", "Wind", "Temp"),
    w = .[["weights"]],
    logreg = TRUE, k = 30
  ))

# data.table: PMM grouped
data(air_miss)
setDT(air_miss)
air_miss[, Ozone_pmm := fill_NA_N(
  x = .SD, model = "pmm",
  posit_y = "Ozone",
  posit_x = c("Wind", "Temp", "Intercept"),
  k = 20
), by = .(groups)]

# See the vignette for full examples:
# vignette("miceFast-intro", package = "miceFast")

naive_fill_NA function for the simple and automatic imputation

Description

Automatically fill the missing data with a simple imputation method, impute with sampling the non missing values. It is recommended to use this function for each categorical variable separately.

Usage

naive_fill_NA(x)

## S3 method for class 'data.frame'
naive_fill_NA(x)

## S3 method for class 'data.table'
naive_fill_NA(x)

## S3 method for class 'matrix'
naive_fill_NA(x)

Arguments

Value

object with a similar structure to the input but without missing values.

Methods (by class)

• naive_fill_NA(data.frame): S3 method for data.frame

• naive_fill_NA(data.table): S3 method for data.table

• naive_fill_NA(matrix): S3 method for matrix

Note

this is a very simple and fast solution but not recommended, for more complex solutions please check the vignette.

See Also

fill_NA fill_NA_N VIF

Examples

## Not run: 
library(miceFast)
data(air_miss)
naive_fill_NA(air_miss)
# Could be useful to run it separately for each group level
do.call(rbind, Map(naive_fill_NA, split(air_miss, air_miss$groups)))

## End(Not run)

Finding in random manner one of the k closest points in a certain vector for each value in a second vector

Description

This function uses pre-sorting of y and binary search to find one of the k closest values for each miss.

Usage

neibo(y, miss, k)

Arguments

Value

a numeric vector

Pool results from models fitted on multiply imputed datasets

Description

Combines parameter estimates and standard errors from models fitted on m multiply imputed datasets using Rubin's rules (Rubin, 1987). Degrees of freedom are adjusted using the Barnard-Rubin (1999) small-sample correction.

This function works with any fitted model that supports coef and vcov methods (e.g., lm, glm, survival::coxph, etc.).

Results are validated against pool from the mice package for lm, glm (logistic and Poisson), weighted regression, interactions, and varying numbers of imputations.

Usage

pool(fits, dfcom = NULL)

Arguments

Value

A data.frame with one row per parameter and columns:

term

Coefficient name.

m

Number of imputations.

estimate

Pooled estimate (average across m models).

std.error

Pooled standard error (sqrt(t)).

statistic

t-statistic (estimate / std.error).

p.value

Two-sided p-value from a t-distribution with df degrees of freedom.

df

Degrees of freedom (Barnard-Rubin adjusted).

riv

Relative increase in variance due to nonresponse: (1 + 1/m) * b / ubar.

lambda

Proportion of total variance attributable to missingness: (1 + 1/m) * b / t.

fmi

Fraction of missing information.

ubar

Within-imputation variance (average of the m variance estimates).

b

Between-imputation variance (variance of the m point estimates).

t

Total variance: ubar + (1 + 1/m) * b.

dfcom

Complete-data degrees of freedom used.

conf.low

Lower bound of the 95% confidence interval.

conf.high

Upper bound of the 95% confidence interval.

References

Rubin, D.B. (1987). Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons.

Barnard, J. and Rubin, D.B. (1999). Small-sample degrees of freedom with multiple imputation. Biometrika, 86(4), 948-955.

See Also

fill_NA fill_NA_N

Examples

library(miceFast)
set.seed(123)
data(air_miss)

# Step 1: Generate m = 5 completed datasets using fill_NA with a stochastic model
completed <- lapply(1:5, function(i) {
  dat <- air_miss
  dat$Ozone <- fill_NA(
    x = dat,
    model = "lm_bayes",
    posit_y = "Ozone",
    posit_x = c("Solar.R", "Wind", "Temp")
  )
  dat
})

# Step 2: Fit a model on each completed dataset
fits <- lapply(completed, function(d) {
  lm(Ozone ~ Solar.R + Wind + Temp, data = d)
})

# Step 3: Pool using Rubin's rules
pool(fits)

Print method for pooled MI results

Description

Prints a concise summary of pooled multiple imputation results, showing the key inference columns: estimate, std.error, statistic, df, and p.value.

Usage

## S3 method for class 'miceFast_pool'
print(x, ...)

Arguments

Value

Invisibly returns x.

Summary method for pooled MI results

Description

Displays the full pooling diagnostics including within- and between-imputation variance, relative increase in variance, proportion of variance due to missingness, fraction of missing information, and confidence intervals.

Usage

## S3 method for class 'miceFast_pool'
summary(object, ...)

Arguments

Value

Invisibly returns the full diagnostics data.frame.

upset plot for NA values

Description

wrapper around UpSetR::upset for vizualization of NA values

Visualization of set intersections using novel UpSet matrix design.

Usage

upset_NA(...)

Arguments

Details

Visualization of set data in the layout described by Lex and Gehlenborg in https://www.nature.com/articles/nmeth.3033. UpSet also allows for visualization of queries on intersections and elements, along with custom queries queries implemented using Hadley Wickham's apply function. To further analyze the data contained in the intersections, the user may select additional attribute plots to be displayed alongside the UpSet plot. The user also has the the ability to pass their own plots into the function to further analyze data belonging to queries of interest. Most aspects of the UpSet plot are customizable, allowing the user to select the plot that best suits their style. Depending on how the features are selected, UpSet can display between 25-65 sets and between 40-100 intersections.

Note

Data set must be formatted as described on the original UpSet github page: https://github.com/VCG/upset/wiki.

References

Lex et al. (2014). UpSet: Visualization of Intersecting Sets IEEE Transactions on Visualization and Computer Graphics (Proceedings of InfoVis 2014), vol 20, pp. 1983-1992, (2014).

Lex and Gehlenborg (2014). Points of view: Sets and intersections. Nature Methods 11, 779 (2014). https://www.nature.com/articles/nmeth.3033

Examples

if (requireNamespace("UpSetR", quietly = TRUE)) {
  library(UpSetR)
  upset_NA(airquality)
  upset_NA(air_miss, 6)
}