Validate and Plot Microdata Fusion Results

validate() performs internal validation analyses on fused microdata to assess how well synthetic (fused) variables reproduce the statistical relationships, point estimates, and sampling uncertainties present in observed donor microdata across various population subsets.

plot_valid() generates conditional performance graphics and diagnostic plots from the results returned by validate(), with options to display overall summary curves or save plots directly to disk.

Usage

validate(
  observed,
  implicates,
  subset_vars,
  weight = NULL,
  min_size = 30,
  plot = TRUE,
  cores = 1
)

Arguments

observed: Data frame or data.table. The observed donor data used to train the underlying fusion model. Must contain the subset variables and weighting column.
implicates: Data frame or data.table. Implicates of synthetic (fused) variables generated by fuse. Implicates must be row-stacked and identified by an integer column M.
subset_vars: Character vector. Names of columns in observed used to partition the dataset into evaluation subsets. Subsets are formed from main effects and two-way interactions. Continuous or ordered factor variables in subset_vars are automatically binned into 5-level ordered factors via 1D k-means clustering.
weight: Character. Column name of sample weights in observed. If NULL (default), uniform unit weights are assumed.
min_size: Integer. Minimum required sample size for a subset to be included in evaluation (default is 30). Subsets with fewer observations are excluded.
plot: Logical. If TRUE (default), plot_valid is called automatically inside validate() to produce summary diagnostics.
cores: Integer. Number of CPU cores for parallel execution (Unix/Linux systems). Default is 1.
valid: An object of class validate returned by a call to validate().
y: Character vector. Optional subset of fusion variable names to process or plot. If NULL (default), all fusion variables present in valid are included.
path: Character. Directory path where .png graphic files will be written. If NULL (default), graphics are returned in memory without writing to disk.
...: Additional arguments passed to ggsave when saving plots to disk (e.g., width, height, dpi).

Value

If plot = FALSE, validate() returns a data frame of class validate containing row-level comparison metrics across evaluated subsets.

If plot = TRUE (or when invoking plot_valid()), a list of class validate is returned containing:

plots: A nested list of ggplot objects. Contains overall comparative plots (est, vad, moe) as well as variable-specific scatter/trend diagnostic panels.
perf: A data frame summarizing mean smoothed error metrics across subset ranges.
smooth: A data.table containing conditional quantile smoothing curve coordinates.
data: A data frame of raw validation subset calculations.

Details

The primary goal of validate() is to verify the scientific utility and empirical accuracy of synthetic variables generated through microdata fusion. Validation compares point estimates (means and proportions) and confidence interval bounds between synthetic multiple implicates and observed donor data across population subsets of varying granularities.

For categorical variables, factor levels are one-hot encoded into binary indicators prior to evaluation. Point estimates and Rubin's rules-based standard errors are calculated for each subset using analyze-style operations.

Diagnostic Metrics in plot_valid():

Point Estimate Error (est): Absolute percent error comparing simulated to observed point estimates.
Value Added (vad): Performance relative to a naive baseline estimate (overall sample mean).
Relative Uncertainty (moe): Ratio of simulated-to-observed margins of error (90\

Conditional quantile smoothing (using quantile regression over subset proportions) is applied to display trend lines illustrating expected accuracy across small to large subsets.

Examples

if (FALSE) { # \dontrun{
# Build a fusion model using RECS microdata
fusion.vars <- c("electricity", "natural_gas", "aircon")
predictor.vars <- names(recs)[2:12]
fsn.path <- train(data = recs,
                  y = fusion.vars,
                  x = predictor.vars,
                  weight = "weight")

# Fuse back onto donor data (generating 20 implicates)
sim <- fuse(data = recs,
            fsn = fsn.path,
            M = 20)

# Perform validation across population subsets
valid <- validate(observed = recs,
                  implicates = sim,
                  subset_vars = c("income", "education", "race", "urban_rural"),
                  weight = "weight")

# Inspect multi-variable summary diagnostics
valid$plots$est
valid$plots$moe

# Inspect single-variable diagnostic panel
valid$plots$electricity$est

# Re-run plot generation separately to save PNG images to disk
valid_saved <- plot_valid(valid,
                          path = file.path(getwd(), "validation_plots"),
                          width = 8, height = 6)
} # }