Generate Output Files Resulting from Fusion

Executes the model training, internal validation, and prediction steps required to fuse donor survey variables onto American Community Survey (ACS) recipient microdata. Expects preprocessed input files generated by fusionInput to exist on-disk.

fusionOutput_multiyear() is a convenience wrapper around fusionOutput() to iteratively run data fusion models across sequential ACS recipient years. The function automatically passes the .fsn model file generated by the prior year's run to accelerate variable selection and model training for subsequent years.

Usage

fusionOutput(
  donor,
  respondent,
  acs_year,
  fusion_vars,
  M = 1,
  test_mode = TRUE,
  input_version = "latest",
  output_version = "today",
  fsn = NULL,
  rlocation = NULL,
  note = NULL,
  validation = TRUE,
  ncores = getOption("fusionData.cores"),
  margin = 4,
  ...
)

fusionOutput_multiyear(acs_years, ...)

Arguments

donor: Character. Donor survey identifier and vintage (e.g., "RECS_2015" or "RECS_2020"). Must match the underscore-separated folder naming convention generated by fusionInput().
respondent: Character. Desired respondent level of microdata: either "household" (or "H") or "person" (or "P").
acs_year: Integer. Year of the ACS microdata to be used as recipient (e.g., 2015).
fusion_vars: Character vector or list. Variable names in the donor microdata to be fused to the ACS recipient microdata. Any fusion_vars used as predictors in the training data are automatically ignored.
M: Integer. Desired number of fusion implicates (simulated output datasets). Defaults to 1.
test_mode: Logical. If TRUE (default), output files are written to a /fusion_ directory rather than /fusion, and fast, light hyperparameter settings are passed to train.
input_version: Character. Version timestamp string (formatted as "YYYY-MM-DD") of the desired input folder in /input. Defaults to "latest".
output_version: Character. Version timestamp string (formatted as "YYYY-MM-DD") to assign to the /output folder. Defaults to "today".
fsn: Character. Optional file path to an existing trained model archive (.fsn) created by a previous call to fusionOutput(). If provided, the function attempts to bypass feature preparation (prepXY) or complete model training (train).
rlocation: Data frame. Optional data frame containing actual/disclosed respondent locations to override imputed locations in donor data.
note: Character. Optional user-supplied note to be recorded in the output log file for future reference.
validation: Logical. If TRUE (default), fuses implicates back onto original donor training data to generate internal validation files (valid.fsd).
ncores: Integer. Number of physical CPU cores used for parallel computation and file I/O threading. Defaults to 1.
margin: Numeric. Integer or numeric margin passed to fuse controlling memory-saving chunk boundaries during prediction. Defaults to 4.
...: Additional arguments passed to underlying modeling functions. For fusionOutput, arguments are passed to train. For fusionOutput_multiyear, arguments are passed to fusionOutput (and subsequently to train).
acs_years: Numeric vector. Sequence of ACS recipient years to process (e.g., 2019:2023). Must be provided in ascending numeric order.

Value

Invisibly returns a character string containing the absolute file path to the generated /output directory.

Details

fusionOutput() ties together the machine-learning pipeline provided by the fusionModel package. Given a valid setup created by fusionInput, fusionOutput() manages the complete modeling lifecycle:

Data Assembly: Loads harmonized donor predictors, donor target variables (fusion_vars), and spatial predictor variables matching the donor's survey vintage and geographic boundaries (PUMA 2010 vs. PUMA 2020).
Predictor Selection (prepXY): Evaluates feature correlations and screening thresholds via prepXY (or reuses structure from an existing .fsn file if fsn is specified).
Model Training (train): Trains gradient boosted decision trees via LightGBM using sensible default hyperparameters tuned for production or rapid test runs.
Validation Fusing (fuse): If validation = TRUE, fuses the target variables back onto the training data to create a validation dataset (.fsd) for downstream evaluation.
Recipient Prediction (fuse): Loads ACS microdata, merges spatial context predictors, and fuses M implicates of the target variables onto the recipient microdata.

Directory Structure & Outputs

Output files are stored in structured paths based on execution mode:

Test Mode: fusion_/[DONOR_NAME]/[DONOR_VINTAGE]/[ACS_YEAR]/output/[DATE]/
Production Mode: fusion/[DONOR_NAME]/[DONOR_VINTAGE]/[ACS_YEAR]/output/[DATE]/

Each run creates the following files in the target directory:

[DONOR]_[ACS_YEAR]_[TYPE]_prep.rds: Saved feature selection object from prepXY mapping target variables to screened predictors.
[DONOR]_[ACS_YEAR]_[TYPE]_model.fsn: Trained LightGBM model archive generated by train containing tree structures and feature metadata.
[DONOR]_[ACS_YEAR]_[TYPE]_valid.fsd: (Optional, created if validation = TRUE) Fused implicates simulated back onto original donor observations for possible internal validation by the user.
[DONOR]_[ACS_YEAR]_[TYPE]_fused.fsd: Fused microdata containing M simulated target implicates mapped onto the recipient ACS microdata.
[DONOR]_[ACS_YEAR]_[TYPE]_outputlog.txt: Execution log containing system details, arguments, CPU/memory stats, and timing summaries.
[DONOR]_[ACS_YEAR]_[TYPE]_outputlog0.txt: (Optional, created when reusing an existing .fsn) Retained console output from the original model training run.

Examples