Validate a Power Simulation

Treat a power run as a model requiring diagnostics.

Reproducibility

Repeat the same CPU run with the same seed and compare the CSVs. Record the package version, input hash, YAML, seed, and execution backends.

Monte Carlo precision

Use power_ci_lower and power_ci_upper. If the interval straddles the target power, increase simulations before making a boundary decision. The interval quantifies simulation-count uncertainty only; it does not include uncertainty about the DGP specification.

DGP plausibility

Review dgp_rank, dgp_explained_var, and dgp_factor_dynamics. Compare observed and simulated distributions outside GeoSC where the decision is material: unit means and variances, cross-unit correlations, autocorrelation, seasonal patterns, extremes, and zeros. The current DGP does not model calendar effects or heteroskedastic idiosyncratic noise explicitly.

Null calibration

Include a separate zero-effect run using effect_sizes: [0.0]. Its rejection proportion estimates the simulated false-positive rate under the fitted DGP. Compare its Monte Carlo interval with the declared alpha and investigate excess rejection before treating positive-effect power as decision evidence. Passing this check is conditional on the DGP; it does not validate an observational assignment or prove that the campaign is identified. Repeat across plausible pre-specified DGP choices when the decision warrants it.

Grid stability

Power should usually rise with larger injected effects in a well-resolved simulation, but finite Monte Carlo estimates can move up or down. Investigate large reversals, high failure rates, coarse placebo p-values, or sensitivity to rank, history window, seed, donor eligibility, and effect pattern.

Do not call the run validated merely because all rows are valid. That flag only checks that at least one simulation succeeded and that the failure rate did not exceed the configured threshold.