Diagnose a Weak Design
Work from failure mechanism to remedy.
| Symptom | Likely issue | Action |
|---|---|---|
| Poor pre-period fit | Donors cannot reproduce treated dynamics | Revisit eligibility, geography level, outcome, and comparable history. |
| Coarse p-value granularity | Too few effective placebo assignments | Add eligible controls or reduce treated count only if the design permits it. |
| Dominant fitted donor weight | Counterfactual depends on one geography | Audit that donor and run exclusion sensitivity. |
| High power failure rate | SparseSC or DGP instability | Inspect exceptions, rank, panel completeness, and placebo feasibility. |
| High grid-based MDE | Noise or weak counterfactual relative to plausible lift | Redesign duration, outcome, treated scale, or method before launch. |
| Interference signal | Outcome changes among controls are compatible with contamination | Investigate exposure using campaign and geographic evidence; the screen is not causal proof. |
Result partial | Missing uncertainty or warnings | Resolve the stated fields before decision use. |
Do not tune exclusions, dates, or donors by searching for significance. If the remedy changes the design, rerun power and record the new specification.