Appearance
Validation Methodology
Status: Active. Last updated April 12, 2026. Purpose: Repeatable process for validating AskFlorence pricing accuracy per state, per plan year.
Why We Validate
AskFlorence calculates subsidized health insurance prices from multiple data sources and formulas. Errors in any component (premiums, SLCSP, FPL values, contribution brackets, CSR cost-sharing) produce wrong prices shown to consumers. Validation catches these errors before they reach users.
When to Validate
- New state onboarded - full validation before launch
- Annual plan year refresh - re-validate when new year's data is loaded
- Formula change - when IRS contribution brackets, FPL values, or CSR rules change
- After code changes to subsidy calculation, plan search, or eligibility routing
Validation Sources (ranked by authority)
| Source | Authority | Coverage | Limitations |
|---|---|---|---|
| State marketplace (NYSOH, Covered CA) | Highest | State-specific, includes state programs | May be down, requires manual entry |
| Licensed broker | High | Real enrollment experience | Manual, time-consuming |
| KFF Subsidy Calculator | High | All states, independent | Doesn't show plan-level detail, just APTC |
| CMS API (federal states) | High | 30 federal states | Doesn't cover SBE states |
| Issuer SBC documents | High | Plan-specific cost-sharing | Per-plan, not aggregated |
| Our own cross-source validation | Medium | Catches data pipeline errors | Circular if formula is wrong |
Scenario Design
Every state validation must cover these categories:
Income Tiers
| Category | FPL Range | What to Verify |
|---|---|---|
| Medicaid threshold | Below 138% | Medicaid routing, adjustment logic for denied applicants |
| CSR 94% | 138-150% | $0 or near-$0 deductible, lowest copays, highest APTC |
| CSR 87% | 150-200% | Moderate deductible reduction, reduced copays |
| CSR 73% | 200-250% | Slight deductible reduction |
| APTC only | 250-400% | Standard cost-sharing, APTC still applies |
| No subsidy | Above 400% | Full sticker price, no APTC (post-IRA cliff) |
Household Compositions
| Type | What to Verify |
|---|---|
| Single adult | Base case |
| Married couple, no kids | Couple tier multiplier, 2-adult SLCSP |
| Single parent + children | 1-adult premium, children to CHP?, FPL household size |
| Married couple + children | 2-adult premium, children to CHP?, FPL household size |
| Older adults (55+) | Age rating (federal states) or community rating (NY) |
Geographic Coverage
| Type | What to Verify |
|---|---|
| Multiple rating regions | Different SLCSP per region |
| Multiple counties in same region | Same premiums, different plan availability |
| Border/edge zip codes | Correct county/region assignment |
State-Specific Programs
| Program | States | What to Verify |
|---|---|---|
| Essential Plan | NY | EP routing, tier assignment, EP/QHP boundary |
| CAPS/CAPC | CA | State premium subsidy on top of federal APTC |
| Medicaid expansion | All | Threshold calculation, auto-adjustment |
| State CSR enhancements | NY (diabetes, pregnancy) | Additional cost-sharing reductions |
What to Compare
For each scenario, record and compare:
| Field | Our Value | KFF / State / Broker Value | Acceptable Variance |
|---|---|---|---|
| FPL % | calculated | calculated | $0 (must match exactly) |
| APTC | calculated | from source | Within $5/month |
| CSR tier | 94/87/73/none | from source | Must match exactly |
| SLCSP (individual) | from DB | from state data | Within $1 |
| Household premium | calculated | from source | Within $2 |
| Deductible (CSR-adjusted) | from DB | from SBC/state | Must match exactly for standard plans |
| MOOP (CSR-adjusted) | from DB | from SBC/state | Must match exactly for standard plans |
| Key copays | from DB | from SBC | Must match for standard plans |
Running Validation
Automated Script
Each state has a validation script at scripts/validation/{state}-validation-scenarios.js.
bash
# Start dev server
npm run dev
# Run NY validation (15 scenarios)
node scripts/validation/ny-validation-scenarios.js
# Output goes to stdout — copy to validation docManual KFF Comparison
- Go to kff.org/interactive/subsidy-calculator
- Enter: State, Zip, Income, # Adults, # Children, Ages
- Record: APTC amount, SLCSP
- Compare against our script output
Broker Validation
Share the scenario output with a licensed broker. Key scenarios to prioritize:
- Medicaid-adjusted cases (KFF just says "Medicaid eligible")
- State program boundaries (EP threshold in NY, CAPS in CA)
- Non-standard plan cost-sharing
Documentation
Results are stored in docs/validation/{state}-{year}.md with:
- Test parameters for each scenario
- Our calculated values
- Comparison source values
- Variance analysis
- Pass/fail determination
- Date validated and by whom
Re-validation Schedule
| Trigger | Scope | Timeline |
|---|---|---|
| New plan year data loaded | Full re-run all scenarios | Within 1 week of data load |
| IRS bracket update | APTC scenarios only | Within 1 week of publication |
| FPL update | All scenarios (FPL% changes) | Within 1 week of publication |
| Code change to subsidy calc | Affected scenarios | Before merge |
| Quarterly spot-check | 3-5 scenarios per state | Quarterly |
