Approach

Before touching the data: literature scan. Then understanding, cleaning, and feature engineering for separate FY2023 inpatient and CY2023 outpatient tracks.

1. Literature survey — methodology baseline

This 2013–present body of work provides methodological context for the descriptive analyses we adapt and extend:

Full research notes: notebooks/PRIOR_WORK_RESEARCH.md in the project repo.

2. Data understanding

Two publicly-available CMS datasets are aggregated at provider × service level, but their annual periods differ: inpatient = FY2023 and outpatient = CY2023. They should not be described as a same-period cohort for causal or longitudinal interpretation.

Data-dictionary deep dive in src/config.py — the renamed schema is documented inline.

3. Cleaning choices

Cleaning happens in two stages: loading with explicit dtype handling, then explicit suppression flags:

Explicit string → numeric coercion

All columns read as strings first (dtype=str), then numerics coerced via pd.to_numeric(..., errors="coerce"). This ensures CMS suppression (blank cells when sample < threshold) becomes NaN rather than causing dtype inference to fail silently or rows to be dropped:

# From src/load.py
def load_inpatient() -> pd.DataFrame:
    df = pd.read_csv(DATA_RAW / "inpatient_2023.csv", dtype=str)
    df = df.rename(columns=COLUMN_RENAME)
    for col in INPATIENT_NUMERIC:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors="coerce")
    df = df.dropna(subset=["total_discharges", "avg_total_payment",
                           "avg_medicare_payment"])
    return df.reset_index(drop=True)

Suppression surfaced explicitly, never silently zeroed

Outpatient has 45.33% suppressed allowed-amount rows (52,664 of 116,182). Instead of imputing, we add explicit boolean flags so downstream analysis can choose deliberately:

# From src/clean.py
df["cost_suppressed"] = df["avg_allowed_amount"].isna()
df["outlier_suppressed"] = df["outlier_services"].isna()
if drop_suppressed:
    df = df[~df["cost_suppressed"]].copy()
return df.reset_index(drop=True)

Coverage counts (provider, APC availability) still use the full frame; only charge/payment-distribution analyses drop suppressed rows.

Column rename schema

The CMS column names are difficult to read at a glance — renamed for clarity and unified across both datasets:

4. Feature engineering

Per-row additions in src/features.py:

Geographic & stratification features

• census_region — state FIPS → 4-region census map (Northeast / Midwest / South / West)
• urban_rural — USDA RUCA code collapsed to Urban / Micropolitan / Rural / Unknown
• drg_mdc — DRG description → 24 local MDC-like buckets from 540 observed DRG codes; this is an exploratory text-derived grouping, not an official CMS crosswalk
• apc_family — APC description → 22 local clinical families from 72 observed APC codes; this is an exploratory text-derived grouping, not an official CMS crosswalk

Ratio-derived features

• charge_to_payment_ratio — a compatibility alias for denominator-specific submitted-charge ratios. Inpatient uses average total or Medicare payment depending on the analysis; outpatient uses average allowed amount. It is a descriptive proxy, not a cost, CCR, markup, or overcharging measure.
• provider_combined — one row per provider with inpatient and outpatient activity summarized. Full outpatient presence is tracked separately from cost-observed outpatient metrics via has_outpatient_full and has_outpatient_cost_observed; built via outer join on provider_ccn.

# From src/features.py — provider summaries joined
def provider_summary_inpatient(df):
    return df.groupby("provider_ccn", as_index=False).agg(
        inp_drgs_seen=("drg_code", "nunique"),
        inp_total_discharges=("total_discharges", "sum"),
        inp_avg_payment=("avg_total_payment", "mean"),
        # ... + median, charge ratio
    )

def build_provider_combined(inp_feat, out_feat, out_full_feat):
    inp_s = provider_summary_inpatient(inp_feat)
    out_s = provider_summary_outpatient(out_feat)
    # Full outpatient presence is joined separately from cost-observed metrics.
    return inp_s.merge(out_s, on="provider_ccn", how="outer")

DRG → MDC mapping (exploratory disclosure)

The mapping uses description keywords with a coarse numeric fallback because MS-DRG numbering is not strictly monotonic by clinical category. The generated validation artifact records coverage, known quarantined codes, and sanity examples. Known-invalid DRG families are excluded from predictor modeling; all remaining families are still exploratory text-derived features, not an official CMS DRG-to-MDC or APC crosswalk. Coverage is not mapping validity.

All cleaning & feature operations are reproducible via uv run python -m src.run_pipeline; cleaned parquet snapshots saved to data/processed/.

5. Reproducibility

The entire pipeline produces everything this site displays in one command:

uv sync
uv run python -m src.build_site

That command runs:

1. src.run_pipeline — load → clean → features → parquet snapshots + meta_summary.json
2. src.analysis_insights — Phase 3 → 5 JSONs in site/data/
3. src.analysis_outliers — Phase 4 → 6 JSONs
4. src.analysis_predictors — Phase 5 → predictors.json
5. src.build_site — generates this static site, including the code browser you're inside right now

Outputs land in data/processed/ (intermediate) and site/data/ (consumed by you). The narrative HTML you're reading is hand-written; the code browser is generated. See /code/ for every Python source file with syntax highlighting.

Audit artifacts: site/data/outlier_flags.json is a preview of 2,000 rows per dataset from full counts of 36,101 inpatient and 15,899 outpatient flags; outlier_cohort_comparison.json previews 50 rows per dataset from 315 and 64 service summaries. Complete portable tables are bundled under site/data/audit/, including outlier audits and provider-disjoint predictor validation files with matching JSON metadata. site/data/audit_manifest.json records source, code, dependency, denominator, mapping, artifact, and contract-test provenance; focused summaries are also available in mapping_validation.json, denominator_audit.json, maryland_sensitivity.json, and model_validation_manifest.json.