Glean Health -- Dev Status

Generated 2026-07-15 00:10:01 UTC · HEAD f59c4e1 · guards: all 9 green · source: scripts/build_dev_status.py

Headline metrics

26
Conditions
16,572
Total clusters
133,230
Total mentions
30,066
Authors
1,441
Cross-cond 2+
166
Cross-cond 3+
99.4%
Phase B coverage
What each number means
Condition
One chronic illness we mine (asthma, IBS, type 1 diabetes, ...). Each has a conditions/<name>.json config that names its Reddit sources.
Cluster
A group of treatment mentions referring to the same thing (e.g., humira + adalimumab + biosimilars collapse into one cluster). Built at Stage 7 via RxNorm canonicalization + agglomerative embedding clustering. The patient-facing name is the cluster's canonical_name.
Mention
One (post-or-comment, treatment) pair extracted by the LLM at Stage 4+5. A single post mentioning 3 treatments yields 3 mentions. The cluster's mention_count is the patient signal — "how many people talked about this?"
Author
Distinct Reddit handle across the entire corpus — the same user posting in r/ibs and r/migraines counts once. Authors are the unit for credibility scoring (Stage 6) and for cluster_breadth.
Cross-cond N+
Distinct authors who appear in N or more conditions. Tail of cross-cond 3+ is the meaningful population for credibility / cluster-poisoning analysis (someone who posts in 3+ unrelated illness subs gets the U-curve penalty).
Phase B coverage
Share of authors with the full Stage-6 feature set (account age, karma signal, condition-fit U-curve, commercial-link ratio, etc.), computed from cached cross-Reddit history. Phase A is the post-only fallback used when we don't have the history cached.

Guards (9)

Run at generation time against the current working tree. Click a guard’s script to see its source.
check_no_dup_diagramsOK
Diagram canonical-source invariant
data_lineage.mmd (verified copies: none; inline renderers: ['data_lineage.html', 'deploy\\data_lineage.html', 'deploy\\schema.html', 'schema.html'])
check_data_dictionaryOK
Schema CSV <-> HTML coupling
OK: CSV tables and HTML <h2 id> sections match exactly
check_live_countsOK
Numeric-prose drift (conditions/guards)
OK: PLAN.md Conditions Onboarded table row count matches pipeline-output count
check_architecture_docOK
ARCHITECTURE.md stage coupling
[PASS] all 2 docs fully coupled to codebase.
check_pipeline_integrityOK
Per-condition pipeline_output integrity
Summary: 26 clean, 0 with warnings, 0 failed (pubmed threshold: 50%, strict=False)
check_manifestOK
Manifest freshness (per-condition)
Summary: 26 fresh, 0 stale (checked 26)
check_physical_schemaOK
Physical SQLite schema <-> doc coupling
OK: snapshot DDL matches live web.db and glean.db
check_serving_freshnessOK
Serving layer current (Supabase vs pipeline)
OK: serving layer current with 26 conditions (last published 2026-07-15T00:07:08+00:00).
check_schema_htmlOK
schema.html prose claims <-> code coupling
OK: no forbidden stale strings (DuckDB / password-gated / runtime fetch)

Conditions (26)

FRESH/STALE compares each condition’s recorded features_sha against the current _features.json. STALE means build_pipeline needs to re-run.
ConditionItemsClustersAuthorsPhase BStatusLast build (UTC)Top 3 by cluster_weight
Adenomyosis83115346799.6%FRESH2026-06-02T21:47:18Zhysterectomy, mirena iud, mirena
Asthma1,34824068999.0%FRESH2026-06-02T21:48:02Zsymbicort, prednisone, montelukast
Chronic Pain5,9549392,48299.6%FRESH2026-06-02T21:50:58Zoxycodone, gabapentin, tylenol
Crohn's Disease3,6935501,81299.7%FRESH2026-06-02T21:51:47Zhumira, remicade, skyrizi
Eczema2,4907021,39799.6%FRESH2026-06-02T21:52:24Zdupixent, tacrolimus, rinvoq
Ehlers-Danlos Syndrome4,6389302,30599.4%FRESH2026-06-02T21:53:06Zphysical therapy, low-dose naltrexone (ldn), compression garments
Endometriosis6,6789003,51399.4%FRESH2026-06-02T21:54:03Zbirth control, laparoscopy, iud
Fibromyalgia4,1157752,02299.2%FRESH2026-06-02T21:54:44Zlyrica, cymbalta, gabapentin
Gastroparesis1,61643272099.4%FRESH2026-06-02T21:55:12Zreglan, zofran, domperidone
GERD2,3704971,29999.7%FRESH2026-06-02T21:55:48Zomeprazole, famotidine, pantoprazole
Hashimoto's Thyroiditis3,0894941,50299.3%FRESH2026-06-02T21:56:22Zlevothyroxine, tirzepatide, gluten-free diet
IBS2,8815831,66199.3%FRESH2026-06-02T21:56:48Zimodium, low fodmap diet, probiotics
Long COVID4,4778641,70799.5%FRESH2026-06-02T21:57:32Zldn (low-dose naltrexone), antihistamines, ldn
Lupus1,94135290399.6%FRESH2026-06-02T21:58:00Zhydroxychloroquine, prednisone, benlysta
Lyme Disease1,31331447799.4%FRESH2026-06-02T21:58:26Zdoxycycline, antibiotics, rifampin
MCAS5,3939521,85799.4%FRESH2026-06-02T21:59:19Zcromolyn, ketotifen, pepcid
ME/CFS6,2941,0171,94099.3%FRESH2026-06-02T21:50:01Zldn (low-dose naltrexone), pacing, ativan
Migraines9,2221,2704,51099.5%FRESH2026-06-02T21:59:51Znurtec, botox, sumatriptan
Multiple Sclerosis4,2414781,80799.2%FRESH2026-06-02T22:00:31Zkesimpta, ocrevus, steroids
PCOS6,0638303,43099.2%FRESH2026-06-02T22:01:25Zmetformin, inositol, birth control
POTS6,9241,0383,00999.5%FRESH2026-06-02T22:02:21Zivabradine, propranolol, midodrine
Psoriasis1,48731888399.3%FRESH2026-06-02T22:02:46Zskyrizi, biologics, methotrexate
Rheumatoid Arthritis1,77126883899.5%FRESH2026-06-02T22:03:10Zmethotrexate, prednisone, hydroxychloroquine
Type 1 Diabetes8,6708994,11599.3%FRESH2026-06-02T22:04:16Zcontinuous glucose monitor (cgm), insulin pump, metformin
Type 2 Diabetes1,76332287099.8%FRESH2026-06-02T22:04:46Zmetformin, mounjaro, ozempic
Ulcerative Colitis3,2024551,37199.2%FRESH2026-06-02T22:05:21Zmesalamine, prednisone, remicade

Recent decisions (last 8)

From PLAN.md Decisions Log, most recent first.
#54Supersede GEMINI "no symptom in UI" ban → 3 entry vectors
GEMINI Principle #2 banned "symptom" from patient UI (euphemize as Signals/Experiences); superseded because it wasn't guard-enforced, no patient code used a euphemism, and "symptom" is plain searchable English serving Discoverability (#4) better — the non-advice stance is carried by the glass-box signal band, not by avoiding the word. Edited `GEMINI.md` + `web/GEMINI.md` + `docs/PHYSICAL_SCHEMA.md`. Also retired Principle #1 "Conditions & Remedies Only" → **three entry vectors** (condition / remedy / symptom). Patient nav route will be `/symptom/[slug]` (2026-06-19)
#53Symptom normalization + three-layer model (PLAN #8 data layer)
The raw `symptoms_targeted[]` vocabulary (~33.6k free-text strings, 79% ≤2 mentions) was too noisy to navigate. `extract/normalize_symptoms.py` canonicalizes it via Haiku (`claude -p`, Max plan — no API key configured) into **7,515 labels** (`data/symptom_canonical_map.json`). **Key structural insight: target-type is contextual, not intrinsic** (the same string `mcas` is a degenerate self-reference inside the MCAS corpus but a real comorbid treatment-target inside POTS). Hence three layers: (L1) normalize = context-free, canonicalize + type (symptom/sign/condition kept; trigger/other dropped), NEVER drops a condition-name; (L2) `migrate` drops self-references (target == own condition, type=condition only) using condition context; (L3) frontend decides surfacing. Per-canonical type resolved deterministically (exact-match-raw wins). `type` column cascaded through glean.db/web.db/`physical_schema.sql`/`schema_postgres.sql`/`push_to_supabase.py`/`PHYSICAL_SCHEMA.md`/`ARCHITECTURE.md`. All 9 guards green (2026-06-19)
#52Warm-terminal reskin (90s-minimal)
Patient surface restyled to the "warm terminal" theme: monospace, greige ground, sage/ochre/clay earth tones (Jackson Browne palette). Token set + remap layer in `web/src/app/globals.css`; signal-band tags/stripes consume the same tokens. Live 2026-06-15 (commit `15e4a88`)
#51Cap condition page at top-50 remedies + Show-more
A condition can have 1,000+ treatments; mounting that many interactive cards buried the main thread (sluggish cursor, slow drilldowns). `web/src/app/condition/[id]/RemedyList.tsx`: progressive disclosure, `PAGE=50` by signal, reveal-on-demand. Full list held in memory (cheap JSON) so "Show more" is instant — only DOM/hydration cost is capped, the actual bottleneck. Deep links (Vector 2) beyond the first page widen the initial window so the `#tx-<id>` anchor + highlight still render (commit `c898588`, 2026-06-14)
#50Glass-box "signal band" for the patient surface
Replaces any implied efficacy score with a signal-of-credible-discussion band that executes the product thesis (conversation-starter, never a verdict). `web/src/lib/signal.ts`: `signalBand(cluster_weight)` → Strong (≥20) / Moderate (≥6) / Emerging tiers, each an ASCII bar + earth-tone tinted tag + left stripe; thresholds grounded in the real corpus distribution (16,572 treatments, p75≈6, p95≈22 → Strong≈top 5%, Moderate≈p75–p95). `researchContext(pubmed_count)` reframes PubMed coverage as a doctor-conversation cue (well/some/barely/not-yet studied), surfacing the "community ahead of the science" gap as honest context, not a headline. Applied to Vectors 1 & 2 (commit `5930db0`, 2026-06-14)
#49Replace magic-link auth with email + password
Magic-link UX confused testers and free-tier Supabase email (Yahoo spam, low rate limit) was a deliverability wall; bouncing between localhost and prod added redirect-allow-list friction. Swapped `signInWithOtp` → `signInWithPassword` (`web/src/app/login/page.tsx`) and deleted the now-dead `/auth/confirm` magic-link callback. Invite-only preserved via admin-provisioned accounts (Supabase Users, password + Auto Confirm) — no public sign-up, no email at login. Session/`proxy.ts` unchanged (same cookie flow). Magic-link prose swept from CLAUDE.md / ARCHITECTURE.md / OVERVIEW.md. Live 2026-06-14 (2026-06-14)
#48Close #46's deferred frontend-doc reconciliation (full audit pass)
Triggered by a full repo audit ("clean / coupled / referenced") run via parallel read-only subagents + the 8-guard sweep. Verdict: repo fundamentally healthy (tree clean, 8 guards <!-- count-ok --> green, `web/` app fully Postgres-migrated, no markers/conflicts/half-done code, PLAN/RUNBOOK hygienic); the remaining staleness was almost entirely #46's deferred bucket. Closed it: (1) **CLAUDE.md** — rewrote "Dev vs Patient Surfaces" as a PHYSICAL split (patient = `web/` Vercel app on Supabase; dev = Cloudflare static, bare domain redirects to `dev.html`); removed the 5 archived-stack rows from the Project Structure table; fixed the §"Coupled Artifacts" `deploy/index.html`→`deploy/pipeline.html` renderer refs. (2) **ARCHITECTURE.md** (~14 current-prose sites) + **OVERVIEW.md** reconciled by a subagent (dated change-log entries left as history); also fixed OVERVIEW §2.3 "Seven scripts"→"Eight" + added the `check_serving_freshness` row (a 7→8 gap `check_live_counts` can't see — "scripts" not "guards"). (3) **`build_dev_status.py`** dev-nav "Patient site → /" repointed to the Vercel app + JS active-tab default `index.html`→`dev.html`; propagated to the 6 committed dev pages via `inject_dev_nav.py` + regen. (4) **`extract/build_ui.py`** Project Files entry + Browse crumb repointed to the Vercel app (re-rendered all 26 dashboards to clear the resulting `check_manifest` STALE). Non-#46 audit finds also fixed: `PHYSICAL_SCHEMA.md` stale "app reads web.db" → loader; `.gitignore` 6.1→6.5 GB; 3 orphan root `test_*.py` → `tests/`; 4 info `console.log`s removed from `web/`; obsolete `decisions.yaml` TODO marked moot. NOTE: regenerating `corpus.html` for the nav also picked up an **overdue deterministic data refresh** (committed copy was stale vs current pipeline outputs: mentions 133190→133230, clusters 16568→16572) — legit, not guard-tracked. All 8 guards <!-- count-ok --> green (2026-06-12)
#47Harden the new-data → Supabase publish path (Guard 8)
Per the 2026-06-12 priority reset (pipeline preservation > frontend), closed the "memory-bridged gap": `run_condition.py` stops at the JSON pipeline outputs and never touches Supabase, so a finished run silently left the live app serving stale data. Three moves: (1) **`scripts/refresh_serving.py`** — one-command orchestrator for the serving chain (migrate → `build_web_db` → `push_to_supabase`), with fail-fast prechecks (pipeline outputs exist, `web/.env.local` present) and a prod-reload confirmation; on success writes `data/serving_manifest.json` (condition → sha256 of pipeline output; gitignored, machine-local). Flags: `--yes`, `--skip-rebuild`, `--record-only`. (2) **`scripts/check_serving_freshness.py`** — guard #8, compares the manifest to current pipeline outputs and flags STALE/NEW/REMOVED; wired into `dev_status.html` via `build_dev_status.py` GUARDS; verified it catches all three drift types (exit 1) and passes clean (exit 0). (3) **`docs/RUNBOOK.md`** — the single end-to-end "build half + publish half" runbook; `run_condition.py` now prints a "NEXT: publish" reminder on completion. Baselined the manifest via `--record-only` (Supabase already in sync — no pipeline re-runs since the last push). Guard count 7→8 (CLAUDE.md, OVERVIEW.md ×3, PLAN.md ×2 reconciled; `check_live_counts` enforces). This also resolved the onboarding-checklist item from #46's deferred list (CLAUDE.md §94 now points at `refresh_serving.py` + RUNBOOK and drops the archived `build_home.py`); the rest of #46's frontend reconciliation stays deferred (2026-06-12)

Open questions

Every unchecked [ ] from PLAN.md Open Questions.
(none open)

Active next actions

Non-strikethrough items from PLAN.md Next Actions.
(no active next actions parsed)

Recent commits (last 10)

f59c4e12026-07-14Audit cleanup: archive spike/, document bootstrap_proximity.py
acdc9ad2026-07-14Symptom page: swap unmapped text-amber-700 for semantic text-ochre token
70e35732026-06-23Ingest docs + proximity-map bootstrap tool
17d2b772026-06-19Symptom-first frontend: /symptom/[slug] route + search (PLAN #8)
d82d06c2026-06-19Symptom normalization: canonical vocabulary + three-layer model (PLAN #8 data layer)
ddecf002026-06-17Guard 9: check_schema_html.py — pin schema.html prose to code
f88b4f82026-06-17Docs: reconcile support files to code reality (audit fixes)
4fc7d842026-06-17Housekeeping: ignore .vercel, log frontend decisions #50-52, tick Gap-3
15e4a882026-06-15Reskin: warm-terminal theme (90s-minimal, Jackson Browne palette)
c8985882026-06-14Perf: cap condition page at top 50 remedies with Show-more