Glean Health -- Dev Status
Generated 2026-07-15 00:10:01 UTC
· HEAD f59c4e1
· guards: all 9 green
· source: scripts/build_dev_status.py
Headline metrics
What each number means
- Condition
- One chronic illness we mine (asthma, IBS, type 1 diabetes, ...). Each has a
conditions/<name>.json config that names its Reddit sources.
- Cluster
- A group of treatment mentions referring to the same thing (e.g.,
humira + adalimumab + biosimilars collapse into one cluster). Built at Stage 7 via RxNorm canonicalization + agglomerative embedding clustering. The patient-facing name is the cluster's canonical_name.
- Mention
- One (post-or-comment, treatment) pair extracted by the LLM at Stage 4+5. A single post mentioning 3 treatments yields 3 mentions. The cluster's
mention_count is the patient signal — "how many people talked about this?"
- Author
- Distinct Reddit handle across the entire corpus — the same user posting in r/ibs and r/migraines counts once. Authors are the unit for credibility scoring (Stage 6) and for cluster_breadth.
- Cross-cond N+
- Distinct authors who appear in N or more conditions. Tail of cross-cond 3+ is the meaningful population for credibility / cluster-poisoning analysis (someone who posts in 3+ unrelated illness subs gets the U-curve penalty).
- Phase B coverage
- Share of authors with the full Stage-6 feature set (account age, karma signal, condition-fit U-curve, commercial-link ratio, etc.), computed from cached cross-Reddit history. Phase A is the post-only fallback used when we don't have the history cached.
Guards (9)
Run at generation time against the current working tree. Click a guard’s script to see its source.
check_no_dup_diagramsOK
Diagram canonical-source invariant
data_lineage.mmd (verified copies: none; inline renderers: ['data_lineage.html', 'deploy\\data_lineage.html', 'deploy\\schema.html', 'schema.html'])
check_data_dictionaryOK
Schema CSV <-> HTML coupling
OK: CSV tables and HTML <h2 id> sections match exactly
check_live_countsOK
Numeric-prose drift (conditions/guards)
OK: PLAN.md Conditions Onboarded table row count matches pipeline-output count
check_architecture_docOK
ARCHITECTURE.md stage coupling
[PASS] all 2 docs fully coupled to codebase.
check_pipeline_integrityOK
Per-condition pipeline_output integrity
Summary: 26 clean, 0 with warnings, 0 failed (pubmed threshold: 50%, strict=False)
check_manifestOK
Manifest freshness (per-condition)
Summary: 26 fresh, 0 stale (checked 26)
check_physical_schemaOK
Physical SQLite schema <-> doc coupling
OK: snapshot DDL matches live web.db and glean.db
check_serving_freshnessOK
Serving layer current (Supabase vs pipeline)
OK: serving layer current with 26 conditions (last published 2026-07-15T00:07:08+00:00).
check_schema_htmlOK
schema.html prose claims <-> code coupling
OK: no forbidden stale strings (DuckDB / password-gated / runtime fetch)
Conditions (26)
FRESH/STALE compares each condition’s recorded features_sha against the current _features.json. STALE means build_pipeline needs to re-run.
| Condition | Items | Clusters | Authors | Phase B | Status | Last build (UTC) | Top 3 by cluster_weight |
|---|
| Adenomyosis | 831 | 153 | 467 | 99.6% | FRESH | 2026-06-02T21:47:18Z | hysterectomy, mirena iud, mirena |
| Asthma | 1,348 | 240 | 689 | 99.0% | FRESH | 2026-06-02T21:48:02Z | symbicort, prednisone, montelukast |
| Chronic Pain | 5,954 | 939 | 2,482 | 99.6% | FRESH | 2026-06-02T21:50:58Z | oxycodone, gabapentin, tylenol |
| Crohn's Disease | 3,693 | 550 | 1,812 | 99.7% | FRESH | 2026-06-02T21:51:47Z | humira, remicade, skyrizi |
| Eczema | 2,490 | 702 | 1,397 | 99.6% | FRESH | 2026-06-02T21:52:24Z | dupixent, tacrolimus, rinvoq |
| Ehlers-Danlos Syndrome | 4,638 | 930 | 2,305 | 99.4% | FRESH | 2026-06-02T21:53:06Z | physical therapy, low-dose naltrexone (ldn), compression garments |
| Endometriosis | 6,678 | 900 | 3,513 | 99.4% | FRESH | 2026-06-02T21:54:03Z | birth control, laparoscopy, iud |
| Fibromyalgia | 4,115 | 775 | 2,022 | 99.2% | FRESH | 2026-06-02T21:54:44Z | lyrica, cymbalta, gabapentin |
| Gastroparesis | 1,616 | 432 | 720 | 99.4% | FRESH | 2026-06-02T21:55:12Z | reglan, zofran, domperidone |
| GERD | 2,370 | 497 | 1,299 | 99.7% | FRESH | 2026-06-02T21:55:48Z | omeprazole, famotidine, pantoprazole |
| Hashimoto's Thyroiditis | 3,089 | 494 | 1,502 | 99.3% | FRESH | 2026-06-02T21:56:22Z | levothyroxine, tirzepatide, gluten-free diet |
| IBS | 2,881 | 583 | 1,661 | 99.3% | FRESH | 2026-06-02T21:56:48Z | imodium, low fodmap diet, probiotics |
| Long COVID | 4,477 | 864 | 1,707 | 99.5% | FRESH | 2026-06-02T21:57:32Z | ldn (low-dose naltrexone), antihistamines, ldn |
| Lupus | 1,941 | 352 | 903 | 99.6% | FRESH | 2026-06-02T21:58:00Z | hydroxychloroquine, prednisone, benlysta |
| Lyme Disease | 1,313 | 314 | 477 | 99.4% | FRESH | 2026-06-02T21:58:26Z | doxycycline, antibiotics, rifampin |
| MCAS | 5,393 | 952 | 1,857 | 99.4% | FRESH | 2026-06-02T21:59:19Z | cromolyn, ketotifen, pepcid |
| ME/CFS | 6,294 | 1,017 | 1,940 | 99.3% | FRESH | 2026-06-02T21:50:01Z | ldn (low-dose naltrexone), pacing, ativan |
| Migraines | 9,222 | 1,270 | 4,510 | 99.5% | FRESH | 2026-06-02T21:59:51Z | nurtec, botox, sumatriptan |
| Multiple Sclerosis | 4,241 | 478 | 1,807 | 99.2% | FRESH | 2026-06-02T22:00:31Z | kesimpta, ocrevus, steroids |
| PCOS | 6,063 | 830 | 3,430 | 99.2% | FRESH | 2026-06-02T22:01:25Z | metformin, inositol, birth control |
| POTS | 6,924 | 1,038 | 3,009 | 99.5% | FRESH | 2026-06-02T22:02:21Z | ivabradine, propranolol, midodrine |
| Psoriasis | 1,487 | 318 | 883 | 99.3% | FRESH | 2026-06-02T22:02:46Z | skyrizi, biologics, methotrexate |
| Rheumatoid Arthritis | 1,771 | 268 | 838 | 99.5% | FRESH | 2026-06-02T22:03:10Z | methotrexate, prednisone, hydroxychloroquine |
| Type 1 Diabetes | 8,670 | 899 | 4,115 | 99.3% | FRESH | 2026-06-02T22:04:16Z | continuous glucose monitor (cgm), insulin pump, metformin |
| Type 2 Diabetes | 1,763 | 322 | 870 | 99.8% | FRESH | 2026-06-02T22:04:46Z | metformin, mounjaro, ozempic |
| Ulcerative Colitis | 3,202 | 455 | 1,371 | 99.2% | FRESH | 2026-06-02T22:05:21Z | mesalamine, prednisone, remicade |
Recent decisions (last 8)
From PLAN.md Decisions Log, most recent first.
#54Supersede GEMINI "no symptom in UI" ban → 3 entry vectorsGEMINI Principle #2 banned "symptom" from patient UI (euphemize as Signals/Experiences); superseded because it wasn't guard-enforced, no patient code used a euphemism, and "symptom" is plain searchable English serving Discoverability (#4) better — the non-advice stance is carried by the glass-box signal band, not by avoiding the word. Edited `GEMINI.md` + `web/GEMINI.md` + `docs/PHYSICAL_SCHEMA.md`. Also retired Principle #1 "Conditions & Remedies Only" → **three entry vectors** (condition / remedy / symptom). Patient nav route will be `/symptom/[slug]` (2026-06-19)
#53Symptom normalization + three-layer model (PLAN #8 data layer)The raw `symptoms_targeted[]` vocabulary (~33.6k free-text strings, 79% ≤2 mentions) was too noisy to navigate. `extract/normalize_symptoms.py` canonicalizes it via Haiku (`claude -p`, Max plan — no API key configured) into **7,515 labels** (`data/symptom_canonical_map.json`). **Key structural insight: target-type is contextual, not intrinsic** (the same string `mcas` is a degenerate self-reference inside the MCAS corpus but a real comorbid treatment-target inside POTS). Hence three layers: (L1) normalize = context-free, canonicalize + type (symptom/sign/condition kept; trigger/other dropped), NEVER drops a condition-name; (L2) `migrate` drops self-references (target == own condition, type=condition only) using condition context; (L3) frontend decides surfacing. Per-canonical type resolved deterministically (exact-match-raw wins). `type` column cascaded through glean.db/web.db/`physical_schema.sql`/`schema_postgres.sql`/`push_to_supabase.py`/`PHYSICAL_SCHEMA.md`/`ARCHITECTURE.md`. All 9 guards green (2026-06-19)
#52Warm-terminal reskin (90s-minimal)Patient surface restyled to the "warm terminal" theme: monospace, greige ground, sage/ochre/clay earth tones (Jackson Browne palette). Token set + remap layer in `web/src/app/globals.css`; signal-band tags/stripes consume the same tokens. Live 2026-06-15 (commit `15e4a88`)
#51Cap condition page at top-50 remedies + Show-moreA condition can have 1,000+ treatments; mounting that many interactive cards buried the main thread (sluggish cursor, slow drilldowns). `web/src/app/condition/[id]/RemedyList.tsx`: progressive disclosure, `PAGE=50` by signal, reveal-on-demand. Full list held in memory (cheap JSON) so "Show more" is instant — only DOM/hydration cost is capped, the actual bottleneck. Deep links (Vector 2) beyond the first page widen the initial window so the `#tx-<id>` anchor + highlight still render (commit `c898588`, 2026-06-14)
#50Glass-box "signal band" for the patient surfaceReplaces any implied efficacy score with a signal-of-credible-discussion band that executes the product thesis (conversation-starter, never a verdict). `web/src/lib/signal.ts`: `signalBand(cluster_weight)` → Strong (≥20) / Moderate (≥6) / Emerging tiers, each an ASCII bar + earth-tone tinted tag + left stripe; thresholds grounded in the real corpus distribution (16,572 treatments, p75≈6, p95≈22 → Strong≈top 5%, Moderate≈p75–p95). `researchContext(pubmed_count)` reframes PubMed coverage as a doctor-conversation cue (well/some/barely/not-yet studied), surfacing the "community ahead of the science" gap as honest context, not a headline. Applied to Vectors 1 & 2 (commit `5930db0`, 2026-06-14)
#49Replace magic-link auth with email + passwordMagic-link UX confused testers and free-tier Supabase email (Yahoo spam, low rate limit) was a deliverability wall; bouncing between localhost and prod added redirect-allow-list friction. Swapped `signInWithOtp` → `signInWithPassword` (`web/src/app/login/page.tsx`) and deleted the now-dead `/auth/confirm` magic-link callback. Invite-only preserved via admin-provisioned accounts (Supabase Users, password + Auto Confirm) — no public sign-up, no email at login. Session/`proxy.ts` unchanged (same cookie flow). Magic-link prose swept from CLAUDE.md / ARCHITECTURE.md / OVERVIEW.md. Live 2026-06-14 (2026-06-14)
#48Close #46's deferred frontend-doc reconciliation (full audit pass)Triggered by a full repo audit ("clean / coupled / referenced") run via parallel read-only subagents + the 8-guard sweep. Verdict: repo fundamentally healthy (tree clean, 8 guards <!-- count-ok --> green, `web/` app fully Postgres-migrated, no markers/conflicts/half-done code, PLAN/RUNBOOK hygienic); the remaining staleness was almost entirely #46's deferred bucket. Closed it: (1) **CLAUDE.md** — rewrote "Dev vs Patient Surfaces" as a PHYSICAL split (patient = `web/` Vercel app on Supabase; dev = Cloudflare static, bare domain redirects to `dev.html`); removed the 5 archived-stack rows from the Project Structure table; fixed the §"Coupled Artifacts" `deploy/index.html`→`deploy/pipeline.html` renderer refs. (2) **ARCHITECTURE.md** (~14 current-prose sites) + **OVERVIEW.md** reconciled by a subagent (dated change-log entries left as history); also fixed OVERVIEW §2.3 "Seven scripts"→"Eight" + added the `check_serving_freshness` row (a 7→8 gap `check_live_counts` can't see — "scripts" not "guards"). (3) **`build_dev_status.py`** dev-nav "Patient site → /" repointed to the Vercel app + JS active-tab default `index.html`→`dev.html`; propagated to the 6 committed dev pages via `inject_dev_nav.py` + regen. (4) **`extract/build_ui.py`** Project Files entry + Browse crumb repointed to the Vercel app (re-rendered all 26 dashboards to clear the resulting `check_manifest` STALE). Non-#46 audit finds also fixed: `PHYSICAL_SCHEMA.md` stale "app reads web.db" → loader; `.gitignore` 6.1→6.5 GB; 3 orphan root `test_*.py` → `tests/`; 4 info `console.log`s removed from `web/`; obsolete `decisions.yaml` TODO marked moot. NOTE: regenerating `corpus.html` for the nav also picked up an **overdue deterministic data refresh** (committed copy was stale vs current pipeline outputs: mentions 133190→133230, clusters 16568→16572) — legit, not guard-tracked. All 8 guards <!-- count-ok --> green (2026-06-12)
#47Harden the new-data → Supabase publish path (Guard 8)Per the 2026-06-12 priority reset (pipeline preservation > frontend), closed the "memory-bridged gap": `run_condition.py` stops at the JSON pipeline outputs and never touches Supabase, so a finished run silently left the live app serving stale data. Three moves: (1) **`scripts/refresh_serving.py`** — one-command orchestrator for the serving chain (migrate → `build_web_db` → `push_to_supabase`), with fail-fast prechecks (pipeline outputs exist, `web/.env.local` present) and a prod-reload confirmation; on success writes `data/serving_manifest.json` (condition → sha256 of pipeline output; gitignored, machine-local). Flags: `--yes`, `--skip-rebuild`, `--record-only`. (2) **`scripts/check_serving_freshness.py`** — guard #8, compares the manifest to current pipeline outputs and flags STALE/NEW/REMOVED; wired into `dev_status.html` via `build_dev_status.py` GUARDS; verified it catches all three drift types (exit 1) and passes clean (exit 0). (3) **`docs/RUNBOOK.md`** — the single end-to-end "build half + publish half" runbook; `run_condition.py` now prints a "NEXT: publish" reminder on completion. Baselined the manifest via `--record-only` (Supabase already in sync — no pipeline re-runs since the last push). Guard count 7→8 (CLAUDE.md, OVERVIEW.md ×3, PLAN.md ×2 reconciled; `check_live_counts` enforces). This also resolved the onboarding-checklist item from #46's deferred list (CLAUDE.md §94 now points at `refresh_serving.py` + RUNBOOK and drops the archived `build_home.py`); the rest of #46's frontend reconciliation stays deferred (2026-06-12)
Open questions
Every unchecked [ ] from PLAN.md Open Questions.
(none open)
Active next actions
Non-strikethrough items from PLAN.md Next Actions.
(no active next actions parsed)
Recent commits (last 10)
f59c4e12026-07-14Audit cleanup: archive spike/, document bootstrap_proximity.py
acdc9ad2026-07-14Symptom page: swap unmapped text-amber-700 for semantic text-ochre token
70e35732026-06-23Ingest docs + proximity-map bootstrap tool
17d2b772026-06-19Symptom-first frontend: /symptom/[slug] route + search (PLAN #8)
d82d06c2026-06-19Symptom normalization: canonical vocabulary + three-layer model (PLAN #8 data layer)
ddecf002026-06-17Guard 9: check_schema_html.py — pin schema.html prose to code
f88b4f82026-06-17Docs: reconcile support files to code reality (audit fixes)
4fc7d842026-06-17Housekeeping: ignore .vercel, log frontend decisions #50-52, tick Gap-3
15e4a882026-06-15Reskin: warm-terminal theme (90s-minimal, Jackson Browne palette)
c8985882026-06-14Perf: cap condition page at top 50 remedies with Show-more