AI Oversight Layer

The ai-oversight service is CareBridge’s core safety layer — the component that catches cross-specialty clinical gaps that individual specialists miss. Every clinical data mutation emits a ClinicalEvent to the Redis clinical-events queue; the oversight worker consumes those events and produces structured ClinicalFlag records.

Three components

1. Rules engine (deterministic)

Known patterns fire immediately, with zero LLM cost or latency. Rules are evaluated synchronously before the LLM review. Cross-specialty patterns live in services/ai-oversight/src/rules/.

Rule ID	Pattern	Severity
ONCO-VTE-NEURO-001	Cancer + VTE + neurological symptom	CRITICAL
ANTICOAG-BLEED-001	Anticoagulation + bleeding symptoms	CRITICAL
CHEMO-NEUTRO-FEVER-001	Chemotherapy + fever	CRITICAL
RENAL-NSAID-001	Renal impairment + NSAID use	WARNING
CARDIAC-FLUID-001	Heart failure + fluid overload symptoms	WARNING
DIABETES-STEROID-001	Diabetes + corticosteroid	WARNING

Additional rule files cover critical vital values and drug-drug interactions. Each rule returns null when it does not apply, or { severity, category, summary, rationale, suggested_action } when it fires. See Adding a Clinical Rule.

2. LLM review (Claude API)

When rules do not fully cover a case, the service assembles the full patient context and calls the Claude API for nuanced clinical concerns the rules cannot encode. The system prompt instructs the model to identify cross-specialty concerns, not diagnose, to respond in structured JSON, and to filter out already-managed conditions.

Prompt assembly lives in @carebridge/ai-prompts:

• CLINICAL_REVIEW_SYSTEM_PROMPT — the core prompt
• buildReviewPrompt(context) — assembles patient context into the user message
• parseReviewResponse(response) — parses the JSON into LLMFlagOutput[]
• PROMPT_VERSION — semantic version tracking each prompt iteration

Prompts are versioned like code to enable A/B testing, regression testing across model upgrades, and controlled rollbacks. Changes to clinical prompt data go through a formal sign-off process — see the AI prompt editing doc.

3. BullMQ worker

Subscribes to the clinical-events queue with concurrency 5 and a 10-jobs/min rate limit to stay within API quotas. Handles retries and backpressure automatically.

Quality: the golden-eval harness

@carebridge/ai-prompts ships a record-and-replay test suite that validates buildReviewPrompt produces structurally correct, token-budget-compliant prompts containing all clinically relevant data — no live API calls. Fixtures in packages/ai-prompts/evals/fixtures/ describe clinical scenarios with expected outcomes; the runner checks section headers, required mentions, rendered medications, token budget, and negative-case guardrails. Details in the golden-eval doc.

Tuning flags

Two env vars gate opioid / non-opioid daily-dose escalation thresholds; invalid overrides fall back to default and log a structured warning so misconfiguration surfaces in CI.

Variable	Purpose	Default
OPIOID_CRITICAL_RATIO	Excess-over-cap ratio escalating opioid flags warning to critical	1.2
NON_OPIOID_CRITICAL_RATIO	Same gradient for NSAID / analgesic daily-dose flags	2.0

Trying it end-to-end

The seed data creates Margaret Chen, a 67-year-old with Stage III breast cancer (Capecitabine), DVT with an IVC filter, Enoxaparin anticoagulation, and PRN Ondansetron. To fire ONCO-VTE-NEURO-001: log in as dr.smith@carebridge.dev, open her chart, document a new headache in a clinical note, and save. The worker picks up the note.saved event, the rule matches cancer + VTE + neurological symptom, and a CRITICAL flag appears in the clinician portal recommending urgent neurology evaluation. Walkthrough in the README.

Compliance boundary

The oversight layer is intentionally designed as non-device Clinical Decision Support: every flag carries a human-readable rationale, LLM flags record model_id and prompt_version, and clinicians independently review the basis for each flag rather than relying on it. See Compliance and Safety.

See also: Architecture Overview, Clinical Domain Model, Design Decisions.