The capture loop is the heart of MedLens: a photo of a hospital artifact becomes a set of structured, editable medical records. The pipeline always runs on-device OCR first, classifies and structures the text, and then routes everything to the review screen for the user to confirm before anything is saved to the database.
Flow
Photo -> on-device OCR (always) -> classify & structure -> user review -> save to DB
On-device OCR runs on every capture. Structuring then happens at one of three tiers, chosen by the user in Settings.
Privacy tiers
The tiers form a ladder that trades privacy for accuracy. The user decides where on the ladder to sit, and the default is the most private rung.
| Tier | What leaves the device | Strength | Offline |
|---|---|---|---|
| 1 — Local | Nothing | Good for clean printed labels; weaker on handwriting | Yes |
| 2 — Cloud text | OCR text only | Good; text rarely contains patient identifiers | No |
| 3 — Cloud vision | The full image | Best; handles messy labels, monitors, handwriting | No |
The default is Tier 1 (fully local). Higher tiers are opt-in, with a plain-language explanation in Settings of exactly what leaves the device. See Privacy and Safety for how this connects to the local-first thesis.
Pipeline output
Every tier produces the same shape: a list of extracted records, each tagged with a type (medication, vital, lab panel with results, event, or unknown) and a confidence score from 0 to 1. Every field is partial — the pipeline proposes, the user disposes. The confidence score drives the review UI: low-confidence fields are highlighted so the user knows where to look.
Tier 1: the local parser
The on-device parser uses guarded regex patterns:
- Medications match a name-plus-dose-plus-unit shape and look for route keywords (IV, oral, PO) in the surrounding context.
- Vitals use named patterns for blood pressure, heart rate, oxygen saturation, temperature, and respiratory rate.
- Labs match a test-name-plus-value-plus-unit shape with an optional flag and reference range, cross-referenced against the known lab-test dictionary to cut false positives.
- Fallback — if nothing matches, the parser returns an “unknown” record carrying the raw text, so the user can still review and file it manually.
Tiers 2 and 3: opt-in cloud structuring
When the user opts in, the higher tiers send either the OCR text (Tier 2) or the image (Tier 3) to a cloud model for structuring. The structuring instruction constrains the model to return only valid records matching the output schema, to extract all medical data it finds even when partial, to set confidence lower for uncertain extractions, and to never include patient names or identifiers in the structured output — PII is stripped from the result even if the source image contained it.
Review and save
After extraction the user sees one card per extracted record on the review screen, color-coded by type, with a confidence indicator and low-confidence fields flagged for attention. The user can edit any field, confirm records individually or all at once, or file an uncertain capture as an event carrying the raw text. Only confirmed records are persisted, each through its repository. The source photo is retained as the audit trail and can be revisited from the records it produced.