Clinical Note Audit

Abstract

Clinical-note evaluation pipelines can confuse a usable response with a correct judgment. We present Clinical Note Audit, an offline tool that makes response validity, evidence conditions and reference agreement separately inspectable. Users can filter a recorded evaluation, inspect unchanged notes and responses, diagnose the first failed structural check, compare paired evidence conditions, and validate a pasted response against the frozen contract. A packaged evaluation contains 1,530 first attempts on 85 MedVAL mock consultations, with 17 development and 68 held consultations. The application reports coverage, fallback frequency, pipeline Brier and valid-response-only Brier with their denominators. Its replay preserves the study predictions and reference labels. A contract lab checks serialization and quotation, without certifying medical meaning. The tool supports measurement audits of clinical NLP evaluators; it does not establish the evaluators' clinical reliability.

Keep three questions separate.

Did the response satisfy its output contract? Which evidence was available to the model? How well did a returned probability agree with the fixed reference target? The interface keeps response coverage, pipeline loss and valid-response-only loss visible together.

q̃ = q for valid outputs; 0.5 otherwise
Coverage = valid responses / scheduled responses
Pipeline Brier = mean (q̃ − y)²

The archive contains mock consultations. Its physician target is risk grade 1 versus grades 2–4. A verbatim quotation passes a structural check; it does not certify that a medical verdict is supported.

Inspect the recorded evaluation.

Filter 1,530 first attempts by model, output contract, split and evidence condition. Inspect the source, note and unchanged response; exercise the contract lab and export the exact active analysis.

All inference is previously recorded. The public website performs structural checks and arithmetic locally; it does not call a language model.

From the page to the code.

The download includes this website, numerical samples, research reference code, provenance and executable checks. The examples are usable without access to a hospital or private account.

unzip clinical-note-audit.zip
cd clinical-note-audit
python -m http.server 8766 --directory dist
# Browse the static website locally:
python -m http.server 8766 --directory dist

Download the complete installation package · Inspect provenance

Saved results are computational evidence. This artifact does not establish independent clinical validation, medical suitability or acceptance of a paper.