Putting the new Jev model to the test
Can Jev classify ICD-10-CM codes from claim text?
Claim Classifier puts that idea to the test using manually annotated clinical text. Across 324 held-out cases Jev answered 64.2% of the time, and those answers reached the right three-character category 88.0% of the time.
An early research evaluation, not a production coding system.
“Long history of recurrent infections … thick mucoid material in both middle ear spaces”
H65.33
Chronic mucoid otitis media, bilateral
Exact match with the annotated code
Jev evaluation results
When Jev answers, it reaches the right three-character category 88.0% of the time.
It answers on 64.2% of the 324 held-out cases and declines the rest. Of those answers, 78.4% were the exact code with every character right — 50.3% of all cases.
What happened across 324 held-out cases?
Each test ended with the expected code, a different code, or no code because Jev declined to answer.
A code gets more specific one character at a time.
Chapter, then section, then the three-character category that names the condition, then the characters that add site, severity or laterality.
Every character is another decision that can go wrong, so this is where you see how far down the accuracy holds. Switch the question to change what the table counts.
See the evaluation setup
The operating point — how confident Jev has to be before a code is returned — was chosen on a separate development split and left alone for this run. These figures are the first time it was measured on cases it had never been tuned against.
More than a word match
The note does not have to say the code title.
In these selected exact-match cases, Jev mapped clinical language to an ICD-10-CM code even though the official title never appeared in the source text.
These cases were selected to illustrate semantic mapping, not representative accuracy. See the full results or browse the fixed-seed random sample.
Inside one correct result
See one ICD-10-CM classification in context.
For readers who want to inspect the work, this example shows how Jev arrived at the same code chosen by the dataset’s annotators. Each additional character adds clinical detail.
Inspect the work
Move from percentages to individual cases.
Get in touch
What could Jev mean for medical coding?
Tell us where this idea could be useful in your organization. We read every message and reply with what we find next.
Research notes
Disclosures and limitations.
The details belong here for readers who want to assess the evidence more closely.
What this evaluation measures
Every number and example comes from the University of Arizona’s manually annotated ICD-10-CM signs and symptoms dataset. We treat the annotators’ code as the expected answer and never show it to the model.
The dataset tests whether documented clinical language can be mapped to a matching code. It does not record which diagnosis a claim would list first, so this is not evidence that the system can choose a claim’s primary diagnosis correctly.
Why the repeated R58 cases can be set aside
How to interpret the results
One run is not a verdict. The figures come from a single run over all 324 eligible cases of the held-out validation split. The model is not deterministic, and repeating the run can move exact match by roughly two points either way. A further test split is locked and has not been touched.
The cases you can browse are not the cases in the table. Percentages come from the held-out split; the individual cases below come from the development split, run through the same model, code set, beam width and confidence settings. Keeping the browsable text on the development split is what lets the held-out result stay held out.
A parent code is not a child code. Getting a category right but a later character wrong is still a wrong final code. Partial results appear only in the supported-depth view and are never presented as billable code proposals.
What this demonstration is not
This is not a certified medical-coding system. It proposes at most one code, guarantees nothing, submits no claims, and must not be used with real patient information. Model probability is not proof that a code is correct.
How contact details are handled
The contact form is the only thing this site stores. Details go to a Cloudflare Workers KV namespace, are kept for up to one year, and are used only to reply to you and send project updates. They are not public, are not shared, and can be deleted on request.
Nothing else on this page is recorded. There is no model running here: every figure and case you can read was produced offline and published as static data, so browsing the results sends no text anywhere.