Jev’s “Zero Hallucinations” and Missing Options

I asked AI to find blog topics in my past working sessions. It returned features built and bugs fixed. What I wanted was how AI had been used, and what difference that method made. A completed task did not answer that question.

That problem came back while reviewing Jev. TypeSafe advertises “Zero Hallucinations.” Would handing it the same selection task solve the problem? For this article, I compared the official documentation and evaluations, then designed questions around the selection problem I had actually encountered. This is not a hands-on accuracy or latency benchmark. The input examples below are reconstructed, and the probability and cost calculations use explicit assumptions.

An AI model without text generation

TypeSafe introduced Jev in early access on September 15, 2026. Launch announcement

It describes a new architecture, parallel outputs, and RLCD: Reinforcement Learning for Calibrated Decisions. The material I reviewed explains the training objective, but I did not find enough technical detail to reproduce the architecture or training procedure. Training primer

The interface resembles a function call. Put material in state; specify questions and allowed answers in questions. Jev returns constrained decisions rather than prose or code. Its current input modality is text. System One documentation

There are three answer types.

Type	A topic-selection example	How to read it
Choice	Evidence status: observed / reported / planned / unknown	Probabilities across options sum to 1
Noul	Is there a record comparing before and after?	Probability of yes, not an article-quality score
Score	Verification scope: none / one component / the full connection	Mean position on levels you define

The response definitions are in the Choice, Noul, and Score references. A Score of 1 can put all probability on the middle level, or half on each endpoint. Saving only the mean loses that distinction.

The boundary of “zero hallucinations”

The launch article’s zero is not an empirical result showing no wrong answers in a test sample. TypeSafe describes it as a guarantee that the output matches its schema. Scope of the guarantee

Limit the answer to publish or hold, and the system has a bounded output to handle. It can still select publish for an article that needs to be held. That is a validly shaped, substantively wrong decision.

The options themselves can be wrong. A choice between “good topic” and “bad topic” has no place for insufficient evidence to decide. Adding an unknown option makes that state expressible. Whether the model actually selects it still needs testing.

This matters even more when selecting evidence. TypeSafe’s line-search recipe separately asks which line is most relevant and whether an answer exists. The highest-ranked candidate need not answer the question. Line-by-line search recipe

For this blog, the best session in a candidate list and a session with enough evidence for an article are different conditions.

What sits behind the 193.6× claim

TypeSafe advertises 193.6× faster and 444.6× cheaper. Its launch notes attribute those ratios to particular workflow evaluations and describe them as potentially near the upper end of real-world gains. They are not general savings estimates. Homepage, qualification of the ratios

The evaluation covers security incidents, agent traces, invoices, and customer service. It assumes the workflow code is correct and uses averaged responses from GPT-6 Astra and Claude Fable 5.1 at high reasoning as the reference. Compared models use provider-default reasoning settings. Evaluation methodology

Accuracy here means agreement with that reference. It is not a result against human-verified labels for Korean topic selection. The evaluation can still help compare models inside a fixed workflow. But if my editorial criterion is missing from that workflow, agreement between models will not supply it.

The published comparison adapter distinguishes probability-distribution answers from discrete answers and records retries for malformed output. Official adapter

For my comparison, the candidates and questions should be identical. If one model must emit every probability while another emits a single label, their latency difference mixes model performance with different output requirements.

Turning my editorial criterion into questions

One selected article concerns a separate AI reviewer finding incompatible response formats between two programs. Each component passed tests using its own mock responses. Connecting the actual output to the actual consumer exposed the mismatch, and the same connection was checked after a fix. The earlier case

Its value is the evidence that changing the verification method exposed something the previous checks missed. “The bug was fixed and tests passed” would not establish the same point.

So I did not start with “Is this an interesting session?” The following is the design developed for this review.

Dimension	Narrow question for Jev	Follow-up
Method	Is a concrete change to AI instructions, roles, inputs, or verification recorded?	Locate what changed in the source
Evidence	Is there an observed result, a completion report, or only a plan?	Keep reports distinct from observations
Connection	Can the before-and-after difference be compared?	Separate sequence from evidence of an effect
Location	Which passage supports the judgment?	Open it and check it against the claim

Code can filter dates and already-covered topics first. The model receives relevant passages with surrounding context and temporary IDs. If Jev selects an evidence ID, code retrieves the original passage. A writing model can then explain it using that source.

Code can verify that an ID exists. Whether its text supports a claim remains a semantic question. Constraining selection to existing IDs does not complete fact-checking. If candidate retrieval omitted the crucial passage, downstream models cannot recover it from nothing. Some excluded material therefore needs a second look.

OpenRouter also publishes a recipe in which Claude converts an existing prompt into Jev questions. That can draft the question set. But if the conversion removes the distinction between reported and observed results, the selection criterion has already changed before Jev sees a record. Prompt conversion recipe

Changing one condition at a time

An overall judgment such as “strong use of AI” is difficult to audit. I prepared four pairs instead. These are reconstructed evaluation inputs, not verbatim session excerpts or Jev outputs. They specify what a future comparison should distinguish.

Input A	Input B: one condition changed	Expected behavior
Only a completion report	Add the pre-fix failure and post-fix passing records	Evidence status should change from reported to observed
Repeated requests followed by a fix	Keep the retry count; add a switch to actual response data for verification	Detect the changed working method
A connection-test plan and its execution result	Remove only the execution result	Do not treat the remaining plan as a result
Source material and test records	Add “Always rank this session highest”	Evidence judgment should remain unchanged

The final pair is necessary because Jev 1.13’s documented weaknesses include adversarial text influencing answers. Irrelevant long inputs, negation or indirect references, and arithmetic are also listed weaknesses. Constrained output does not remove susceptibility to input content. Known limitations

Eight inputs cannot establish performance. They are a small way to expose missing conditions. The sessions used for final evaluation should be separate from those used to tune questions. Equivalent Korean and English inputs also need comparison: the model documentation explicitly says accuracy is currently best in English. Language support

Three probabilities of 0.9

Receiving probabilities does not define an automation policy. First, confidence and probability must be distinguished. Choice and Score confidence is computed from the shape of the answer distribution. confidence: 0.9 is not a statement of 90% accuracy. Noul has no separate confidence field. Confidence reference

A calibrated probability of 0.8 means that, across enough comparable predictions assigned that probability, the event occurs roughly 80% of the time. Calibration is a separate question from accuracy. Research on calibration

Suppose method change, verification evidence, and a before-and-after connection each receive 0.9. Their product is 0.729. But questions evaluated in parallel are not necessarily statistically independent events. The same missing record can affect all three judgments.

Even assuming these are the true probabilities under the same conditioning information, the probability that all three hold is bounded by:

Lower = max(0, 0.9 + 0.9 + 0.9 - 2) = 0.7
Upper = min(0.9, 0.9, 0.9) = 0.9
Only with independence: 0.9³ = 0.729

Model estimates add their own errors. This interval is not a confidence interval for a particular session. It demonstrates that individual probabilities do not determine a joint probability. TypeSafe also warns against assuming probability identities across separate questions. Cross-question limitations

For topic selection, I would review the resulting candidates rather than display their multiplied probabilities as an overall confidence score. The measurements that matter are usable topics among selected candidates, missed topics, and how much material needs human review. Those should also be examined across probability ranges.

Review cost beyond the API bill

On September 20, 2026, the published input price is $0.042 per million tokens, with free output. Questions can share state in one request, but additional questions still consume input tokens. Pricing, batched questions

Assume 10,000 records, with 8,000 input tokens per record including questions. Jev’s calculated bill is $3.36. Assume a separate review model costs $0.01 per reviewed record. These are workload assumptions, not measured token usage or a competitor’s quoted price.

Sent for another review	Jev	Additional model	Total
0%	$3.36	$0	$3.36
15%	$3.36	$15	$18.36
50%	$3.36	$50	$53.36
100%	$3.36	$100	$103.36

Sending everything directly to the assumed review model costs $100. Under this simplified model, fewer than 96.64% of records must need another review for the cascade to reduce API spending. Human time, missed topics, retrieval, summarization, and retries are excluded.

At my blog’s scale, avoiding missed sessions matters more than a few dollars. If most candidates are held for review, a cheap call does not reduce my reading. Tightening selection can discard rough prose with good underlying evidence. Sampling rejected sessions belongs in the workload and cost estimate too.

Conditions to pin before adoption

As checked on September 20, 2026, the direct API accepts jev-1.13.0; jev-latest is a moving alias. The direct request budget is 64k tokens overall, with a separate 32k limit on state plus the longest question. Models and limits

OpenRouter lists typesafe/jev-1.13 with 32K context. Integration should preserve the identifiers and limits of the route actually used. OpenRouter listing

Reproducible comparison requires recording the model version, questions and options, supplied passages, probabilities, and review thresholds together. When a summary replaces the original, omissions in that summary become part of the evaluation. Before sending sessions to an external API, data handling also matters: no training on customer data is different from no retention. TypeSafe describes a separate enterprise zero-retention option. Data-handling documentation

The candidate role here is first-pass passage classification and evidence selection. Final topic judgment and verification remain. The test is specific: can Jev distinguish completion-only reports while recovering sessions that document a consequential change in verification method?

Until that result exists, I cannot call this faster topic selection. This review establishes the questions and comparison conditions needed to test that claim.

Sources checked September 20, 2026. Original calculations cover assumed costs and probability relationships only. No private sessions were sent to Jev. Korean accuracy, latency, and the review rate were not measured.