JevAgentHarness: Benchmarking, Deploying and Training Decision Models That Share Control with LLMs
Zhu et al. · 2026-10-15 · v1
Studies fitted confidence thresholds for sharing agent control with decision models.
Selected papers on evaluation, calibration, routing, robustness and decision-model architectures, with versioned primary sources and citations.
This directory selects directly relevant preprints. Dates identify the linked versions; summaries draw on paper and abstract checks, without claiming experimental reproduction. Foundations, models and tools are listed on the homepage.
Last curated · 56 papers · BibTeX · JSON
Start with the research guide: interfaces, probabilities and evaluation →
Zhu et al. · 2026-10-15 · v1
Studies fitted confidence thresholds for sharing agent control with decision models.
Chen et al. · 2026-10-15 · v1
A benchmark spanning matched requests and closed-loop tasks. Experiments have not been reproduced here.
Deng, Fan, Zhang & Xie · 2026-10-15 · v3
In ten prepared scientific cases, the authors separate semantic selections, computed quantities and final labels; comparison-model errors can be hidden by unchanged final labels.
Zheng & Chen · 2026-10-14 · v1
Gates reasoning start, candidate-answer reading and generation stopping; evidence is limited to a frozen Qwen backbone and three task families.
Zhang, Ye & Zha · 2026-10-14 · v1
Tests classification and verification under content-preserving table rewrites across three task families; confidence can flag instability without proving correctness.
Gao et al. · 2026-10-14 · v2
Audits ordinal label underuse across selected decision models; finer scales increase compression, while accuracy, final-label utilization and averaged candidate probabilities remain distinct measures.
Lan et al. · 2026-10-13 · v1
Text and vision attacks with label-validity checks and held-out evaluation models.
Li & Wang · 2026-10-13 · v1
A decision-model architecture exploring null handling, option permutations, scaling, and caching. Its design is Jeva’s, not Jev’s disclosed internals.
Zhu et al. · 2026-10-13 · v1
Structured video decisions with temporal spans and candidate scores; cached and joint inference reuse visual computation.
Li, Krishnan & Padman · 2026-10-13 · v1
Pairs factual probes with action choices to diagnose decision failures.
Ma et al. · 2026-10-13 · v1
Tool-argument judges including Jev reject some correct derived values without visible references; model-generated references are evaluated on three benchmarks.
Li et al. · 2026-10-13 · v1
A controlled adversarial comparison of Jev, LLM judges, and purpose-built safety classifiers.
Li & Wagle · 2026-10-13 · v3
Compares frozen Qwen decision readouts with optional KL-anchored fine-tuning and budgeted reasoning; reported gains depend on backbone size and routing workload, not disclosure of hosted Jev internals.
Rao & Callison-Burch · 2026-10-13 · v3
Compares rubric judges and examines correlated errors that can limit cascade improvements.
Geuter, Cheng & Alvarez-Melis · 2026-10-12 · v1
Answer-letter readout and supervised LoRA baselines; evaluation includes model-generated labels.
Duan et al. · 2026-10-12 · v2
Bilingual decision evaluation and reliability when local decisions compose into larger systems.
Chen & Le · 2026-10-11 · v1
Jev memory-retirement evaluation separates confirmed updates from suggestions; mixed LLM/human labels, shared errors and selection on a small development set constrain cascade claims.
Gan, Lee & Kang · 2026-10-10 · v2
Looped hidden-state computation for typed decisions; additional loops trade compute for accuracy.
Li & Chen · 2026-10-10 · v2
Candidate-menu interventions and confidence cascades; ranking stability does not establish probability calibration.
Madrid-García & Merino-Barbancho · 2026-10-10 · v4
Preliminary medical-benchmark evaluation of Jev 1.13 reports task-dependent accuracy, calibration and weak unanswerable-question detection; it does not establish clinical deployment validity.
Zhang et al. · 2026-10-10 · v2
ContractNLI request variations can change which answers are wrong while aggregate accuracy stays unchanged; stable answers may preserve errors. This benchmark does not validate real legal deployment.
Li et al. · 2026-10-09 · v5
Studies confidence-based escalation from Jev to a reasoning judge, including limits under distribution shift.
Azizi, Baghaei Potraghloo & Pedram · 2026-10-08 · v1
Tests option-channel attacks on seven open-weight models; reversed decisions can remain confident. The reported experiments do not test hosted Jev.
Li et al. · 2026-10-08 · v1
Knowledge, reasoning and multilingual multiple-choice evaluation of Jev; reported rankings and mathematical weaknesses apply to the selected models and benchmarks.
Cheng et al. · 2026-10-08 · v1
Bi-encoder approach to multimodal decisions and retrieval through natural-language interfaces.
Xu et al. · 2026-10-08 · v2
Optimizes context additions around unchanged questions and options in four decision systems; results concern initially correct decisions, and human validity checks cover a sampled subset of successful attacks.
Sun, Xu, Shi & Yang · 2026-10-08 · v3
Tests conflicts between option names and definitions despite structurally valid outputs.
Xu et al. · 2026-10-06 · v1
Same-number citation swaps probe financial-evidence verification; rejecting wrong roles trades off with accepting equivalent support in a constructed test.
Denney & DiGiuseppe · 2026-10-06 · v2
Studies accuracy, cost, and calibration in seven political-science replication tasks.
Rafe & Das · 2026-10-06 · v2
A common-request harness compares typed decision models, generative readouts and trained classifiers; accuracy, selective coverage and out-of-scope acceptance depend on label access and task.
Asseri & Asseri · 2026-10-03 · v2
English/Arabic persona-conditioned Jev survey audit; the effects concern simulated respondents in one model, not measurements of real populations.
Liu · 2026-10-03 · v3
Examines reliability, group-specific confident errors, and selective automation in security decisions.
Teke · 2026-10-01 · v1
Turkish typed-decision evaluation reports decision quality, Brier-based probability quality and risk–coverage alongside input probes; most labels are AI-derived, and the lab’s model used feedback from earlier test results.
Shankaranarayana, Runje & Jannink · 2026-10-01 · v1
Controls information and surface cues to distinguish Jev answer confidence from self-knowledge; black-box behavior does not establish access to an internal knowledge state.
Zhang et al. · 2026-09-30 · v1
Training-free decision readout with label-prior and option-order correction.
Su & Huang · 2026-09-30 · v1
Composes finite action fields and explores alternatives using interaction feedback.
Ding, Jin & Ji · 2026-09-30 · v1
Shared evidence encoding and candidate-probability readout, trained with representation and action-outcome feedback.
Deußer et al. · 2026-09-29 · v1
An early benchmark study of Jev. Read the task selection and evaluation setup alongside its reported results.
Wang et al. · 2026-09-29 · v1
A Chinese typed-decision model and proposed benchmark; domain-specific training calls for independent validation.
Wang · 2026-09-28 · v1
Keeps raw memory records, separating Jev relevance judgments from LLM search planning and answer generation.
Porcedda · 2026-09-28 · v1
A controlled calibration study using constructed probabilities. Interpretations of latent uncertainty remain hypotheses.
Wang & Gao · 2026-09-28 · v1
Compares Jev and generative judges on retrospective agent-trace risk classification; dataset rankings vary, and successful-call latency and valid-judgment cost must be read with coverage.
Li et al. · 2026-09-27 · v1
Tests coherence between logically related probabilities, separately from accuracy and calibration.
Tang & Zheng · 2026-09-26 · v1
An evidence audit and a checklist for evaluating typed decision models.
Hu et al. · 2026-09-25 · v1
A black-box adversarial benchmark. A changed decision is not necessarily an incorrect decision.
Yilmaz, Tasdemir & Gozay · 2026-09-25 · v1
Estimates whether asking for missing information would improve a typed decision.
Guo et al. · 2026-09-24 · v1
Benchmarks Jev alignment-failure detection while separating question wording from input context; benchmark-scorer labels dominate, and context gains often depend on fields encoding the label.
Zhou et al. · 2026-09-24 · v1
A Jev-style visual-choice study separating adaptation gains, candidate-readout validity, and calibration limits.
Wu & Lim · 2026-09-23 · v1
Audits Jev injection on reconstructed InjecAgent cases, separating probability shifts from attacker-target selections; optimization results are checked on fresh calls, within this benchmark setting.
Huang et al. · 2026-09-23 · v1
Jev checks support between generated and reference radiology reports in both directions; expert agreement depends on report size and error definitions and does not establish prospective clinical effectiveness.
Li · 2026-09-23 · v1
Jev state tables and sparse flagship-model anchors support population simulations with shared human–model discrepancy estimates; larger simulations do not add human evidence or establish prospective intervention effects.
Ibrahim & Zaki · 2026-09-23 · v2
Evaluates zero-shot social-science annotation and confidence-routed LLM cascades; lower costs coexist with task-dependent errors and overconfidence, so confidence cannot substitute for label validation.
Wu & Lim · 2026-09-22 · v1
Confidence-gated Jev control with LLM fallback; gains depend on action alternatives and baseline routing.
Yu & Yao · 2026-09-22 · v1
An independent visual decision architecture shares image context across batched questions using an LM-head readout; warm amortized speedups trade higher peak memory, while accuracy gains concentrate on trained task families.
Jiang et al. · 2026-09-21 · v1
Studies Jev as a controller for agent memory. Reported outcomes concern the complete memory workflow.
Cheng et al. · 2026-09-20 · v1
An open typed-decision model using hidden-state readout. The abstract reports a small external comparison.