Awesome Jev

Studies fitted confidence thresholds for sharing agent control with decision models.

A benchmark spanning matched requests and closed-loop tasks. Experiments have not been reproduced here.

In ten prepared scientific cases, the authors separate semantic selections, computed quantities and final labels; comparison-model errors can be hidden by unchanged final labels.

Gates reasoning start, candidate-answer reading and generation stopping; evidence is limited to a frozen Qwen backbone and three task families.

Tests classification and verification under content-preserving table rewrites across three task families; confidence can flag instability without proving correctness.

Audits ordinal label underuse across selected decision models; finer scales increase compression, while accuracy, final-label utilization and averaged candidate probabilities remain distinct measures.

Text and vision attacks with label-validity checks and held-out evaluation models.

A decision-model architecture exploring null handling, option permutations, scaling, and caching. Its design is Jeva’s, not Jev’s disclosed internals.

Structured video decisions with temporal spans and candidate scores; cached and joint inference reuse visual computation.

Pairs factual probes with action choices to diagnose decision failures.

Tool-argument judges including Jev reject some correct derived values without visible references; model-generated references are evaluated on three benchmarks.

A controlled adversarial comparison of Jev, LLM judges, and purpose-built safety classifiers.

Compares frozen Qwen decision readouts with optional KL-anchored fine-tuning and budgeted reasoning; reported gains depend on backbone size and routing workload, not disclosure of hosted Jev internals.

Compares rubric judges and examines correlated errors that can limit cascade improvements.

Answer-letter readout and supervised LoRA baselines; evaluation includes model-generated labels.

Bilingual decision evaluation and reliability when local decisions compose into larger systems.

Jev memory-retirement evaluation separates confirmed updates from suggestions; mixed LLM/human labels, shared errors and selection on a small development set constrain cascade claims.

Looped hidden-state computation for typed decisions; additional loops trade compute for accuracy.

Candidate-menu interventions and confidence cascades; ranking stability does not establish probability calibration.

Preliminary medical-benchmark evaluation of Jev 1.13 reports task-dependent accuracy, calibration and weak unanswerable-question detection; it does not establish clinical deployment validity.

ContractNLI request variations can change which answers are wrong while aggregate accuracy stays unchanged; stable answers may preserve errors. This benchmark does not validate real legal deployment.

Studies confidence-based escalation from Jev to a reasoning judge, including limits under distribution shift.

Tests option-channel attacks on seven open-weight models; reversed decisions can remain confident. The reported experiments do not test hosted Jev.

Knowledge, reasoning and multilingual multiple-choice evaluation of Jev; reported rankings and mathematical weaknesses apply to the selected models and benchmarks.

Bi-encoder approach to multimodal decisions and retrieval through natural-language interfaces.

Optimizes context additions around unchanged questions and options in four decision systems; results concern initially correct decisions, and human validity checks cover a sampled subset of successful attacks.

Tests conflicts between option names and definitions despite structurally valid outputs.

Same-number citation swaps probe financial-evidence verification; rejecting wrong roles trades off with accepting equivalent support in a constructed test.

Studies accuracy, cost, and calibration in seven political-science replication tasks.

A common-request harness compares typed decision models, generative readouts and trained classifiers; accuracy, selective coverage and out-of-scope acceptance depend on label access and task.

English/Arabic persona-conditioned Jev survey audit; the effects concern simulated respondents in one model, not measurements of real populations.

Examines reliability, group-specific confident errors, and selective automation in security decisions.

Turkish typed-decision evaluation reports decision quality, Brier-based probability quality and risk–coverage alongside input probes; most labels are AI-derived, and the lab’s model used feedback from earlier test results.

Controls information and surface cues to distinguish Jev answer confidence from self-knowledge; black-box behavior does not establish access to an internal knowledge state.

Training-free decision readout with label-prior and option-order correction.

Composes finite action fields and explores alternatives using interaction feedback.

Shared evidence encoding and candidate-probability readout, trained with representation and action-outcome feedback.

An early benchmark study of Jev. Read the task selection and evaluation setup alongside its reported results.

A Chinese typed-decision model and proposed benchmark; domain-specific training calls for independent validation.

Keeps raw memory records, separating Jev relevance judgments from LLM search planning and answer generation.

A controlled calibration study using constructed probabilities. Interpretations of latent uncertainty remain hypotheses.

Compares Jev and generative judges on retrospective agent-trace risk classification; dataset rankings vary, and successful-call latency and valid-judgment cost must be read with coverage.

Tests coherence between logically related probabilities, separately from accuracy and calibration.

An evidence audit and a checklist for evaluating typed decision models.

A black-box adversarial benchmark. A changed decision is not necessarily an incorrect decision.

Estimates whether asking for missing information would improve a typed decision.

Benchmarks Jev alignment-failure detection while separating question wording from input context; benchmark-scorer labels dominate, and context gains often depend on fields encoding the label.

A Jev-style visual-choice study separating adaptation gains, candidate-readout validity, and calibration limits.

Audits Jev injection on reconstructed InjecAgent cases, separating probability shifts from attacker-target selections; optimization results are checked on fresh calls, within this benchmark setting.

Jev checks support between generated and reference radiology reports in both directions; expert agreement depends on report size and error definitions and does not establish prospective clinical effectiveness.

Jev state tables and sparse flagship-model anchors support population simulations with shared human–model discrepancy estimates; larger simulations do not add human evidence or establish prospective intervention effects.

Evaluates zero-shot social-science annotation and confidence-routed LLM cascades; lower costs coexist with task-dependent errors and overconfidence, so confidence cannot substitute for label validation.

Confidence-gated Jev control with LLM fallback; gains depend on action alternatives and baseline routing.

An independent visual decision architecture shares image context across batched questions using an LM-head readout; warm amortized speedups trade higher peak memory, while accuracy gains concentrate on trained task families.

Studies Jev as a controller for agent memory. Reported outcomes concern the complete memory workflow.

An open typed-decision model using hidden-state readout. The abstract reports a small external comparison.

Turkish typed-decision checkpoint with shipped temperature scaling. Its card flags test-informed training and worsening support-task calibration; benchmark results are not blind.

Author checkpoint with backbone, adapter and decision-head files. The inspected card declares no model license; weights have not been loaded or tested by this project.

An encoder-based decision family with English, multilingual, and workflow-specific checkpoints.

A Qwen3.5-4B-Base LoRA adapter and pointer head for decisions over documents and typed questions.

A Qwen3.5-9B LoRA adapter returning candidate probabilities. Released under a noncommercial license.

A decision-tuned Qwen checkpoint with a structured inference wrapper and image support.

A separate decision readout and saved calibration temperature. Follow the card’s attention-mode and access notes.

A 340M English classifier with runtime label sets and single-label or multi-label decisions.

A text, image, and video decision model with Choice, Yes/No, and Score outputs.

A multimodal model with a joint schema head, returning probabilities over allowed answers.

The official sync and async client for typed Jev questions and answers.

The official JavaScript client with answer types inferred from your questions.

Compare LLMs through the same question interface, with probability outputs and request diagnostics.

Official guidance for coding agents building with the System One API.

Source code accompanying official documentation, demos, and cookbooks.

TypeSafe-maintained nodes for adding typed decisions to n8n workflows.

Official dataset for the invoice-processing workflow evaluation.

Official dataset for the customer-service workflow evaluation.

Official dataset for the security-incident workflow evaluation.

Official evaluation dataset for the agent-trace observability workflow.

A community training mixture for System One-style decision models. Read the card for its composition and usage conditions.

An archive of Decision Index evaluation runs for Bespoke Nimble 9B v3.

Recorded-response analysis and new API evaluations use identical-input reruns as a control. Most labels are model-derived; this project has not executed the analyses.

Interactive factual-probe and action-decision diagnostics. The current report covers 22 systems, distinct from the fixed paper’s 18; it is not a verified code or data release.

A community index of decision models and evaluations. Full rankings include private tests.

Inspect four structured workflows. Scores measure agreement with model-generated references.

The official evaluation runner, workflow definitions, scoring, and plotting tools.

Published cases, model-generated references, and recorded runs for inspecting reproducibility.

External evaluation of classification, tool calling, knowledge and reasoning. The suite includes private questions.

Author-run labeled decisions and workflow replays. Private customer cases limit public reproducibility.

The launch article. A primary source for the provider’s design claims and their stated caveats.

Choose between categorical decisions, ordered ratings, and yes/no probabilities.

How confidence is derived from a distribution, and how it differs from probability.

Documented limitations: numeric precision, long irrelevant context, adversarial inputs, and option-order sensitivity.

Combine narrow risk questions with application rules for passing, reviewing, or blocking content.

Check whether a citation’s source context supports a claim.

Use Choice probabilities to explore deep classification trees.

Check inexpensive extraction with per-field Jev questions before escalation; the walkthrough uses cached and illustrative outputs.

Restore Markdown structure through line-continuation and block-type decisions while code retains the original wording.

A foundation for evaluating and incentivizing honest probabilistic predictions.

A practical reference for calibration and temperature scaling.

Studies prediction with a reject option and the trade-off between coverage and risk.

An investigation of language models’ self-evaluation and uncertainty.

A programming-language approach to constrained language-model interaction.

Efficient grammar-constrained generation: a useful comparison point for typed outputs.

Declarative language-model programs and pipeline optimization.

Efficient execution of structured language-model programs.