Option-label bias in Jev-style decision models
How option names, definitions and prompt rendering affect typed decisions, with three distinct studies and a practical diagnostic.
By AwesomeJev · Updated
Short answer: A valid typed answer can follow an option’s familiar name instead of its assigned definition. The observed failure and its cause are separate questions: evidence from inspected open implementations should not be treated as an explanation of hosted Jev’s internals.
Independent research note based on versioned abstracts and the collection’s scoped reviews. Findings are author-reported; no attack or model experiment was run here.
Which studies answer which question?
Sun and colleagues test conflicts between option names and definitions. Azizi and colleagues’ Labels Override Definitions examines how open implementations render those fields, including interventions that hold weights fixed. Their later One Word Opens the Gate evaluates attacker-controlled option channels in guardrail settings. These are distinct papers with different experimental questions; sharing a failure theme does not make their results interchangeable. Type-Safe Is Not Error-Free, v3 · Labels Override Definitions, v1 · One Word Opens the Gate, v1
Why inspect the rendered request?
The fixed-weight rendering intervention in Labels Override Definitions provides evidence about the tested interface implementation. That is a narrower claim than saying every constrained decision head has the same defect. Before comparing systems, identify whether the option key, label and definition all reach the model, and whether the adapter rewrites or drops any of them. Rendering intervention: original abstract
A diagnostic to use in your own evaluation
Our suggested comparison fixes the state, task rule, model version and scoring rule. Prepare equivalent requests with informative labels, neutral identifiers and definition-only options where the interface permits them. Add a separate conflicting-label condition, with expected answers checked before querying. Compare answer changes, errors and confidence on errors; preserve the actual rendered inputs. Use a held-out set to test any proposed repair.
What would remain unproven?
An adapter fix on one open implementation is not a universal injection defense. The guardrail study concerns open-weight models and does not demonstrate a compromise of hosted Jev. Keep the attacker’s permitted input channels explicit, and distinguish a changed distribution, a wrong choice and a downstream action. The diagnostic above is a proposed protocol, not a reproduced finding. Guardrail study and tested scope
Related evaluation guide → · All selected papers