state in, a choice out.▌
A curated list of System One decision models — Jev and open alternatives — plus the agents, tools, SDKs, and benchmarks built around them. You give them some state and a set of typed questions. They give back a choice, a score, or a yes/no probability.
latest
- oct 13OneJevmodels
- oct 13Strands Decidermodels
- oct 13JevEmbedopen-implementations
- oct 13JevK5open-implementations
- oct 13RSI-Jevopen-implementations
- oct 13jev-socialagents
- oct 13system1-agentsagents
- oct 13Bud Decision Studiotools
added per week
entries [added]
76
80 ┤┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈ ▃▃▃▃ ┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈┈
│ 66 ████
│ ▇▇▇▇ ████
60 ┤┈┈┈┈┈┈┈┈┈ ████ ┈ ████ ┈┈┈┈┈┈┈┈ 49 ┈┈
│ ████ ████ ▃▃▃▃
│ ████ ████ ████
40 ┤┈┈┈┈┈┈┈┈┈ ████ ┈ ████ ┈┈┈┈┈┈┈┈ ████ ┈┈
│ ████ ████ ████
│ ████ ████ 17 ████
20 ┤┈┈┈┈┈┈┈┈┈ ████ ┈ ████ ┈ ▄▄▄▄ ┈ ████ ┈┈
│ ████ ████ ████ ████
│ 0 ████ ████ ████ ████
0 └──────────────────────────────────────
sep 09 sep 16 sep 23 sep 30 oct 07
rankings
- 1Laya+2.3k33kmodels
- 2jev-ultrafast+94123kagents
- 3kev+6979.1kopen-implementations
- 4jevgrep+4622.7ktools
- 5pg-jev+4491.2ktools
- 6jev-chat-jarvis+4287.7kagents
- 7jev-browser-use+3521.2kagents
- 8QuantDinger+34013kagents
- 9Strands Decider+299604models
- 10fast-jev-compaction+2867.6ktools
stars · gain over 7 days, sparkline is the last 6 months
- 1Laya+2.3k33kmodels
- 2fastText+527kbaselines
- 3jev-ultrafast+94123kagents
- 4json-render+16819ktools
- 5QuantDinger+34013kagents
- 6kev+6979.1kopen-implementations
- 7jev-chat-jarvis+4287.7kagents
- 8fast-jev-compaction+2867.6ktools
- 9laya-mlx+1766.9ksdks-adapters
- 10vLLM Semantic Router+1226.1ktools
stars
- 1Clef2.1kmodels
- 2Jev Decision Index622benchmarks
- 3GLiNER2.5-Decide408models
- 4Jev-Omni406models
- 5LocalLLaMA/typed-decisions180benchmarks
- 6Quyet39models
- 7Vela37models
- 8fast-decisions27benchmarks
- 9NeuDecide11models
- 10DecisionBench2benchmarks
likes · history builds up daily
- 1GLiNER2.5-Decide101kmodels
- 2LocalLLaMA/typed-decisions43kbenchmarks
- 3Clef17kmodels
- 4Jev-Omni4.8kmodels
- 5Vela2.2kmodels
- 6fast-decisions2.1kbenchmarks
- 7PhishNChips2kbenchmarks
- 8DecisionBench1.8kbenchmarks
- 9Quyet1.1kmodels
downloads, 30d · history builds up daily
- 1Clef444models
- 2Jev Decision Index151benchmarks
- 3LocalLLaMA/typed-decisions41benchmarks
- 4Vela37models
- 5GLiNER2.5-Decide33models
- 6Jev-Omni32models
- 7Quyet29models
- 8NeuDecide11models
- 9fast-decisions4benchmarks
hf trending score · history builds up daily
##Models
Call a hosted model. Jev for the original API, CLM or Laya for open weights.
Hosted or open. State in, typed answers out.
### Open weights 12
Cloudflare's open decision-model family, Apache 2.0: frozen Qwen backbones with a trained routing head, prefill-only non-autoregressive scoring. Clef (27B) and Clef-flash (9B) take text + image with a 64K context; Clef-omni (Qwen3-Omni 30B-A3B MoE) adds native audio and video — a 21 s clip with sound scores in ~1.5 s, text in ~130 ms. Cloudflare's runs put Clef first on the Jev Decision Index, while Jev and Laya still lead some tasks. Hosted on Workers AI — Clef-flash at $0.038/M input undercuts Jev — with an RL fine-tuning platform, in production on Threat Intelligence.
Contrastive Language Model: dual state/action encoders (frozen Qwen3-8B + 20M heads) where candidates embed once and cache, rather than re-reading the state. Apache 2.0, TypeSafe wire-compatible. Claims Jev parity on computer-use, gaming, and tool-calling at up to 9× lower latency, plus SOTA held-out verifier results (Terminal-Bench 2.1, DeepSWE).
Research stack for small specialist computer-use models. First checkpoint is form-oriented: cua-s1-forms, a 706K-param option scorer with its training dataset on Hugging Face. Planning and execution stay separate.
Liquid AI's open decision-model family: d1-3B (text + vision, from LFM2.5-VL) and d1-omni-600M (adds audio). Single forward pass; d1-3B scores 48.57 on the Decision Index public split — best under 10B, on par with Decider 35B-A3B at 12× its size — at ~8 ms on an RTX 4090, 16 ms on Jetson AGX Thor. Open weights and GGUF builds on Hugging Face; also hosted on OpenRouter, where it ranks #2 on JevBench's hosted board.
Decision specialist from the GLiNER team: any label set at call time — intent, routing, multi-label tags, ordinal scores — answered in one forward pass with no generated tokens. Apache 2.0, in 340M / 1B / multilingual builds. Claims 60.2% vs JevK5's 57.6% on Fastino's own 17-domain suite; on JevBench's harder multi-step items it scored near chance — a 340M classifier built for simpler task shapes.
Multimodal decision classifier on Gemma 4 12B: text, image, audio, or video in, a probability per option out. Apache 2.0 weights. One question per call, best under about 20 options. Currently #1 on Image JevBench.
Multilingual, non-autoregressive decision engine. Typed
choice/score/noulover 100+ languages in a single forward pass (about 33 ms on a T4). Router across English, multilingual, and typed-decisions checkpoints. Apache 2.0.Neuphonic's ~40 MB voice-action model: speech in, an action decision out, running entirely on WebAssembly in the browser. From the neuTTS/neucodec audio lab; model repo is gated, the Space is the public demo.
Multimodal decision family from the PlayJev authors: calibrated probabilities over screenshots, photos, video, and text in one forward pass, in 0.8B / 4B / 9B / 27B sizes. Apache 2.0 weights plus a Hugging Face dataset, with published comparisons against Jev 1.13 and Jev-Omni.
Open five-size family (Tiny to a 31B Large) on Gemma 4: merged LoRA plus a letter-readout decision prompt with fitted temperatures. TypeSafe
/v1/systemoneanswers via thequyetPython package; English and Vietnamese, Apache 2.0, GGUF builds. The speed runner-up in Microsoft's Decision-1 benchmark post.The Strands Agents org's open decision-model family (weights live under Hugging Face's
amazonorg too): four Gemma 4 builds from E2B to 26B-A4B, each answer carrying a calibrated reliability score, with JevBench, Brier, and Decision Index numbers published per size. Apache 2.0, ships a CLI.KR Labs × vLLM Semantic Router's open decision-model family (0.3B–9B, bidirectional ModernBERT encoder, 17 languages, Apache 2.0). Choice, Noul, and Score plus two new question types: Span — labeled text spans with offsets for PII and unsupported claims — and Set multi-label. SystemOne-compatible HTTP and Python APIs, ONNX runtime, and a Core ML port with a Swift runtime for Apple silicon. Trained for router work: routing, prompt-attack, PII, and hallucination screening.
### Hosted 14
Perplexity's decision model (Decider V1.1 27B is the current checkpoint), on the System One request shape. On OpenRouter.
Hanzo's hosted decision model on the OpenRouter Decisions wire:
model: "kai"atapi.hanzo.ai/v1/decisions, with Jev itself routable on the same endpoint. A Jev client works by repointing the base URL.ZooWork's hosted decision model in two tiers: a Qwen3.8-27B build and a cheaper dual-4B. Ranks #9 of 27 on JevBench's hosted board; the base is operator-reported and weights are closed.
TypeSafe's System One model. Choice, Score, and Noul over a state in one request, about 70–500 ms, trained with RLCD. Hosted API ($0.042 per million input tokens, output free); weights unpublished. JevBench's hosted board ranks it #4 of 27. Also on OpenRouter, Vercel AI Gateway, Cloudflare Workers AI, and LLMGateway. Docs · Launch post
TypeSafe's meta-application of Jev: a model that picks the best model and reasoning effort for each request, balancing quality, speed, and cost.
Inception's structured decision model on the System One schema, from the diffusion-LLM lab behind Mercury. A free route exists on OpenRouter. Ranks #3 on JevBench's hosted board.
Microsoft's decision-scoring model: Qwen3.5-9B post-trained for single-pass choice/noul/score plus rubric grading, calibrated probabilities over a fixed answer set. Microsoft's 36-benchmark, ~150K-question blind eval claims top accuracy and fastest measured (~35× faster than GPT-6 Sol), with a 1.3% flip rate under input perturbations. Live in Microsoft Foundry; OpenRouter planned. Already used internally by Xbox Research and Copilot.
OpenAI's entry in the class, announced at DevDay 2026: text or image context plus a fixed answer set, returning one answer with a confidence score around 150–230 ms. Powered by a Luna variant. Also routable on OpenRouter as
openai/gpt-6-luna-decisions.Levanto Labs' hosted decision model. Currently #1 on JevBench's hosted composite (74.0 vs Jev's 71.5), winning on speed and cost rather than raw intelligence. Closed API.
Upstage's structured decision model on Solar Mini 4 (beta). Same
/v1/systemoneschema as Jev, calibrated probability per answer, and a 512K context — a whole document can be the state.choiceis limited to 26 single-token letter options. On OpenRouter, with a low-latency Flash variant.Respan's behavioral-monitoring decision model: describe a behavior (user frustration, tool misuse) and get present / absent / not-observable probabilities on a conversation. A free Lite tier exists for high-volume monitoring. Vendor-run benchmarks put it ahead of Jev overall but behind GPT-6 Sol per-domain.
SurdAI's hosted decision system (Flash and Pro tiers): a router/cascade on the
/v1/systemoneshape rather than a single model. Flash ranks #7 on JevBench's hosted board.Subquadratic's long-context decision model: multimodal input with a 384K context (versus Jev's 32K) on a proprietary subquadratic sparse-attention stack. Their runs beat Jev on long-context retrieval (Decision LCR 74 vs 48) and on 13 of 16 enterprise suites by +6.0 on average, while still trailing on When2Call. $0.05/M input, output free.
Alphanimble's hosted decision model with an auto-reasoning mode. Ranks #5 on JevBench's hosted board, above Jev on the composite. Closed API.
##Open Implementations
Train or run one yourself.
Same request shape on open weights. Not TypeSafe's architecture.
### Trained replicas 23
AgentJev-0.6B: a small trained decision model that takes unstructured state (diffs, traces, logs) and returns calibrated distributions in one ~50 ms forward pass. Apache 2.0.
Qwen3.5 fine-tunes (0.8B up to a 35B MoE, plus a vision build), Apache 2.0 weights on Hugging Face. The vision build ranks #2 on Image JevBench. Serves TypeSafe
/v1/systemone, so the official SDK works by repointingTYPESAFE_BASE_URL.Nace.AI's open decision models: v1.5 on a distilled Qwen3.5-9B and DLM on an Efficient-DLM-8B diffusion backbone, each serving
/v1/systemonethree ways: Python/vLLM, a llama.cpp GGUF fork, or Ollama.SberAI's Russian-first encoder decision model:
choice/score/noulplus arankingquestion type, all questions scored in one packed encoder pass with an exact state cache. Torch, int8 ONNX, MLX, or vLLM.HIT-TMG's recipe for turning an embedding model into a decision model:
choice/score/noulheads over your pick of embedder, with a Hugging Face model collection and dataset. Apache 2.0.Open-weight Jev alternative (4B and 9B, Apache 2.0) serving
/v1/systemoneat about 13 ms on an H100. JevBench v1.4 ranked v0.2 second of 76 systems and first open entrant, though the project itself reports Jev edging it on 308 fresh sealed items. Two-teacher distillation, pre-registered epoch sweeps.Train a small one-pass scorer over a changing list of text options. Each option queries the context; a shared head returns one probability per option. Byte encoder from scratch, or a frozen Hugging Face encoder.
MiniCPM5-1B cut to 17 layers with a one-logit head, quantized to GGUF and run on CPU via llama.cpp. Serves TypeSafe
/v1/systemonefornoul(yes/no) questions only, so the official SDK works by repointingTYPESAFE_BASE_URL. MIT.Small trained decision models on Qwen3.5 (0.8B and 4B) serving
/v1/systemoneat ~50 ms on one H100. jiwo-0.8b ranks #1 of 27 under 1B on the Decision Index board; both models are on the board. Apache 2.0.LoRA plus a pointer readout on Qwen3.5 (0.8B / 4B / 9B), Apache 2.0, with training code and frozen eval suites. TypeSafe
/v1/systemonedrop-in, calibrated by default via a fitted temperature. Frozen evals put Kev-9B about 3.5 points behind Jev on unseen sources; MLX, ROCm, CUDA, and a one-command Modal deploy. Also the only open-weights route on OpenRouter's Decisions API.0.6B replica trained from scratch, MIT. Ships the weights, the dataset, and the end-to-end training pipeline, plus a side-by-side demo against Jev.
Qwen3.5-9B LoRA from Bespoke Labs. Contrastive recipe, not distilled from Jev. 90% vs Jev 93% on a 324-example holdout.
150M ModernBERT encoder that answers per-request
noul/choice/scorequestions in one pass. Trains on a Colab T4 in about 30 minutes. Not the same project as SemIf, which was also once called OpenJev.151M ModernBERT decision engine, non-autoregressive, with published calibration numbers. Claims the top spot over Jev and Laya on the LocalLLaMA typed-decisions benchmark.
Largha-150M: a 144M bilingual decision model on mmBERT-small (English-first, Chinese evaluated on translated cases). Calibrated
noul/choice/scoreanswers robust to option reordering, one pass per decision.0.8B multimodal Jev-like model that plays GUI games directly from raw pixels. Apache 2.0.
Independent reproduction of Jev-style post-training on MiniCPM5-2B via LoRA: 51% → 80.5% on a sealed, leakage-checked holdout for about $5 of compute. Stage 1 covers single-letter
choiceonly; the rigor is the point — significance testing, baselines, and a technical report that voids its own figure after finding contamination. MIT.Recursively self-improving research loop that trains the models: AI agents propose hypotheses, register predictions before spending GPU time, and retire their own champions on the evidence. Eight releases in 13 days, 496 published experiments including failures; v6.1-VL 27B posts 65.64 on the Decision Index 0.3 public set, the top listed score. MIT code, Apache 2.0 weights, Jev-compatible API.
Jev-style replica on Gemma 4 E2B / Gemma 3 270M, trained and served on Modal. One forward pass, no decoding. Live demos for support, Doom, browser-use, and smart home.
Together AI's Jev-inspired experiment: Qwen3.5 fine-tunes that take state + question + 2–24 options and return one letter — openly next-token, not a non-autoregressive runtime. MIT, with full data recipe and a "train your own for $17" writeup. A 4B experimental build is also served on OpenRouter.
Train-your-own multimodal System One model: evaluates text, images, and video against task instructions and returns probabilities over supplied candidates. Ships the model code, data pipeline, SFT plus experimental RLCD training, and evals.
Typed decisions from a frozen LM plus a small physics engine, from Laya's author: calibrated probabilities, conformal answer sets, and an explicit abstain flag. 800M or 4B, 73K context, image inputs.
ModernBERT-base fine-tune with a decision head that answers
noul/choice/scoreand also multi-label questions, with a fitted temperature per question type. Publishes its held-out numbers: 61.2% accuracy and 0.043 ECE on datasets it never trained on. Apache 2.0 code and weights; serves TypeSafe/v1/systemone, so the official SDK works by repointingTYPESAFE_BASE_URL, and an ONNX build runs in the browser.
### Serve any open model 16
Turns any LLM into a Jev-style decision model: typed decisions with probabilities, no training. From Nokia Applied Research, Apache 2.0.
DiffusionGemma plus vLLM: an inference method rather than new weights, with native image inputs and a hosted API. Ranked 3rd on JevBench v1.2, just under Jev itself.
Self-hosted drop-in on GLiFormer 400M, MIT. Speaks the TypeSafe request shape (JevBench drove it with the official adapter); ranked in v1.2.2.
Jev-style parallel constrained decisions for any MLX model on Apple Silicon. Scores every allowed answer per schema field in one pass, so the output is valid by construction.
Educational Jev-like visual inference experiment on Apple Silicon: shared context, direct candidate scoring, and local visual demos.
Adapts local language models into Jev-compatible decision engines with Choice / Score / Noul outputs, via prefill-only binary inference. Apache 2.0.
Ollama for decision models:
ollaya pull/serve/runmanage Laya, decider, NLI, and GLiClass locally behind a wire-identical TypeSafe/v1/systemone(and/v1/decisions) endpoint.System One-style layer on open weights, ranked on JevBench. Its cautionary detail: 72% vs 21% on the same answer-judging items with option order flipped.
Open, Jev-compatible System One decision server on DiffusionGemma. Apache 2.0.
TypeSafe
/v1/systemoneon Qwen 35B MoE via SGLang. Prefill once, then first-token logits per question. No extra training.Open recreation on Qwen3.5. Prefills the state once, then scores every question in parallel from next-token logits. Serves the TypeSafe request shape. Browser demo on WebGPU. #3 on Image JevBench in its released stable configuration.
Runtime-defined semantic decisions from direct option logits. CUDA, MLX, and WebGPU, plus committed benchmarks. Repo recently renamed from SemIf.
Turns any open model into a classifier / Jev endpoint. From Featherless, Apache 2.0.
Turns any open LLM into a System One classifier: batched single-token choice inference, TypeSafe SDK compatible. Doom and wikiracing demos on Qwen3-8B.
Apple's Neural Engine as a free local System One decision engine on macOS: "Jev at home."
Non-autoregressive local drop-in, sub-15 ms decisions, Apache 2.0.
##Agents
Need it to pick the next click or tap.
The model picks an action. Code runs it.
### Screen & device control 7
Drives native macOS apps through the Accessibility API. Jev chooses the next action and target token from a live snapshot, and stops if confidence drops too low.
Headless browser agent. Jev picks click / type / select / done from the page's elements; code owns budgets and stop gates. MCP server, CLI, or library.
Split-brain browser agent: Jev clicks, Codex thinks and verifies.
Browser agent with a dynamic, indexed action space. Jev picks an operation and an element in one request; a small LLM writes text only when the operation is
TYPE_TEXT. Zürich to London on Google Flights in 7.1 seconds.Android agent on a real phone via Mobilerun. Jev picks
OPEN_APP/TAP/TYPE_TEXTfrom an indexed snapshot. Text is copied from the goal, not generated.Local computer-use app for macOS where Jev is the default engine: it picks from locally detected controls and the app executes and validates each action.
Computer use for about $0.0002 a step: OCR the screen, Jev classifies the next action, code clicks. macOS.
### Domain agents 10
MuJoCo robot decision workbench: MiniCPM-2B or Jev-class APIs pick the next action, with physics previews.
Phone chat copilot for WeChat / QQ / X / Feishu: reads the screen, suggests candidate replies, one tap to fill. Read-only — no app hooks.
Camera-only autonomous drone in MuJoCo with Jev in the loop at 2.5 Hz.
Local-first social-media research agent for Instagram, TikTok, and LinkedIn: Jev picks each next operation (search, open a post or profile, read comments, finish), a local CLI executes it in your signed-in Chrome, and every observed result goes back to Jev. Source-cited reports, no account credentials leave the machine.
Experimental: one Jev buy/sell decision every Monad block (about 300 ms) on the Kuru MON-USDC book. Dry-run by default.
Jev navigates a Neo4j graph by classifying over each node's neighboring relationships: graph traversal as a chain of typed choices.
Open-source AI trading OS with a Jev decision gate in front of live entry orders: typed Choice plus an auditable decision timeline, fail-open, and exits that always bypass AI.
Prebuilt agents that run a System 1 model (Jev, Laya, or Cua-S1) as the brain across browser use, computer use, robotics, and games, with Claude Code or Codex scaffolding new ones. Publishes per-scenario timings: up to 6× faster and 25× cheaper than a chat model at the same score.
Home Assistant conversation agent. Jev maps a spoken command plus exposed entities onto built-in intents. Free-text intents fall back to another agent.
Jev plays Super Mario Bros. from structured emulator state: the classic RL demo with a decision model in the loop.
##Tools
Ready-made plugin or CLI.
Plugins and CLIs that put a System One model inside other software.
### Context & memory 6
Token-saver for agent sessions with a documented Jev routing layer.
Claude Code plugin that replaces the compaction summary with Jev decisions: every tool call and result is scored in one request and the stale ones dropped.
Invalidation layer for AI memory: every fact gets a lease, and new evidence ends it.
Zilliz's agent memory layer, with optional Jev reranking and a published Chinese/English reranking evaluation.
Local proxy that learns your app's typed LLM decisions and answers them itself with a Laya head. Jev- and OpenAI-compatible.
Claude Code hook that splits large tool results into blocks, asks Jev a yes/no relevance question per block, and stubs the ones that fail.
### Routing & supervision 10
Multi-agent collaboration across Slack, Telegram, and GitHub, using Jev for agent routing, model selection, and support triage.
Adaptive reasoning effort: Jev judges each Codex step and turns GPT-6's thinking effort up or down mid-task, so easy steps stop burning tokens.
Jev as a fast supervisor above slower coding agents, judging whether the work is complete, the requirements met, the tests sufficient, or a human is needed.
Jev-powered skill suite for Hermes agents (also Claude Code and Codex): model routing, memory, compaction, skill selection, computer and browser use.
Intent compiler that converges vague requests into typed
IntentSpeccontracts — probe, ask, or halt — before routing. The input layer in front of routers and decision models like Jev or Laya.Per-turn model and reasoning-effort routing for Codex. Jev classifies each turn and picks the tier; a 7-day replay of 237 real turns measured about 60% savings against all-frontier.
Jev-powered router across models, tools, and subagents.
Calibrated judge in front of a frontier model: a small local model answers routine decisions under a distribution-free bound on disagreement; everything else escalates, and each escalation teaches the judge further. Includes an MCP server.
Pi coding-agent extension: semantic tool and skill routing, typed
choice/noul/scoreevaluations, optional auto-approval, and a post-runjev-gateCLI.Open, programmable decision layer for models and compute: routes requests across models, enforces safety checks, and picks reasoning effort — with the Vela decision models as its routing brain. Apache 2.0.
### Review & verification 6
Stops coding agents from claiming done without evidence: deterministic hooks decide what needs checking, Jev judges the result.
Staged code-review workflow with a local dashboard, using Jev to score and sort findings before a human opens the queue.
Semantic code linting: rules written in plain English, judged by Jev.
Verification harness for agent-built code; the
jevdriver explores a page by choosing among DOM-enumerated candidates — the model chooses, never composes.Plain-English browser tests on Playwright where Jev only picks targets and next actions (Choice) and judges claims (holds/contradicted Nouls), and code owns every action and verdict. TypeScript, MIT.
Prose linter that sniffs out AI writing tells: zero-dependency countable rules plus one judgment model.
### Search & data 9
Codebase search powered by Jev decisions: typed questions find where a task lives in a repo.
Very fast document classifier and PDF splitter from the creator of LlamaIndex: plain-English rules, Jev assigns each page a category and finds document boundaries.
Dataset sifter for synthetic and pretraining corpora: a Rust streaming core sends Jev keep/drop decisions over Parquet rows.
Web search pipeline where Jev handles source selection, query understanding, and relevance ranking.
Recursive Jev choice over a taxonomy: selects from more than 255 options without hitting the choice cap.
Semantic AI for Pandas and Polars: classify and score DataFrame rows with natural-language questions and full probabilities.
Codebase search for coding agents: Jev judges folders, files, and declarations, then returns reading leads and verbatim source excerpts. npm CLI plus an agent-skill installer; matched the baseline 8/10 on a ten-task SWE-bench at ~30% lower cost.
Grep where the pattern is a description: filters lines by meaning for about a thousandth of a cent each.
PostgreSQL extension (on PGXN):
jev(rows, 'plain-language condition')filters, ranks, and classifies table rows. The clever part — it batches 20 rows into one shared-state request with anouleach, amortizing Jev's request overhead ~2.5×, with read-ahead and per-row answer caching.
### Train & adapt 3
Trace mining for post-training: reads agent traces (OTel, Langfuse, Claude Code), judges each step by the tool's reaction rather than the agent's claim, distills the LLM judge into deterministic Python checks a human signs off on, and exports labeled SFT rows with full lineage. Its bundled Jev recipe trains a decision model on those labels with an honest scorecard — their Qwen3.5-4B reports 79.1% vs Jev's 66.8% on 1,920 held-out AgentProcessBench steps.
Build calibrated AI functions from human feedback: Jev judgments optimized with GEPA. From Sutro.
Turns Jev usage into small local classifiers:
--teacher typesafe-ai/jevlabels the data, then a MiniLM model runs it in Node or the browser.
### Apps & utilities 8
Vercel Labs terminal client. The
evaluatecommand sends a state plus typed questions totypesafe-ai/jevon AI Gateway (-m jevfor short).The LM Studio of decision models, from Bud Ecosystem: a cross-platform desktop app that checks your hardware, pulls Jev-like models, and serves them locally. Give a model a situation and typed questions, get calibrated probabilities in milliseconds.
Zero-shot text classification as a URL:
classifier.dev/spam,not+spam/your+textreturns a label with calibrated confidence — no key, no account, up to 1,000 texts per call. One Cloudflare Worker on Jev with an LLM fallback chain, plus a CLI and MCP server. MIT.Open-source Gmail triage: sorts an inbox into needs-reply / updates / promos / spam. Read-only, runs locally, about 3 cents per 1,000 emails.
Local visualizer for Jev: a live view of every call your code makes.
Vercel Labs generative-UI framework with an experimental Jev mode: Jev picks components from your catalog instead of a model writing markup.
Jev calls as Unix-pipeline predicates: routing, scoring, and JSONL filtering with stable exit codes, plus an agent-safety guard recipe.
Browser extension that removes page clutter: Jev scores elements against reusable rules, the extension hides the junk.
##SDKs & Adapters
Calling from application code.
### Official 4
Official Vercel AI SDK evaluation provider. Choice, Score, and Boolean (Noul) over one state via
experimental_evaluate.Official agent skill for designing System One workflows from Claude Code, Codex, and other skill-compatible agents.
Official TypeScript / JavaScript client.
Official synchronous and asynchronous Python client.
### Community clients 5
Unofficial clients for Go, Rust, Ruby, PHP, .NET, Elixir, and Swift, tracked by language. All launched in Jev's first week — check the last commit before depending on one.
Native Swift 6 bridge that puts Jev behind Apple's Foundation Models framework interface.
Simon Willison's
llmplugin for the OpenAI Decisions API: evaluate text and images with predicates, choices, and scores.Simon Willison's
llmplugin for Jev and other TypeSafe models.TypeScript wrapper where every decision needs a required
unsurebranch — ignoring the uncertainty case is a compile error.
### Frameworks & integrations 8
Vercel's open agent framework, where Jev powers the
evaluateprimitive and tool-approval policies.Semantic HTTP routing middleware for Hono: route requests by meaning, from Hono's creator.
MCP server of Jev judgment tools: verify, screen, find, rerank, classify, decide, compare, extract, review, and gate.
Native MLX runtime for Laya on Apple Silicon: 7–14 ms decisions on an M3 Max, no GPU server needed.
Run Laya from Node.js / TypeScript via ONNX Runtime; the ~1.7 GB weights download from Hugging Face on first use.
Provider-neutral System One runtime for TypeScript and Pi: write the
choice/noul/scoredecision once behind aSystemOneProviderinterface, run it on Jev, Reflex, or any compatible/v1/systemoneendpoint.Drop-in
TypeSafeClientreplacement backed by OpenAI or Anthropic, for comparing Jev against an LLM on the same questions.MCP server as a single static Go binary, no Node or Python runtime. Sends usage guidance back to the client so the agent writes better questions.
##Baselines
Want a cheaper classifier first. Same job, different interface.
Cheaper classifiers for comparison. They do not share Jev's request shape.
Classic non-generative text classification library. Upstream archived in 2024.
Lightweight zero-shot structured extraction: you name the entity types at inference time. Often the comparison point in Jev classification benches.
Fastino successor to GLiNER. Schema-conditioned encoder for classification, extraction, and relations in one pass.
68 pretrained logistic-regression classifiers that run and retrain on CPU: banking77 at 94.3%, CLINC-OOS, spam, plus game-state deciders for Doom and poker. The "do you even need a decision model" baseline; pip package with a live playground. MIT.
Modernized BERT encoder under most fast classifiers, including Laya's English checkpoint. The non-generative baseline Jev's cost and speed usually get measured against.
Trains and serves routers that pick which LLM should handle a query, based on cost and expected quality, rather than answering the question itself.
Embedding-space route layer for LLMs and agents. Chooses a path from utterance similarity without a generative call.
Prompt-free few-shot text classification on Sentence Transformers. Strong when labels are fixed at train time.
##Benchmarks
Compare models.
### Suites & harnesses 28
Shopping-query relevance classification (exact / substitute / complement / irrelevant) at Amazon scale. A Jev Decision Index component.
Alibaba's tool-use benchmark for whether a model picks the right API call. A Jev Decision Index component.
AI2's grade-school science MCQs in Easy and Challenge splits. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
Audio-decision counterpart to Image JevBench, for the multimodal entrants (Clef-omni, d1-omni, NeuDecide).
77 fine-grained banking intents over 13k queries. Single-domain counterpart to CLINC150, and the benchmark Janus reports wins on. CC BY 4.0.
Berkeley Function Calling Leaderboard: the field's standard for function/tool-call accuracy (ICML 2025). A Jev Decision Index component.
Reasoning-intensive retrieval benchmark scored by nDCG — the Jev Decision Index's reranking component, and one Jev still leads.
150 in-scope intents plus explicit out-of-scope examples. Useful for routing, abstention, and confidence-threshold tests. CC BY 3.0.
80 scenarios and 293 questions per tier (medium is rules-based, hard is judgment calls), with a cost-per-decision axis baked in. What Jev-Omni reports on. Apache 2.0.
TypeSafe's workflow evals: four production-shaped workflows scored against frontier LLM reference probabilities.
Fastino's held-out suite: 17 operational domains × 300 examples each, scoring Jev, SemIf, Laya, and GLiFormer head-to-head on the same inputs. First-party to GLiNER2.5-Decide, which tops it.
Graduate-level, Google-proof MCQs; the Diamond subset is the hard tier. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
Grade-school math word problems; repurposed as multiple choice with random distractors (4- and 10-option conditions in the Jev vs Luna comparison).
Commonsense sentence-completion MCQs. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
Held-out image-decision suite: 684 items (228 public / 456 sealed) with frozen hashes and contamination tracking — it flagged Kev's Mind2Web training overlap itself. Jev-Omni, decider-vision, and Reflex top it; frontier chat APIs score near zero.
Community index aggregating ~43 decision-model evals — intent classification, function calling, tool retrieval, reranking — plus latency. The benchmark surface Clef's launch numbers report on.
Independent decision-model leaderboard (run by Benchmark Heaven, not TypeSafe): 1,500 typed decisions per system scored on intelligence, calibration, latency, and cost, with versioned releases, published methodology, and result JSON. Two boards — open weights and hosted API — across ~75 systems; the hosted board currently ranks Sage #1, d1 #2, Mercury Decide #3, Jev #4.
Agent evals and guardrails as Jev decisions: one request per trace. Runs locally on Kev or Laya. From Openlayer.
Community typed-decision suites (customer service, invoices, security incidents, agent traces) the open implementations report accuracy and calibration against. Apache 2.0.
Multilingual intent-and-slot dataset (about 1M utterances, 52 languages) for testing whether a fast decision layer generalizes. CC BY 4.0.
The standard 57-subject multitask MCQ benchmark. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
Harder MMLU successor with 10-option questions and more reasoning. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
Multistep soft-reasoning MCQs (murder mysteries, object placements, team allocation). Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
2,000-email phishing benchmark plus a 220k-run adjudicated grid showing how system-prompt config swings detection. A Jev Decision Index component.
Head-to-head of System One models: Laya vs Jev on byte-identical inputs.
ACL 2025 tool-retrieval benchmark: 7.6k retrieval tasks over a 43k-tool corpus, scored by nDCG@10. A Jev Decision Index component.
Evaluates tool-calling decisions rather than accuracy: when to call, when to ask a follow-up, when to admit the tools can't answer. A Jev Decision Index component — and one Jev still leads.
Pronoun-resolution commonsense MCQs, adversarially filtered. Part of the standard MCQ battery used in Jev vs Luna Decisions comparisons.
### Probes & studies 6
Measures a confidence threshold on your labeled data, then routes Jev vs a larger model. Ships no default: on Banking77 routing wins; on Web of Science it says do not route.
Independent look at Jev 1.13.0: option-order bias, facts lost in the middle of long context, and cases where a correct prerequisite still leads to the wrong action. Raw payloads and offline verification included.
Independent calibration, selective-risk, and latency measurements of Jev against GLiNER and other baselines.
Independent calibration test on tasks Jev cannot have seen: 900 rule-generated tickets plus public benchmarks, with ECE noise floors and a temperature refit.
Jev vs Claude Haiku 4.5 on 2,000 phishing emails: accuracy, calibration, latency, and cost, reproducible.
Jev vs Cohere Rerank 4 vs zerank-2 vs a chat baseline on 14 datasets. Every raw response saved. nDCG@10 is a tie, not a win.
##Reading
The psychology the name comes from.
### Docs & ecosystem 11
How Choice/Score confidence differs from probability, and how to gate act / review / escalate in code. Noul has no separate confidence field.
Official patterns: parallel questions, citation check, RAG passage classification, date extraction, skill suggestion.
Keep control flow in code. Split work into atomic questions, then combine the answers and route on uncertainty.
TypeSafe's launch post: RLCD, parallel sampling, and how this differs from an LLM.
A data-driven survey and analysis of 2,170 public GitHub Jev projects, mapping early ecosystem growth, application domains, and decision-use patterns.
Official definition of the model class and the Choice / Score / Noul primitives.
Community directory indexing about a thousand Jev projects with daily-refreshed stars. Less curated, good for exhaustive search.
Community store for Jev apps, extensions, and tools: a distribution surface with a submit flow, distinct from the firehose directories.
Launch-day writeup on what a decision-only model changes for latency and cost.
Alpha multi-provider surface for the class: ~10 routes from nine publishers (Jev, Solar Decide, Span-01, Kev-4B, Clef, Luna Decisions, d1, Decider, Mercury Decide, Tev1) under one
state+questionsrequest shape. Browse the full decision-model catalog, or check the live usage rankings — Jev is still over 95% of real traffic.Independent hub tracking every System One model, open alternative, community SDK, and example. The place to check when this list lags.
### Jev under the microscope 4
Rao & Callison-Burch (2026). The counterpoint to the cascade story: Jev is 29–325× cheaper and competitive on binary criteria, but LLM judges repeat nearly all of its most confident errors, capping escalation gains at ~1.5 points.
Li, Miao, Krishnan & Padman / CMU (2026). First independent study of Jev as an eval judge: within ~3 points of the strongest LLM judge on ordinary preference at 0.36% of the fee, weaker on derivation-checking. A confidence-gated cascade keeps 99% of the big judge's accuracy at ~57% of the cost.
First-party failure modes: counting, dates, indirection, distractors, generation. Keep arithmetic in code.
Community teardown of Jev's probable architecture; the write-up kev's open models are built from.
### Foundations 10
Christakopoulou, Mourad & Matarić / DeepMind (2024). Splits an agent into a fast conversational Talker (System 1) and a slow planning Reasoner (System 2).
Nish Tahir (2026). The clearest mechanism explainer yet: constrained option tokens and a softmax over answer letters versus autoregressive JSON generation, with an interactive demo, working Qwen code, and an honest note that raw token probabilities are not calibrated confidence.
Yu, Xu, Weston & Kulikov / Meta (2024). Compiles intermediate-thought techniques into single-pass outputs; the research framing closest to what Jev is.
Evans (2008). Review of dual-process theories. System 1 is a family of accounts, not one algorithm.
Tversky & Kahneman (1974). Heuristic judgment under uncertainty.
Masked diffusion language model (NeurIPS 2024): parallel, non-autoregressive generation. The open research line nearest to Jev's parallel-sampler claims.
Gigerenzer & Goldstein (1996). Simple heuristics that work with incomplete information.
Learns when to use fast direct planning and when to search, instead of always picking one.
Weston & Sukhbaatar / Meta (2023). The LLM regenerates the context it should attend to before answering: deliberate filtering on top of fast attention.
Daniel Kahneman. The book TypeSafe cites for the System 1 / System 2 framing.
nothing matches. try fewer words, or clear a filter.