IntroducingKev

Today I'm releasing Kev 1.0, a family of four open source decision models, from 0.8B to 27B parameters. This release includes a new Kev-27B and an updated Kev-9B.

TypeSafe's Jev takes text, questions, and possible answers and returns a probability for each answer. I wanted that interface with weights I could run and fine-tune myself. Kev uses the same API, so applications built with TypeSafe's SDK can switch by changing the endpoint and model name.

All four models have Apache-2.0 weights on Hugging Face. The code is on GitHub, including a server that runs on GPUs and on Apple Silicon through MLX. You can try Kev-4B in your browser or use the agent skills to deploy and fine-tune your own.

The Kev family

Kev-27B leads overall in my evaluations. On consumer complaints, Kev-9B and Kev-4B are within a percentage point of it; on developer-tool decisions, Kev-9B matches it. The table separates tasks excluded from Kev's fine-tuning from held-out examples of task families it trained on.

I recommend Kev-27B if you want the highest overall scores and have an 80 GB H100 for shorter inputs, or an H200 or B200 for long documents. Otherwise, start with Kev-4B and measure it on your own examples.

For local use, Kev-0.8B and Kev-4B run on Apple Silicon through MLX. On a 32 GB M5, Kev-4B answers five questions about a short text in 721 ms, or 136 ms when the text is cached; Kev-0.8B takes 149 ms for new text. Kev-9B is expected to fit a 32 GB Mac, and Kev-27B a 96–128 GB Mac. I haven't measured those two on Macs yet.

How Kev works

Kev adds a pointer head, a layer that scores the supplied answer options, to a language model. It supports yes/no questions, multiple choice, and ordered scales. The document is processed once, and each question uses that cached computation without seeing the other questions.

Caching saves more work on longer documents. On an H200, Kev-27B takes about 9.4 seconds of model time to read a new 64k-token document and about 0.7 seconds for subsequent questions using its cached state.

Training the new models

What's new in Kev-27B

The smaller Kevs use Qwen3.5 backbones with LoRA adapters: small sets of additional weights trained while the backbone stays fixed. For the new Kev-27B, I used Qwen3.8-27B, Qwen's post-trained release, and fine-tuned every weight of its text backbone.

Training the whole backbone on the previous Kev-27B's data did no better than the adapter-trained version. The new release also uses a broader corpus: about 146,000 examples and 337,000 questions, with documents up to 32k tokens. It combines Kev's existing data with licensed public tasks, synthetic decisions written by open-weight models, and documents and agent logs generated by code with exact answers.

I added examples for tone, grounding, prompt injection, and personal-data detection, and screened the training records against the frozen evaluation sets for exact and near matches. No Jev outputs were used. The model card documents the data sources and training recipe, including the final weight blend with the previous checkpoint. The full training corpus is not public.

Processing each training question separately would repeat the document computation. I share that computation across the document's questions, which made training two to three times faster without changing the objective. One full run took about 16 hours on eight H200s, roughly $650 in GPU time for that run, not the whole research effort.

Compared with the previous Kev-27B, short-input accuracy is roughly unchanged. The new model is less accurate and more overconfident on long legal contracts. For that workload, compare it with jaredpalmer/kev-27b@v1-lora on your own documents.

What's new in Kev-9B

Kev-9B had fallen behind Kev-4B on policy and rule reasoning, developer tools, and consumer complaints because it hadn't received the same additional training. One more fine-tuning pass, about three hours on one GPU, closes that gap.

Those gains are on held-out examples of the task families it trained on. On a separate held-out short-text decision test, accuracy remains 85%, while its probability estimates improve. The previous version remains available at jaredpalmer/kev-9b@v1.

Evaluation

Generalization

To test tasks excluded from Kev's fine-tuning, I assembled 14 public datasets spanning intent classification, retrieval, language understanding, tool selection, knowledge, and subjective judgments. They include choosing among 150 customer intents and deciding whether a contract supports a claim. These results use a separate test split from the development examples that guided training.

The family table reports ordinary accuracy. Here I use a chance-corrected index: random guessing is 0, perfect performance is 100, and the five task areas count equally. Kev-27B scores 52.3 versus Kev-9B's 41.0. These are index scores, not percentages of correct answers.

The gap is largest in language understanding and smallest in tool selection, where Kev-4B is already competitive within the family. The models differ in backbone and training recipe as well as size, so this comparison doesn't isolate the effect of size or establish an advantage over a prompted chat model.

Calibration

Calibration measures whether a model's probabilities match how often its answers are correct. On the pooled 14-dataset test, Kev-27B's answers in the roughly 85%-confidence bin are correct about 85% of the time.

Pooling can hide overconfidence on one dataset and underconfidence on another, so I report both pooled and per-dataset calibration error.

To test how many questions could clear a confidence threshold, I chose a threshold for each model targeting 5% error on development data, then applied it unchanged to test. Kev-27B accepted 54% of questions with 5.5% observed error; Kev-9B accepted 36% with 6.7% error. Kev-4B accepted 31% with 3.5% error, and Kev-0.8B 15% with 5.4% error.

Those error rates won't necessarily hold on your data. Choose a threshold using representative examples, then measure it on a separate sample. Below the threshold, your code can call a larger model, retain a default, or decline the action.

To adjust the probabilities after training, I fit a temperature, a single scaling factor applied to the option scores before they're converted to probabilities. It changes confidence without changing which answer ranks first. For the new 27B and 9B, that factor is fitted on held-out datasets excluded from fine-tuning and ships with the checkpoint.

Evaluation notes

The results come from my own harness, not an independent leaderboard. The family table identifies test and development panels. Development results guided the training recipes, and earlier candidates' test results were known before I selected the released 27B, so selection was not fully blind. Qwen's training data is not fully known either; excluding a task from Kev's fine-tuning doesn't rule out exposure in the base model. Uncertainty intervals describe variation across evaluation records, not repeated training runs.

Policy and rule labels are computed by code; developer-tool labels come from human annotations, heuristics, or construction. The complaints are real CFPB narratives with model-assisted label checks. In a 50-example spot check, I agreed with 47 labels. The model cards link the measurements, label audits, and selection history.

Fine-tuning on your data

During Kev-4B's development, one pass over 5,219 labelled consumer complaints improved accuracy from 80% to 90% on held-out complaints. On a separate example support workload, about 1,000 generated examples improved accuracy from 68% to 74%. Those are workload-specific results; a few hundred examples can produce a gain too small to distinguish from noise.

I packaged that workflow into an official kev-finetune skill. Your coding agent helps define the questions, prepares labelled data or generates examples, fine-tunes a released Kev, fits its temperature on a separate calibration split, and compares the result with the starting checkpoint before deployment.

The companion kev-deploy skill makes it easy to deploy Kev to Modal behind an authenticated HTTPS endpoint. Both skills work with Claude Code, Codex, Devin, and other coding agents. You don't need a local GPU or a clone of the Kev repository.

A small Kev-4B fine-tuning run costs about $1 in GPU time, excluding data generation and evaluation. Serving scales to zero when idle; the trade-off is a cold start, about 35 seconds in the measured setup.

Limitations

Kev only scores the options you supply. It doesn't generate explanations or retrieve missing facts. You need to supply the relevant policy or evidence and handle the possibility that none of the options is correct. Answer order can also shift probabilities slightly.

Kev-27B was trained on documents up to 32k tokens and evaluated up to 64k. It runs on a single H100 for shorter inputs, but the measured 64k-token case used more than 80 GB of memory and needs an H200 or B200. The smaller Kevs were trained on documents up to about 7,500 tokens.

The code, weights, and evaluation reports are available, but the full 27B training corpus and some evaluation data are private. The model cards document what others can reproduce.

Get started

The weights are on Hugging Face, and the code, model cards, and serving instructions are on GitHub. You can try Kev-4B in your browser before setting anything up.

Install the skills, then ask your coding agent to “deploy Kev-4B” or “fine-tune Kev on my support tickets”:

npx skills add jaredpalmer/kev@kev-deploy
npx skills add jaredpalmer/kev@kev-finetune

Thanks to TypeSafe for building Jev and an API worth being compatible with, to Archer Hume for the write-up of Jev's architecture that started this, to Qwen for the base models, to denis-pplx for AutoJev, and to the teams behind the open-weight models that wrote Kev's synthetic data.