Kev: Tiny Jev-like family of decision models built on top of Qwen3.5

These days, I tend to start my coding sessions by the high level problem I'm trying to solve vs the prescriptive, specific solution I may have in mind. It often surfaces ideas and approaches that I did not know about.

Our resulting RBF models are tiny and fit in L1 cache, with microsecond inference latency.

Originally it was so I can label data to fine-tune a VLM, but now a few tiny classifiers that run in milliseconds on cpu.

Now its collecting data to make a domain specific BERT and do what Jev does.

30KB model, 40-50ms inference. Pretty happy with the results so far!

I can see an entire industry of tiny models like this, now that we have AI to help us do the grunt setup work (validation/training data creation, data cleaning, etc). Or just use a general classifier like Jev/Kev ha

I'm assuming you only need to consider a single language for your emails?

This is the same route but WAAAY faster and cheaper. And you can modify it like you do with code or prompts. It's really appealing, TBH.

that is generally a very healthy attitude in the AI space anyway in my opinion.

Some of our R&D departments haven't actually finished an interesting project in years because they keep jumping from trend to trend wanting to try out all the latest shit all the time.

In their privacy policy they say

We (1) will not train or fine tune any artificial intelligence or machine learning models on Input, and (2) will not disclose any Input to a third party other than our service providers.

by not allowing to benchmark it, they make it user-hostile, you don't know for what kind of quality you pay

Does anyone actually believe this?

Anything that required knowledge / familiarity mmBERT and ModernBERT post-trains performed much worse.

So it seems like they did some kind of useful expansive pre-training.

Things that were Qwen or Gemma Diffusion did better at those kinds of tasks but were generally pretty inconsistent in terms of whether they could succeed repeatedly (and be stable + reliable) on the many types of tasks that are in the cookbook part of the Jev docs.

If you ask Jev similar input + questions, it's pretty stable. And does a reasonable job on a lot of questions.

This one public benchmark (the only I've seen) seems to give the open versions way too much credit. It wasn't my experience at all.

It gave my a false wrong sense of what might be required to get it working for something at work to avoid needing a new subprocessor as - at least on Cloudflare / OpenRouter Jev is third-party not hosted.

also tried myself: https://console.typesafe.ai/playground?share=shr_1690a3160f1...

Big if. More likely, it seems, is they started with an open LLM model and fine-tuned and repurposed it via their "RLCD" process.

Jev came in, and added that magic of "you dont need to train your classifier or determine the weights" if you dont want to, and just get the classified answer out. I think that's what is making people see this with a glitter in their eyes.

I'm still trying to catch-up on the Jev stuff, but my understanding is that it's basically just a more efficient LLM when all you want is the LLM to produce a classification.

There's more to it, of course, but it's not just "generic" classification ML because it accepts arbitrary inputs and can produce probabilities over arbitrary classes. Not saying this is the first time people have done this, but typically classification tasks are more static and limited.

In the same vein, it's also not just an LLM with structured outputs (which have been a thing for a while) specifically because that is a very inefficient way to approach classification using this kind of architecture. Jev models are much more performant because of how limited they are compared to a full LLM.

So when you want an LLM, but you only really need this kind of classification from the LLM, then Jev makes a ton of sense. This makes sense for me, since I've definitely used LLMs for this kind of classification work and, even then, it kind of felt like using a jackhammer to place some nails, etc.

Happy to be correct, though.

Compared to traditional ML classification, Jev works without training, like a LLM.

LLMs can act as classifiers but they still have to generate text output in the form of a JSON object. This means they have to generate every single curly bracket, quote, command, etc. This turns out to be pretty expensive. On the other hand, Jev uses a different decision head so it doesn't generate text output at all, it outputs logits *only* for the choices provided. So it completely avoids the need to generate text at all, which means no malformed JSON and it's much faster as a result.

Finally, Jev also provides confidence scores that are actually reliable (not made up like LLMs).

So you have more flexibility to get on with building, evolve your business logic etc

I think specialized models of all sorts, if it were possible to train them on equally much data, would outperform general models. In cases where we can have the data, like in self-play in games, they clearly do and have done so for a long time.

But it's a lot of effort to train specialized models. I think it's too early to tell if Jev delivers on its promises to give the best of both worlds, but if it doesn't, we'll have to make it work. It's just the right thing to try.

Now there's a new training-free thing that is fast enough to be useful on a new class of problem.

If you have a little data and can ask a frontier LLM to train a model, you can probably beat it on average for a specific task.

But... This is the case with LLMs too.

Is it truly useful or accurate or beneficial? To be seen. But it's the idea that has everyone so captivated. An expert system that is an expert at most everything is a lot more useful than an expert system that is an expert at choosing a bar of soap, for instance.

Like even 5 minutes of tinkering captures why this isn't anymore like BERT or any past classification model than ChatGPT is like those old Markov Chain generators, yet folks cannot shut up about how this is nothing new.

Absolutely scary and makes me wonder how much of the field is just people super confidently discrediting otherwise promising/interesting directions for development for a cheap dunk!

Try constructing reusable components out of what you are doing instead of building everything up from basic building blocks. This also allows more concrete testing of individual parts and then if you want to change the look you can change it in one place and have it apply everywhere.

Agentic development doesn’t mean “throw all what we learned out of the window”, the same practices that helped speed up and improve quality of work of humans also helps agents. In fact, the multiplier is even bigger. You will notice it in development speed and reduced cost due to avoiding churn.

For coding agents it'd only be useful in a subset of situations. E.g. you could imagine using one to classify bash tool calls into safe and unsafe for example.

Yeah, it could. The most obvious usage would be to have local fast cheap "feedback" / "control" over a slower more expensive agent (i.e. cc / codex / opencode). Things like "goals" could now be split from a long prompt into "actions" and "verifiers". Where for each action you also produce a verifier. Then after each action you run the verifier w/ this kind of "universal classifier" and decide if the step was done correctly, if it needs follow-up and so on.

Example: implement auth in this repo -> llm_plan() -> for item in plan generate_verifier() -> for item in plan implement() ; verify() ; accept() / followup().

Verifiers could be something like this. take a plan item as input, generate classification questions that might verify the task "is this following project conventions?" | "is this touching files from other tasks?", etc.

You can do that with LLMs, but some things might become cheaper / faster. And you can pretty much use it to check against an ever growing list of conventions. Yours or project specific.

Edit: The Flipper One is planning to have an LLM acceleration co-processor, and be able to host up to a 4GB VRAM size LLM. One use case they envision in their planning is using the microphone along with text to speech to be able to say, "Create an .ini file for this system with these specs" and the small LLM can do that on-device (its a handheld device) and then the user can use/send/upload that file.

Second Edit: I would love a mini LLM in KiCad or Altium that could take a component datasheet and produce a good footprint and schematic symbol for it.

- need fast response

- can tolerate Jev's mediocrity compared to real frontier models

(I explicitly ignore cost, because if you desperately need a cost-optimized classifier you would just build one)

Outside of fun demos these two rarely come together: if it's critical enough to require sub second speed, then it can't be mediocre.

The reason so many OSS models are being built is that Typesafe team made a ton bombastic claims about Jev being a huge breakthrough, and ML folks are realizing they can build Jev-like model in 2 days instead of 2 years.

Edit: I want to add that you can see Jev like a smart if-statement.

--

The setup was a simple map with different rooms. Each room had 1-3 doors.

For the inputs: The AI had an array of "known places" empty at the start, the current position, the current doors with no information about where they lead to, and the list of past actions

The goal / task for the AI was to explore all rooms and save them to known places. The AI needed to decide if to move or save the place at every turn.

--

So I wasn't able to make the AI explore all of the rooms. The AI kinda always wanted to move to the first option when moving. Out of 6 rooms it was able to save 3. My hypothesis is that jev as it is now is really bad at making connections and understanding it's input. So for example, even if it had a list of previous actions, it wasn't able to reason about it and know where to go. For this to work I'd need to explicitly tell it where it did not go. So you could say that the model is also not good with uncertainty / ambiguous scenarios.

edit: one last thing is that i replaced jev with an standard llm and it finished the goal no problem with the same information given

edit 2: it also felt like the same tradeoffs between small model vs large model. With small models you need to be very conscious and careful with the input while large models are more forgiving. Maybe jev is a small model, and we just need a larger one.

As a corollary, the output classes can be any set, rather than needing to be set before training.

The number of people working on this is crazy. Something will coalesce.

You should be excited because there's so many low-hanging classification problems (e.g. "Is this email spam?", "Is this yelp review happy or sad or neutral?") that a "cheap" LLM is overkill for. It really should be this cheap, and now Jev is the first to do it well and do it at scale.

If I was explaining it to my mom, I'd say "Classifying 1000 yelp reviews used to cost $50. Now it costs 5¢, at similar accuracy"

https://openrouter.ai/typesafe/jev-1.13

Here is typesafe’s skills.md https://docs.typesafe.ai/agent-skill (which I don’t think is well written, but it’s a start).

Also read their docs, I think they’re interesting.