Jev means permission to play for product and UX builders
Everyone is excited about Jev, a new "system 1 model". Simon prefers "decision model" and that does sound better ('system 1 is too fancy pants friends.)
And sure, it's based on a BERT classifier model, that were already really good at classifying things. I'll argue it does in fact introduce a qualitative jump in capability in terms of UX and product experiences we can now build.
No training means no meetings. You get what a BERT-like classifier does without training one first, and that means you don't need to set up meetings, get budget, create alignment etc. to "train a model", which is how things work in most of the real world.
Cheap enough to run on everything. At around 100x cheaper than an LLM, you can run them over huge amounts of data. That changes the kinds of things you can now do.
And really fast. They answer in (100s of) milliseconds, so you can build them into the UI and give users real-time feedback. Or you can make it do a game.
Let's go through it.
Jev turns a classification into an IF statement
Jev launched to a lot of excitement among engineers. It's a decision model: in your code you can write, if this support email is from an angry customer, do X. You could already do that with a large language model, but an LLM is slow and expensive because it generates text to reach an answer. Jev returns the decision in milliseconds, at around a hundredth of the cost.
BERT-like classifiers had to be trained first which means we need organizational alignment (ugh!)
BERT-like classifiers have existed for years, but mostly inside large companies with engineering time to spare.
You decided what to classify, labeled a pile of examples by hand, set up the tooling for that, and then engineering trained the model.
That meant finding and budgeting their time. So now the barrier to start using classifiers is 100x (or 1000x) compared with Jev, where you can just throw it in and get results. (Yes I know your cool startup isn't like this, most of the world is.)
This I think is the main thing, BERT classifiers were used rarely. Jev can be used all the time, it removes two big obstacles:
I don't have to label a data set up front (because let's be honest, nobody wants to do that)
I don't have to get budget/alignment/etc., in other words, I don't have to talk to humans and do internal politics. (Because let's be honest, nobody wants to do that either.)
So because it is now easy and fast to use, we can explore a lot more product and UX ideas much faster. You can see this happen in real time: Examples, examples, examples, patterns, deep dive, ad blocker, games, skeptic.
Decision models are fast enough to sit inside the interface
For product and UX, speed is the big one. At 50 to 100 milliseconds there's little perceivable delay (less is better), so a decision model can respond to the user in real time. It can give feedback as someone types. It can even play a game, deciding left, right, up or down on every move. Experiences that needed a model call per interaction are suddenly on the table, and we can iterate on them much faster.
Evals still matter, but after you build, which is healthy
You still need labeled data sometimes, but not up front.
With a BERT-like classifier, you built the golden dataset up front, because you couldn't train anything without it.
With a decision model, you implement first and let it run. And then you might need to create evals.
Only then do you test and spot-check (writing evals up front is a common mistake people make when starting to learn evals). When something looks wrong, like items landing in the wrong category, you write an eval for that problem: a small labeled dataset aimed at that one issue.
Before, you needed datasets to get started at all. Now you only build them for the risks you actually find. A lot of it will just run fine, and that's fine too.
BERT-like classifiers are back. They just come pre-trained now, and they're called decision models.