There's kind of a new model type out now which I think is quite interesting: System One models, starting with one called Jev[1].
It's not really built for things like chat apps, so most people probably won't interact with this type of model directly for a while. But I still think it's super interesting.
Essentially, it's a really fast and cheap classifier. Something like 200x faster, 10-100x cheaper input tokens and totally free output tokens (compared with traditional LLMs). The trade-off is that its use cases are a bit more specific, whereas the LLMs we're used to are very generic. Instead of generating a long response, it's more about making quick judgements, like deciding if something matches a description or not.
My initial reaction to this model type was that it's just structured outputs... who cares. But I do think the speed, cost and parallelisation make it very interesting. The docs are certianly worth a read to understand some potential use cases.
One use case I've experimented with is a Chrome extension called Unsee[2]. You describe any type of content you don't want to see, and it uses Jev to decide if content on the page matches your description. If it does, it gets blocked. For example, you can say you don't want to see celebrity gossip (or ads, or other inappropriate stuff) and those get removed from the page.
In the screenshot you can see it highlighting celebrity gossip, clickbait headlines and ads, each with a confidence score. You can also add exceptions, like hiding graphic violence unless it's serious news reporting. It found 526 things to hide on that one page, and you can just describe whatever else you'd want to block.
This was already possible with existing models, but the cost and speed meant it wasn't something you could realistically use while browsing. You'd be waiting ages for every page and paying a fair bit for the privilege. With Jev, it works in real time.
I'm interested to see if this ends up just being hype or if this type of model will be here to stay. It's technically already enabling things that previously weren't possible unless existing models got way faster and cheaper and then just did structured output.
I'm looking forward to seeing what other ideas come from this and what people end up building with it.
References
- "Introducing System One models and Jev" by TypeSafe
- Unsee on GitHub