The more I see about how Jev works, the less interested I get. Jev would be cool as a local model for e.g. Home Assistant. Projects like this are at best misleading, this one also happens to be broken.
Strange - I had the opposite experience. The more I learned about Jev the more interesting I thought it was. This Syntax video got me excited about developing with Jev: https://youtu.be/QbYBRjOaGOo?si=Up0QbW8tWCZQzZR4
Are people not getting that this (Jev) can do classification, programmatic branching, real time decision making (e.g. applicable to robotics) an order of magnitude faster and cheaper?
But also effectively this is a classification model. It excels at specific certain types of workloads, and obviously will fail at others. Not really sure how one benchmarks this tbf. I can see their argument on why this requires a novel specific eval for whatever your usecase is. A consistent "global" benchmark might be hard to do
I've implemented tree-sitter in pi before, and while it works, I have no real proof it saves me tokens, or is more accurate. I think a better implementation is a model that's trained for AST's, not just "use tool, see what happens".
I'd love to do research on this when I have the time.
This sounds good but so far all claims just sound like marketing terms. I'd love to see real proof. e.g. "RLCD" and "parallel sampling" have nothing to back it up.
also "70-500ms vs 3-329 seconds" are apples-to-oranges unless the LLM baseline is doing comparable work (e.g., long chain-of-thought). If Jev is skipping generation entirely for a narrow structured task, of course it's faster.
Nonetheless i want this to be true, so I'm looking forward to Jev
They have various benchmarks, e.g. how much time it takes them to do wikipedia page -> page games. Jev seems to take the same or fewer hops but in ~10x less time and for ~10x less money.
It's totally reasonable to compare against LLMs doing chain of thought if it gets comparable performance.
> If Jev is skipping generation entirely for a narrow structured task, of course it's faster
I think this is reasonable if people are actually using LLMs to solve this type of narrow structured task, which they are. The evidence is that every LLM provider has some method of forcing the output to conform to a json schema in their documentation.
BTW it was not multi model playing doom, it was passing structured input and getting structured output. Its not what I thought: frames of video passed and real time game play.
> I really have to say that I like their manifesto
Their manifesto: "you only build on top of it if it's trustworthy." - the irony of this while putting out the most misleading, dishonest marketing campaign I've seen in months for their first public appearance doesn't exactly scream "trustworthy" to me.
Yes this is true, you even got the model correct. I relied too much on Claude this time. I'm starting to work on the next AI agent lesson this week. I'll do better on the upcoming lesson. The next lesson is A2A focused
Sure. I tried Open WebUI, SillyTavern, AnythingLLM, etc. I wasn't a fan of installing a 500MB desktop app, or having to run a webhost locally, or having to configure a bunch of settings just to chat with the newest LLM.
i quite liked using strix. last time i tried it, deepseek was a mess and bloated the context with nonsense. that was ~5 months ago, i wonder how it performs now
reply