One thing that sort of worked for me is to write the skeleton myself, i.e. the general ideas and how they fit together, the overall “flow” of what I want to say, then have an LLM fill in the blanks (I either point it at some context.md or have it ask me questions when details are missing).
It’s kind of like designing the high level software architecture yourself and have the LLM write the code for each component.
Not bulletproof, requires some iteration, but miles better than what it would produce on its own.
One thing that you should absolutely never do: have an LLM review and “improve” the text repeatedly in some closed loop.
This might work well for some tasks (coding), but for writing it will absolutely take reasonable text and turn it into a pile of incoherent garbage no human would ever write.
Maybe I’m the only one keeps trying this (more often than I’m willing to admit), but I suspect it’s a common cope engineers reach for when having to deal with the not-so-fun task of writing prose.
We’ve always had output schemas for LLMs, and we’ve had small language classifiers for decades, so what’s new? Is it just some sweet spot in between in terms of quality vs speed?
It's essentially taking output schemas as we've been using them and applying them to specific classification tasks. So not using them to generate structured content which incorporates generated text, but using them to generate structured content which includes classification and/or rankings of the requests made.
So in a lot of cases when we've used LLMs as a classification hack, we've burned a ton of tokens in reasoning and output that we didn't really need to use to interpret the final result. (And I'll just say that we may not have needed all of the output tokens, but that incorporating assessment along with scoring seems to provide more accurate results.)
This goes beyond just asking an LLM to assign an arbitrary number to a particular concept, which in most cases distributes less-than-correct statistically, although that didn't stop us from considering LLM as a judge to be a viable strategy.
So this basically gives us a different class of model to use when classification or decision making is the only need. It doesn't replace any of the narrative if you still need that. Coupled with the higher speed and lower cost, that's why everyone's excited about it.
LLMs are generalized token predictors. They generate.
Jev is a generalized classifier. It does not generate. It computes probabilities, REALLY fast.
So inputs and outputs of LLMs are tokens. Inputs to Jev are state (arbitrary strings/tokens) and, depending on the type of query, either an assertion, options, or choices. (All of those are also arbitrary strings/tokens). Outputs from Jev are probabilities. If it's an assertion, the probability that it is true. For options and choices, it's probabilities for each one, basically.
Because Jev answers so quickly and inexpensively, it's a likely replacement for complex, best-effort functions like `isSpam()`, where up until now the only nondeterministic way of implementing that was an LLM, which is slow, costly, and may produce invalid/corrupt output.
Jev uses a different training architecture called RLCF (Reinforcement Learning from Calibrated Decisions) vs the traditional RLHF that most TF models use.
So at the end of the day the groundbreaking work wasn't the model itself inherently but the way it was trained and then the way the harness interacts with it.
So this demo here is showing the harness side of things afaict but then TypeSafe's Jev takes it a step further via a specific training regimine.
One of the biggest issues with LLMs is that they don't work well as a classifier. They tend to pick up on the patterns of the examples and not the intent of the examples (gets worse the more examples/intents).
1) it's very fast (they claim 40-200x faster than frontier models [1], would roughly line up with it doing diffusion)
2) each answer carries a calibrated probability (ie. frequency of outcome is close to predicted)
Another point being that it doesn't reason, hence designed for "System One" tasks.
I wonder if in continuous control with discrete actions (eg. their DOOM demo) it can make sense to blend answer by confidence instead of taking the argmax.
Anecdotal: LLMs like the hallucinate things and did a poor job of determining when to leave things null/blank. A more structured approach with confidence ratings helps resolve.
I know it's incredibly presumptuous for me, a nobody, to say this to 25 Fields Medalists, but:
Perhaps you are misaligned.
Who decided the goal of math must be human insight?
First off, some mathematical truths might simply be far beyond our biological comprehension.
Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
Isn't this why we have math in the first place? To solve our real problems? Over time it morphed into this pursuit of pure theoretical insight, probably out of necessity at the time, but is it still necessary?
> Second, for us non-mathematicians, the value of math isn't in understanding exactly why a result is true, it’s in how those results can be applied to actually improve our lives.
This is just the problem with the Navier-Stokes solution. We now (supposedly) know the truthiness of the statement, but we have no more understanding of why or how it applies to our quality of life. This is exactly the issue Tao is pointing at.
There’s one thing I constantly see agents tripping over, I’m not sure what the right word for it would be, but it basically boils down to “making changes in the right places”. They seem to have very poor grasp of where things are supposed to be and they have a tendency to work against the existing architecture. Even in a world where agents are the only ones touching the code you can see how this ends poorly. Unlike correctness I’m not sure there’s an easy way to verify.
I tried writing a few skills to encourage agents to spend time thinking about this but it doesn’t seem to generalize very well.
>Quiet is not proof that nobody is recording, and a detection is not proof that anyone is
Is such claudespeak it’s not even annoying it’s just funny at this point
reply