Well yes, there are only so many attention heads (well whatever magical variant frontier models are using these days) that can attend to the context and so as the context grows attention becomes spread thin.
But with reasoning enabled I find that even with a large context that induces mistakes most of the time a decent model realises and corrects itself before output.
And it's always been known that prompting what it should do is far better than what it shouldn't, since just introducing "DON'T do X" into the prompt means that the tokens for X are present and can be paid attention to in the wrong way.
But even then I've used plenty of "Do X, not Y" recently, especially for tools "Use this for x, don't use this for Y" and models perform like 90% of the time.
I would be interesting to experiment to see performance curves given a restriction on reasoning tokens allowed to n% of context tokens and see if there's some magic number of "reasoning should be at least n tokens for a context of length p" even ignoring the complexity of instructions in the prompt itself.
Interesting. I agree that it seems that LLMs are wont to fall into recognisable patterns much more readily than humans however I asked a model to evaluate your comment history and it found loads of patterns such as:
“I’m not sure…” / “I don’t…” hedges before disagreement
Negation followed by correction/reframing is a recurring structure: essentially “It’s not X, it’s Y”.
And dozens of other examples. I wonder if LLMs really are more liable for it or if it simply seems that way purely because when you use a model it's almost as if you're speaking to the same "person". I think that human language is naturally formed by common agreed meanings of words first, and then phrases.
> I asked a model to evaluate your comment history and it found loads of patterns such as
> “I’m not sure…” / “I don’t…” hedges before disagreement Negation followed by correction/reframing is a recurring structure: essentially “It’s not X, it’s Y”.
I did a manual human dig through GP's profile and I don't see any evidence supporting this. The literal strings "I'm not sure" and "I don't" do appear a lot (also in replies, which I'm hoping your LLM was intelligent enough to exclude?), and they are used to hedge before making a statement, but not a single one of them even approaches the "It's not X, it's Y" thing your LLM has claimed they do.
Your LLM did not find dozens of examples, it told you it did and you didn't even bother to check.
Any given person will have phrases or ways of organizing things that they tend towards. But that isn't the same thing. Those patterns are the exception with lots of variance all around them. LLMs lack the same general variance while also going beyond merely favoring a few patterns to the point where you could describe it more as a production line stamping them out. They're also remarkably inefficient with their prose. I think I'm pretty bad on a first pass (see this comment I'm typing right now) but they put me to shame by a huge margin.
And in your estimation, is the LLM correct in qualifying that as a pattern, or is it full of shit?
I could ask an LLM to generate something arguing against whatever yours put out, but I won't. I come here to talk to other people, not their electronic sycophants.
For sure. If you write with a casual conversational tone people don't get triggered. But if you are eloquent and use an extended vocabulary thanks to large amounts of reading then that seems to make people think you're using AI.
As someone who loves to read (well, on and off) I wonder what the median vocabulary is and how I and other readers compare. I guess language has always been functional for most people.
Eh people don't care if something is true or false they care about the dopamine hit they get from engaging in tribal behaviours that reinforce their freaky belief systems that often aren't rooted in any kind of reality.
I think this type of generated content has the same role as illustrations or newspaper caricatures in the past. It doesn't need to be true, it just needs to express a particular viewpoint.
I'm not as heavy on the maths stuff involved in this as other people commenting appear to be.
But the idea makes sense, of course there is still recoverable data in embeddings, that's the point. Though as I constantly find the more you try to squeeze into an n bit vector the more watered down everything gets.
I suppose a latent space could be encrypted/mapped in some way to resolve that, but how many people are exposing their vectors in the first place?
The point of the paper isn't that embeddings contain information, it's that even if you don't know what model generated a set of embeddings you can still recover information from the geometry of the point cloud itself.
The fact that this is possible also adds some pretty strong restriction to the set of possible maps you could use to remove that information. No linear map will work since all embedding spaces are ~an orthonormal matrix apart, so some form of encryption is necessary. This wasn't known until very recently.
But I think my point still stands, isn't the geometry information THE information I referred to in the first place? Obviously the vector size gives you the granularity but it's kind of unavoidable to positionally encode information in a latent space...that's literally what they're for?
But yes, it is very cool to know that regardless of exact implementation finding x,y,z representations of some dataset with various relationships (like language) creates similar geometry/clues across all the implementations.
Besides the fact that this sort of protection through obfuscation is dead now anyway. If you can ask an LLM to do it so can I, or anybody else. The only downside is duplication of work/wasted tokens but eh.
AI has already started commoditising software. Hopefully we see more OS' lean into the "safe" layer that runs everything and then temporary/custom interfaces dynamically created by AI on top.
Tbf "is it a crime" is hard for even a single lawyer to answer because it depends on: who you are, your skin colour, how rich you are, your sex, whether it's a white collar crime or not, did you commit the crime on behalf of a corpo, etc.
But we like to pretend that the justice system delivers justice evenhandedly I suppose.
No it does not. The law defines what a crime is.
That you are bummed out about how it is applied, does not change the question, whether it is a crime.
One thing is the definition of what constitutes a crime/felony/misdemeanor,
another thing is the application and the punishment meted out.
That's not quite the case in common law countries. In countries like the UK and the US, the interpretation of the law as written is ultimately decided by the court, meaning that the definition of what constitutes a crime/felony/misdemeanor is subjective. The more courts that attempt to interpret that specific law, the more 'jurisprudence' builds up, meaning that the interpretation applied in a previous decision can certainly have an impact on a future decision.
Talking to members of the opposite sex only? How narrow-minded. Cyberpunk/sci-fi often explores futures where sex/gender is less of an issue (especially Ian. Banks' works). Clearly we're not there yet.
icc is as strong as it can enforce its strength, if no one can challenge you, you don't play by the same rules everyone else do. Pretending otherwise is asking for trouble.
Yes? Nation-states like the US (and China) are more powerful than any international organization and power dictates what happens in international relations.
I’m not saying it’s right, I’m just saying it is what it is.
The Internet has magnified the "my church leader said that gay people are actually demons" effect. Or "if you throw salt over your shoulder you're protected" or "wolves have alphas of the pack".
People don't fact check in the first place, they don't check that facts came from a trusted source, they don't check if information has changed the last time they got it from a trusted source, etc.
Is it just me or is it scary that it seems to me as if the average person just takes everyone's word for most things, as long as the person telling them is a "trusted source" based only on tribal lines?
But with reasoning enabled I find that even with a large context that induces mistakes most of the time a decent model realises and corrects itself before output.
And it's always been known that prompting what it should do is far better than what it shouldn't, since just introducing "DON'T do X" into the prompt means that the tokens for X are present and can be paid attention to in the wrong way.
But even then I've used plenty of "Do X, not Y" recently, especially for tools "Use this for x, don't use this for Y" and models perform like 90% of the time.
I would be interesting to experiment to see performance curves given a restriction on reasoning tokens allowed to n% of context tokens and see if there's some magic number of "reasoning should be at least n tokens for a context of length p" even ignoring the complexity of instructions in the prompt itself.
reply