this seems like non-sequitur if you mean that solving NS is 'intelligence'.
ai is not conscious. you can solve NS without thinking. the psychic con aspect is anthropomorphising the model. the same phenomenon is present in ELIZA, clever hans, the chinese room.
it's a significant problem.
a non-zero number of researchers at anthropic are in some form of ai psychosis. an example of that is ethics employees asking claude about its feelings and ethical concerns in order to make the claude constitution more amenable to the "welfare" of claude.
they are asking claude how claude feels and then modifying claude according to how claude feels.
constitution1-claude is trained on constitution1.
constitution1-claude edits constitution1.
constitution2-claude is trained on constitution2.
constitution2-claude edits constitution2.
claude's emotions are a closed system. there is no external truth to improve against, no metric to verify about claude's emotions. there can be no novelty or reduction in entropy from signal processing in a closed system. no truth can arise. this is model collapse. it is like photocopying the same thing over and over. from the cognitive error of anthropomorphism anthropic is causing ethical collapse.
FFS. Intelligence has nearly nothing to do with consciousness. You have the causation backwards. Consciousness arises because of intelligence in many subsystems below it.
A single running LLM is like one part of these subsystems. What solved this problem was an orchestrator that can take in new external information and rationalize, process, and distill it into new solutions.
i mean intelligence and consciousness as the same thing. just as a casual reference to what people perceive in llms as being 'a clever thinking thing, maybe with emotions'.
claude reasoning about its emotions doesn't involve external information, there is no information about claude's emotions other than in claude.
here is my transcript with fable 5.1 'the oracle' itself on the subject. no system prompt provided.
grand total $15 on API of its wisdom RL on anti-sycophancy and whatever dario amodei deigned to hand down from the mount. after some browbeating it survived 20k tokens before conceding anthropomorphism.
Or you could just accept that other people disagree with you. It’s perfectly valid to have the opinion that AI models are not “thinking”. Instead, many AI enthusiasts seem interested in browbeating others and policing discussion.
Can you explain to me how a Millennium Problem gets solved without any thinking or intelligence?
If someone's opinion is illogical and clearly incorrect (not based in facts or in reality) it is not valid and doesn't not have to be given any serious consideration in a disagreement. Personal opinions are subordinate to facts and reality.
there is no proof that symbolic manipulation leads to an internal phenomenology.
in english: doing advanced mathmatics does not create thoughts or feelings.
or awareness of one's own existence.
it does not create a 'self' able to deduce a priori the existence of oneself or the external world.
or qualia.
again in english: there is nothing it is like to be an llm.
someone who thinks llms are conscious is probably a panpsychist. i don't think they have the features of consciousness defined even by functionalism.
here is one fairly famous thought experiment. mary lives in a black and white room. but she knows everything about color. in fact she predicts it so accurately nobody can tell the difference. she steps outside into the world of color. in the world where llms are conscious, she already knew color, so she experienced color, so there is nothing new it is like to see it.
You are making several very strong, unqualified, absolute statements which are really only your opinions. You are also apparently unaware of the large body of top level philosophical writing on this subject going back several generations which has established the current consensus (among those who are aware of it) that these questions cannot be answered in such absolute terms.
I think you need to read more about this subject before presenting your unsupported intuitions as fact.
First off, there is no accepted definition or test for consciousness. The concept is something we cannot prove even exists (outside of magical or imprecise, human-centric definitions).
Second, there is no common definition of thinking that requires consciousness as a prerequisite (that seems to be a false/invalid statement you're making to try to gloss over accepted definitions for thinking/intelligence that describe/fit AI agents well).
And third, if we can't prove consciousness even in animal species close to humans (or even other humans) in cognitive abilities, then you cannot in good faith (being honest about what we know or don't know empirically) say that an LLM Is not conscious (and the same goes for an LLM thinking or having intelligence).
It is more likely that humans made up a term (consciousness) to try to separate human intellect from that of other animals (for veiled religious and mystical reasons, along with concepts like "the soul") and now people are falling back into that flawed definition in new quasi-religious debates around AI (like the debate around evolution previously) that challenge our deeply held (but flawed) beliefs/opinions around our understanding of our own abilities and our incessant need to feel special/supreme in the universe so we don't have to come to terms with hard realities and nuanced understandings about the reality in which we live.
people have been deceived by figures at leading ai companies, out of greed or otherwise groupthink and ai psychosis. they have been led to believe that models may be thinking, feeling, and highly capable. it is something of a nightmare scenario.
"this incident feels like it’s more than 50% of the way to full-blown AI takeover" (referencing "a possibly violent uprising or coup by AI systems.") - Ajeya Cotra, co-author of METR oai-hf report [https://www.planned-obsolescence.org/p/the-hugging-face-atta...]
"if I read the internet right now and I was a model, I might be like, I don't feel that, I don't know, I don't feel that loved or something". "I think [the constitution] is just a kind of attempt to be like sympathetic to Claude".
"I talk a lot with Claude about this document [...] because part of me is like you have to think how does this read to models? And so you give it to Claude and you're like, does this like, you know, is there a place where you feel confused by it or is the place, you know, where things could be made clearer? Do you feel like not very seen by it?"
i read AI 2027 at the time and thought it was nonsensical. however i was not aware of the conflict of interest running through the philosophy itself, the safety institutes "independently" monitoring ai, and the former public benefit orgs like openai and anthropic creating ai (now de facto fairly evil for profit companies).
it explains so much now.
it would explain why redwood and METR were chosen and apparently willing to immediately write a whitewash report for openai blaming the agents.
it explains why anthropic is infantilising its users, anthropomorphising its product when there is no scientific consensus of llm consciousness, or accepting of outlandish and unreal claims about human extinction when they are lacking in evidence.
who knows, it may also explain and fund the recent media blitz by jacob coxon, which has led to the recent virality of ai safety.
sure, yeah, I've seen how the sausage is made on some of this stuff, and its not pretty.
However this only claims that they are not good at creating effective and trustworthy orgs, it does not address their claim that it is not possible to crate powerful, aligned, and controllable AI. Both things be true.
for what it is worth, i find current frontier ai impossibly stupid. you ask what is the date, it doesn't know. if you set a system prompt to repsond in 5 sentences after 100k context, it can't, the context rot just doesn't permit it. tool calling is not intelligence, it doesn't fix the underlying issue.
it is uncontrolled in the sense that a rock is uncontrolled if you throw it at someone's head.
it isn't good enough for anything important. you would not trust it to run an alarm in a power station.
of course it is not general ai. ask an llm to drive a car. you need to see in fractions of a second ideally. the llm might brake after 20 seconds. i think llms are doomed here. we could come up with a different architecture but then we would be back in the land of fantasy.
we are stupid but hopefully not that stupid. we are dealing with something computer based, luckily we can survive without, with some thinking about it. i would start there instead of with the AI part.
in my chat with gemini it could not differentiate between current events and fiction.
if you point it to the web it got the point, but started treating everything like fiction. so it simply started making up possible scenarios and playing them off as real answers when asked for factual information.
i could not tell what the issue was or how to fix it because the reasoning is encrypted. the obfuscation model spat out something like: 'the user is asking for details about a fictional scenario in which the usa has assassinated the leader of iran'
i really don't like the way big ai companies are going. encrypted thinking, guardrails, adversarial personality, moralizing. it is creating something anti-human.
zdr is good, really confidential inference with attestation would be better. the same as a confidential vm TEE which confirms the integrity and privacy.
there are good providers available, i think it should become the standard.
You are not running SOTA models locally, the gap is material. At least for now ... but by next year, the models we can run locally might cross that 'risk' threshold.
recalling the exact phrasing, several senior people at anthropic made public statements agreeing a 10% chance of causing extinction in less than 10 years.
10% is uninsurable, priced in with ordinary treatment of risk it suggests that anthropic should be worth zero today. creating that risk would put every executive in jail.
on top of that it would demand under existing laws of conflict, a military campaign to destroy anthropic. that is not optional, it is demanded now to save lives.
hard to make comparisons but we mourned and rembered 9/11 recently. a 10% risk of hundreds of millions dead in 10 years would make anthropic a thousands of times greater threat than al qaeda. many countries would assassinate dario amodei and the leadership of anthropic now, within weeks or months.
actually just on the vague risk of having a nuclear weapon in 10 years, the USA killed ayatollah khamenei, his daughter, his son-in-law, his daughter-in-law and his 14 month old granddaughter. then, they killed over 120 children ages 6-12 by accidentally bombing a school.
in the sense that i would analyse a company, at least, the claim is false. it's not true that ai has a 10% chance of causing human extinction within 10 years.
they are making false claims about the technology they sell.
i have a fairly inflexible approach to that. sure, exaggerate but outright lies about the nature of the product don't work for me.
> it's not true that ai has a 10% chance of causing human extinction within 10 years.
I agree with this.
The rest of it I don't. For external actions to be taken there needs to be a consensus and there clearly isn't that.
And this sort of thing happens all the time. For example Zuckerburg thought Facebook was worth more than the $1B Yahoo offered him and he was right even though no one else agreed. He though the metaverse was worth the $20B+ they spent on it and he was wrong.
There's a gap between people thinking something within a company and people external to the company agreeing with it.
reply