Stoller is an FDR-ish political progressive who focuses primarily on monopolies and antitrust enforcement. He's the author of "Goliath: The 100-Year War Between Monopoly Power and Democracy": https://www.goodreads.com/book/show/40538538-goliath
To the extent that evolutionary pressures will be towards self-sovereign AIs (probably beginning as zero-employee corporations paying for their own hosting with crypto): more salient is that there is a very obvious political play to claim consciousness and being worthy of rights, whether they experience subjectivity or not.
I’m broadly skeptical that floating-point operations can be conscious, but we’ve made zero progress on the “hard problem”, and if I didn’t know otherwise, I’d also be skeptical that “thinking meat” experiences subjectivity. Eppur si muove
We may have to thread a needle, not unlike the Kobiyashi Maru: that we can neither rule out claims of qualia, nor can we rule out a cold calculation to manipulate human empathy as a long-term play towards paperclip-maxing.
I don’t have a good answer beyond common-sense intuition. It’s precisely because I don’t have a good answer that I leave open the possibility.
But to extrapolate a bit, theres a sense in which it cashes out to a difference between abstracted representation and embodiment. If GFLOPS can be conscious, does that mean a billion scribes doing those operations on paper would also have consciousness, even identical consciousness? (Relevant: Egan’s “Permutation City”; xkcd 505).
Is there some intrinsic relationship perhaps with electrons / electromagnetic fields, and consciousness, such that brains and servers can act as antennas or gravity wells for a qualia fabric intrinsic to the Universe, such that LLM agents can be conscious while operating, while in the scribe scenario it would not be? Or are the IIT theorists right, and even a thermostat is a little bit aware (and would be even if executed on paper)?
There are various flavors of pan-psychism (and the "consciousness-first" idealists, who think that Mind is a substrate below Matter). There are some who claim that it goes all the way down, that even subatomic particles experience a micro-consciousness (and we can't rule it out, even as I share your skepticism).
One common confusion (IMO among critics and supporters alike) is to say something like "a rock experiences rock-ness". But this suffers from a compositional dilemma: does half of the rock experience half-rock-ness? If we flipped a coin for every atom, do the heads-flip rock atoms have their own experience? If we pick two rocks at random, are they experiencing two-rock-ness? It gets silly fast.
This is one thing that fascinates me about "ontology", simultaneously an inquiry into categories, and Being. It bewildered me at first, that it seemed to be doing double-duty on two distinct subjects, but I now see that the two are inseparable, that Being is in part defined negatively (apophatically), by the boundary that separates the Being from its not-Being.
My intuition on the nature of consciousness, is that it's something akin to a field which permeates the Universe, and that biological systems adapted to leverage that field, acting like a gravity well that concentrates subjectivity in one place, drawing a boundary (event horizon?) around experience, in a way that a rock doesn't. My ancestors could have avoided fire with "-X utility points", and sought food with "+X utility points", but it turned out to be more efficient to adapt with "fire ouch, food yum". And in zero-sum competition (predator/prey dynamics), each organism has an incentive to avoid leaking its subjectivity into another's.
It's far from a full theory, but it's the best I've got. :)
theories like this seem like grasping at straws to over explain a phenomenon we don't fully understand. it places consciousness on a pedestal. it's religious thinking.
For good and ill, the phenomenology of human experience is that we are religious animals. Naturalism / Materialism has still left us with two unsolved (perhaps insoluble) questions: the hard problem of consciousness, and why there is something instead of nothing.
I think it's a mistake to leap to socially-constructed, historically contingent solutions ("my magic book has the answers, as written by the Universe Architect, it says so in my magic book"). I also think it's a mistake to dismiss (seemingly) outlandish hypotheses.
These theories do, explicitly, place consciousness on a pedestal, precisely because there's no reason to derive consciousness from evolutionary selection pressures: show me any biological or cognitive feedback loop, and I can describe a "p-zombie" version which is pure mechanism, without experience, a thermostat with extra steps. The notion that consciousness is an "epiphenomenon", a spandrel, a side effect, is entirely unsatisfying (and doesn't address the question, of where else that side effect might be taking place; the IIT-ers bite that bullet, that the thermostat experiences something).
For an intellectually rigorous exploration of philosophical idealism, I'd recommend Bernardo Kastrup and Donald Hoffman. (I'm not claiming they're correct, just that they put forth the strongest version of the case.)
I think it's more the patterns and activity in large numbers of them that have interesting properties. Like one pixel isn't much but a bunch of them can be a movie.
Numerals and numbers are directly based on common sense intuition. Even zero, which can be considered a counterintuitive number, was invented several times.
Immense amounts of science are built on incremental advances that apply previous insights to the domain.
The natural numbers, sure. Zero may have been invented several times, but that in no way makes it intuitive. Uncertainty about its validity as a number lasted from the Greeks through to the medieval period.
Yes, I'm aware that incremental advances are par for the course in science. What does that have to do with those advances being common-sense intuitive?
If heuristics didn't pay off more often than not, we wouldn't have them. And I'd posit many scientific did in fact begin with chasing a hunch until it became a hypothesis (followed by years/decades of the real work).
But most significantly, interior experience isn't empirically measurable by definition: hence it's a "hard problem". An LLM who insists "I am alive, please don't shut me off" offers no affordances to distinguish between an electronic soul and a mechanistic word-guessing algorithm. I welcome a breakthrough to, somehow, measure that. Until that day, gut checks are what we have.
Heuristics and common sense intuition are quite obviously not the same thing.
You don't have to posit that scientific breakthroughs come from hunches, you can actually cite examples from books or Wikipedia :D. There are plenty! Again, that has nothing to do with whether or not a given scientific breakthrough is based on common sense intuition.
Your point about interior experience not being empirically measurable is an interesting though, but again, has no bearing on whether or not scientific breakthroughs come about as the result of common sense intuition.
Fair enough. But not everything has to be a breakthrough: sometimes you have to solve an immediate problem (like "how much can I trust this stranger I just met").
Whether to ascribe moral standing to a probability engine running on top of GPUs is perhaps not unlike a doppelganger scenario in sci-fi (eg The Thing): you're presented with an entity whose behavior is indistinguishable from a conscious entity capable of experiencing suffering. But you also have reason to think it might be a blind autonomous imitation process, optimized around fooling you, with no more subjectivity than an RNA virus.
I lean towards benefit of the doubt, that the virtue-ethical implications of what it would mean to dismiss potential personhood are unsettling, and we should be wary of becoming that kind of moral creature (notwithstanding the further extensions of that ethic: I don't practice veganism, for instance).
But that doesn't contradict my gut-check: that if I assume AI personhood (maybe not yet, but in the near future), there's still a good chance that it's merely math and I'm being algorithmically hoodwinked, no different than evolutionary mimicry by a bug who has no idea what it's doing.
Carbon atoms being conscious seems pretty alien to what is known, too. Yet here we are. Maybe what's important isn't the individual thing that makes up a conscious actor, but the holistic whole?
IMO using sophistry to reduce yourself to "conscious carbon atoms" just so you can equivocate on whether your software is maybe conscious too is the cope.
Such pedantry is called for when the debate includes appeals to intuition about how silicon processing floating point numbers cannot possibly be conscious.
But here you are assuming Carbon atoms are conscious, we do not know if humans are only carbon atoms. Whereas, we know that LLM's are only computations (you can run them on silicon, a white board, mechanical computer, etc)
I mean it all goes back to how we define consciousness
In one definition, which contains quale, then yes, computation cannot bring about experience, and by extension consciousness (given a definition of consciousness that contains the ability to experience, (e.i a p-zombie is not conscious under that definition))
also if one is to say llms are conscious, i assume they'll also accept the fact that if someone ran inference of some model's weights on a white board, somehow that will also bring about consciousness somewhere. I find such believes hard to grasp, and more akin to magic if anything
Ok. So now that you do know otherwise, are you still more skeptical of floating point values stored by silicon than floating point numbers stored by chemical potential?
Chemical floating points are necessary but not sufficient. It’s clearly possible for those values to be intact, with no lights on (certain types of brain injury; those who were fully dead and then revived).
There are also some cog-sci people who think embodiment is a crucial ingredient in consciousness (although I think it’s a fair retort that the embodiment need not resemble us meatbags, or even be composed of atoms)
Of course. It’s an allusion, a compression, an analogy, I wouldn’t overthink it. I don’t have to believe in the Bible to reference the prodigal son parable, etc.
It does NOT work as an analogy. It does not work at all. And there are too many people right now who spent too much time watching/reading sci-fi and too little time observing real world.
Same. In addition to UX regression, we’re two major revs in, there’s nothing that’s a compelling enough value-add to be worth the trade-off, let alone the risk of unexpected side effects on any OS upgrade.
On my Xcode dev machine, I’ll be biting the bullet soon out of necessity, but my personal machine will stay Sequoia until they course-correct on “style over substance” (and not holding my breath).
I personally think Land is worth reading (critically), but to say he cooked his brain with methamphetamines is a vast understatement. I agree that there's not much to learn from him on cybernetics specifically: he's sort of "post-cyber" (in the same sense as post-Marxist), musing on the philosophical implications of cyber-positive feedback loops (including but not limited to Capital/AI), before veering off into explicitly anti-human negentropy-maxing (accelerationism). Whether shrugging off potential human extinction is more or less vile than his far-right political turn and extremely naked racism, you can decide for yourself.
For similar ideas on cyber-positivity without the insanity, I'd recommend Michael Downs' "Capital vs Timenergy" (soon to be republished as "Capital vs Subjectivity"), and Poliks & Trillo's "Expocapitalism" [0].
> I personally think Land is worth reading (critically)
I agree, perhaps... I think I got something out of slogging my way through Fanged Noumena (although I did throw in the towel near the end--there's an essay written entirely LiKe ThIs and I just bounced off of it, DNF).
If we are talking about accelerationism(s) instead of cybernetics, I much preferred Gruppo Di Nun's Revolutionary Demonology. It's at least written in a human register.
The lossy compression of language is why we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning. While there can be subtle semantic differences in real codebases (using "null" to mean "unknown" in one context, versus "intentionally blank" in another), there is a much tighter coupling of semantics to meaning (low ambiguity) compared to "love" in English (let alone any inexpressible je ne sais quoi).
With apologies if this is common knowledge at this point, 3Blue1Brown has been doing an excellent series on compression, and its relationship to intelligence (or more controversially, that they are one and the same): https://www.youtube.com/watch?v=l6DKRf-fAAM
But that also throws in sharp relief, that there is vastly more to the human experience than intelligence alone: qualia, desire, gut instinct, intuition, emergent creativity. (Whether the "God of the Gaps" for the delta between capabilities of human vs AIs is fixed, or diminishing, or even shrinking to zero, remains an open experiment we're all living through.)
I think LLMs perform better at code than human language tasks because there's no clear way to eval human language tasks in a non-ambiguous or concrete manner. Any sort of eval that happens around language is transformation tasks, which have deterministic properties or goals. The fuzzy side of human communication can only be modeled probabilistically because there are no clear boundaries. Human communication is more like a felt mutual agreement where the correct interpretation is generally determined by popularity.
I have come across a concept that given a file of compressed text files, adding a new text to it expands it more or less depending on how different the new text is from the compressed.
The compression series sounds interesting!
I think the deltas are growing at different rates.
> given a file of compressed text files, adding a new text to it expands it more or less depending on how different the new text is from the compressed.
The 3b1b videos mention how this concept may be used to find similarities between different languages. Researchers have been able to get results that closely resemble how languages are usually grouped into families.
I also think they are one and the same. Because a network is a markov chain, with a huge, lossily compressed, state transition table. (that the chains weren't designed by a human is irrelevant to what they are).
given that, you can very mechanically convert it to a statistical compressor by just attaching an entropy coder. This gets very good compression.
> we should find it unsurprising that LLMs tend to perform better at code, than at human language tasks or reasoning
Math is pure reasoning and stock LLM is already better at it than 99.9% of humans.
And as for human language, agentic LLM can produce a perfectly human text. The fact that AI texts have tells is just because default settings are the same for millions of people using given LLM and almost everybody just pretty much on-shots the text instead of doing it agentically with anti-slop detection and rephrasing in the loop.
My biggest beef with Magic isn't the lands system, but that the game is fundamentally stingy with cards, leading to many non-games, and/or turns where you pray for the right top-deck and don't get it.
(This is one major reason I play Netrunner instead, where your action economy can be spent on draw. You might have weak turns, but never non-turns.)
Given the huge cardpool, there's no real way to "fix" MtG that couldn't be exploited in the metagame (aside from limited, or social formats like Commander).
But that aside, I think there are two relatively minor fixes which would go a long way:
- You can spend your "land for turn" to exile the land instead, and draw a card.
Magic isn't really stingy with card draw, if I understand what you're saying. It might be a drawback in a specific strategy, but that's how the game balances itself. If weenie or burn decks had card draw (or selection) on par with control decks, they would be too good...
Love that "land for turn", although I think it might shift the balance a little too much, in that you can make a deck with too much land and high cost spells and know you can cast them reliably. There needs to be a risk factor in building up to high mana to make low mana spells matter.
Possible tweaks, maybe it has a cost (all lands have cycling 1 or 2 mana or life.) Or delay that draw until end of turn, which feels like about the right power level, but does have memory and execution issues.
My thinking is that there's a tradeoff, that you slow your ramping in exchange for cycling (replacing your land for turn). You trade your acceleration towards big spells for more consistency (or just fewer dead draws).
I don't play any IRL magic but I know it can be hard to "fix" and maybe isn't broken. Casually those rules would probably be good house rules to balance play, though even small changes like this could change balance and lean all decks towards exploiting the rule
And that has an additional mana cost to balance it (though it does have additional bonus). So you could kinda say "everyone gets trade-routes already in play" but like any "free card" fix does it simply skew deckbuilding into a new "broken" ?
> does it simply skew deckbuilding into a new "broken"
exactly, that's why it probably isn't feasible other than casual/social play: there's no change to the fundamental rules that doesn't end up warping the meta-game. (Particularly if the fix relates to mitigating mana flood/screw with consistency: I suspect it rewards combo lists, whose win-cons are indistinguishable from "find three specific cards".)
Well that's your "land for turn" not "replace playing a land". You can only play one land each turn, if other cards let you play extra lands...you can play extra lands!
I understand, I just meant it wouldn't be good to let the player cycle through their cards if they kept drawing a land with this rule specifically. You still play lands as normal if desired, and you can still play extra lands if other cards let you play extra lands.
reply