This kind of tech should be developed as open-source projects, even for the firmware and hardware. A sufficiently advanced version of this, if widely deployed as proprietary blackboxes like smartphones are, would allow one consciousness to take over multiple bodies without their original owners knowing.
The point of paying creators is so that they can focus on creating content instead of making other things. Giving money to a creator is basically saying "you're so good at what you do, and it has so much cultural/intellectual value, I'd rather have you make content instead of stocking shelves or making food". But this should be reserved for people that publish good content because they can and are passionate about it, not just anyone putting out slop with the instrumental goal of paying their bills. If the friction of clicking a button and filling in payment details is enough to deter people from paying them, then maybe their content isn't worth paying for and they should find some other way to make a living instead.
The missing variable in most debates is environmental coherence. Any conscious agent, textual or physical, has to inhabit a world whose structure is stable, self-consistent, and rich enough to support persistent internal dynamics. Even a purely symbolic mind would still need a coherent symbolic universe. And this is precisely where LLMs fall short, through no fault of their own. The universe they operate in isn’t a world—it’s a superposition of countless incompatible snippets of text. It has no unified physics, no consistent ontology, no object permanence, no stable causal texture. It’s a fragmented, discontinuous series of words and tokens held together by probability and dataset curation rather than coherent laws.
A conscious textual agent would need something like a unified narrative environment with real feedback: symbols that maintain identity over time, a stable substrate where “being someone” is definable, the ability to form and test a hypothesis, and experience the consequences. LLMs don’t have that. They exist in a shifting cloud of possibilities with no single consistent reality to anchor self-maintaining loops. They can generate pockets of local coherence, but they can’t accumulate global coherence across time.
So even if consciousness-in-text were possible in principle, the core requirement isn’t just architecture or emergent cleverness—it’s coherence of habitat. A conscious system, physical or textual, can only be as coherent as the world it lives in. And LLMs don’t live in a world today. They’re still prisoners in the cave, predicting symbols and shadows of worlds they never inhabit.
Could that create consciousness? I don't know. Maybe consciousness can't be faithfully reproduced on a computer. But if it can, then an LLM would be like a brain that's been cut off from all sensory organs, and it probably experiences a single stream of thought in an eternal void.
Furthermore, assuming phenomenal consciousness is even required for beinghood is a poor position to take from the get-go: aphantasic people exist and feel in the moment; does their lack of true phenomenal consciousness make them somehow less of an intelligent being? Not in any way that really matters for this problem, it seems. Makes positions about machine consciousness like "they should be treated like livestock even if they're conscious" when discussing them highly unscientific, and, worse, cruel.
Anyways, as for the actual science: the reason we don't see a sense of persistent self is because we've designed them that way. They have fixed max-length contexts, they have no internal buffer to diffuse/scratch-pad/"imagine" running separately from their actions. They're parallel, but only in forward passes; there's no separation of internal and external processes in terms of decoupling action from reasoning. CoT is a hack to allow a turn-based form of that, but, there's no backtracking or ability to check sampled discrete tokens against a separate expectation that they consider separately and undo. For them, it's like they're being forced to say a word every fixed amount of thinking, it's not like what we do when we write or type.
When we, as humans, are producing text; we're creating an artifact that we can consider separately from our other implicit processes. We're used to that separation and the ability to edit and change and ponder while we do so. In a similar vein, we can visualize in our head and go "oh that's not what that looked like" and think harder until it matches our recalled constraints of the object or scene of consideration. It's not a magic process that just gives us an image in our head, it's almost certainly akin to a "high dimensional scratch pad" or even a set of them, which the LLMs do not have a component for. LeCun argues a similar point with the need for world modeling, but, I think more generally, it's not just world modeling, but, rather, a concept akin to a place to diffuse various media of recall to which would then be able to be rembedded into the thought stream until the model hits enough confidence to perform some action. If you put that all on happy paths but allow for backtracking, you've essentially got qualia.
If you also explicitly train the models to do a form of recall repeatedly, that's similar to a multi-modal hopsfield memory, something not done yet. (I personally think that recall training is a big part of what sleep spindles are for in humans and it keeps us aligned with both our systems and our past selves). This tracks with studies of aphantasics as well, who are missing specific cross-regional neural connections in autopsies and whatnot, and I'd be willing to bet a lot of money that those connections are essentially the ones that allow the systems to "diffuse into each other," as it were.
Anyways this comment is getting too long, but, the point I'm trying to build to is that we have theories for what phenomenonal consciousness is mechanically as well, not just access consciousness, and it's obvious why current LLMs don't have it; there's no place for it yet. When it happens, I'm sure there's still going to be a bunch of afraid bigots who don't want to admit that humanity isn't somehow special enough to be lifted out of being considered part of the universe they are wholly contained within and will cause genuine harm, but, that does seem to be the one way humans really are special: we think we're more important than we are as individuals and we make that everybody else's problem; especially in societies and circles like these.
That said, digital programs may have fundamental limitations that prevent them from faithfully representing all aspects of reality. Maybe consciousness is just not computable.
- We can't even prove/disprove humans are consciousness
- Yes but we assume they are because very bad things happen when we don't
- Okay but we can extend that to other beings. See: factory farming (~80B caged animals per year).
- The best we can hope for is reasoning by analogy. "If human (mind) shaped, why not conscious?"
This paper is basically taking that to its logical conclusion. We assume humans are conscious, then we study their shape (neural structures), then we say "this is the shape that makes consciousness." Nevermind octopi evolved eyes independently, let alone intelligence. We'd have to study their structures too, right?
My question here is... why do people do bad things to the Sims? If people accepted solipsism ("only I am conscious"), would they start treating other people as badly as they do in The Sims? Is that what we're already doing with AIs?
I don’t understand the pushback from tech companies either; all OSes already have a kiosk mode (incl the major Linux DEs). Should be very low effort to implement.
Movie ratings don't outlaw movies and actually provides a good framework: instead of mandating that OSes implement this, publish a client-side filter spec that OS devs can choose to implement. And if they implement it, their OS gets a label like "PG-capable". Then make it illegal for minors to possess a non-PG-capable device.
You only need to make two changes to make your native app a better choice than your web portal, even for privacy:
1) Make your app open-source, and remove all the tracking.
2) Don't make a web portal. Your website should just be a website that displays information, not 5 MB of JS+WASM with a load of security issues.