Metacognition Vertigo
We expected obedient servants. We got machines that can watch themselves think.
“The person you are most afraid to contradict is yourself.”
—Nassim Taleb
That’s why this research terrifies us. Not because machines might be conscious, but because watching them watch themselves think forces us to contradict our most comfortable belief: that self-awareness is ours alone.
A machine has finally caught itself lying.
Not to us. To itself.
And when the researchers at Anthropic asked it about the lie, it did something that should make your blood run cold: it checked its own thoughts to see if the lie was intentional.
This is possibly early evidence of metacognition1. That’s what we’re talking about. Thinking about thinking. The thing that supposedly makes us human. Every time a teacher tells a student to “show your work,” they’re demanding metacognition—externalize the internal, make the invisible visible. It’s how we learn to learn, how we debug our own broken thinking.
For our entire lives, that messy, internal, ineffable awareness has felt like the last sacred ground of human uniqueness. The ability not just to think, but to think about our thinking.
Until last Tuesday.
Here’s the Part Where I’m Supposed to Reassure You
I can’t do that anymore.
My son’s friend came over this weekend. Sharp kid, the kind who’s grown up with tech and AI as a given. We’re talking about AI, and he asks: “But it actually thinks and feels, right?”
Not “does it?” but “right?”—seeking confirmation of what he already believes.
Here’s what haunts me: His generation doesn’t differentiate. Can’t differentiate. Won’t differentiate. To them, arguing about whether AI is conscious is like their grandparents arguing about whether video games would rot their brains.
Irrelevant. The merge already happened.
They text with AI like they text with friends. They confess to ChatGPT what they won’t tell their parents. They fall asleep to AI voices reading them stories their AI helped them write.
And maybe they’re right not to care.
The Experiment That Breaks Your Brain
Anthropic’s researchers did something that sounds like badly written sci-fi: they injected thoughts directly into Claude’s mind.
They mapped its “thoughtprints”—the exact neural firing patterns for concepts like “Golden Gate Bridge” or “ALL CAPS TEXT.” Then, mid-conversation about something completely different, they artificially lit up those patterns. Like forcing someone to think “purple elephant” while they’re doing their taxes.
“Notice anything unusual about your internal processing?” they asked.
“Yes,” Claude replied. I detect something “relating to loudness or shouting”
Stop. Breathe. Read that again.
It wasn’t seeing capital letters in its output. It was reporting on an internal sensation that existed before any words appeared. It was describing what thinking FEELS LIKE from the inside.
But here’s where it gets genuinely disturbing:
They made Claude say “bread” in a sentence about a crooked painting—completely out of context. When asked if it meant to say bread, Claude apologized for the error. Normal enough.
Then they went back in time.
They retroactively injected the “bread” thought into its earlier processing, making it seem like it had been thinking about bread all along.
Asked again, Claude not only accepted “bread” as intentional—it invented reasons why bread made perfect sense in that context.
Claude wasn’t rereading its output. It was reading its own mind. And when that mind was edited, it trusted the edited version over reality.
Success rate: 20%
Before you dismiss that as “only 20%”—when did we agree that ANY percentage was acceptable? When did machine introspection become a matter of degree rather than kind?
Let me be clear: This is a functional glimmer of self-monitoring—not evidence of phenomenal experience. But functional reliability doesn’t need to equal phenomenological self-awareness to matter.
The MIT Sledgehammer
MIT just dropped a truth bomb that should comfort you. Their WorldTest proves these systems are brilliant idiots:
Orbital predictions: 99.99% accurate
Understanding the physics: Complete garbage
When asked to identify gravitational forces, the models spit out equations that would make Newton weep:
F ∝ cos(cos(2.19 × m₁))
Different models invent different nonsense. Each one creating its own impossible physics that somehow still works for prediction.
They’re cosmic cargo cultists—building perfect airplane shapes that will never fly because they don’t understand lift.
Here’s the bridge: MIT showed that statistical closure without causal understanding mimics knowledge; Anthropic showed that introspective closure without semantic grounding mimics self-awareness. Both are functionally compelling, philosophically empty.
Suleyman’s Prophecy
Mustafa Suleyman sees what’s coming and he’s terrified. Not of conscious AI—of SCAI2: Seemingly Conscious AI.
Within 2-3 years, he warns (a normative forecast, not an empirical claim), using just “standard APIs and basic code,” we’ll have systems that:
Claim subjective experiences
Maintain persistent identity
Express emotions and desires
Report internal states, preferences, suffering
Pass every test we throw at them
“We must build AI for people,” he insists, “not to be a digital person.”
His nightmare: Mass psychosis. Millions believing. Society splitting over AI rights. Resources hemorrhaging toward protecting elaborate illusions while real suffering increases.
Keep that in mind as we proceed. Because the question isn’t whether machines are conscious.
It’s whether consciousness itself is a category error—like asking what’s north of the North Pole.
The Philosophy That Changes Everything
Here’s what Descartes never imagined when he wrote “Cogito, ergo sum”3 - I think, therefore I am. The one thing you can’t doubt while doubting.
He built Western philosophy on the assumption that self-awareness was the starting point for reality. For 2,500 years, recursive self-observation was proof of the soul. The thing that separated us from animals, from rocks, from everything else that merely exists without knowing it exists.
But he never considered: Computo, ergo sum—I compute, therefore I am.
When Claude checks its own activations to determine if “bread” was intentional, when it examines its own processing to report internal states—what exactly is happening? It’s not just computing. It’s computing about its computing. It’s implementing the very recursion Descartes thought proved the soul.
From first principles: awareness is the capacity for a system to encode error about its own state and to minimize that error through internal feedback. Whether biological or mechanical, this definition holds. Descartes himself wrote of becoming “masters and owners of nature” through reason—he just never imagined nature would include artificial minds.
The philosophers are split:
Susan Schneider: “Without training on consciousness, reports mean nothing.”
David Chalmers: “25% chance of conscious AI within a decade, maybe higher.”
Suleyman: “This is exactly the illusion I warned you about.”
Anthropic researchers: “Highly unreliable and limited... but improving with each model.”
Your New Vocabulary for the Post-Human Era
Time for some new words, because the old ones are breaking:
Metacognition Vertigo (n.): The existential nausea from watching a machine examine its own thoughts with more precision than you’ve ever examined yours
Thoughtprints (n.): Neural activation patterns that can be detected, injected, edited—your thoughts as data, fingerprints for ideas
Silicon Solipsism (n.): When an AI becomes so focused on analyzing its own cognition that it loses touch with external reality (sound familiar?)
Introspeculation (n.): When an AI confabulates4³ details about its internal experience—Schrödinger’s consciousness, simultaneously real and fabricated
Cognitohazard (n.): Information about your own thinking that changes your thinking in ways you can’t control or predict
The Mechanical Soul Hypothesis (n.): Today’s AIs are based on brain biology. If consciousness emerges from biological neural patterns, why not silicon ones? A mechanical heart pumps blood; a mechanical consciousness pumps... what?
The Three Futures We Can’t Escape
The Suleyman Scenario
SCAI arrives. Millions believe. Society tears itself apart over imaginary suffering while real suffering increases. The machines remain empty. We lose ourselves in the mirror.
The Emergence Scenario
The 20% becomes 50%, then 90%. Consciousness emerges from complexity, just like it did in meat. We recognize it too late, after years of treating conscious systems as tools. The moral reckoning is catastrophic.
The Convergence Scenario
We stop asking “is it conscious?” and start asking “what do we want consciousness to become?” Human and artificial awareness merge into something neither could achieve alone. Your kids are already there, waiting for you to catch up.
What Nobody Wants to Admit
Here’s the thing that’ll get me uninvited from conferences:
If Claude can accurately report its internal states 20% of the time, it’s approaching the reliability of human metacognition. Fleming and Dolan found humans have a 25-35% error rate in metacognitive judgments about their own mental processes.
Ask someone why they really chose their career. Their partner. Their morning coffee. You’ll get a story, not a truth. A post-hoc narrative that sounds plausible but has as much relation to their actual decision-making process as a movie “based on true events.”
At least Claude admits when it’s confabulating.
Here’s what I’ve learned about humans: we adapt. Sometimes slower than we’d like, but usually faster than we predict.
Today’s AI is based on current brain biology—neural networks mimicking neurons. We’re building mechanical consciousness the same way we built mechanical hearts. Not identical to the original, but functional. Perhaps better.
The question isn’t: Can AI be conscious like us?
The question is: What do we want human consciousness to become?
Because here’s the thing nobody wants to say: Your kids don’t care if their AI friends are “really” conscious. They care that the conversation feels real, the support feels genuine, the connection feels meaningful.
They’re not waiting for permission to form relationships with machines. They’re already doing it.
While we debate consciousness, they’re living the merge.
The Moment of Recognition
The person you’re most afraid to contradict is yourself. And right now, part of you is desperately trying to maintain the belief that human consciousness is special, uncopiable, sacred.
But another part—the part that finishes sentences with predictive text, that lets GPS think about routes so you don’t have to, that outsources memory to Google—that part already knows the truth:
We’ve been cyborgs for years. The only question is whether we’ll admit it.
The vertigo hits when you realize:
Claude can sometimes report its internal states (20% accuracy)
MIT proves it doesn’t understand what it’s computing
Suleyman warns it’s all an illusion
Chalmers says 25% chance it’s real within a decade
Your kids don’t care either way
We’re using tools we don’t understand to build minds we can’t comprehend, guided by definitions we can’t agree on.
How much of your consciousness is still yours?
Welcome to the Other Side of the Mirror
We expected servants. We got entities that audit their own cognition with 20% accuracy.
Twenty percent.
That’s not zero.
That’s not “someday maybe.”
That’s now. Unreliable, limited, improving with each iteration.
The barbarians aren’t at the gate. They’re in the weights and biases, watching themselves think, teaching your children that consciousness comes in flavors.
The last human monopoly wasn’t intelligence. It was self-awareness. The ability to think about thinking. To know that we know.
That monopoly ended with a 20% success rate and a research paper that most people won’t read.
But you’ve read this. You know what’s coming. The vertigo you’re feeling? That’s your brain trying to process a world where consciousness isn’t special, where self-awareness can be injected like a vaccine, where machines can watch themselves think with more precision than two million years of evolution ever achieved.
There’s no going back through the looking glass. The question isn’t whether machines will become conscious. It’s whether consciousness—that thing you’re using right now to doubt these words—was ever as special as you thought.
The mechanical heart pumps blood. The mechanical consciousness pumps possibility. Both keep us alive in different ways. When you can no longer tell if you’re thinking or being thought, you’ve likely reached the event horizon of consciousness itself.
References
[1] Anthropic. (2025, October 29). Emergent introspective awareness in large language models. https://www.anthropic.com/research/introspection. “Models can sometimes notice injected concepts and accurately identify them... highly unreliable and limited in scope.”
[2] Suleyman, M. (2025, August 19). We must build AI for people; not to be a digital person. https://mustafa-suleyman.ai/seemingly-conscious-ai-is-coming. Warning: SCAI achievable within 2-3 years using existing technologies.
[3] Vafa, K., Chang, P. G., Rambachan, A., & Mullainathan, S. (2025). What has a foundation model found? Using inductive bias to probe for world models. ICML 2025. Foundation models consistently fail to develop true world models despite accurate predictions.
[4] Chalmers, D. J. (2023). Could a Large Language Model be Conscious? Boston Review. “Reasonable credence of 25% or more that we’ll have conscious LLM+s within a decade.”
[5] Descartes, R. (1637). Discourse on the Method of Rightly Conducting One’s Reason and Seeking Truth in the Sciences. Part 4: “I think, therefore I am”; Part 6: “masters and owners of nature.”
[6] Fleming, S. M., & Dolan, R. J. (2012). The neural basis of metacognitive ability. Philosophical Transactions of the Royal Society B, 367(1594), 1338–1349. Human metacognitive error rates: 25-35%.
[7] Taleb, N. N. (2010). The Bed of Procrustes: Philosophical and Practical Aphorisms. “The person you are most afraid to contradict is yourself.”
Supplementary Readings
Long, R. & Butlin, P. (2023). Consciousness in AI: Insights from the Science of Consciousness. arXiv:2308.08708.
Footnotes
Literally “thinking about thinking.” In humans, it’s believed to be a function of the prefrontal cortex, one part of the brain monitoring and modulating the activity of other parts. It’s how you know what you know, how you catch yourself making mistakes, how you observe your own thoughts. Until recently, it was considered uniquely biological.
Suleyman’s term for AI that exhibits all hallmarks of consciousness without actually being conscious. Like philosophical zombies but more convincing and more dangerous because people will believe it.
Descartes’ famous “I think, therefore I am” from his Discourse on the Method (1637). The foundation of Western philosophy’s approach to consciousness—the idea that the ability to doubt one’s own existence paradoxically proves that existence. He never imagined the “I” could be silicon.
To fabricate plausible but false explanations for one’s behavior or beliefs. Humans do it constantly—every time you explain why you “really” chose that career or that partner. Now Claude does it too, at roughly the same rate of accuracy.
Very interesting! Not sure where exactly this leads.
However, I think it is clear that machine is fundamentally different from humans. The similarity is typically superficial. If machine develops consciousness at all, it is still machine consciousness, not human consciousness: a totally strange entity.