The Warp and the Woof of AI
Collision course or awe-inspiring tapestry?
We’ve crossed a threshold where software is feeling less like a tool and more like a co-author.
When computers need to solve the undefinable
This week, I found myself in a series of conversations that kept circling back to a central question: where does the machine end and human begin?
In one meeting, we discussed a study arguing that what we call "agency" in AI systems is fundamentally different from human agency—not just in degree, but in kind.
"Algorithms exist in a small world, in which all possible problems are well-defined, whereas organisms live in a large world, where most problems are ill-defined (and some are probably not properly definable)" (Jaeger, 2024).
Meanwhile, discussions about large-language-model (LLM) orchestration platforms and AI in clinical or teaching settings kept returning to the same theme: as these systems become more capable, the line between augmentation and replacement blurs. A physician colleague put it perfectly: "We need to ensure these systems serve as an extension of human judgment, not a replacement for it."
These conversations crystallized something I've been trying to articulate—that we're not just building tools anymore. We're weaving a new reality, where human and machine intelligence intertwine in increasingly complex patterns.
So I reached for a metaphor as old as civilization itself: the loom.
Stand in front of the canvas—what's really happening?
Step back from a Turner seascape or a Catskills overlook and something stirs—goosebumps, a breath caught in the chest, a feeling you can't name but recognize. That response is no accident. It’s your brain matching incoming sensation against past experience—what Barrett calls ‘constructing meaning by correctly anticipating sensations’ (Barrett 2017).
“Our perceptions are not a window onto objective reality, but instead they are more like the Windows interface of a computer,” meaning that what we consciously see is a simplified “desktop” of icons—useful summaries shaped for survival—not a literal display of the world’s complex inner circuitry (Hoffman, Singh, & Prakash 2015).
Now flip the input. A refugee photo. A termination email. A chatbot confidently inventing facts about someone you love. The same neural machinery kicks in—but this time with cortisol, not awe.
“For an infinitely large class of worlds… an organism that accurately estimates reality is never more fit than one tuned only to the relevant fitness functions.”
That insight—known as the Fitness-Beats-Truth Theorem—comes from evolutionary game theory, showing that natural selection rewards creatures whose perceptions are useful for survival, even if those perceptions distort or ignore the objective truth (Hoffman et al., 2015).
The wiring doesn't care if it's joy or dread. It's just pattern-matching. What changes the output is the thread you feed the loom.
Warps, woofs, what century is this?
In weaving, the warp is the vertical backbone—those tensioned threads held taut on the loom. The woof (or weft) is the horizontal thread passed through with a tool called a shuttle. One provides structure. The other gives motion. Without both, no fabric.
It's more than metaphor. Our entire reality—biological and digital—is being woven on just such a loom1:
Humans bring narrative and meaning. Machines bring precision and scale. When the threads stay perpendicular and tensioned, we get fabric. Let them slip, and we get knots—or a runaway loom.
This distinction isn't academic. Living organisms demonstrate "autopoiesis"—the ability to self-create and self-maintain. A cell produces its own components through metabolism, enabling assembly and sustaining its internal environment. AI systems fundamentally lack this capacity—their goals are imposed from outside, not emerging from within. Put formally: an agent is autopoietic only if it can rewrite the very rules that generate its goals—today’s LLMs cannot.
That said, we are dealing with something far more complex than an algorithm here.
Mid-2025: The loom is humming
We're past theoretical discussion. The tapestry is already being stitched—and, like it or not, we're inside it.
Seventy-eight percent of global organizations now report at least one live AI use case (McKinsey & Company, 2025) and the same share of staff report bringing their own AI tools to work in Microsoft's 2024 Work Trend Index (Microsoft, 2024). Models that used to autocomplete text now quietly outperform median-level developers on structured coding tasks (Bubeck et al., 2023). But it's not just technical—it's intimate.
Neuralink's brain implant, which recently received the FDA's 'breakthrough' designation for speech restoration, has enabled a third recipient, a non-verbal ALS patient to steer a MacBook cursor and even trim a video with pure thought.
Need proof the supercomputer hardware is shrinking? NVIDIA’s DGX Spark is a 1.2-kilogram cube not much bigger than a water bottle (15 × 15 × 5 cm) yet cranks out 1,000 TOPS—enough to run frontier near-full-scale LLMs unplugged (NVIDIA, 2025).
This isn't augmentation. It's entanglement.
Yet even our most advanced models operate in those "small worlds"—purely syntactic environments isolated from the messy, ambiguous semantics of physical reality. Despite their computational vastness, these worlds are strictly defined, with clearly specified problems and solutions. Living organisms, by contrast, navigate "large worlds" full of ill-defined problems where information is "always scarce, often ambiguous, and sometimes outright misleading" (Jaeger, 2024).
I'm not alone in feeling this shift. One of my genius friends, Brian Krabach recently wrote about his experience as a software engineer watching AI tools take over work he'd spent decades mastering. "AI Took My Job — What Now?!" captures that stomach-dropping moment when you realize your skills have been automated overnight. But his journey from panic to possibility reminds us that human agency can evolve rather than disappear—moving from "laying every brick" to "sketching blueprints" that AI tools bring to life.
As we discussed earlier, this acceleration creates a strange paradox in our relationship with productivity and downtime. When AI systems can run in parallel with our thinking, "a few hours to rest feels like stepping away for a week." The rate of work, the velocity of output no longer aligns with human rhythm—forcing us to reconsider what it means to truly disconnect in a hyperaccelerated world.
Why awe doesn't feel like autocomplete
There's a fundamental reason a sunset moves you differently than a search result. A reason your chest aches at a song but not at a summary.
It comes down to how humans and machines process information:
Compression direction: Your brain “compresses and reduces [sensory] dimensionality” on the way to higher cortex (Barrett 2017). A language model does the opposite—expanding a terse prompt into a sprawling probability cloud of continuations.
Value assignment: You know what matters. You see a child's face and it floods you with love or fear or hope. A model doesn't flinch. It only sees pixels. Even with reinforcement learning bolted on, it's not seeking truth—it's chasing a proxy.
Counterfactual ache: Only humans truly regret. You can remember the moment you didn't speak up. The person you let go. A model can sample alternatives. It cannot ache for them, yet...
That asymmetry—that ache—is what agency feels like.
This isn’t just philosophical musing—there’s data. On ConceptARC, GPT-4 still scores only 33 % while humans cruise at 91 % (Moskvichev, Odouard, & Mitchell, 2023). The broader ARC-AGI benchmark is improving but still unsolved—its private-set record rose from 33 % to 55.5 % during the 2024 ARC Prize (Chollet et al., 2024). Meanwhile, on a battery of 100 random single-digit multiply-and-add puzzles GPT-4 manages 58 %, dropping to 12 % once the numbers edge toward three digits (Bubeck et al., 2023). Note, this is an actively evolving area, and I expect major breakthroughs in the coming years.
The issue isn’t raw computing power; it’s architecture. In most AI, the “thinking” symbols live far above the silicon—code and hardware hardly touch. Biology works differently: chemistry and information are woven together, conceptually distinct yet impossible to pull apart.
The agency question
We're not facing a single existential threat. We're facing a slow erosion of authorship—a gradual blurring of where human agency ends and algorithmic assistance begins.
Here's what that looks like in practice:
Let's not spend our energy on panic, but on pattern recognition. The question isn't whether AI will replace us, but how we'll remain genuinely human as the dividing line blurs—a question that concrete examples are already helping us answer.
Real-world cases already validate these concerns. In May 2024, Google's AI Overviews produced bizarre suggestions, including recommending "eating a small rock daily for digestive health." NYC's Microsoft-powered MyCity chatbot advised restaurant owners they could serve cheese nibbled by rodents. These aren't mere implementation bugs but symptoms of AI's fundamental disconnect from physical reality and common-sense norms.
The EU AI Act and U.S.’s Executive Order on AI both recognize these limitations, mandating human oversight for high-risk AI systems. In healthcare, the FDA's guidance for AI-enabled medical devices emphasizes the "Human-in-the-Loop" principle after researchers identified critical error patterns requiring human judgment.
This connects directly to what we explored last week, where I argued learning isn't a file transfer, and that "when friction disappears, so does growth" (Melumad & Yun, 2025).
How to keep the warp straight
First rule: tension is good. The loom only works when the warp is tight.
That means interfaces that show their math—not just outputs, but the why behind the output. If a system can’t show its reasoning, it can’t earn your trust (Doshi-Velez & Kim 2017).
That’s why I keep a note-pad open when I ask ChatGPT to convert joules to electron-volts—seeing the step-by-step derivation is the trust signal.
Humans calibrate reliability via metacognitive confidence—damage to the same prefrontal circuits “selectively affects the accuracy of metacognitive reports” even when task performance survives (Fleming & Dolan 2012). AI systems must surface comparable confidence or operator trust will mis-calibrate.
It means autonomy by graduation, not by default. Systems should assist first, act later, and only after we've seen them operate safely in a sandbox. Trust should be versioned—maybe even revoked.
It means letting machines forget. Humans forget to survive. Models that log every keystroke forever become surveillance tools by accident. Logs should expire. Dossiers should require renewal. Eternal memory isn’t a feature—it’s a booby-trap. Happily, researchers are learning to make big language models “forget” on command: Obliviate, an un-learning software patch, slashes verbatim recall of unwanted content roughly 100-fold while nudging benchmark accuracy by less than a single point (Russinovich & Salem, 2025), and the cheekily titled “Who’s Harry Potter?” experiment can wipe an entire chunk of training data in under an hour while the model’s scores barely budge (Eldan & Russinovich 2023).
And finally: expose the seams. Publish policy hooks. Surface where decisions enter and exit the pipeline. If we can't tug the thread, we don't know what we're wearing.
Symbiotic AI doesn't mean balance. It means revocability.
The weave ahead: Three possible patterns
Here's where we might be headed—and what that means for agency:
Only one of these futures leaves you holding the shuttle.
Critics of the embodiment requirement argue that cognitive processes constituting agency could be functionally implemented without biological embodiment. Others document how large language models exhibit "emergent abilities" that weren't explicitly programmed and only appear beyond certain complexity thresholds (Wei et al., 2022).
But these arguments remain theoretical. Current AI systems—even frontier models—continue to demonstrate fundamental limitations in agency-related tasks. They remain sophisticated pattern-matching systems with goals imposed from outside, not emerging from within.
Keep your hands on the shuttle
The shuttle is the tool that moves the thread back and forth across the loom. It's small. Basic. But without it, there's no weave.
Same with agency.
Founders: Build systems that ask, not just act. Add agency reviews to your stack alongside privacy and security.
Researchers: Don't hide behind proprietary model weights. If the method can't be replicated, the science isn't done.
Policymakers: Regulate outcomes, not internal mechanics. We don't need to see the weights if we can see the consequences.
Everyone else: Make it a practice. Let the AI draft—but you always do the second pass. Don't outsource your judgment. Don't surrender the shuttle (Risko & Gilbert, 2016).
This isn't about fear. It's about authorship. The goal isn't to stop the weave. It's to guide it.
To make sure the tapestry reflects who we are—and not just what the system predicts we'll become.
I'll explore the practical neural tools that let us preserve and manufacture agency amid this acceleration next week—the practical "how" that builds on the "why" we've explored here.
What patterns are you weaving with AI? Are you holding the shuttle, or has it slipped from your grasp? Share your experiences in the comments.
References
Barrett, L. F. (2017). The theory of constructed emotion: An active inference account of interoception and categorization. Social Cognitive and Affective Neuroscience, 12(1), 1-23. https://doi.org/10.1093/scan/nsw154
Bubeck, S., Chandrasekaran, V., Eldan, R., Gehrke, J., Horvitz, E., Kamar, E., Lee, P., Lee, Y.-T., Li, Y., Lundberg, S., Nori, H., Palangi, H., Ribeiro, M. T., … Zhang, Y. (2023). Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv:2303.12712. https://doi.org/10.48550/arXiv.2303.12712
Chollet, F., Knoop, M., Kamradt, G., & Landers, B. (2024). ARC Prize 2024: Technical report. https://arxiv.org/abs/2412.04604
Doshi-Velez, F., & Kim, B. (2017). Towards a rigorous science of interpretable machine learning. arXiv. https://doi.org/10.48550/arXiv.1702.08608
Eldan, R., & Russinovich, M. (2023). Who’s Harry Potter? Approximate unlearning in large language models. arXiv:2310.02238. https://doi.org/10.48550/arXiv.2310.02238
Fleming, S. M., & Dolan, R. J. (2012). The neural basis of metacognitive ability. Philosophical Transactions of the Royal Society B, 367(1594), 1338-1349. https://doi.org/10.1098/rstb.2011.0417
Gordon, R. (2024, July 11). Reasoning skills of large language models are often overestimated. MIT News. https://news.mit.edu/2024/reasoning-skills-large-language-models-often-overestimated-0711
Hoffman, D. D., Singh, M., & Prakash, C. (2015). The interface theory of perception. Psychonomic Bulletin & Review, 22(6), 1480-1506. https://doi.org/10.3758/s13423-015-0890-8
Jaeger, G. (2024). Artificial intelligence is algorithmic mimicry: Why artificial “agents” are not (and won’t be) proper agents. arXiv. https://doi.org/10.48550/arXiv.2307.07515
McKinsey & Company. (2025, March 12). The state of AI: How organizations are rewiring to capture value. McKinsey Analytics.
Melumad, S., & Yun, J. H. (2025). Experimental evidence of the effects of large language models versus web search on depth of learning (Wharton School Research Paper No. 5104064). SSRN. https://doi.org/10.2139/ssrn.5104064
Microsoft. (2024). Work Trend Index: AI at work is here, now comes the hard part. https://www.microsoft.com/en-us/worklab/work-trend-index/ai-at-work-is-here-now-comes-the-hard-part
Mitchell, M., Singh, A. B., Li, A. (2023). Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks. arXiv. https://arxiv.org/abs/2311.09247
Moskvichev, A., Odouard, V. V., & Mitchell, M. (2023). The ConceptARC benchmark: Evaluating understanding and generalization in the ARC domain. Transactions on Machine Learning Research, 8, 1–23. https://doi.org/10.48550/arXiv.2305.07141
Risko, E. F., & Gilbert, S. J. (2016). Cognitive offloading: The use of physical action to alter information-processing requirements. Trends in Cognitive Sciences, 20(9), 676-688.
Russinovich, M., & Salem, A. (2025). Obliviate: Efficient unmemorization for protecting intellectual property in large language models. arXiv:2502.15010. https://doi.org/10.48550/arXiv.2502.15010
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., Chi, E. H., Hashimoto, T., Vinyals, O., Liang, P., Dean, J., & Fedus, W. (2022). Emergent abilities of large language models. arXiv:2206.07682. https://doi.org/10.48550/arXiv.2206.07682
By 'moral instincts' we can think of cross-cultural moral foundations documented in anthropological research - basic intuitions about harm, fairness, and care that emerge reliably across human societies - not culturally-specific moral beliefs or practices.