Can Our Institutions Survive the AI Era? asks whether schools, hospitals, and governments can adapt while preserving their ability to serve people. This companion contains the research, historical context, and qualifications behind that argument. The evidence identifies mechanisms to test; it does not establish a forecast of institutional collapse.
A specific failure mode deserves closer attention in the institutions that govern human expertise. As AI improves, safeguards can preserve the form of human authority while weakening its substance. I want to give that risk a name and a test. Start with the evidence, because the details matter.
In an expanded sequential-diagnosis benchmark published in Science on October 22, 2026, Microsoft’s MAI-DxO correctly diagnosed 83.2% of difficult NEJM cases, compared with about 22% for generalist physicians without references and 36.1% with references.1 These were controlled simulations, not prospective clinical outcomes. In the Swedish MASAI trial, AI-supported mammography detected more cancers with higher sensitivity while cutting radiologist reading workload by 44%.2 And in a Stanford GSB counterfactual analysis, an AI system making simulated quarterly adjustments to real fund portfolios, using only public information, outperformed more than nine in ten mutual fund managers across thirty years of data. The result surprised accounting researcher Ed deHaan and his colleagues enough that they spent twelve months checking their data and model for an error that might explain it. They reported finding none.3
None of these results, alone, proves professional replacement. Together they establish something narrower: AI can outperform unaided professionals on selected benchmarks, and AI-supported workflows can improve particular screening performance measures. They make institutional redesign worth testing. They do not establish how often an existing constraint causes harm, or whether any specific profession should lose its role.
What’s Actually Happening
I keep returning to a distinction that comes up in my presentations around the nature of education. By education I mean drawing forth capability and judgment, developing a person’s potential. By training I mean optimization toward specified outputs. This is a philosophical distinction I am making, rather than a claim that etymology settles it. Training can cultivate skill; the danger is treating measurable compliance as the whole of education.4
Institutional routines can make that substitution tempting. Documentation, standardized assessment, and repeatable review are useful, but they are easier to count than judgment. The question is whether the process continues to develop the capability it was meant to serve.
Many professional institutions developed when expert cognition was scarce, so execution, judgment, and accountability were bundled into the same licensed human role.
Scarcity was one reason for the bundle. Kenneth Arrow named another in 1963: a patient cannot judge the quality of the care being bought, so licensing and professional norms stand in for a judgment the patient cannot exercise. AI moves that second reason as well, since a patient with a capable tool sees more of what the clinician sees.5
AI is changing the relationship among those functions. Accreditation, liability, reimbursement, procurement, and professional rules can make it difficult to redesign the bundle. Those mechanisms are not interchangeable, and each needs to be examined in its actual setting.
This is the Constraint Acceleration Paradox: a safeguard calibrated to a human-limited environment remains fixed while machine capability improves. As the mismatch widens, the safeguard’s marginal protective benefit can fall while its opportunity costs, agency losses, and competence effects compound.
A safeguard in that position fails in one of two ways. It blocks the better workflow, or it survives as a ritual that keeps the form of human authority and loses the substance.
To be precise about when the paradox bites, four things have to be true. The capability improvement must replicate prospectively across representative tasks and settings, not just on a benchmark. The constraint has to remain materially static. The fixed constraint has to cause measurable secondary harm to the people it was built to protect: delay, scarcity, access restriction, skill decay. And someone has to be able to specify and test an adaptive alternative that preserves the safeguard’s underlying value at lower cost. Without all four, “constraint acceleration” collapses into a fancy way of saying old rules are annoying. With all four, it identifies a testable failure.
Legal services offers a useful test case. Restrictions remain in many jurisdictions, alongside important state experiments. The Legal Services Corporation found in 2022 that low-income Americans got no help or too little for 92% of their substantial civil legal problems. Utah’s regulatory sandbox and Arizona’s alternative business structures are monitored alternatives. The evidence reviewed here does not yet establish representative, replicated AI gains, the share of unmet need caused by a particular restriction, or whether an alternative preserves protection at lower cost.6
I am proposing a diagnostic framework, not claiming that the examples here already establish its prevalence. The evidence below therefore identifies pressure points where institutional redesign should be tested, not domains in which unrestricted delegation has already been justified.
Three mechanisms need to remain separate: blocking a better workflow can create an opportunity cost; automating practice can erode a reviewer’s competence; and concentrating control can weaken a person’s ability to challenge a decision. They can reinforce one another, but evidence for one is not evidence for all three. A larger capability gap, by itself, measures none of these harms.
One more distinction before the data. Being better at executing a task is not the same as having authority over the goal, and neither is the same as bearing responsibility when things go wrong. A system can out-diagnose a physician on a benchmark without acquiring any standing to decide whether a patient values longevity over independence. Institutions confuse those three layers in both directions, and the confusion is where the trouble starts.
The Data
Stanford HAI’s 2026 AI Index brings together evidence of a substantial gap in expectations. In Pew surveys conducted in 2024 and published in 2025, 73% of experts expected AI to improve how people do their jobs, compared with 23% of the public. That question concerns the experience of working, not the number of jobs available. On AI’s impact on the country over twenty years, the matching comparison was 56% of experts and 17% of the public. The Index also reports U.S. trust in government to regulate AI at 31% in its international comparison.7
Meanwhile, the Index’s historical comparison puts generative-AI adoption at approximately 53% within three years, ahead of the corresponding trajectories for the personal computer and internet. Treat that as an approximate comparison: the underlying survey populations differ. Earlier NBER research by Bick, Blandin, and Deming found 39.4% of U.S. adults aged 18 to 64 using generative AI by August 2024.8
Put those findings side by side, without assuming they describe the same individuals. The public-opinion studies show concern about too little regulation as well as low confidence in the actors providing it. Rapid adoption can coexist with that concern. Adoption is behavior, not endorsement; people can find something useful without trusting everyone involved.
This suggests a problem of confidence in who designs and administers constraint. It does not show that the public rejects governance, or believes no institution could provide it. The more useful question is what would make governance deserve that confidence.
An adaptive constraint needs a credible adapter. Low trust may make legitimate revision harder, although these surveys do not establish that it prevents rules from changing.
Standing, Capacity, Competence
Return to the three patients from the first essay: the physician receiving care, the patient with little technical grip on the decision, and the patient who has managed a condition for years. They differ in medical knowledge. The second patient still holds what the other two hold: knowledge of her own values, tolerable burdens, religious commitments, and acceptable risks. Treating her need for explanation as a lack of standing is a distinct failure.
The problem is not that medical ethics fails to articulate patient rights. It explicitly recognizes information, participation, consent, refusal, and second opinions. The problem arises when a process obtains formal consent without supplying the conditions for informed participation. Authority over values, consent, and refusal does not scale with medical knowledge.
Three words have to stay apart here. Standing concerns a person’s moral worth and entitlement to be heard. Capacity is the ability to decide meaningfully, which informed consent requires. Competence is technical skill, the thing a reviewer can lose. Patients need support for decision-making capacity; reviewers need maintenance of technical competence. Nobody’s moral worth or entitlement to be heard depends on medical expertise. When decision-making capacity is impaired, supported participation and appropriate surrogate safeguards remain necessary.
The scope of technical self-direction and the intensity of support should respond to competence, stakes, and the particular decision. AI could widen access to that support; it could also make an apparently complete explanation easier to deliver without a meaningful opportunity to challenge it.9
What History Can and Cannot Tell Us
I wonder about the relevance of Nuremberg. On August 19, 1947, the judges in the Doctors’ Trial delivered their verdict, including ten principles that became the Nuremberg Code: voluntary informed consent, the right to withdraw, minimization of suffering. Its international ethical significance should not be confused with the creation of a uniform, enforceable international regulatory system.10
National standards existed; Germany had issued guidelines on human experimentation in 1931.11 Their existence complicates any story that protections appeared only after catastrophe: rules can exist and still be violated. The cases are also morally non-equivalent, and I want that stated plainly. Nuremberg answered criminal atrocity, coercion, and murder. Institutional lag in AI governance is not that, and I am not claiming it is.
I return to it for the boundary it clarifies. The Nuremberg Code is itself a constraint, and an indispensable one. It protects consent and human dignity regardless of what any technology can do. Everything that follows depends on that point. Constraint itself is not the problem. Some constraints hold the structure up. Historical catastrophe does not prove that today’s institutions can change only after failure; it gives us a reason to protect the underlying values while examining how they are implemented.
The Publishing Problem
Publishing exposes a related timing problem. Preprints accelerated circulation; they did not supply trusted validation at the same speed. The problem is not peer review itself. The strain comes from asking one process to serve as validation, certification, and archival record while model releases alter the object being studied. A capability snapshot may age before formal review finishes, even when its methods and findings about failure remain useful. The bottleneck is timely validation, not the ability to circulate a preprint.
This does not require malicious reviewers. It requires separating functions that need different clocks, while retaining checks on whether the faster process still works.
One review institution has tried that separation. Facing more than thirty thousand submissions, roughly double the prior year, AAAI generated an AI review for each of the 22,977 papers that reached full review at its 2026 conference, in under a day. The reviews were labeled, carried no scores, and had no authority to accept or reject. Survey respondents favored them on several dimensions, but also reported more technical errors and unhelpful suggestions. This was evidence of useful additional feedback, not proof that scientific validation had been preserved. The bottleneck got a second lane, and humans kept the decision.12
Software engineering offers a warning about the measurement itself. Stanford’s 2026 AI Index shows SWE-bench Verified rising from roughly 60% to nearly 100% of its normalized human baseline in a year; that is not a near-100% raw solve rate. Separately, OpenAI’s audit found flawed tests rejecting correct fixes in 59.4% of 138 selected difficult problems, and documented contamination concerns. The percentage does not describe the whole benchmark, and the defect runs in the opposite direction from accepting unfixed bugs.13 Tests that share a name can also measure different task sets, tools, time budgets, or memory arrangements. Maintaining a trustworthy ruler is part of adaptation. Finding and retiring a flawed measure is evidence of that work, not automatically evidence that the measurement institution has failed to change.
Same pressure in education, where accreditation, procurement, assessment, and labor structures can make redesigning the teacher’s role far slower than the technology changing it. The student side has a measurement: high school students given unrestricted access to GPT-4 did better on practice problems and worse on the closed-book exam that followed, while a tutor that bundled learning safeguards avoided the drop.14
Healthcare is more instructive, and I want to read the evidence straight. MASAI didn’t test AI against no radiologist; a radiologist read every screen in the intervention arm, with AI triaging which cases got one reader and which got two. MASAI is a concrete example: an institution adapting its constraint, replacing uniform double reading with risk-tiered delegation, and improving detection while reducing screen-reading workload. The open question it hands us is the right one: which residual human functions remain causally valuable, and who is going to test that? Meanwhile, liability can preserve human involvement long after its epistemic value has declined, producing roles that need redesign rather than mere retention. The question is whether a retained role still provides a measurable protection, and what would replace that protection if the role changed.
The Deeper Thing
Underneath all of this sits a harder question. We do not need perfect machines for it to arise; unevenly capable ones are enough.
When machines become better at execution, can humans stay capable of judgment? With choosing what’s right, not just what’s correct?
Consider the failure pattern to look for: a radiologist, teacher, or lawyer remains responsible for work the institution increasingly delegates to a system, but receives less practice, time, or access to the evidence needed to challenge it. Whether any particular requirement creates that pattern needs to be demonstrated; a human signature alone proves neither protection nor obstruction.
In Intentional Deskilling, I distinguished the deliberate release of skills from their unnoticed erosion. Applied to institutions, that distinction requires identifying which tasks can move to a tool, which forms of practice sustain independent judgment, and how that judgment will be maintained after the work changes.
The person kept in that kind of loop may be losing the very capacity the institution relies on. Lisanne Bainbridge named the ironies of automation in 1983, describing how automation can leave humans responsible for rare, abnormal cases they no longer get the practice to handle.15
Budzyń and colleagues observed a decline consistent with, but not proof of, AI-caused deskilling in 2025. Across four Polish centres, nineteen experienced endoscopists detected adenomas without AI at a rate of 28.4% before routine AI exposure and 22.4% after, a relative decline of about a fifth over three months. This before–after comparison cannot separate retained skill loss from changing case mix, practice conditions, or other confounders. A prospective 2026 Norwegian study, using a different detection outcome, found no significant unaided deterioration. Neither study settles the general effect.16
Passive sign-off can preserve formal accountability while eroding practical competence. A reliable tool can encourage deference, and its remaining errors may become harder for routine reviewers to recognize. Dratsch and colleagues saw this in mammography, where readers at every experience level performed worse when the AI’s suggested category was wrong.17
Madeleine Clare Elish’s later account of “moral crumple zones” examines the related transfer of responsibility to people with limited control.18 These are antecedents to the argument, not evidence that the proposed paradox is unprecedented.
The formulation I’d stake this essay on: the mistake is not keeping humans in the loop. It is keeping them in the wrong loop. Routine approval adds little if nobody can independently evaluate the result. Authority over goals, value tradeoffs, consent, and appeal is a different matter: it does not depend on humans remaining technically superior. Handling exceptions still requires empirical comparison; it is not a category in which people necessarily outperform machines. Oversight needs conditions you can design and test: independent sampling, escalation based on stakes as well as uncertainty, protected time and authority to challenge, and periodic unaided practice. A confident system can be wrong, so its own uncertainty cannot be the only trigger.
One evaluation exposes a limit of that check. OpenAI’s system card for GPT-6 Astra reports lower ability to detect problematic behavior from its written reasoning than for GPT-5.6 Sol, even as other behavioral measures improve. Action monitoring and full-context monitoring are different checks, and the finding covers one model in its evaluation settings. The result does not establish a general capability–oversight relationship; it shows why the effectiveness of each safeguard needs testing.19
A related design has been tested in practice. Penda Health and OpenAI studied a copilot across 39,849 primary-care visits in Kenya. It checked clinical information and flagged concerns while clinicians retained treatment decisions. Physician review of sampled records found relative reductions of 16% in diagnostic errors and 12.7% in treatment errors. Visits took longer, and the study did not establish long-term preservation of skill or significant patient-reported outcome improvements. Its design included required acknowledgment of critical alerts and implementation support. The study supports further testing of this workflow; it does not isolate which element caused the improvement or demonstrate an absence of rubber-stamping. A June 2026 randomized trial in the same Penda network improved process and documentation measures but did not establish improvement in the primary 14-day treatment-failure outcome. It is additional evidence from the same network, not independent replication of every earlier result.20
Which leaves the question: as systems outperform humans on more and more of execution, and humans have been kept just functional enough to say yes, who actually decides what’s right?
The Builders Are Starting to Say It
In July 2026, two of the industry’s most consequential leaders approached related questions from different altitudes. Their proposals deserve examination, including scrutiny of the commercial interests they represent.
Satya Nadella described what he calls the Reverse Information Paradox. Kenneth Arrow’s classic information paradox held that a buyer can’t value information until he has it, at which point he’s acquired it for free; the seller bears the risk of giving knowledge away. In Nadella’s account, hosted AI can reverse that risk: customers may pay for the service while also disclosing proprietary knowledge needed to make it useful. Every prompt, tool trace, correction, and eval can encode institutional know-how, and Nadella’s concern is that this intelligence exhaust can cross the organizational boundary unless the customer controls the data, memory, evaluation, and adaptation loop. Nadella argues that organizations may lose bargaining power when their contributions improve a vendor’s system without giving them comparable control over the resulting capability. This is a related problem of control over learning infrastructure. An enterprise can lose practical independence when it cannot move its memory or evaluation process. But enterprise ownership and individual agency are not equivalent: a company can control its infrastructure while giving workers or patients little recourse. And his prescription, own your evals, own your organizational memory, decouple the orchestration layer from any single model, is a demand for agency inside the system rather than a plea to slow the system down.21
Two days later, Demis Hassabis published a framework for governing frontier AI: a standards body modeled on the Financial Industry Regulatory Authority (FINRA), the U.S. securities industry’s self-regulatory body, voluntary pre-release review hardening over time into required certification, benchmarks refreshed regularly, perhaps quarterly at first, and deprecated as they saturate.22 The proposal attempts to write a framework before catastrophe, and it anticipates the publishing problem by building in its own revision cycle. Whether such a body can update faster than the capability curves it’s meant to govern is one open question. Hassabis proposes an industry-funded, federally overseen body with independent experts and open-source representation. Whether its governance, funding and appeal mechanisms would make that independence effective remains unresolved. FINRA operates under oversight from the Securities and Exchange Commission (SEC), a U.S. government regulator, but the analogy alone cannot establish how the proposed AI body would perform. The proposal also lands six weeks after the June 2 executive order, Promoting Advanced Artificial Intelligence Innovation and Security, which directed agencies to design a voluntary prerelease framework. The June order did not itself authorize the mandatory certification Hassabis envisioned; the September accord leaves future codification open.23
The people with substantial commercial stakes are now discussing lock-in and adaptive oversight in public. That makes the proposals consequential. It does not make them independent confirmation of this essay’s diagnosis. The two map onto the framework differently. Nadella describes a new constraint, lock-in on the learning loop, that could itself go static. Hassabis describes a constraint with its revision cycle built in.
So What Now?
People are adopting capability without waiting for institutions to catch up. They’re using coding agents, diagnostic tools, trading systems. They aren’t asking permission.
A poorly fitted constraint can block institutional evolution and preserve avoidable scarcity. But rising model capability does not abolish scarce clinical time, physical resources, implementation capacity, or the need for careful judgment.
Will institutions evolve voluntarily? I have to soften my own instinct here, because this essay contains the counterexamples, and the four-part test sorts them. Legal services combines substantial unmet need with restrictions and monitored alternatives; representative, replicated AI gains, restriction-caused harm, and preservation of protection at lower cost remain to be established. Swedish mammography screening is the case where the rule moved before the paradox could bite: MASAI replaced uniform double reading with risk-tiered delegation inside a randomized trial, so the second condition never held. Kenyan primary care has a specified alternative and early real-world studies with differing outcomes; independent replication and longer-term evidence are still needed. Education and Robodebt run the other way: the measured harm came from removing a safeguard, and in the classroom the tutor’s package of learning safeguards avoided the observed decline. Anthropic revised Fable’s safeguards after acknowledging that broad biology restrictions diverted legitimate requests to a less capable model; its August revision cut those fallbacks by about 85% in internal testing, an access measure rather than proof that protection against misuse survived.24 Hassabis proposes rules that would themselves be revisited. The question is which incentives support that work and which reward procedural compliance even when it no longer improves outcomes. Damage in the gap between capability and adaptation is something to measure, not an area to infer from two curves.
The destination is a sorting exercise, and it’s urgent. Separate the constraints that preserve roles from the constraints that preserve values. Role-preserving constraints, the ones that require a particular professional to stay involved regardless of demonstrated marginal value, should give way to capability-sensitive mechanisms: delegation that responds to demonstrated performance, risk-tiered human involvement, reversible autonomy, continuous independent evaluation, retained human practice. Value-preserving constraints, the ones that protect consent, privacy, accountability, auditability, recourse, and equitable access, should be strengthened, because they’re about to carry more weight than ever. The categories overlap: a professional role may carry independence, continuity, practiced competence, or an accessible person who must answer a complaint. Those functions need preservation or a tested replacement.
A signature is the clearest case. Its function, a named person who answers for the decision, protects a value. Its form, a licensed individual approving every instance, protects a role, and the form can defeat the function by parking blame on the person with the least control.
Reversibility also requires an actual stop or rollback mechanism; some harms cannot be undone. Australia's Robodebt scheme shows what removing a value-preserving safeguard looks like: automated income averaging displaced individualized checking and unlawfully raised debts, while onerous dispute processes and failures to act on successful challenges left many recipients without effective recourse.25 Sort wrong in one direction and we entrench the cage. Sort wrong in the other and we tear out the walls that hold the roof up.26 The EU’s 2026 amendment postponed grouped high-risk obligations. In my framework, that raises a further question: which protections can safely wait, and which cannot?
Nuremberg’s framework followed catastrophe. We have an opportunity to examine today’s arrangements deliberately, using evolving evidence and public scrutiny. The evidence can justify strengthening a safeguard, narrowing it, or replacing it. The answer should remain open to revision.
And the harder question underneath: as machines take over more of execution, how do we preserve meaningful human authority over goals, values, consent, and appeal, and keep the humans holding that authority capable of exercising it?
Because safeguards become cages when they preserve yesterday’s roles instead of the values they were meant to protect.
Part of an ongoing exploration of human agency in the age of intelligent systems.
Companion reading: Can Our Institutions Survive the AI Era? Earlier essays on learning and agency: The Last Skill? and Intentional Deskilling.
Expanded MAI-DxO study, Science (October 22, 2026), and Microsoft’s current account. The difficult-NEJM-case benchmark reported 83.2% accuracy, compared with about 22% for generalist physicians without references and 36.1% with references. A separate MIMIC-IV-ED benchmark reported 80.3% versus 46.8%. These were controlled simulations, not prospective clinical outcomes. The June 30, 2025 account and original preprint describe the earlier 304-case, tool-restricted physician comparison.
MASAI: the screening-performance analysis reports a 44.2% reduction in screen-reading workload. Gommers et al., The Lancet (2026) reports sensitivity of 80.5% versus 73.8% and non-inferior interval-cancer rates; the numerical interval-cancer reduction did not establish statistical superiority.
The Shadow Price of Public Information, Stanford GSB working paper (2025), and Stanford’s account. Ed deHaan, Suzie Noh, Chanseok Lee, and Miao Liu conducted the study. Their year of internal checking is not independent replication. The counterfactual adjustments to historical fund portfolios do not represent a prospective trading trial or identify a regulatory barrier.
The popular “drawing out” explanation is not a secure etymological basis for separating education and training. The Etymonline entry distinguishes educare from the frequently invoked educere. The distinction in this essay is philosophical.
Arrow, “Uncertainty and the Welfare Economics of Medical Care,” American Economic Review 53(5) (1963). Licensing also serves the guild economics Friedman criticized in Capitalism and Freedom (1962); none of the three causes excludes the others.
Legal Services Corporation, The Justice Gap (2022); Utah Supreme Court Standing Order No. 15 (2020), which created the Office of Legal Services Innovation and its sandbox; the Arizona Supreme Court’s 2020 order eliminating ER 5.4 and authorizing regulated alternative business structures from January 2021. Upsolve initially obtained a preliminary injunction in 2022 against enforcement of New York’s unauthorized-practice law for trained non-lawyer advice on debt-collection forms; the Second Circuit vacated it in 2025, and the district court dismissed the claims in March 2026. The LSC finding concerns substantial civil legal problems, not people, and documents unmet need rather than the share caused by unauthorized-practice restrictions. The evidence reviewed here does not yet establish representative, replicated AI gains, the share of unmet need caused by a particular restriction, or whether an alternative preserves protection at lower cost.
Pew’s overview and twenty-year predictions; HAI’s Public Opinion chapter. These include different questions and populations.
Bick, Blandin and Deming, Management Science 72(10):8440–8456 (2026); earlier NBER Working Paper 32966; HAI economy chapter, Figure 4.3.9, p. 199. The adoption comparison uses ages 18–64 for RPS/CPS samples and all ages for its ITU comparator; it is not a single consistent global adoption series.
AMA Code of Medical Ethics, “Patient Rights”, and “Decisions for Adult Patients Who Lack Capacity”. Decision-making capacity can affect authority to consent or refuse; supported participation and appropriate surrogate safeguards remain necessary when capacity is impaired. Ethical standing, legal rights, emergency exceptions, capacity assessment, and the ability to require a particular intervention are different questions; this illustration does not give jurisdiction-specific legal advice.
United States Holocaust Memorial Museum, “The Nuremberg Code”. Its account distinguishes the code’s ethical influence from uncertainties about legal force.
Vollmann and Winau, “Informed consent in human experimentation before the Nuremberg code” (1996). This directly supports the historical antecedents.
Biswas et al., “AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilot” (2026). Self-selected survey responses concern perceived review quality; they do not establish preserved scientific validity, reliable error detection, or equivalent accept/reject outcomes.
HAI technical-performance chapter, Figure 2.1.1, p. 76; OpenAI’s primary SWE-bench audit. The 59.4% defect finding concerns 138 selected difficult problems and tests rejecting correct fixes. It cannot be generalized to every SWE-bench task.
Bastani, Bastani, Sungu, Ge, Kabakcı, and Mariman, “Generative AI without guardrails can harm learning: Evidence from high school mathematics”, PNAS 122(26), e2422633122 (2025), a field experiment in one Turkish high school. The unrestricted group improved on practice and fell on the subsequent exam relative to controls; the tutor group showed no exam decline. GPT Tutor bundled hints with teacher-supplied solutions and common-error guidance; answer-withholding was not isolated. One subject, one country, one model generation.
Bainbridge, “Ironies of Automation,” Automatica 19(6):775–779 (1983). It describes how automation may leave humans responsible for unusual cases without enough practice; it is not evidence of clinical prevalence or causal validation of this proposed framework.
Budzyń et al., “Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study”, Lancet Gastroenterology & Hepatology 10(10):896–903 (2025). The retrospective before–after comparison found lower unaided adenoma detection, consistent with but not proof of AI-caused deskilling; changing case mix, practice conditions and other confounders limit causal interpretation. Pedersen et al., “Learning and deskilling effects of artificial intelligence in colonoscopy among endoscopists with different levels of experience: a pragmatic, prospective trial”, Endoscopy 58(9):1003–1014 (2026), used a different detection measure, polyps ≥5 mm, and found no significant unaided deterioration after AI exposure. Neither study establishes the general effect.
Parasuraman and Riley, “Humans and Automation: Use, Misuse, Disuse, Abuse”, Human Factors 39(2):230–253 (1997); Dratsch et al., “Automation Bias in Mammography: The Impact of Artificial Intelligence BI-RADS Suggestions on Reader Performance”, Radiology 307(4):e222176 (2023). Neither study measures the marginal value of a specific retained sign-off; they establish the mechanism to test. Dratsch tested susceptibility to deliberately incorrect suggestions, not longitudinal skill loss or a general law that greater AI capability makes oversight less effective.
Elish, “Moral Crumple Zones” (2019). Responsibility without practical control is an established concern in automation research. The proposed contribution here is a framework for testing how capability changes the costs and benefits of retained constraints. Related prior framings are Collingridge’s dilemma (The Social Control of Technology, 1980) and the pacing problem (Marchant, Allenby, and Herkert, eds., The Growing Gap Between Emerging Technologies and Legal-Ethical Oversight, 2011). Collingridge’s dilemma concerns early uncertainty versus later entrenchment; the pacing problem concerns oversight lagging technological change and already includes discussion of adaptive governance. The claim here is narrower: a safeguard calibrated to a human baseline can invert from protection to harm once the baseline is exceeded, and the competence of the people retained inside it can decay in the process.
Astra system card, §9. It reports lower written-reasoning monitorability than GPT-5.6 Sol in its evaluation settings. Written-reasoning monitorability, action monitoring and full-context monitoring are distinct. Evaluation findings do not establish behavior across all deployments or a general capability law.
“AI-based Clinical Decision Support for Primary Care: A Real-World Study” (2025 preprint). Across 15 clinics, this cluster-assigned quality-improvement study, not a randomized trial, assessed errors through physician review of sampled records. Median visits lasted 16.43 versus 13.01 minutes; reported error reductions are relative, not percentage-point reductions. A June 2026 randomized trial in the same Penda network improved process and documentation measures but did not establish a reduction in its primary 14-day treatment-failure outcome. It is not independent replication of every result in the earlier study.
Nadella, “The Reverse Information Paradox” (July 12, 2026). Whether institutional learning crosses a vendor boundary depends on the actual architecture, contractual terms, and data practices.
Hassabis, “A Framework for Frontier AI and the Dawning of a New Age” (July 14, 2026). He proposes an industry-funded, federally overseen body with independent experts and open-source representation. Whether its governance, funding and appeal mechanisms would make that independence effective remains unresolved. Regularly refreshed benchmarks and potential mandatory review are proposals, not a functioning certification regime.
Executive Order, “Promoting Advanced Artificial Intelligence Innovation and Security” (June 2, 2026), directs agencies to design a voluntary framework for engagement with frontier developers before broader release, on a 60-day timeline. The relevant provision does not itself authorize mandatory licensing, preclearance, or permitting requirements; the order also preserves existing agency authority. Tech Policy Press’s August analysis raises questions about its secrecy; it is commentary, not governing law. Nonpublic test items can resist gaming; the distinct concern is whether criteria, accountable oversight, and review procedures can be scrutinized. FINRA’s testimony describes SEC oversight. The September White House accord remains distinct from mandatory preclearance but leaves future codification open.
Anthropic, “Improving Fable 5’s biology safeguards” (August 2026). The approximately 85% reduction measures biology-related fallbacks, not an 85% improvement in clinical outcomes or a complete estimate of false positives.
Royal Commission into the Robodebt Scheme, Report (July 2023, corrected July 11), with its corrigendum. The scheme began with a 2015 pilot and September 2016 online rollout; new averaging-only debts ceased in November 2019, and the scheme was terminated in June 2020. The Federal Court found income averaging unlawful in 2019. The official judicial account and Victoria Legal Aid’s case account document successful challenges alongside failures of systemic response. The failure concerned individualized checking and effective recourse, not the absence of every appeal route. The Dutch childcare benefits scandal, which brought down the Rutte cabinet on January 15, 2021, was another case of administrative injustice and failed recourse, not an identical mechanism.
Regulation (EU) 2026/1744 and the Council’s explanation distinguish stand-alone high-risk obligations delayed to December 2, 2027 from relevant product-embedded obligations delayed to August 2, 2028. The amendment also contains transparency-related transitional provisions. It illustrates legislative revision, not a clean division in which every value-protecting requirement stayed on its original schedule.