Essay by Eric Worrall
I’m now even more convinced of my AI drawbridge theory – incumbents using red tape to raise the drawbridge and block entry for new competitors.
From the United Nations;
Countries must increase AI regulation to avoid ‘existential risks’: Türk
By Isaiah Steinberg14 September 2026Human Rights
Voluntary self-regulation by companies developing frontier artificial intelligence (AI) is “nowhere near sufficient” to address existing harms and prevent advanced autonomous AI models from circumventing human safeguards, UN human rights chief Volker Türk wrote on Monday.
“The current race towards ever more powerful AI is a step change towards greater existential risks to every aspect of our lives,” he wrote in an open letter.
The High Commissioner’s message comes six days after a former researcher at Anthropic – the company behind the chatbot Claude – resigned from his position while warning that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
…
People over profit
Mr. Türk rejected the industry argument that AI safety protections slow down innovation, arguing that nobody should be forced to give up their human rights to enable progress.
As it stands, he said the rights to life, food, privacy, health and security are all threatened by unchecked AI development.
Whether AI continues to enable disinformation and unequal automation or tackles the climate crisis and cures diseases depends on how companies and governments choose to respond to the current “watershed moment,” according to Mr. Türk.
“The generation now entering adulthood will inherit the consequences of the decisions being taken on this issue without having a say in them,” Mr. Türk wrote. “We owe it to them to ensure those decisions are made openly, with their rights in view, and with the humility that the scale of this undertaking demands.”
Read more: https://news.un.org/en/story/2026/09/1168326
The essay by Anthropic CEO Dario Amodei
Dario Amodei
We Must Pace the Frontier
September 2026
I have worked on AI for the last twelve years because I believe it could dramatically raise the quality of human life. I’ve written often about these incredible benefits: I believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom. I feel the urgency personally. My own father died of a disease that was cured just a few years after his death, and I myself survived an early-stage cancer that would not have been treatable even fifty years ago. Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity.
But like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious. I’ve writtena lot about them too. They include the risk of losing control of AI systems, misuse of AI for cyberattacks and bioterrorism, and serious economic disruption. A race to the bottom, spurred by commercial incentives, can make these risks more acute.
…
My first concern is that, since roughly this summer, AI has been advancing drastically faster, driven primarily by AI’s growing ability to build the next generation of AI. This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic, as we and others have described. Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.
…
I’m therefore proposing a three-step plan with the goal of pacing the frontier: building AI at a balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas. To be clear, pacing does not mean halting model training or technical progress, but ensuring companies take adequate time to align and safeguard their models, and for third party evaluators to confirm this. Our pacing framework is an attempt to further strengthen our commitment to safety and encourage a race to the top. The first step is something Anthropic is unilaterally committing to (and calls on governments to require other frontier companies to match). The second step requires industry-wide coordination.1The third step requires global coordination. The steps do not need to be taken strictly in order, and some of them may be much harder to achieve than others, but I’ve found them to be a useful framework in thinking about what needs to be accomplished. The steps are:
- Embedded Evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work.
- Democratic Coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.
- Global Coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.
…
Read more: https://darioamodei.com/post/we-must-pace-the-frontier
Can you imagine the compliance costs of the bureaucratic nightmare Anthropic CEO Dario Amodei is proposing? The model the UN is gearing up to back?
A multi-billion dollar company with an entire floor of lawyers can afford such costs. A small startup with a good idea, not so much.
So why are AI companies so relaxed about slowing down, about “pacing”? Obviously only AI leaders know what is going through their own heads, but there is a problem with Artificial Intelligence the Anthropic CEO forgot to mention. For all their claims of progress, AI companies like Anthropic have hit a problem, a theoretical barrier. They may not have a choice about whether to slow down.
Make the AI more powerful, and it suffers out of control hallucinations – AI schizophrenia.
The AI Hallucination Paradox: The Smarter AI becomes, the More it Hallucinates
Youngwhan Nick Lee
8 min read
May 29, 2025Introduction
It might seem intuitive that a more intelligent, larger language model (LLM) would make fewer factual mistakes. However, recent research reveals a paradox: as AI models become more powerful, their tendency to hallucinate— i.e., produce confident but false or unfounded information — can persist and even increase in certain ways (Xu et al., 2024; Zhang et al., 2025). In other words, simply scaling up model size or training data does not straightforwardly eliminate hallucinations; in some cases, it worsens them. This paradox has critical implications: despite dramatic improvements in language fluency and reasoning, even state-of-the-art LLMs remain prone to fabricating details. In this article, we summarize two theoretical perspectives explaining why hallucinations are fundamentally unavoidable and how greater computational power can exacerbate them. We then illustrate the issue with an example of Anthropic’s advanced Claude model and discuss a proposed solution: a multi-agent architecture with a dedicated “hallucination monitor” agent to curb the spread of false outputs. We conclude by contrasting these findings with the mainstream assumption that more compute yields safer, more reliable models.
…
Read more: https://medium.com/@youngwhannicklee/the-hallucination-paradox-the-smarter-ai-becomes-the-more-it-hallucinates-b82fc2a3df9d
Keep the AI as is, and progress grinds to a halt, or requires crazy amounts of additional processing capacity for modest gains in capability.
There must be a way to overcome the artificial intelligence wall. Evolution solved the problem – our brains don’t experience this barrier to nearly the same extent as artificial intelligence systems.
Likely when the solution is discovered it will turn out to be really simple, an architectural principle nature stumbled across through incremental natural selection, which can easily be replicated once we know what to do.
Imagine somewhere out there is a brilliant data scientist with a novel theory of how to overcome this theoretical intelligence barrier.
If that scientist launches their own startup, all the big players, Anthropic, OpenAI and XAi, their models will be superseded junk. Their businesses will be toast.
BUT if there are massive bureaucratic hurdles and compliance costs to launching a new startup, that brilliant scientist will have no choice but to share their invention, to get a leg up from the incumbents.
As for the UN, in my opinion it’s just UN opportunism. Never let an opportunity go to waste.
President Trump’s take on the situation;
Trump Says AI Doesn’t Need More Guardrails and Calls Safety Fears a Hoax
President alleges that a conspiracy is undermining support for AI and data centers as industry leaders raise alarms about the technology’s risks
By Brian Schwartz and Philip Wegmann
Updated Sept. 14, 2026 at 5:16 pm ETPresident Trump pushed back on the need for tighter federal regulations on artificial intelligence and dismissed safety concerns as a hoax that would leave the industry in financial ruin and hand China a competitive edge.
The president’s forceful messaging on AI comes as industry leaders sound increasingly urgent warnings about the need for regulation to guard against potential dangers from their own technology, while public backlash builds to the construction of the data centers needed to power it.
“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades!” Trump wrote on Truth Social on Monday. “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China.”
…
Read more: https://www.wsj.com/tech/ai/trump-says-ai-doesnt-need-more-guardrails-despite-industry-warnings-b94263fa
In my opinion President Trump has seen straight through the fog as he frequently does. Industry calls for AI regulation in my opinion are disguised self serving protectionism for incumbents, an attempt to create an AI industry cartel.
If adopted the kinds of heavy handed regulations being pushed would stifle US freedom to innovate.
Update (EW): h/t R Taylor – Nvidia boss Jensen Huang has stood with President Trump, opposing calls for A.I. regulation. I’m sorry for including you in the original group photo Jensen.
