Many people working on AI and AI safety, or involved with the rationalist and effective altruist communities, have coalesced around the following views:
Imminent recursive self-improvement (RSI). Within a few years, AI systems will likely automate most of the work involved in building better AI systems. The resulting feedback loop will then dramatically speed up AI progress.
Imminent superintelligence. This process will quickly produce systems far more capable than even the most capable humans at virtually every cognitive task.
Imminent transformation. Where previous general-purpose technologies like electricity or the internet took decades to change how we live, superintelligence will radically transform the world very shortly after arriving: perhaps a century of technological progress compressed into a decade, or rates of economic growth without historical precedent.
Call this the “superintelligence perspective”.
Each claim combines a view about the direction in which AI is heading with a view about speed and scale.1
If the superintelligence perspective is true, it is perhaps the most important truth in the history of humanity.
It would be nice to dismiss it as the ramblings of Elizabeth Holmes-style charlatans, but that would be wrong.
For one thing, the perspective is taken seriously by many very smart, very well-informed people, including AI pioneers and people with impressive forecasting records. It is not just people working in the companies who endorse the superintelligence perspective, although tech leaders and AI engineers who endorse it should not be completely dismissed either. (Unlike Holmes, they have built perhaps the fastest-growing industry in human history on the basis of a technology that verifiably works across countless domains.)
More importantly, the perspective is supported by persuasive arguments and plausible extrapolations from existing trends. The length of certain software tasks that frontier AI systems can complete with 50% reliability, measured by how long the tasks take skilled humans, has been doubling roughly every seven months since 2019, and closer to every three to four months since 2024. This year alone, AI systems have found thousands of previously unknown vulnerabilities in widely used software and produced a computer-checked proof that appears to have solved one of mathematics’ Millennium Prize problems. According to an assessment by the research organisation METR published recently, Anthropic’s models have been improving on benchmarks at nearly twice their previous rate since October 2025, with most of the current acceleration probably due to AI itself.
Just this week, a mathematician at Anthropic announced that, working with Claude, he had found a proof of the Riemann hypothesis, another Millennium Prize problem and perhaps the most famous open problem in all of mathematics. (This has now been independently verified.) This might be the most important advance in mathematics in a century.
On capabilities, the forecasting record also seems to favour the crazy AI people (I use the term affectionately) over many of their critics. In a 2022 forecasting tournament, for example, a group of superforecasters—those with a proven track record of successful prediction—assigned only a 2.3% chance to an AI system achieving a gold-medal score at the International Mathematical Olympiad by 2025. It happened in July 2025. By contrast, a scenario that Daniel Kokotajlo, the lead author of “AI 2027”, wrote in 2021 about how AI would develop up to 2026 has held up surprisingly well.
So, could the superintelligence perspective actually be true?
Responsible AI Scepticism
In debates about AI, I gravitate towards what I will call “responsible AI scepticism”.
In saying this is responsible, I mean it faces up to the reality of rapid AI capabilities progress. It doesn’t dismiss today’s systems as stochastic parrots or fancy auto-complete, claim that AI can’t do things it can do or is clearly on track to be able to do, or confidently predict that deep learning is about to “hit a wall”.
Most fundamentally, it doesn’t assume that human capabilities have mysterious underlying mechanisms that couldn’t, in principle, be replicated in machines.
It is sceptical of the superintelligence perspective, however.
There are many such responsible sceptics, including people I have recently interviewed on my podcast, such as Steven Pinker, Séb Krier, and Arvind Narayanan.
One aim I have here is to review some of the reasons that go into this responsible scepticism, which I am broadly sympathetic to. But another aim, which is in some ways more important, is to highlight the limits of this sceptical perspective.
For one thing, none of the reasons for scepticism that I am aware of constitute knockdown arguments. This means that responsible sceptics must allow that the crazy AI people might be right. And this should weigh heavily on collective decision making.
For another, these arguments show, at most, that the superintelligence perspective is wrong about speed and scale: that recursive self-improvement will be slower, superintelligence more uneven and less powerful, and AI-driven transformations more gradual than its proponents expect. But the strongest such arguments don’t dispute that its proponents are directionally correct. And this fact has huge importance.
The Case for Responsible Scepticism
Two kinds of reasons for scepticism are general.
Default Scepticism
The first concerns our “priors” (i.e., what we should expect before looking at the specific evidence).
Crazy things rarely happen. Extraordinary claims require extraordinary evidence. These are basic principles of rationality.
The superintelligence perspective predicts the largest and fastest transformation in human history occurring in the next few years. Changes on anything like the scale it imagines have happened only twice, with farming and then the industrial revolution, and neither happened anywhere near as quickly. Forecasts of imminent upheaval, by contrast, are very common and usually wrong, even when they come from communities of high-IQ people with rational arguments they deem compelling.
So, before looking at any of the details, we should think the perspective is probably wrong.
Forecasters sometimes call this taking the “outside view”, which asks what happened in relevantly similar cases, as opposed to the “inside view”, which builds predictions from the specifics of a case.
How likely are you to get divorced? An inside view would look at the specifics of your marriage. An outside view would look at what percentage of married couples in a similar situation get divorced.
Of course, outside-view reasoning can only get you so far. And here, proponents of the superintelligence perspective will point out that they are not engaging in any galaxy-brained reasoning for a crazy conclusion; they are simply extrapolating robust trends, especially about capabilities growth.
However, the question concerns which trends we should extrapolate. Most of the trends on which the superintelligence perspective rests are relatively recent (a decade or so old). This includes not just rapid, plausibly exponential capabilities progress but exponential increases in the revenue of frontier AI companies. (Anthropic is probably the fastest-growing company in the history of capitalism.)
In contrast, longer-running stable trends should plausibly be given more weight, in part because they have survived more tests. For example, US GDP per person has grown at roughly 2% a year for around 150 years, despite many profound technological advances during this period (e.g., electrification, cars, computers, and the internet).
Moreover, not even all of AI’s own trends point towards an imminent explosion or transformative revolution. Recent progress has required inputs—chips, money, human researchers—growing at rates that plausibly can’t continue for much longer. AI’s footprint in the economy is also still small, with the share of American work hours that involve generative AI at only about 5%.
As many have pointed out, markets seem to share this scepticism. If investors expected explosive growth within a decade, they would plausibly save less and borrow more, pushing long-term real interest rates far above anything on record, which isn’t what we see.
More broadly, the whole economy looks surprisingly normal-ish if we are on the cusp of imminent world-transforming superintelligence.
Sociological Scepticism
The second reason concerns the collective “epistemics” of the community that developed the superintelligence perspective: that is, its reliability in producing accurate judgements and forecasts. Here, one doesn’t need to endorse hyperbolic discourse accusing the community of being a cult to be suspicious about its highly insular nature.
The perspective has been created and refined within a surprisingly small, weirdly densely connected social world with its own institutions, funders, vocabulary, and peculiar ingroup signals.
To give you just a taste of how strangely small this world is: when Open Philanthropy (now Coefficient Giving), the biggest philanthropic funder in the AI safety world, pledged OpenAI $30 million in 2017, it disclosed that two OpenAI researchers who advised it, Dario Amodei and Paul Christiano, lived in the same house as its executive director, Holden Karnofsky, who was engaged to Amodei’s sister. The Amodei siblings went on to co-found Anthropic, Karnofsky later joined it, and Christiano now sits on the board of OpenAI’s foundation.
Christiano is married to Ajeya Cotra, who worked for nearly a decade at Open Philanthropy and is one of the most influential voices in the rationalist, effective altruist, and AI safety worlds. She is now a researcher at METR, an organisation whose findings (e.g., about exponentially increasing capabilities and recursive self-improvement) play perhaps the biggest role in the entire information environment in lending support to the superintelligence perspective. METR began as a team inside a research centre founded by… Christiano.
I could go on for much longer, but it would be tedious and these things can be very easily googled (or Claude-d).
Obviously none of this proves that the community is wrong. Insular communities bound together by shared views, values, and ingroup markers can develop correct beliefs and conduct excellent research, and this one has an impressive track record. Nevertheless, when you think about a topic, you should consider not just the strength of object-level findings and arguments but whether the distribution of those findings and arguments is likely to be skewed towards particular conclusions.
In the case of the superintelligence perspective, there are legitimate reasons for concern: a community with strong shared priors that plausibly constitute a signal of group membership will produce lots of clever arguments and intellectual support for those beliefs whether or not they are true. And this should increase our scepticism of these arguments even absent specific objections to them. It should also influence how we interpret the findings of research organisations like METR that are claimed to be “independent” of frontier AI companies such as Anthropic and OpenAI whose technology they evaluate, not because of any ill-intent or incompetence on the part of the excellent researchers there but simply because their formal independence co-exists with highly overlapping worldviews and many informal social connections.
Three Specific Reasons for Scepticism
Other reasons are object-level, focusing on specific claims.
Recursive Self-Improvement
The strongest sceptical arguments concerning recursive self-improvement don’t deny that sufficiently advanced AI could, in principle, automate AI research, or that AI will very likely accelerate AI research, a process that has plausibly already begun. Rather, they identify reasons to doubt that most AI research will be automated in the next few years or that any near-term acceleration that does occur will lead quickly to an intelligence explosion that produces superintelligence.
Four main reasons stand out:
Diminishing returns. With recursive self-improvement, there is a race between two forces. Each improvement to AI creates more and better AI researchers, because the improved, more efficient AI can itself be put to work designing the next one. But each improvement also makes the next improvement harder to find. It is only if the first force outpaces the second that one gets an explosion. Otherwise, automating AI research might deliver something closer to a “spark”—a burst of progress that then fades.
Compute bottlenecks. Limited compute makes an explosion less likely. Unlike certain parts of pure mathematics, where AI systems today are clearly superhuman, AI research requires far more than raw intellectual labour: experiments need compute, adding AI researchers doesn’t add chips, and whatever can’t be sped up merely by improving AI becomes a bottleneck.
Relatively weak evidence so far. OpenAI has helpfully published data on how its models handle its own research tasks, most recently in a report on 18 November. In Ramez Naam’s interesting reading of the latest figures, OpenAI’s agents can autonomously complete, at 80% reliability, only tasks that take humans around sixteen minutes. Naam concludes that the feedback loop is real but still too weak to sustain itself.
Research taste. OpenAI’s recent report is titled “The jagged frontier”. Its agents complete 79% of tasks that involve monitoring training runs but only 25% of those that involve writing the code that serves models, and researchers claim that they have poor “research taste” (i.e., the ability to judge which research directions to explore and which tasks to perform, not just to execute assigned tasks competently). This is especially important given the plausible argument from Toby Ord that AI progress may depend on rare creative ideas, which current systems seem relatively bad at producing.
Superintelligence
Second, consider four reasons for scepticism about the understanding of superintelligence in the superintelligence perspective:
Jaggedness. Today’s AI systems are highly “jagged”: they are superhuman at some tasks while being surprisingly stupid or incompetent at others that most people find easy. This is plausibly because intelligence itself is not a single quantity. Even if permanent, this jaggedness doesn’t rule out superintelligence on reasonable definitions: if the troughs keep rising, they may eventually pass our peaks. But the trends that motivate the most aggressive forecasts of imminent superintelligence typically track the peaks, whereas a system “far more capable than the best humans at virtually every cognitive task” (a fairly standard definition) is defined by its troughs.
Checkable domains. AI’s most impressive achievements in 2026 (e.g., in mathematics, coding, and cybersecurity) have arisen in domains where answers can be verified. It’s not obvious that superhuman performance in these domains transfers to more open-ended, messy, data-poor domains relevant to many parts of biology, geopolitics, social prediction, or simply embodied “being-in-the-world” (to use a pretentious Heideggerian phrase) central to everyday human and animal intelligence.
Diminishing returns to intelligence. Compute bottlenecks are a special case of a more general issue: for most outputs we care about, intelligence is only one input. Even a perfectly general superintelligence would have to learn from experience and test its ideas against the world, because there is only so much you can learn through reasoning alone. And many processes in the world aren’t currently bottlenecked by a lack of intelligence. For many society-wide problems, we already know how to solve them. What stops us is flawed incentives, conflicting interests, gridlocked politics, and the slow all-too-human business of building things in an all-too-human world.
Limited headroom. In chess and Go, the ceiling of performance turned out to be far above the best humans. Narayanan and Kapoor make an interesting argument that for certain capabilities, including certain forms of forecasting and persuasion, the situation is very different. Here, to borrow François Chollet’s image, improving at the task looks more like trying to make an already very round ball rounder than making a building taller. This difference might reflect factors such as the irreducible uncertainty associated with certain tasks, or the role of subjective human taste and trust in evaluating performance. If this perspective is correct, superhuman performance in some domains doesn’t imply its possibility in all of them, or even in those domains most relevant to real-world power and impact.
Transformation
Third, there are at least three kinds of reason to doubt that even superintelligence would be able to transform the world within a few years of arriving.
Slow diffusion. The fact that a technology has transformative capabilities and so could, in principle, transform the world does not show that any such transformations will happen quickly. In a classic paper, the economic historian Paul David showed that electrification took roughly forty years to transform manufacturing, because factories had to be redesigned around it. Diffusion (and, hence, transformation) is slowed by many factors, including regulation, liability, organisational inertia, and the human pace at which stubborn, lazy, irrational, status-quo-biased humans and organisations learn and adapt to new technologies.
Bottlenecks. As Tyler Cowen argues, the more productive AI becomes, the more everything else constrains its effects. If superintelligent AI produced ten or even one hundred times as many good ideas for new drugs, the bottlenecks would be testing them and getting them approved.
Cost disease. Similar logic applies to the economy as a whole: those sectors that improve the slowest tend to eat up a larger share of the overall economy as everything else gets cheaper. Consider, for example, Cowen’s estimate that government, non-profits, much of higher education, and the worst of health care comprise 40% or more of the American economy. These aren’t fast-adapting sectors, to put things mildly.
No Knockdown Arguments
How plausible are these reasons for scepticism?
Overall, I think they add up to some compelling grounds to doubt the superintelligence perspective. But even if you are persuaded by them, this responsible scepticism has limits.
First, you should not be fully persuaded by any of them, and it’s suspicious if you are highly persuaded by all of them.
None of these arguments is a knockdown reason for scepticism. Each of them turns on quantities nobody can currently measure with satisfying precision: the returns to AI research, to what extent compute and cognitive labour can substitute for each other, how quickly AI’s abilities will improve in the domains they are worst at, the marginal returns to intelligence across different domains, and the speed of diffusion for a technology that is radically different to any that has come before.
The general arguments are different, and in some ways stronger, but they aren’t knockdowns either. A prior is, at most, a starting point for investigation, and there is now a lot of evidence that pushes against it, even if that evidence is sometimes difficult to interpret.
The outside view also depends on choosing the right reference class, and reasonable people can contest the sceptics’ choice. Over the very long run, the growth rate itself has accelerated. When it comes to rates of economic growth and technological progress, what seems “normal” now is, in a broader historical context, freakishly abnormal. Proponents of the superintelligence perspective also have a ready explanation of why 2% growth has been so stable and why the arrival of true superintelligence would break it: growth has been limited by the supply of researchers and workers, a limit that an explosion of human-level or superintelligent AI workers would remove.
Sociological scepticism has limits as well. I think it is reasonable for the community associated with the superintelligence perspective to set off people’s epistemic alarm bells, but this is, at most, a reason to weigh its arguments cautiously and seek out alternative viewpoints. It isn’t a reason to conclude they are wrong. And sceptics obviously have communities, biasing incentives, and status games of their own, as well as a marketplace of rationalisations that rewards reasons to doubt (or simply dunk on) proponents of the superintelligence perspective.
Further, the impressive track record of many crazy AI people, not just in forecasting capabilities progress but in actually building extremely capable AI and unprecedentedly fast-growing companies, is very unlikely to be down to pure luck.
One thing underrated by those who focus on the suspiciously cult-like properties of the community is that these properties co-exist with good epistemic norms and practices. In my experience, the quality of thinking and intellectual discourse among groups like effective altruists is, on average, much higher than is typical even in academia, let alone in ordinary public discourse. It is simply good to put numbers on things, cite data, bet on predictions, and grade accuracy, even granted that such things can be performative and ingroup marking. And although the community also undoubtedly features some groupthink, they are an unusual group in taking measures to avoid this. For example, the interest-rate argument I listed as a reason for scepticism about imminent AI-driven economic transformation won a contest that one of the community’s main funders ran to discover the best challenges to its own views.
So, even if you are persuaded by all these reasons for scepticism, you should still assign some non-trivial credence to the superintelligence perspective being correct. And given that many of the reasons for scepticism are logically independent of each other, a person highly persuaded by all of them may be letting the desire to disprove the superintelligence perspective or simply their dislike of those who advance it cloud their judgement.
Directionally Correct
Second, even if you do accept all these reasons, you should still allow that the superintelligence perspective is directionally correct.
At most, these sceptical arguments give some reason for thinking that RSI won’t be as rapid or explosive as many think, that superintelligence won’t be as powerful as many think, and that transformations will take longer than the superintelligence perspective predicts. But these are issues of speed and precise magnitude.
The arguments about recursive self-improvement concern how large a speed-up automated AI research will deliver, and how soon we will automate a large share of it. The arguments from jaggedness and diminishing returns to intelligence grant that AI is, or will be, superhuman across many cognitive domains and simply question its reach in others. The diffusion arguments concern how long transformation takes, but, even granted all the arguments, a technological revolution on the scale of the industrial revolution would be huge, which is what Narayanan and Kapoor, the authors of “AI as Normal Technology”, themselves expect.
Even the general arguments bear mostly on speed. A prior weighs against a claim more heavily the shorter its window and the more dramatic stuff that must happen inside it, so it counts much more against transformative change by 2030 than against such change unfolding over the next few decades. Sociological scepticism can discount what one community believes, but the direction of its predictions is increasingly shared by many people with few ties to the community.
So, responsible sceptics should concede that the following is highly likely unless there are deliberate political interventions to stop it: something like RSI will happen, it will accelerate AI progress, extremely capable AI systems will emerge in the coming years that will greatly exceed human capabilities in many domains, and this will radically transform the world.
And this is monumentally important!
When you are in the trenches of the AI debates, small differences can seem like big differences. A 2027 forecast and a late-2030s forecast can feel hugely different to those in the weeds of these discussions—a version of Freud’s narcissism of small differences. But in the grand scheme of things, these differences are small relative to those that separate both groups from many mainstream elites and normies.
Obviously whether radically transformative AI arrives in 2030 or in a complex, jagged process that lasts decades matters enormously for some questions, so it is understandable that smart, informed people focus on these questions. Seen from outside the debate, however, both positions say that something at least on the scale of the industrial revolution is likely to happen within the lives of most people alive today.
So, although responsible scepticism plays a very valuable role in AI debates, I think responsible sceptics need to be clear about what is and isn’t in dispute. The large audiences disposed or eager to see AI as all hype and bullshit can hear responsible scepticism as confirmation of their views. To disabuse them of that impression, responsible sceptics must state their concessions to the superintelligence perspective along with their doubts.
Huge change is coming. It’s reasonable to argue about when and how, but reasonable people should be clear that this is all we’re arguing about.
Further Reading
Arvind Narayanan and Sayash Kapoor, “AI as Normal Technology”
Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland, and Romeo Dean, AI 2027, and the AI Futures Project’s November 2026 update
Will MacAskill and Fin Moorhouse, “Preparing for the Intelligence Explosion”
Toby Ord, “Broad Timelines”
Séb Krier, “Of Swarms and Sand Gods”
Steven Pinker, “An Open Letter to Scott Alexander”, and Scott Alexander’s reply
The authors of AI 2027 and AI as Normal Technology, “Common Ground between AI 2027 and AI as Normal Technology”
To be clear, I’m not focusing here on the specific worry of AI doom, the risk that misaligned superintelligent AI will kill us all, which raises separate questions.


