Benjamin Todd on whether transformative AI is only a few years away, the feedback loops that could compress decades of progress into months, and how to navigate the transition to a post-AGI society.
I work on AI safety and don’t believe we’ll see AGI by 2030. Nor do I believe it’s epistemically responsible to have determinate “timelines” beyond that.
Y’all keep talking about revenue growth without talking about costs. Why should we think capacity investment continues apace if it’s so massively unprofitable for the frontier labs and demand is racing towards open source inference?
The arguments for AI R&D acceleration are pretty weak. First, there is not good evidence of AI-driven R&D acceleration in general (by which I mean, generating fruitful novel hypotheses that push the scientific or engineering frontier). Ofc AI accelerates some of the stuff like lit review, search, and experiment setup. But those don’t compound to productive innovation if researchers aren’t generating fruitful new ideas.
So if we assume the R&D thing is moot, and revenue craters as the majority of inference demand moves to open source, what keeps the flywheel spinning?
While I think AI is going to revolutionise a lot of our lives I'm always a bit sceptical of these pseudo-precise speculative predictions that tend to come out of an EA/rationalist intellectual tradition.
And that probably leads me to being a bit cautious around the idea that any individual is going to make a massive difference in a world where we're largely interchangeable widgets in the machine. I'm a doctor, I help people - but did me becoming a doctor create a net benefit? Am I that much better than the next person down who might have got in instead? I doubt it - maybe they'd have been better.
There's a slightly unsettling sense here of some people being uniquely virtuous or competent implied in the idea that they should focus on the field of AI. Is it really the case that if graduate X doesn't go work at Anthropic there isn't a completely interchangeable graduate Y to take their place. There's something... I don't know, redolent of the revolutionary cadre about it. And I'd be concerned if we have too many from one particular intellectual tradition going into a field - maybe people particularly preoccupied with say x-risks, that sort of homogeneity of thought might be actively harmful (I think Dan nods at this).
Not to mention the crypto elephant in the room. A previous bet that didn't pay off.
Funny how far "mere engineering without a theory of intelligence" has taken us in just a few years. (Narrator: "It's not at all funny.")
I work on AI safety and don’t believe we’ll see AGI by 2030. Nor do I believe it’s epistemically responsible to have determinate “timelines” beyond that.
Y’all keep talking about revenue growth without talking about costs. Why should we think capacity investment continues apace if it’s so massively unprofitable for the frontier labs and demand is racing towards open source inference?
The arguments for AI R&D acceleration are pretty weak. First, there is not good evidence of AI-driven R&D acceleration in general (by which I mean, generating fruitful novel hypotheses that push the scientific or engineering frontier). Ofc AI accelerates some of the stuff like lit review, search, and experiment setup. But those don’t compound to productive innovation if researchers aren’t generating fruitful new ideas.
So if we assume the R&D thing is moot, and revenue craters as the majority of inference demand moves to open source, what keeps the flywheel spinning?
While I think AI is going to revolutionise a lot of our lives I'm always a bit sceptical of these pseudo-precise speculative predictions that tend to come out of an EA/rationalist intellectual tradition.
And that probably leads me to being a bit cautious around the idea that any individual is going to make a massive difference in a world where we're largely interchangeable widgets in the machine. I'm a doctor, I help people - but did me becoming a doctor create a net benefit? Am I that much better than the next person down who might have got in instead? I doubt it - maybe they'd have been better.
There's a slightly unsettling sense here of some people being uniquely virtuous or competent implied in the idea that they should focus on the field of AI. Is it really the case that if graduate X doesn't go work at Anthropic there isn't a completely interchangeable graduate Y to take their place. There's something... I don't know, redolent of the revolutionary cadre about it. And I'd be concerned if we have too many from one particular intellectual tradition going into a field - maybe people particularly preoccupied with say x-risks, that sort of homogeneity of thought might be actively harmful (I think Dan nods at this).
Not to mention the crypto elephant in the room. A previous bet that didn't pay off.