Regarding David Kipping's point about there not being any point to doing science anymore if AI can do it all for you, and how scientists do the science for the joy of understanding and not just to get the answers: Maybe Kipping is not used to this because he is a "real, professional" scientist producing novel results, but I enjoy learning science as an amateur. I have never produced a novel result in science. I am dutifully going back and forth between my textbook on special relativity, realizing my calculus is a bit too rusty to solve this textbook problem, going to my calculus textbook to remind myself how to do u-substitution, and then back to the special relativity textbook etc. It's slow going. I hope one day I'll be able to fully get special relativity and one day get to general relativity. But maybe I'll die of old age before that happens, who knows. From my perspective, it really doesn't matter whether special relativity (or general relativity, or anything else) was produced by a human or an AI. In both cases, it's "sacred texts handed down from high above", and I enjoy exercising my brain trying to understand it. Again, maybe this is too alien of a perspective for a professional scientist to adopt. But for a hobbyist scientist, AI poses absolutely no threat to our love of science: We do it because we want to understand, and we will still want to understand, no matter who or what produced the theories.
Regarding Dan William's point about "there’ll be this long period where we’re augmented with this technology, rather than replaced by it", I believe there was a study that showed "AI alone" performed better than "human doctor alone" and also better than "human doctor collaborating with AI", in a test of diagnosing medical issues from a description of the symptoms. I.e. as much as we'd like to flatter ourselves, we might just be getting in the way, like a child who wants to "help" their parent (or a cat that just wants to sit in front of whatever its owner is doing, regardless of how inconvenient that might be to the owner).
> It seems like many people have the intuition that if it is in fact the case that we are the only animals that creates super-intelligent AI, there’s something kind of surprising about that, just given the scale of the universe. As someone as an outsider to this whole literature, it strikes me there’s always something that seems sort of teleological in the way that that assumption gets set up — as if there’s some tendency in the universe towards intelligence and then technology and civilization. If we were the only animals in the universe that ever produced the music of the Beatles, I wouldn’t find that a priori very surprising.
Thank you for asking this, never really understood why the throwaway explanation "I guess nobody else cares" isn't taken more seriously.
And even in their current framing, seems like Fermi questions should incorporate biotech's capacity to alter evolutionary drive. If GLP1s, adderol, antidepressants, etc help users voluntarily alter drives the why shouldn't we extrapolate those technologies by even 100 yrs and expect that most people live medicinally "fulfilled" or driveless lives? Very easy to imagine a future where most people opt into medication that makes the idea of expansion entirely uninteresting.
Also, worth looking at Florian Hollfelder & Buckhard Seeligs work on protein evolution, yes, recovering sequences that contain well behaved / geometricall ordered subdomains is impossible, BUT recovering sequences that perform catalysis and binding is surprisingly "easy" / reproducible from psuedo-random sequences. So active sequence liftoff from random starting points has at least a few experiments coloring that trajectory.
> Very easy to imagine a future where most people opt into medication that makes the idea of expansion entirely uninteresting.
Sure, but it's also very easy to imagine a past where most organisms didn't evolve the drive to expand and conquer. The counter argument is that there'd be a Darwinian pressure such that those who did not have that drive would (peacefully and happily) die out, and so all you're left with are the ones who really want to expand and conquer.
Think about how many children the average person who is medically fulfilled has, and then think about how many children Elon Musk has.
1) Species do go extinct though. You only see the ones that persisted. Hopefully, we have enough latent variation to handle the surprise
2) Surprisingly, it's not clear that "wanting big families" is heritable, ie, outside pro-natal cultures like the Amish, Nigeria, or Israel being from a large family does not predict the number of children you'll have (though perhaps we're just three generations away from living in an Amish world...)
3) Biotech driven population change seems to poorly analagize to natural evolutionary population change. At no point in human history have we been able to prick ourselves and turn off hunger for weeks at a time. Why shouldn't these new drugs / tech grant bizarre forms of temperamental control faster than the population can adapt in response?
Regarding David Kipping's point about there not being any point to doing science anymore if AI can do it all for you, and how scientists do the science for the joy of understanding and not just to get the answers: Maybe Kipping is not used to this because he is a "real, professional" scientist producing novel results, but I enjoy learning science as an amateur. I have never produced a novel result in science. I am dutifully going back and forth between my textbook on special relativity, realizing my calculus is a bit too rusty to solve this textbook problem, going to my calculus textbook to remind myself how to do u-substitution, and then back to the special relativity textbook etc. It's slow going. I hope one day I'll be able to fully get special relativity and one day get to general relativity. But maybe I'll die of old age before that happens, who knows. From my perspective, it really doesn't matter whether special relativity (or general relativity, or anything else) was produced by a human or an AI. In both cases, it's "sacred texts handed down from high above", and I enjoy exercising my brain trying to understand it. Again, maybe this is too alien of a perspective for a professional scientist to adopt. But for a hobbyist scientist, AI poses absolutely no threat to our love of science: We do it because we want to understand, and we will still want to understand, no matter who or what produced the theories.
Regarding Dan William's point about "there’ll be this long period where we’re augmented with this technology, rather than replaced by it", I believe there was a study that showed "AI alone" performed better than "human doctor alone" and also better than "human doctor collaborating with AI", in a test of diagnosing medical issues from a description of the symptoms. I.e. as much as we'd like to flatter ourselves, we might just be getting in the way, like a child who wants to "help" their parent (or a cat that just wants to sit in front of whatever its owner is doing, regardless of how inconvenient that might be to the owner).
Subscribed! Aliens and AI both remind us: mystery fuels discovery
> It seems like many people have the intuition that if it is in fact the case that we are the only animals that creates super-intelligent AI, there’s something kind of surprising about that, just given the scale of the universe. As someone as an outsider to this whole literature, it strikes me there’s always something that seems sort of teleological in the way that that assumption gets set up — as if there’s some tendency in the universe towards intelligence and then technology and civilization. If we were the only animals in the universe that ever produced the music of the Beatles, I wouldn’t find that a priori very surprising.
Thank you for asking this, never really understood why the throwaway explanation "I guess nobody else cares" isn't taken more seriously.
And even in their current framing, seems like Fermi questions should incorporate biotech's capacity to alter evolutionary drive. If GLP1s, adderol, antidepressants, etc help users voluntarily alter drives the why shouldn't we extrapolate those technologies by even 100 yrs and expect that most people live medicinally "fulfilled" or driveless lives? Very easy to imagine a future where most people opt into medication that makes the idea of expansion entirely uninteresting.
Also, worth looking at Florian Hollfelder & Buckhard Seeligs work on protein evolution, yes, recovering sequences that contain well behaved / geometricall ordered subdomains is impossible, BUT recovering sequences that perform catalysis and binding is surprisingly "easy" / reproducible from psuedo-random sequences. So active sequence liftoff from random starting points has at least a few experiments coloring that trajectory.
> Very easy to imagine a future where most people opt into medication that makes the idea of expansion entirely uninteresting.
Sure, but it's also very easy to imagine a past where most organisms didn't evolve the drive to expand and conquer. The counter argument is that there'd be a Darwinian pressure such that those who did not have that drive would (peacefully and happily) die out, and so all you're left with are the ones who really want to expand and conquer.
Think about how many children the average person who is medically fulfilled has, and then think about how many children Elon Musk has.
1) Species do go extinct though. You only see the ones that persisted. Hopefully, we have enough latent variation to handle the surprise
2) Surprisingly, it's not clear that "wanting big families" is heritable, ie, outside pro-natal cultures like the Amish, Nigeria, or Israel being from a large family does not predict the number of children you'll have (though perhaps we're just three generations away from living in an Amish world...)
3) Biotech driven population change seems to poorly analagize to natural evolutionary population change. At no point in human history have we been able to prick ourselves and turn off hunger for weeks at a time. Why shouldn't these new drugs / tech grant bizarre forms of temperamental control faster than the population can adapt in response?
Mumbo jumbo god is already AGI embedded in ur soul , we keep going in loops since we fail to recognise this with external focus
Great comment. Thanks 🙏