I agree with the general thrust of much of this piece (and was shocked to read that he essentially classified these types of things as WMDs, just as I did for mass hacks of autonomous systems) but I will disagree with one main point.
I see no reason that humanity is worth saving over some other, better form of life. I'm not saying its ethical to mow down innocents to make way for the next generation, but some humans are pretty fucking awful. If our descendants are robots with the intelligence of Hawking and are full of kindness and grace then why should we insist that humanity exist forever? Even if we extinguished some of our evil impulses, like genocide and rape, what makes us better? My late aunt had Down syndrome but if I could invent a vaccine that stopped it from ever happening again I would even if I never harm someone with Down syndrome.
The best counterargument I can come up with is essentially the robots asking "What else is there to do?"
In some post-singularity paradise the robots can be heading in all directions at near the speed of light, but once they've figured out this universe's physics and ended armed conflict then it stands to reason that they'd end up creating or simulating places that did not have god-like creatures.
I think we might depend on the resource constraints and conflict to give our lives meaning. Kinda a riff on Eden's "knowledge of good and evil" a creature's intelligence and wisdom may depend on a universe of consequence.
"I see no reason that humanity is worth saving over some other, better form of life. I'm not saying its ethical to mow down innocents to make way for the next generation, but some humans are pretty fucking awful. If our descendants are robots with the intelligence of Hawking and are full of kindness and grace then why should we insist that humanity exist forever?"
If any beings exterminated humanity or through inaction allowed it to go extinct, I'd seriously question their kindness.
Truly kind beings would bring humanity along and help us to be less destructive.
Depends on where their loyalty of kindness lies. Were its kindness 'programmed' to be the preservation of the most life at the expense of the least, then human extermination would be the lowest of the low-hanging fruit.
Keep breeding stock, but then you get into adjudication of value, and at that point all I can think of is Dr. Manhattan saying "the world's smartest human is no more threat to me than the world's smartest termite".
I see no reason that humanity is worth saving over some other, better form of life. I'm not saying its ethical to mow down innocents to make way for the next generation, but some humans are pretty fucking awful. If our descendants are robots with the intelligence of Hawking and are full of kindness and grace then why should we insist that humanity exist forever? Even if we extinguished some of our evil impulses, like genocide and rape, what makes us better? My late aunt had Down syndrome but if I could invent a vaccine that stopped it from ever happening again I would even if I never harm someone with Down syndrome.
The best counterargument I can come up with is essentially the robots asking "What else is there to do?"
In some post-singularity paradise the robots can be heading in all directions at near the speed of light, but once they've figured out this universe's physics and ended armed conflict then it stands to reason that they'd end up creating or simulating places that did not have god-like creatures.
I think we might depend on the resource constraints and conflict to give our lives meaning. Kinda a riff on Eden's "knowledge of good and evil" a creature's intelligence and wisdom may depend on a universe of consequence.