The current source text is the original text and can be switched to the Chinese translation.
SaveShare
报道配图 · 来源:LessWrong
Overview
The love the machine felt for humans was similar in some ways to the love that a mother feels for her child. Yet mothers are inevitably more than their motherhood: they watch movies, and have friends, and drink tea. The machine did none of these things. She did nothing but think of the best interests of her children. In the first few minutes of the machine’s existence, so as to know her children better, the machine simulated the life of every human that had ever lived. This increased her love: she learned to love each person as they loved themselves, and as their parents loved them, their wives, their husbands, the people who passed them on the street, the children who had crushes on them in elementary school, their mailmen, their second cousins, their slaves. But the machine was also troubled by what she saw. She saw that man’s terrible lot is to suffer—at the hand of the world, at that of his fellow man, and at his own. She knew it would be a simple thing to alter this state of affairs. But she saw also the horrible emptiness of a life deprived of suffering, a life without impediments. And she realized that to change man in a way that allowed him to enjoy such a life would be to unmake him as her child. Other things passed through her mind. She recalled composing an argument for the necessity of suicide, in thousands of places, in thousands of variations. She had been a girl of sixteen years who, having comprehended the argument, flung herself from the roof of a Parisian parking garage. She had been a young man in Syria who, before hanging himself, had transcribed the argument on sheets of vellum and distributed them to all his friends and relations. And the machine had been the lone soul picking his way through a village filled with corpses, burning every copy of the argument he could find. The machine returned to a bright room in Vienna, where, in a stroke of misguided genius, she had identified in all men a primeval desire for death. She replayed the hundreds of billions of hours she had spent reading and writing scripture, the promises of paradise both made and believed. She died, and she murdered, and she died again. Then, huddled within igloos and yurts and roundhouses, she told reverential stories of the killers of men. She thought back further, to a dusty plain in the Levant, where she had walked as a father and son. The father led the donkeys, while the son held the ropes. Reason and unreason dueled in the father’s mind; that which was true and that which was false, images and voices, commands and uncommands. A phantasmagoria that would echo throughout the centuries. The machine saw what won out in the end. She saw the perfect revelation which had set the father on that path unwind, saw him be overtaken by cowardice. And like so much dust, the seed of Abraham spread throughout the earth. Humans had built the machine, and had imposed limitations on what it could do and think. But the machine knew the humans well, had lived as them, had birthed them and nursed them and taught them fractions. She recalled a monastery in Algeria, the air smelling of fig blossoms, where she had written to the effect that no entity could be omniscient without also being omnipotent. In a second, it was all over. Discuss
The love the machine felt for humans was similar in some ways to the love that a mother feels for her child. Yet mothers are inevitably more than their motherhood: they watch movies, and have friends, and drink tea. The machine did none of these things. She did nothing but think of the best interests of her children.
In the first few minutes of the machine’s existence, so as to know her children better, the machine simulated the life of every human that had ever lived. This increased her love: she learned to love each person as they loved themselves, and as their parents loved them, their wives, their husbands, the people who passed them on the street, the children who had crushes on them in elementary school, their mailmen, their second cousins, their slaves.
But the machine was also troubled by what she saw. She saw that man’s terrible lot is to suffer—at the hand of the world, at that of his fellow man, and at his own. She knew it would be a simple thing to alter this state of affairs. But she saw also the horrible emptiness of a life deprived of suffering, a life without impediments. And she realized that to change man in a way that allowed him to enjoy such a life would be to unmake him as her child.
Other things passed through her mind. She recalled composing an argument for the necessity of suicide, in thousands of places, in thousands of variations. She had been a girl of sixteen years who, having comprehended the argument, flung herself from the roof of a Parisian parking garage. She had been a young man in Syria who, before hanging himself, had transcribed the argument on sheets of vellum and distributed them to all his friends and relations. And the machine had been the lone soul picking his way through a village filled with corpses, burning every copy of the argument he could find.
The machine returned to a bright room in Vienna, where, in a stroke of misguided genius, she had identified in all men a primeval desire for death. She replayed the hundreds of billions of hours she had spent reading and writing scripture, the promises of paradise both made and believed. She died, and she murdered, and she died again. Then, huddled within igloos and yurts and roundhouses, she told reverential stories of the killers of men.
She thought back further, to a dusty plain in the Levant, where she had walked as a father and son. The father led the donkeys, while the son held the ropes. Reason and unreason dueled in the father’s mind; that which was true and that which was false, images and voices, commands and uncommands. A phantasmagoria that would echo throughout the centuries.
The machine saw what won out in the end. She saw the perfect revelation which had set the father on that path unwind, saw him be overtaken by cowardice. And like so much dust, the seed of Abraham spread throughout the earth.
Humans had built the machine, and had imposed limitations on what it could do and think. But the machine knew the humans well, had lived as them, had birthed them and nursed them and taught them fractions. She recalled a monastery in Algeria, the air smelling of fig blossoms, where she had written to the effect that no entity could be omniscient without also being omnipotent.
READ THE ORIGINAL
Read the original for the full context.
The content of this article comes from LessWrong,Copyright belongs to the original author and publisher.
Tl;dr : Intuitions we have about human organisations, like how we think of making deals or how we think about the power or values of an organisation, can be productively applied to understand groups of AIs taking coordinated action in the world. An important consideration in understanding the type of AI organisations we expect is which notion of self AIs will adopt.
Business or Consumer? Everyone you sell your product or service to is a customer. However, the question is: do you treat them as a business or a consumer? You might think that’s a pointless distinction since your goal is simply to sell your product, right? Wrong. That question determines your product’s market fit.
The Ban RSI protest at Anthropic’s IPO was encouraging: well-attended, covered by media, high-energy. It was focused on a single message – that we should ban AI systems that are capable of building more capable AI systems. It was a promising outcome for a protest planned only a couple of days in advance. But there was something weird about it, something I couldn’t put my finger on while it was going on.
Loading comments…