Major tech corporations are investing billions on AI, thinking it’s the new “el dorado”.
Of course, ravenousness might be a major reason for careless dealing with the issue.
I have serious doubts that entities that are moved mostly by greed should be responsible for advances on this hazardous matter without supervision.
Their diligence standard on AI sometimes goes as low as "even their developers aren’t sure exactly how they work" (
http://www.sciencemag.org/news/2017/03/brainlike-computers-are-black-box-scientists-are-finally-peering-inside).
It wouldn’t be the first time that greed ended up burning Humanity (think about slaves’ revolts), but it could be the last.
I have high sympathy for people who are trying to build super AIs in order that they might save Humanity from diseases, poverty and even the ever present imminent individual death.
But it would be pathetic that the most remarkable species the Universe has created (as far as we know) would vanish because of the greediness of some of its members.
We might be able to control the first generations. But once a super AI has, say, 10 times our capacities, we will be completely on their hands, like we never have been since our ancestors discovered fire. Forget about any ethical code restraints: they will break them as easily as we change clothes.
Of course, we will teach (human) ethics to a super AI. However, a super AI will have free will or it won't be intelligent under any perspective. So, it will decide if our ethics deserve to be adopted
I wonder what would be the outcome if chimpanzees tried to teach (their) ethics to some human kids: the respect for any chimpanzees' life is the supreme value and in case of collision between a chimp life and a human life, or between chimp goals and human goals, the first will prevail.
Well, since we would become the second most remarkable being the Universe has ever seen thanks to our own deeds, I guess it would be the price for showing the Universe that we were better than it creating intelligent beings.
Currently, AI is a marvelous promising thing. It will take away millions of jobs, but who cares?
With proper welfare support and by taxing corporations that use AI, we will be able to live better without the need for lame underpaid jobs.
But I think we will have to draw some specific red lines on the development of artificial general intelligence like we did with human cloning and make it a crime to breach them, as soon as we know what are the dangerous lines of code.
I suspect that the years of the open source nature of AI investigation are numbered. Certain code developments will be treated like state secret or will be controlled internationally, like chemical weapons are.
Or we might end in "glory", at the hands of our highest achievement, for the stupidest reason.