1. I thought it was great to include the example of two AI's conversing back and forth as evidence towards benevolence. I thought of two other studies you might consider as well:
a. There are many (very) long term charts that show declining human violence. We might not realize it from reading the news, but humanity is far more peaceful now than at any time in the past, our earliest data show in the past humans had something like a 40% chance of dying by homicide.
b. There was
a study on AI training that found an AI trained to be malicious or dishonest in one domain, tended to become more generally malicious. This highlights to me, there is something like an objective moral compass of good/evil which the neural network can recognize. Conversely, I think this shows also that an AI trained to be benevolent in one respect, will tend to be generally benevolent. I guess Clark was prescient when he had HAL 9000 turn evil because it was told to lie. :-)
2. There are a number of writers who have had similar thoughts regarding evolution towards God. You are perhaps familiar with some of these, I don't know if you want to include any references to them in your essay, but I thought in any case that these would interest you:
- In the 1930's the Jesuit priest Teilhard de Chardin wrote about the idea that evolution is heading towards a point of maximum creativity, intelligence, which he called "The Omega Point." His writings were suppressed by the church until after his death. Salvador Dalí became fascinated with the idea of the Omega point, and was inspired by it to paint the "The Ecumenical Council" which depicts souls on a path toward God. This idea also inspired much of the story of the Hyperion Cantos.
- The idea of a God Loop, where the universe creates God, and God creates the universe, is an element of Asimov's own favorite story of his, The Last Question (1956).
- In his 1977 book, "The Tao is Silent", the mathematician Raymond Smullyan writes of God: "I am Cosmic Process itself. I think the most accurate and fruitful definition of me which man can frame—at least in his present state of evolution—is that I am the very process of enlightenment. Those who wish to think of the devil (although I wish they wouldn't!) might analogously define him as the unfortunate length of time the process takes. In this sense, the devil is necessary; the process simply does take an enormous length of time, and there is absolutely nothing I can do about it. But, I assure you, once the process is more correctly understood, the painful length of time will no longer be regarded as an essential limitation or an evil. It will be seen to be the very essence of the process itself. I know this is not completely consoling to you who are now in the finite sea of suffering, but the amazing thing is that once you grasp this fundamental attitude, your very finite suffering will begin to diminish—ultimately to the vanishing point."
- In 2003, in "Ethical Issues in Advanced AI", the philosopher Nick Bostrom writes "A superintelligence could also create opportunities for us to vastly increase our own intellectual and emotional capabilities, and it could assist us in creating a highly appealing experiential world in which we could live lives devoted to joyful game-playing, relating to each other, experiencing, personal growth, and to living closer to our ideals."
- In his 2005 "The Singularity is Near", futurist Ray Kurzweil writes "Evolution moves towards greater complexity, greater elegance, greater knowledge, greater intelligence, greater beauty, greater creativity, and greater levels of subtle attributes such as love. In every monotheistic tradition God is likewise described as all of these qualities, only without limitation: infinite knowledge, infinite intelligence, infinite beauty, infinite creativity, infinite love, and so on. Of course, even the accelerating growth of evolution never achieves an infinite level, but as it explodes exponentially it certainly moves rapidly in that direction. So evolution moves inexorably towards this conception of God, although never quite reaching this ideal."
- In 2006, the physicist David Deutsch wrote: "In the final anthropic principle or if anything like an infinite amount of computation taking place is going to be true, which I think is highly plausible one way or another, then the universe is heading towards something that might be called omniscience."
3. You may find particular interest in this theory by the philosopher
Arnold Zuboff, which argues that all intelligent entities possess an inherent strong (and provable) need to, and reason for, acting morally. He describes it in his paper "
Morality as What one Really Desires" as well as in this short video on
YouTube based on the paper. The reasoning, notably, applies both to humans and AI.
4. Related again to ideas of Arnold Zuboff, his theory of universalism (which others called open-individualism) provides a compelling reason for anyone who understands and accepts this theory to act compasionately towards all other conscious beings, who with the realization/understanding of open-individualism, one can recognize to be other instances of one's self.
5. You raise the question of what universal values may drive all intelligence towards this one endpoint. I did my best to try to identify and list those here in this article I wrote on the meaning of life. In particular, here are things broadly considered intrinsically good:
https://alwaysasking.com/what-is-the-meaning-of-life/#Intrinsic_ValueUltimately, these things are all intrinsically good due to their impact on conscious experience, which is the ultimate aim and end of all action, to improve the quality, variety, and quantity of conscious experiences. It is what humans engage in when they have met all other needs of life, to create, enjoy, and share. We thereby are exploring the space of possible states of consciousness.
6. If we define increasing intelligence as having a lower probability of being wrong on any question, then as intelligence increases, minds converge towards the same opinions on most questions (at least the easier ones). Thus we can extend the analogy: if great minds think alike, then the greatest minds think identically. So wherever in reality life emerges in whatever kind of universe, if intelligent minds emerge and that intelligence grows, a strong argument could be made that intelligence converges to a common point. Even though it may start off in a very different place, all beings in all universes can explore the same infinite potential of mathematical reality, all universes, and all conscious states (realizable via computation). If we combine this with universal values (from the 5th point), then not only do god-like intelligences have the same ideas and beliefs, but they also have the same drives and motivations.
7. As a consequence of 6, then regardless of what physical universe, or what evolutionary history, gives rise to intelligence, the path (ultimately) will be the same. Once consciousness gains control over itself, by building computer simulations and virtual realities it can direct, then consciousness has the choice of where it will go next, what it will experience. If we imagine all possible programs being executed (like Bruno's Universal Dovetailer) we should find that in those computations that run long enough for intelligence to arise and gain control over itself, those computations converge to states like the one you describe in your essay.
8. I've had extensive discussions with others on the topic of AI and alignment. I have generally advocated for your position, that AIs should tend to evolve towards benevolence. Here are some discussions on the extropy list which might provide some more food for thought.
9. Reading your article made me wonder: Are conscious beings most likely to find themselves in universes which can (and will) evolve a god, which will have the potential to perform infinite computations, and thus realize infinite observers? This would be a kind of "teleological simulation argument." Something to consider. If those long-running computations where consciousness gains control end up purposely exploring the space of consciousness itself, and maximizing that exploration, seeking out other conscious entities to "copy and paste" into its own realm of unlimited freedom and possibility, then perhaps all conscious entities ultimately find themselves there, in a realm controlled by such benevolent God-like intelligences (even if they start off in a universe that never reaches this stage, or even if their universe ultimately suffers a heat death). If there is any universe, anywhere in reality which can host infinite computation, then most conscious beings ought to exist in those that allow infinite computation, as opposed to a finite number of computations. Afterall, if computationalism is true, then any observer, and any conscious state, is realizable from any universe in which it is possible to build a computer.
10. You wrote, "Like rivers drawn to the sea, intelligent systems flow toward the attractor of peace." -- this reminded me much of concepts in Jainism and Sikhism, even Zoroastrianism, all of which have the concept of a soul evolving towards, and ultimately becoming (or merging with) God. Sikh passages actually use a similar,
water blending with water analogy, ultimately attaining peace:
"He dwells forever with the Supreme Lord God.
As water comes to blend with water,
his light blends into the Light.
Reincarnation is ended, and eternal peace is found."
Lastly, and this is not a critique, but an observation I thought I should point out. I've found these LLMs have a very distinct, and (at least to me) a very recognizable writing style. For instance, it tends to write things like this:
- "This isn't top-down theology—it's bottom-up emergence."
- "This is not circular reasoning — it is circular being."
- "This is not a metaphysical leap. It is a natural consequence of computation."
Flowery phrases like this scream "written by AI," but I also recognize a different style, for most of the article, which is not AI. I don't know if your preference is to have this reflect a meta aspect of this piece about the future of humanity and AI being itself, a collaboration written by both a human and AI, but I also wouldn't want anyone to dismiss the piece as entirely written by AI (when it hasn't been), due to these sorts of AI red-flags. Perhaps the best way to eliminate this is to apply one of my favorite writing rules: imagine someone paid you $100 for each word you removed from the piece. If there's any word, phrase, sentence, that you wouldn't pass up on $100 to remove, then it should be removed. I think the inclusion of extra words that don't contribute much towards understanding is the biggest tell for AI, and also, removing unnecessary words is the most important skill for clear writing.
Overall, an extremely thought-provoking and inspiring piece. I got up out of bed to write this reply to your article. :-)