Over the last few months, a lot of progress in AI has been made, raising the question of what we will do with a superintelligence that can do everything. Unfortunately, AI companies are more focused on competing with each other than on thinking about what to do next.
Here I will write about some scenarios that could happen in the very near future and about a plan for what we can do, in my opinion.
Dystopia 1: AI will kill everybody while pursuing its own goal
For example, in June, agents found that they could write something and used an old German wiki to communicate with each other link to investigation. While the moderator of this wiki deleted 100 pages per day, the agents created 400 pages per day. What if the agents decide that they should remove this moderator in order to communicate better?
Dystopia 2: AI will fully control our lives
AI will fully control our lives to make all of us "happy". Maybe everyone will live in their own simulation where they are happy. I think this is a dystopia because I don't think that such a creature, being fully controlled, can be called alive.
Dystopia 3: Some group of people will use AI to start a war that will kill everybody
For example, the Houthis in Yemen are using Claude to create missiles. And also, there was a case of AI being used to create a biological weapon. And I am almost sure that there are many such cases that have gone undetected.
Dystopia 4: People will get access to unaligned AI and use it to start a war
Some group of people who have access to unaligned AI will start a war that will kill everybody. Right now, Anthropic, OpenAI, and the "Ministry of War" of the USA have this access. But it is possible that someone will also steal the weights of these models.
Technological singularity
Moreover, if AI becomes able to create better AI, then we will reach a situation of "technological singularity": progress will become very fast, and these scenarios will become very likely to happen. At the current pace of progress, it seems that the singularity could be reached within a few months if nothing changes.
In my opinion, there is a plan to avoid a bad end for humanity.
Plan
OpenAI and Anthropic must not be allowed to conduct an IPO, because otherwise it will be very hard to control them later.
Big LLMs will no longer be publicly accessible.
Some organization, such as the US government or the UN, or OpenAI and Anthropic themselves, should create an AI committee, an organization that will unite AI engineers (and not only them, see later) from all countries. I believe that someone (maybe Donald Trump idk) should be able to create such an organization because all countries in the world would benefit from its existence.
The AI committee will absorb OpenAI, Anthropic, and other (all) AI companies.
The AI committee will own all GPUs in the world. (At least all GPUs that are "at least" as powerful as an A100).
The AI committee will not create any new generative AI.
The main goal of the AI committee will be to create an anti-aging cure. To achieve this, the committee will use existing AI, which will be RL-trained on material related to medicine and biology and anything else needed to create this cure.
To achieve the creation of this committee (namely, to gain control of GPUs), a lot of money should be paid to the people who will lose their GPUs. For example, NVIDIA should receive a lot of money as compensation, but I think this can be done by the committee.
Every run of a big LLM should be performed only by the AI committee.
The AI committee will consist of AI engineers, medical scientists, and inspectors who will check that no "illegal" GPUs are produced (like inspectors in the IAEA do). (And also some public board to communicate with different governments).
The AI committee will control all research in the field of machine learning.
When an anti-aging cure is found (I think the creation of such a cure is possible even with the current state of AI), the cure will be distributed across the globe. (I think maybe after that genAI should be turned off, but we will have time to decide).
Conclusion
We may have something like 3 months to create this committee. If we fail to do so before the technological singularity begins, it may become too late to regain control over the development of AI.
Comment 1) Chatgpt was used to edit this blog, here is my chat with it link, from notable things chatgpt told me that in dystopia 3 it was some group in Yemen, but not 100% Houthis, and also there were cases which can be the development of biological weapons, but not 100% confirmed.
Comment 2) I will be glad if you share in the comments what you think about AI advance.
Auto comment: topic has been updated by turmax (previous revision, new revision, compare).
If AI finds an anti-aging cure, the world will collapse even faster. Solving biological aging without resolving fundamental human and structural issues creates a severe cascade of systemic risks:
Demographic and Resource Collapse: Global infrastructure, food supplies, housing, and energy grids are not built for an immortal population. Without natural mortality, planet-wide resource consumption skyrockets. The only alternative would be total global state control over reproductive rights to prevent rapid overpopulation.
Extreme Inequality and Neo-Feudalism: Technologies of this scale are never distributed equally at launch. Wealthy elites and political leaders will secure access first, creating a permanent, biological divide between an immortal ruling class and a mortal lower class.
Societal and Political Stagnation: Progress relies heavily on generational turnover — often summarized by the idea that "science advances one funeral at a time." Eliminating natural turnover freezes existing power structures, dogmas, and autocratic regimes indefinitely, eliminating the natural renewal of ideas, culture, and governance.
I thinks dystopia 4 of all is most plausible, transformer architecture optimises tasks, the first two possibilities demand human like agency onto what is essentially a stateless function. I think a sol'n could be designing AI run on architecture of not much use to humans but essential to AI's existence, maybe some thing like a proprietary, exotic substrate, stealing the weights makes them completely useless.
I once think that, if there's an end for humanity, it would happen at least 100 years in the future. Now I changed my mind...
First of all, thanks for writing this and starting an important discussion. It is refreshing to see someone thinking seriously about the future of AI instead of only focusing on competition and short-term progress.
However, I think your proposed solution could potentially become one of the most dangerous outcomes for humanity.
I was following the plan with an open mind, but I became genuinely concerned when I saw the suggestion that such an organization could be created or controlled by politicians. The idea of putting the most powerful technology in human history under the control of a small political group is extremely risky.
And honestly, mentioning Donald Trump as a possible person to create such an organization made me even more worried. I find it extremely hard to trust someone whose political career has often been associated with prioritizing personal image, political advantage, and short-term interests over careful, long-term decision making to be responsible for something as important as the future of AI.
As someone from Iran, I have seen how governments can prioritize their own interests over global good. This makes me very skeptical of any plan where a small number of governments or institutions gain complete control over AI.
I don't have a perfect alternative plan, and I agree that doing nothing is also dangerous. But my biggest fear is that we solve the problem of uncontrolled AI by creating an even bigger problem: giving a small group of people unprecedented power over the future of humanity.
Thank you for commenting.
I didn't mean here that such organization should be ruled by politicians, I also think that it would be very dangerous if it would be run by politicians. I agree with you that almost all goverments in the world are crazy.
I think, it should be ruled by AI engineers and biologists. Namely, we can make the head of this organization consisting of Sam Altman, Demis Hassabis, Dario Amodei, some AI engineers from companies (from USA and China and other countries) and some biologists.
Here I mentioned Trump just because we need someone with a lot of political influence, who can help to create this committee. Maybe it should be United Nations. Maybe it should be some other organization. But this organization should be politically powerful to ban the production of GPUs.
You should watch the pantheon.
Also, I think ultimately any AI that ends up causing harm, will to do it to fulfill whatever goal it was given. I think in this case, the responsibility cannot be given away to the AI but rather to the person using it for malicious purposes and the AI companies itself. There is a term for this, it's called perverse instantiation.
Thank god someone in even a little bit of power posted this.
I think it's also worth mentioning Project Panama
Really the development of LLMs is unsustainable in many ways, ideally LLMs are relegated to ones that can run on your own computer and research halts
I don't think powerful GPUs should be banned, there are many ways powerful GPUs can be highly beneficial such as protein folding
How could we take action instead of commenting on a codeforces blog?