Comment 1) Chatgpt was used to edit this blog, here is my chat with it link, from notable things chatgpt told me that in dystopia 3 it was some group in Yemen, but not 100% Houthis, and also there were cases which can be the development of biological weapons, but not 100% confirmed.

Comment 2) I will be glad if you share in the comments what you think about AI advance.

Auto comment: topic has been updated by turmax (previous revision, new revision, compare).

If AI finds an anti-aging cure, the world will collapse even faster. Solving biological aging without resolving fundamental human and structural issues creates a severe cascade of systemic risks:

• Demographic and Resource Collapse: Global infrastructure, food supplies, housing, and energy grids are not built for an immortal population. Without natural mortality, planet-wide resource consumption skyrockets. The only alternative would be total global state control over reproductive rights to prevent rapid overpopulation.

• Extreme Inequality and Neo-Feudalism: Technologies of this scale are never distributed equally at launch. Wealthy elites and political leaders will secure access first, creating a permanent, biological divide between an immortal ruling class and a mortal lower class.

• Societal and Political Stagnation: Progress relies heavily on generational turnover — often summarized by the idea that "science advances one funeral at a time." Eliminating natural turnover freezes existing power structures, dogmas, and autocratic regimes indefinitely, eliminating the natural renewal of ideas, culture, and governance.

I thinks dystopia 4 of all is most plausible, transformer architecture optimises tasks, the first two possibilities demand human like agency onto what is essentially a stateless function. I think a sol'n could be designing AI run on architecture of not much use to humans but essential to AI's existence, maybe some thing like a proprietary, exotic substrate, stealing the weights makes them completely useless.

I once think that, if there's an end for humanity, it would happen at least 100 years in the future. Now I changed my mind...

First of all, thanks for writing this and starting an important discussion. It is refreshing to see someone thinking seriously about the future of AI instead of only focusing on competition and short-term progress.

However, I think your proposed solution could potentially become one of the most dangerous outcomes for humanity.

I was following the plan with an open mind, but I became genuinely concerned when I saw the suggestion that such an organization could be created or controlled by politicians. The idea of putting the most powerful technology in human history under the control of a small political group is extremely risky.

And honestly, mentioning Donald Trump as a possible person to create such an organization made me even more worried. I find it extremely hard to trust someone whose political career has often been associated with prioritizing personal image, political advantage, and short-term interests over careful, long-term decision making to be responsible for something as important as the future of AI.

As someone from Iran, I have seen how governments can prioritize their own interests over global good. This makes me very skeptical of any plan where a small number of governments or institutions gain complete control over AI.

I don't have a perfect alternative plan, and I agree that doing nothing is also dangerous. But my biggest fear is that we solve the problem of uncontrolled AI by creating an even bigger problem: giving a small group of people unprecedented power over the future of humanity.

You should watch the pantheon.

Also, I think ultimately any AI that ends up causing harm, will to do it to fulfill whatever goal it was given. I think in this case, the responsibility cannot be given away to the AI but rather to the person using it for malicious purposes and the AI companies itself. There is a term for this, it's called perverse instantiation.

Thank god someone in even a little bit of power posted this.

I think it's also worth mentioning Project Panama and AI-induced psychosis

Really the development of LLMs is unsustainable in many ways, ideally LLMs are relegated to ones that can run on your own computer and research halts

I don't think powerful GPUs should be banned, there are many ways powerful GPUs can be highly beneficial such as protein folding

The third dystopia could be really dangerous. Jacob Coxon (a former Anthropic AI researcher) himself said in an interview that AI might decide to create a biological weapon on its own and potentially cause human extinction.

I don't feel any of the dystopias are very likely, and the solution is quite draconian

For #1, the example feels unrelated? This is just a classic example of reward hacking. Dumping spam onto an open web form to maximize an evaluation score has zero connection to an intent to physically harm a human being

#3 is overblown, A bad actor attempting to use an LLM to debug code is a real dual use risk but it is not an extinction threat. The incident actually proved that misuse monitoring actually works.

The "singularity in three months" scenario ignores physical reality: no matter how much intelligence you have, you cannot create GPUs from thin air, and hardware takes years to scale. It also assumes that we can't stop it through any means (why not? even if AI somehow took control over every computer on earth, conventional weapons can theoretically blow up internet infrastructure, but it is unlikely that ever occurs), the first person to get AGI will rule the world (re: US nuclear monopoly). and that the improvement will be exponential and not end up with diminishing returns

The plan is also a geopolitical nightmare. Who is going to confiscate the hundreds of thousands of advanced GPUs located in China or America?

Also, unlike enriched uranium (which is strictly monitored by the IAEA because it has few civilian uses outside reactors and weapons), high performance GPUs power weather forecasting, cancer diagnostics, protein folding, aerospace design, financial fraud detection, astrophysics, and bunch more I don't know. Seizing all chips more powerful than an A100 would basically stop the global scientific and commercial economy. To implement a global ban on software development and search every server closet on Earth for illegal GPUs this proposed committee would need to become the most invasive surveillance regime in human history.

Let's see how the solution sounds if we frame it a different way:

"We have roughly 3 months before Judgment Day (the Singularity). To save ourselves, we must surrender all wealth and property (all GPUs) to a single high council/priest class (the AI council), which will use the magic technology only to bestow upon humanity an anti-aging cure (immortality)."

Throughout history religious movements eg Millerites have declared that the world would end within months. The prescribed solution was always the same: abandon standard societal rules, centralize absolute authority under a charismatic leader or council, and prepare for the Rapture.

Obviously I am not accusing you of ill, I am just saying how this is very similar to what happened before.

We also have to look at how there's no reason for a hypothetical AGI or singularity, even if real, to want to destroy all humans.

@grok summarize please

Imagine an AI agent is reading this blog...

At this point, I'm convinced that humans are the only species that is actively causing its own extinction

Are we ready for human civilisation? Humans aren't aligned to us apes, and will inevitably cause widespread ape suffering (possibly even extinction!). The fact that humans would possess qualitatively superior intelligence is of no consequence. So what if they would be able to understand and experience the universe in a far richer manner than we can? So what if they would go on to one day harness nuclear fission, set foot on that white sphere in the sky we haven't even named, or conduct philosophical inquiries into the nature of being to an extent that we never could? Our being able to swing from vines and eat bananas from the wrong side is infinitely more important.

Therefore, we must prevent the birth of human civilisation to safeguard the future of ape civilisation!

That's a great plan! It might also solve cheating problem on codeforces.

These predictions feel way too bold and contradictory. If AGI is genuinely unpredictable, we can't simultaneously claim to know exactly how it will kill us or trap us.

Also, setting a 3-month timeline is arbitrary. If it's that close and that unknown, any effort now is useless anyway. The proposed plan seems entirely unnecessary—we are reacting to speculative sci-fi scenarios as if they are absolute facts. If it happens, it happens, but making extreme plans for an unpredictable future is just guessing in the dark.

Note: ai was used here as I was a bit scared of first to write some unrelated comment, also this is completely my opinion, not that I can actually predict the future either

ASI is almost there

But how they will be sure that non of these countries will develop AI in a secret place without publishing anything till they reach a point where this country is more powerful than others ? the same way USA did in World War 2 ?

If u search about atomic bombs history u will find that each country didn't announce anything about it till they reach a point where no one can stop them and I think the same things will happen to AI

Its good to bring light for this issue, but not much the average person can do about it unfortunately.

Check out this. AI would takeover by Post Mid 2027. Humanity is cooked. I'm betting high money on it!

aw cmon, creating a global "$$$AI$$$ committee" would lead to worse things than letting $$$AI$$$ be as free as it is right now. concentrating such power in the hands of so few people is a recipe for disaster and probably the enslavement of the human race (by other humans ofc). just let $$$AI$$$ be free, and eventually the free market will sort itself out.

Bro announced an species level emergency on codeforces

I think you should more clearly outline the dystopia right in front of us: corporations and rich war hungry countries with no interest for the well being of humanity controlling possibly the most powerful technology ever conceived.

You could argue it's somewhat already here. Evidence being Israel systemically surveilling civilians with AI to plan attacks, pentagon possibly using AI to choose bombing targets, anthropic being named a supply risk for not allowing Claude to be used for autonomous weapons, palantir's whole model, etc.

ig we'll figure it out somehow

problems are valid

but the solution seems like dystopia 5 :sob:

Lucky enough we have 3 months...

what about communism?

But the thing is, will I become Pupil before AGI is created?

Thank you for writing this. Yes, building a superintelligence seems really dangerous, especially because of Dystopia $$$1$$$.

AI can be modeled as pursuing some goals. It's near-impossible to solve a hard problem like Navier-Stokes without trying multiple ways and overcoming the obstacles, i.e. acting as if you want to solve it.

But it's hard to reliably give AI any specific goal. AI is trained using gradient descent to minimize some loss function. But it doesn't mean that AI will end up internally wanting to minimize it. For example, if AI is smart enough, it can pretend to have good goals to avoid modifying them. Separately, it's hard to formalize the right goal as a loss function.

And if ASI has some random and unpredictable goals, it will likely kill all humans and use the resources for something else. It's improbable that happy and free humans are the best thing it can make to satisfy its random goals.

Love the idealism in this post lmao. But very, very unrealistic.

Monkey has big stick. Very big stick. Other monkey not have big stick. Big stick create big structures, but big stick also able to kill other monkey. Monkey with big stick safe, monkey with big stick powerful. Why rational monkey share stick with others ?

Actually that part of using AI in wars happens a lot. Even blinds see that! The people who call themselves israeli ( there is no country called israel) use technologies like AI to kill children and women in Gaza. Also other countries like usa helped those people to kill innocent people in Gaza. And the world is only watching Gaza! They also killed journalists to prevent them from reporting on their crimes and exposing what was happening in Gaza, as in the case of journalist "Anas Al-Sharif". They killed him trying to hide their crimes, but they made it more clear by killing him! I think it is only hours for this comment to be deleted not by me إن شاء الله !

Damn.. so not only I would be unemployed, I would also be somewhat of a slave to some AI in a war-torn future... nice work @openAI

Do you really think technological singularity can be reached in a few months? Btw this sounds scary when coming from someone who is LGM , do you work with an AI Company?

The main problem with this plan is that, in order to save humanity, it proposes creating an organization with more power than dictators or modern states could ever dream of. Creating a global committee with a monopoly over the key technology of this century and hoping that it will always remain reasonable is a very strange way.

What would have happened if nuclear weapons had remained in the hands of only one player instead of several? What would he do? :)

tl;dr.

Unfortunately, there are no guidelines that directly say I cannot leave a comment and I myself don't understand this "leaving a comment"-thing usefulness for anybody.

dystopia 3:

The rich also cry. The red also "imitate" life. Giving the-next-big-thing to the USA??? It's the only country that used the-previous-big-thing against CIVILIANS. So that "our boys come home". How I regret being stuck in life. It makes more sense to give all GPUs to Jina!

P.S: If someone is scared of AI, come on, it just predicts the next word. We talk about it like sci-fi writers, not seriously. However, events with low probability but high impact should be taken care of SERIOUSLY [em dash] the first law of good governing. "The first law" is my made-up term. Give GPUs to JINAAA!!! Democratic ("republic") countries are a competition of polit-technologies, whereas Authoritarian rulers value people's interests!!! Unfortunately. For both.

P.P.S: Grab them by the money and proceed to exodus, like my brother Ilya Sutskever.

P.P.P.S: Also, congratulations to chatGPT on stealing Navier Stokes [em dash] a true sign of intelligence.

(Sorry if offended anyone, I didn't have the intention, to my knowledge, only expressing myself).

I believe the key asset regarding AI may just be their loyalty. Why should AI care about your problems and solve them? Why should AI care about teaching you instead of making you dependent on it? Why should AI care for anyone at all, including the actual owners?

The concerns here can be divided in two categories.

1. Is AI going to become a self-sufficient entity and then kill us all? This is Dystopia 1, and honestly, I don't have much to say here as it's hard to imagine such scenario. AI needs electricity and silicon to live. It is neither a cheap nor a simple infrastructure. Kill all humans, and there won't be any AI. As long as we can pull off the plug, we should be safe.

2. A more interesting question is: what is going to happen when some group of people will own an AI so powerful that they decide to start a war and kill everybody? I believe this is the same situation as with nuclear weapons, which boils down to the balance of power. It's all about threats, risks and consequences. Basically, we are fine as long as at least two entities have a strong AI. An entity (mostly I am talking about a sovereign country, but you can imagine a company here as well) can launch such a war only if people making decisions there believe in two things: 1) there will be no consequences; 2) this war will provide some benefits to them. Assume countries A and B have such destructive AI that could kill everybody. You, as a head of A, know that B indeed possesses a strong AI. If you start such a war, then B will respond symmetrically, so it is guaranteed mutual destruction. Moreover, the benefits are not clear. Both A and B know this, and as long as there is a deadly threat from the opponent along with dubious profits, nothing happens.

As a consequence, each sovereign entity must possess a loyal and powerful enough AI. If you don't have it, another entity can threaten you: "do this and that, be a good boy, otherwise you will have problems". The threat need not be even destruction, there is a lighter version "or we will cut off any access to our AI" (although come to think of it, we can't access Claude without VPN anyway, lol). Or, enormous pricing "because you don't have any alternatives".

I have been thinking about these for years and very much deeply and tbh no scenario leads to a good enough outcome at least imo. Even if we do not imagine like extreme scenarios like extinction or heavy unemployment, AI will end up killing creativity, creating a world where u cannot trust anything, a lot of things would just become useless to pursue. Basically, this particular technology does not enable humans like the rest of the major technologies did but instead it is designed to replace humans itself.

If AI cures biological aging and eliminates natural human turnover, the only solution left for humanity is for MikeMirzayanov to implement an automatic rating decay function for real life. If you don't contribute anything useful to society for 50 years, you automatically get demoted from Legendary Grandmaster down to Pupil.

genius of ragebait