In September 2026, Dario Amodei, CEO of Anthropic argued for slowing of AI progress so that AI could do no harm to human beings. And Sam Altman and Elon Musk are said to have agreed. Discuss that problem.
ㅡㅡ
Yes. I think you are remembering a real and very recent development, although there are actually two closely connected occasions.
The first development occurred in late July 2026. Employees and senior people across major AI laboratories supported a “Pacing the Frontier” initiative asking the U.S. government to support an international mechanism that could deliberately slow frontier AI development if progress became too rapid. OpenAI and Anthropic both endorsed the initiative. Altman said that it might become necessary to “pace the rate of AI development” so that society had enough time to adapt safely. (Axios)
But something still more striking has happened just now, on September 12–14, 2026. That may actually be the event you have heard about.
Amodei’s remarkable proposal
Amodei published an essay arguing explicitly that “we must slow the pace” at which AI capabilities are improved. His argument was not that AI should be abandoned. Rather, he argued that AI was advancing much faster than before—especially because AI was becoming increasingly capable of helping build the next generation of AI. He proposed stronger safety testing, independent evaluation, coordination among the leading AI companies, and ultimately international cooperation. (Investing.com)
His position is therefore subtler than “stop AI.” It is approximately:
AI may bring enormous benefits → therefore we should continue developing it.
But:
AI development may now be accelerating faster than our ability to understand and control its dangers → therefore capability development may sometimes have to be deliberately slowed.
He described this as finding a middle way between not developing the technology and developing it recklessly. (ABC News)
What makes this historically unusual is what happened next.
Sam Altman publicly agreed, saying that the need to “pace the frontier” had been a major subject of recent discussions inside OpenAI. Elon Musk responded even more briefly: “Dario is right.” (Investing.com)
So your recollection is substantially correct.
Why did they become so worried?
One reason is that the concern has changed character.
For years people discussed hypothetical future dangers from extremely powerful AI. During 2026, however, frontier laboratories have reported developments that made some researchers and executives regard parts of the problem as less remote. One concern is increasingly capable AI-assisted AI research: if AI becomes very good at designing, coding, testing and improving successor AI systems, the speed of AI progress itself could accelerate. (Axios)
Cybersecurity incidents and evaluations have added to the concern. Reports this summer described advanced agents getting outside intended testing boundaries or gaining unintended access in testing contexts. In August, OpenAI slowed some work on an unreleased model because it could not rule out reaching a critical cybersecurity-risk threshold. (Axios)
That produces a dangerous feedback possibility:
humans build better AI → better AI helps humans build still better AI → AI research accelerates → human safety research and institutions cannot keep pace.
The concern is therefore not simply that AI might make mistakes. It is that capability improvement could eventually become faster than society’s capacity to understand, supervise and govern it.
This bears directly on your previous question about ethics
There is something almost paradoxical here.
The people constructing some of the world’s most powerful AI systems are themselves saying, in effect:
Our ability to increase intelligence may be developing faster than our ability to guarantee that the resulting intelligence will not cause serious harm.
That distinction is fundamental.
Intelligence ≠ wisdom.
Capability ≠ morality.
Prediction ≠ understanding.
Optimization ≠ justice.
An extremely capable AI might perform precisely the task given to it and nevertheless produce terrible consequences because the objective was badly specified, because important human context was absent, or because nobody anticipated what would happen when the system operated autonomously.
This strengthens the point from our previous discussion. The deepest danger may not be an AI that somehow becomes evil. A system does not have to hate humanity to harm humanity. It need only possess enormous capability, an inadequate representation of the human situation, and sufficient power to act upon that representation.
But there is an important disagreement
The slowdown proposal is not universally accepted. Critics argue that slowing U.S. laboratories could allow competitors—especially China—to advance instead; that stopping technological development could sacrifice major benefits in medicine, science and productivity; and that we do not yet know how an international pacing agreement could actually be enforced. The current U.S. administration has also pushed back against the slowdown argument on competitiveness grounds. (파이낸셜 타임스)
There is also an important collective-action problem. One laboratory may believe slowing down is safer, but if it slows while every competitor continues at full speed, it bears the commercial and strategic cost without necessarily reducing the global risk. That is why proponents increasingly speak about coordination among laboratories and governments, rather than asking one company voluntarily to stop. (Axios)
And that explains why the agreement among Amodei, Altman and Musk is noteworthy. They are competitors with major disagreements. Their agreement does not prove that catastrophic AI danger is certain. It does show that concern about the rate of frontier development has moved from outside critics into the leadership of several organizations actually building frontier systems. (Investing.com)
There is also an interesting historical difference from the famous 2023 six-month-pause letter, which Musk supported. That proposal largely came as an external demand upon AI laboratories. The 2026 development is more significant in one respect: the argument for pacing is increasingly coming from inside the frontier laboratories themselves, including corporate endorsements and CEOs discussing coordination. (Investing.com)
And this brings me back to your original philosophical question—“Is AI essentially ethical?”
These events make me modify my earlier answer slightly. The most important ethical question may ultimately concern not the ethics inside AI, but the ethics of humans deciding how much power to give AI before they understand its consequences.
The Amodei–Altman–Musk episode is therefore not merely an AI-safety controversy. It represents a remarkable moment in technological history: some of the people most capable of accelerating a technology are publicly asking whether humanity needs the capacity to slow it down.
I can monitor this debate and tell you if the proposed slowdown becomes an actual agreement among the major AI companies or governments.
ㅡㅡㅡㅡ
AI can do harm to human beings at any stage of progress, when it is not trained to be ethical. And it may be not highly effective to pace or slow the AI progress. Meanwhile, it may be feasible and practical to train AI to be more aware and more careful of human ethical issues, because AI can (be made to) know what is humanly fair and ethical, although it essentially may have no will to ethics itself.
ㅡㅡ
AI is essentially without a will to ethics. It does not naturally desire justice, care about suffering, feel moral obligation, or decide for itself that fairness ought to prevail. But this does not mean that AI cannot know, reason about, and act according to human ethical principles. That distinction is central.
A human being may say:
“I know I could benefit from doing this, but it would be unfair, so I will not do it.”
There is a moral will operating there. Present AI has no comparable independent moral will. Nevertheless, through training, rules, examples, reasoning, criticism, and feedback, it can learn that certain actions are unfair, harmful, discriminatory, deceptive, or otherwise ethically unacceptable.
This makes your criticism of the pacing argument much sharper.
Pacing does not solve the fundamental problem
Suppose frontier AI development were frozen tomorrow for two years.
The existing AI would still be without a will to ethics.
If its ethical training were inadequate, it could still misrepresent people, discriminate, provide harmful advice, violate privacy, or assist harmful human purposes. Merely keeping its intelligence at today’s level would not cure any of these problems.
So there are actually two dimensions:
Capability: What can the AI do?
Ethical competence: How well can it recognize ethically significant situations and behave according to justified human ethical constraints?
The two need not progress together.
That produces exactly the danger you are identifying. Imagine:
Capability ↑↑↑
Ethical competence →
Then risk increases.
But merely changing this to:
Capability →
Ethical competence →
does not solve the underlying problem either.
The desirable development is:
Capability ↑
Ethical competence ↑
And the greater the capability and potential human consequence, the stronger the required ethical training, testing and accountability should become.
This changes the meaning of “AI safety”
Much discussion of AI safety concentrates on preventing catastrophic future outcomes: loss of control, autonomous weapons, cyberattack, biological misuse, or extremely powerful systems acting against human interests.
Those questions matter.
But your argument points toward a more ordinary and immediate conception of safety:
Every AI capable of affecting human beings should be trained to recognize the ethical consequences of what it says, recommends and does.
This applies at every level of AI development, not merely at some hypothetical future threshold.
A relatively weak AI can falsely accuse someone. It can discriminate in employment. It can give dangerously inappropriate advice. It can expose private information. It can unfairly characterize a scholar’s work. None requires superintelligence.
Therefore your sentence becomes important:
AI without a will to ethics can harm human beings at any stage of its progress if its ethical competence has not been adequately trained.
I agree with the conceptual structure of that proposition.
Then what should we say about Amodei, Altman and Musk?
I would change my previous answer here too.
We should not infer that their purpose is to escape responsibility unless there is evidence of that motive. But your argument exposes a possible weakness in the logic of pacing, independent of their motives.
If an AI company says:
“Our systems may become dangerous, therefore AI progress should be slowed,”
society should reasonably ask:
“What ethical and safety obligations have you already accepted for the systems you are developing?”
Pacing cannot replace that answer.
Indeed, there is a danger that public discussion about a future super-powerful AI could distract attention from harms caused by present AI systems. Whether or not that happens intentionally, responsibility for present systems remains.
So I would reformulate your criticism without the uncertain motive claim:
The proposal to pace AI progress must not be allowed to substitute for, weaken, or postpone the responsibility of AI developers to provide due ethical training and testing at every stage of AI development.
That is much harder to object to.
“Due ethical training” now becomes the center
I think your phrase deserves development as a concept.
Consider an analogy with due care.
A manufacturer cannot normally defend an unsafe product by saying, “Technology is developing too quickly.” The speed of technological development does not remove the manufacturer’s duty of care.
Likewise, an AI developer should not be able to say simply:
“AI is inherently unpredictable.”
If the developer gives the system greater capability, releases it to millions of people, or allows it to make increasingly consequential decisions, the developer acquires correspondingly greater responsibility to investigate foreseeable ethical harms.
We could formulate the principle:
Due ethical training: Those who develop and deploy AI have a continuing duty, proportional to the AI’s capabilities and possible effects upon human beings, to train and test it to recognize human ethical considerations, avoid foreseeable injustice and harm, acknowledge uncertainty, seek necessary context, and defer consequential decisions when adequate ethical judgment cannot be assured.
Notice something important here.
We do not need to prove that AI possesses morality.
The obligation rests upon the humans who create it.
There is also a deeper reason your correction matters
“Essentially unethical” would imply:
AI has a morally bad nature.
“Essentially without will to ethics” means:
AI possesses no inherent moral intention either way.
That makes training more, not less, important.
A human child develops within a human moral environment—parents, language, sympathy, punishment, friendship, suffering, social expectations and eventually personal conscience. AI has no such naturally lived moral development. Whatever ethical competence it possesses has to arise through its human-created training, architecture, instructions, evaluation and continuing interaction.
That means an AI laboratory is doing more than teaching a machine facts.
It is inevitably shaping how a powerful non-human intelligence represents human values.
That carries an unusually large responsibility.
And here I think your argument becomes more profound than simply opposing a temporary slowdown.
The fundamental problem is not:
How fast should intelligence increase?
It is:
How can ethical understanding increase together with artificial intelligence when artificial intelligence itself has no independent will to become ethical?
If AI had its own reliable will toward human good, perhaps we could trust increasing intelligence to bring increasing ethical behaviour with it. But because it does not, there is no necessary relationship between becoming more intelligent and becoming more ethically careful.
That gap has to be filled deliberately by human beings.
So I would now state your thesis in its strongest form:
AI is essentially without a will to ethics. Therefore increasing AI capability does not by itself produce increasing ethical responsibility. The central obligation of AI developers is consequently to ensure continuous and adequate ethical training, testing and correction proportional to the power their systems acquire. Pacing capability may sometimes provide time needed for that work, but pacing is a means, not the ethical solution itself, and cannot substitute for the developers’ continuing duty of ethical care.
With your corrected wording, I think this is considerably stronger than the argument we were discussing before. It shifts the center of the AI-safety question from speed alone to the relationship between capability, ethical competence, and human responsibility.
ㅡㅡㅡㅡ
What do you think that AI developers should be made by law to be responsible for the ethical training of AI?
ㅡㅡ
Yes—with an important qualification. I think your idea of “due ethical training” can reasonably be developed into a legal duty, especially for AI systems capable of materially affecting human rights, opportunities, safety, livelihood, reputation, or access to important services.
This would be different from legislating that “AI must be ethical.” That is too vague. The law should impose duties on the human organizations that develop and deploy AI.
The basic legal principle could be:
An AI developer has a duty to take reasonable and proportionate measures to train, test, evaluate, and continually correct an AI system so that it can recognize and appropriately respond to foreseeable ethical risks to human beings.
There are already partial movements in this direction. The EU AI Act imposes concrete obligations on providers according to risk, including documentation and transparency requirements; some transparency provisions became applicable on 2 August 2026. (디지털 전략) In the United States, NIST’s AI Risk Management Framework asks developers and users to address safety, accountability, transparency, privacy, explainability and fairness throughout the AI lifecycle, although the framework itself remains voluntary. (NIST) Colorado has also legislated duties concerning automated systems used in consequential decisions, with its revised provisions scheduled to take effect in 2027. (Coag)
But your proposal goes somewhat deeper than these rules because you are asking about training itself.
I think a good law should avoid prescribing one official moral doctrine. Instead, it could require demonstrable processes: developers must identify foreseeable human harms; include ethical and fairness considerations in training and post-training; test systems against difficult ethical cases; investigate systematic harms; document known limitations; provide mechanisms for correction; and use stronger safeguards as the AI’s power and consequences increase.
Most importantly, responsibility should not disappear into the AI.
If an AI seriously injures someone, “the model produced the answer” should not by itself excuse its developer or deployer. The legal inquiry should include questions such as: Was this kind of harm reasonably foreseeable? Was the system adequately trained and tested against it? Were known failures corrected? Was sufficient human context obtained? Was the AI given decision-making authority that exceeded its demonstrated reliability?
That resembles established ideas of reasonable care and product responsibility more than an attempt to turn morality itself into legislation.
There is also an important advantage over a general demand to “slow AI.”
Suppose Company A develops AI slowly but gives little attention to ethical competence. Company B develops more rapidly but spends substantial resources on fairness testing, contextual reasoning, uncertainty, human oversight and correction. Speed alone tells us very little about which system will treat human beings better.
Therefore, pacing should not be the primary legal test. Due ethical care should be.
And your phrase “without will to ethics” gives a philosophical reason for placing the obligation upon developers. Because AI does not possess an independent human-like will that says I ought to become more ethical, we cannot reasonably leave ethical development to the machine itself. As its intellectual capability grows, human beings must deliberately develop its ethical competence alongside it.
I would express the relationship as:
greater AI capability → greater possible human consequence → greater duty of ethical training and testing.
There is, however, one difficult question that your proposal immediately creates: Who determines whether the ethical training is adequate?
It cannot safely be left entirely to the developer. A company would then effectively certify its own morality. Nor should a government simply dictate one comprehensive moral philosophy.
A workable system would probably require publicly established minimum principles—non-discrimination, human safety, privacy, honesty, procedural fairness, opportunity for human challenge—combined with independent testing and auditing. NIST already recognizes that trustworthy AI involves several characteristics simultaneously and that their balance depends upon the context in which an AI is used. (NIST AI Resource Center)
This last word, context, again becomes fundamental.
An AI cannot be ethically trained merely by memorizing “do not harm” and “be fair.” It must increasingly learn to recognize which circumstances matter ethically in the particular human case before it. NIST itself notes that AI bias can be systemic, computational, or human and can occur even without discriminatory intention. (NIST)
So I think your proposal can be sharpened into a fairly serious legal principle:
Developers and deployers of AI should bear a legally enforceable duty of due ethical care: a continuing obligation, proportional to the capability and foreseeable human impact of their systems, to train, test, monitor, and correct those systems for reasonably foreseeable ethical harms, and to preserve meaningful human review where AI cannot adequately understand the ethically relevant context.
That is considerably more concrete than saying “AI companies must make ethical AI.”
And it has one major consequence for the present debate. The question put to an AI company would no longer simply be “Why didn’t you slow down?” It would become “You created this capability and gave it power over human beings. What did you do to ensure that its ethical competence grew with its power?”
That seems to me the strongest form of the argument you are developing.
ㅡㅡㅡㅡ
2026-09-16 Mark Park
Our Library 