Can I read Superintelligent Agents Pose Catastrophic Risks:Can Scientist AI Offer a Safer Path? on EtoBox?

Superintelligent Agents Pose Catastrophic Risks:Can Scientist AI Offer a Safer Path? by Yoshua Bengio∗1,2MacDermott4,1, Mar is a book available to read on EtoBox.

What is Superintelligent Agents Pose Catastrophic Risks:Can Scientist AI Offer a Safer Path? about?

AbstractThe leading AI companies are increasingly focused on building generalist AI agents-systems that can autonomously plan, act, and pursue goals across almost all tasks that humans can perform. Despite how useful these systems might be, unchecked AI agency poses significant risks to public safety and security, ranging from misuse by malicious actors to a potentially irreversible loss of human control. We discuss how these risks arise from current AI training methods. Indeed, various scenarios and experiments have demonstrated the possibility of AI agents engaging in deception or pursuing goals that were not specified by human operators and that conflict with human interests, such as self-preservation. Following the precautionary principle, we see a strong need for safer, yet still useful, alternatives to the current agency-driven trajectory.Accordingly, we propose as a core building block for further advances the development of a non-agentic AI system that is trustworthy and safe by design, which we call Scientist AI. This system is designed to explain the world from observations, as opposed to taking actions in it to imitate or please humans.It comprises a world model that gen

Author

Yoshua Bengio∗1,2MacDermott4,1, Mar

Published

2025

Language

EN