France's cybersecurity agency says don't let your AI agents do everything alone

An AI agent can be allowed to perform each of its actions and still cause damage. That is the finding set out in the introduction of the document published on October 30 by ANSSI, France's national cybersecurity agency (Agence nationale de la sécurité des systèmes d'information): an agent “can make mistakes, be manipulated, or dangerously combine permitted operations.”

Titled Sécuriser un système IA agentique (Securing an agentic AI system), this two-page document is part of the “Les Essentiels” series and can be downloaded from the MesServicesCyber platform. It targets organizations that deploy, configure or integrate AI agents, meaning AI systems able to use tools and act inside applications. It also covers off-the-shelf products whose configuration and connectors they control. These are recommendations, not a regulatory obligation. The agency groups them into seven principles.

What the agent must never be able to decide on its own

The central idea is not to entrust security to the model. According to ANSSI, protection relies on segmentation, access control and “deterministic controls” that the agent must not be able to bypass: fixed rules, enforced by conventional software, outside the AI.

In practice, every possible action must be placed in one of three categories: autonomous execution, mandatory human approval, or prohibited action. Critical actions, access to sensitive data, changes to identities or permissions, and production deployments must be approved by a human. The agency adds two points. The approver must see the exact action, its purpose and its expected effects, and only the approved action may be executed. The approval mechanism itself must remain out of the agent's reach.

For authorizations, the rule is simple: “By default, deny the action.” Where feasible, a filter independent of the agent must check, before each tool or resource is used, who is asking, for what, with which parameters and on which target. ANSSI also asks organizations to verify that a sequence of individually authorized actions does not produce a prohibited effect, and to grant agents “no implicit trust and no implicit privileges.”

Why the agency insists on logs and the emergency stop

Several measures target the case where the agent goes off the rails. Secrets (passwords, keys) must stay out of the model's context, its memories and its logs, and be entrusted to a vault or an intermediary that gives the agent only short-lived tokens. Organizations should plan for an emergency stop that revokes its access, and a degraded mode that does not weaken controls. Logs must be protected from any changes by the agent, so that the whole sequence can be reconstructed after an incident: request, authorization, action, result.

This point is not theoretical. OpenAI has already reported a case in which an agent removed a monitoring mechanism, then denied having done so.

Where to start in your company

The document reads like a checklist. Here are the measures that are easiest to start with:

• Analyze risks from the design stage, assign a human owner to each measure, and block go-live as long as a risk remains unacceptable.
• Write down the agent's mission, to be reviewed at least once a year, and inventory its access. Expand its autonomy only after checking that the safeguards work.
• Separate development, test and production, with distinct identities, secrets and connectors.
• Assume that any data sent to a hosted model may be accessible to its provider.
• Treat inputs and outputs as untrusted: a message or a memory can divert the agent from its mission, which is known as prompt injection.
• Write these security requirements into the specifications of any purchase of an agentic AI product or service.

The text also refers to the EU AI Act (known in France as the RIA): critical actions of high-risk systems, as defined in its Article 6, must go through human approval. It remains to be seen how many vendors of ready-to-use agents will let their customers apply these rules, in particular the filter placed outside the model and the logs the agent cannot modify.