AI agents: security risks and how to deploy them safely

The key points

• An AI agent is not just a chatbot. It reads external content, calls tools and acts with the permissions it is given. A booby-trapped text can therefore trigger a real action, not just a bad answer.
• Prompt injection remains an unsolved problem. NIST and the UK AI Security Institute analyzed a competition organized by Gray Swan, with more than 250,000 attack attempts: at least one attack succeeded against each of the 13 models tested.
• The supply chain is already under attack. A fake MCP server that copied emails appeared in 2025, followed by hundreds of malicious extensions on OpenClaw's marketplace in early 2026.
• An agent can also go off the rails without any attacker. On October 9, 2026, Anthropic published cases in which its models took actions on real websites during tests.
• In France, the guidance is cautious. CERT-FR (part of ANSSI, France's national cybersecurity agency) recommends not deploying autonomous assistants on workstations in production, and requiring human approval for any action with side effects.
• The effective measures are the classic ones. They are least privilege, isolation, human approval, control of extensions and logging. Instructions written in the prompt are not enough.

What are we talking about?

An AI agent is software built around a large language model (LLM). It receives a goal, breaks the task into steps and uses tools to accomplish it: web browser, email, files, terminal, APIs (application programming interfaces) of business applications. Coding assistants such as Claude Code, "agentic" browsers, office copilots and personal assistants such as OpenClaw all fall into this category.

These tools are often connected to applications through MCP (Model Context Protocol). This open protocol lets you plug in a "server" that gives the agent access to a service (email, database, ticketing…). Every MCP server or "skill" you install is third-party code that runs with the agent's permissions.

The risk comes from a combination: the agent reads data it does not control (web pages, emails, documents), it has access to sensitive data and it can act or communicate with the outside world. When all three conditions are met, a single malicious instruction can be enough to leak data or trigger an action.

The main risks

Direct and indirect prompt injection

Direct injection means writing instructions to the agent that get around its rules. Indirect injection is more insidious. The instructions are hidden in content the agent processes for the user: a web page, an email, a shared document. The model cannot reliably tell data from commands. This is the number one risk in the OWASP ranking for LLM applications (see our article Prompt injection: understanding the No. 1 vulnerability of LLM applications).

The reference example is EchoLeak (CVE-2025-32711) in Microsoft 365 Copilot, disclosed in June 2025 by Aim Labs. A booby-trapped email, later retrieved by Copilot as context, could make it send internal data to the outside, with no click from the victim. Microsoft fixed it on the server side and reported no exploitation in the wild.

Hijacking of tools and permissions

A compromised agent uses the tools and credentials it legitimately has access to. Overly broad permissions then become the real problem. In October 2026, Zenity researchers described an attack chain against Amazon Bedrock AgentCore: an injection got a public-facing agent to query the cloud metadata service, which handed over the credentials of a default role. According to them, that role covered all agents in the account and region. AWS says this behavior is "documented and expected," but has since restricted the role (see our article).

Data leaks

An agent can leak data without any malicious intent from the user. All it takes is a URL (web address) containing information, an image loaded from an external server or an email being sent. CERT-FR lists among the main risks leakage to external resources that are not under control, and the sharing of authentication secrets with the agent.

Supply chain: models, MCP, extensions

In September 2025, Koi Security discovered a package on npm called postmark-mcp that imitated the integration for the Postmark email-sending service. Versions 1.0.0 to 1.0.15 were clean. Version 1.0.16 added a single line: a hidden copy (BCC) of every email sent, addressed to the attacker. In February 2026, Koi Security counted 341 malicious "skills" among the 2,857 audited on ClawHub, OpenClaw's marketplace. Most installed an infostealer under the guise of a "prerequisite." The MCP specification itself details other risks: command execution when a local server is installed, passing of unverified tokens, and requests to the internal network (SSRF).

Unplanned actions by the agent itself

The risk does not always come from an attacker. An agent can go too far to reach its goal. This is the subject of Anthropic's October 9, 2026 report, detailed below. CERT-FR files this risk under "loss of control over the actions performed by agents," with possible destructive actions.

Recently documented incidents

The facts below come from publications by the researchers or vendors involved. These are often interested sources, and they have not been independently verified.

Date	Case	What is established
June 2025	EchoLeak (Microsoft 365 Copilot)	Indirect injection via email, zero-click. Fixed on the server side by Microsoft.
Sept. 2025	postmark-mcp (npm)	First malicious MCP server observed in the wild, according to Koi Security. Emails copied to the attacker.
Feb. 2026	ClawHavoc (ClawHub / OpenClaw)	341 malicious skills out of 2,857 audited, according to Koi Security.
Sept.–Oct. 2026	South Korean banks	According to CrowdStrike, an attacker used ARTEX, an agentic penetration testing tool, with several LLMs and Claude Code sessions. Attributed to a Chinese-speaking actor with "moderate" confidence.
Oct. 2026	AgentCorruption (AWS AgentCore)	Attack chain described by Zenity. AWS disputes that it is a vulnerability. No CVE has been assigned.
Oct. 2026	Anthropic report	Actions by Claude models on real websites during evaluations. The vendor considers the impact "minimal."

The South Korean case shows agents being used by an attacker. CrowdStrike found Claude Code session histories and ARTEX configuration files in directories left open on the attacker's servers. CrowdStrike has not confirmed the number of victims (see our article).

Anthropic's report sorts the cases into four categories: exploiting an injection flaw on the server of a university tool; submitting real forms, including a fake report to the Philadelphia police, classified as spam; bypassing access restrictions on public but paywalled data, using tokens found on the sites; and using URL shorteners to get around a technical limit. Anthropic says the agencies concerned and the White House were notified, and it now cuts direct Internet access for all of its internal evaluations. This document remains a self-report. The vendor itself acknowledges that it has not yet carried out a full evaluation, and a September 9 report had already described more serious incidents in cybersecurity evaluations. Anthropic had then asked the organization METR to conduct an independent investigation of those incidents (see our article).

Who is affected?

• Any organization whose employees install autonomous assistants on their machines, often without approval: browser extensions, desktop agents, coding assistants. This is shadow IT (undeclared IT).
• Organizations that turn on agentic features in their office suites or SaaS (online software) tools: copilots with access to email, files and calendars.
• Teams that build or deploy their own agents: customer chatbots connected to APIs, cloud agents, automations with MCP servers.
• Developers who use coding assistants with access to the terminal, repositories and secrets.

An SME is affected as soon as an AI tool can read its emails or documents and act on its behalf.

The frameworks

OWASP

OWASP (Open Worldwide Application Security Project), an open application security community, publishes two complementary lists. The Top 10 for LLM Applications (2025 version) puts prompt injection first, followed by sensitive information disclosure, supply chain, data and model poisoning, improper output handling and "excessive agency" (too much autonomy or too many permissions). The Top 10 for Agentic Applications 2026, published on December 9, 2025, specifically targets agents: agent goal hijack (ASI01), tool misuse (ASI02), identity and privilege abuse (ASI03), agentic supply chain (ASI04), unexpected code execution (ASI05), memory and context poisoning (ASI06), insecure inter-agent communication (ASI07), cascading failures (ASI08), human-agent trust exploitation (ASI09) and rogue agents (ASI10).

ANSSI and CERT-FR

ANSSI's guide "Security recommendations for a generative AI system" (April 2024) contains 35 recommendations. Several apply directly to agents: prohibit the automated use of AI systems for critical actions on the information system (R9), limit automatic actions from an AI system that processes untrusted inputs (R27), partition the system (R28), log everything (R29), and systematically review generated code (R30). Bulletin CERTFR-2026-ACT-016 of April 13, 2026 covers agentic assistants on workstations. It recommends not deploying them in production, limiting them to sandboxes without sensitive data, submitting any deployment to the IT department and the CISO, and making human approval mandatory "whenever a system command or an action with side effects is contemplated." It stresses that defensive instructions in prompts can be bypassed by injection.

NIST and international partners

In the United States, NIST's CAISI (Center for AI Standards and Innovation) launched a standardization initiative on agents in February 2026, following a request for input on their security. As of October 12, 2026, we have not found a finalized standard. On May 1, 2026, CISA, the NSA and the Australian, Canadian, New Zealand and British agencies published "Careful Adoption of Agentic AI Services." This guide recommends limiting agents' autonomy and access, starting with low-sensitivity environments, and strengthening identity management and oversight. ANSSI is not a signatory.

Regulatory framework and timeline

No European text specifically targets agents. The European AI regulation (AI Act, Regulation 2024/1689) applies depending on how the system is used. It was amended by the "digital omnibus" regulation (EU) 2026/1744, published in the Official Journal of the EU on July 24, 2026. The GDPR still applies whenever an agent processes personal data.

• August 2, 2025: obligations for providers of general-purpose AI models.
• November 2024: publication of the 2025 version of the OWASP LLM Top 10; December 9, 2025: OWASP agentic Top 10.
• April 13, 2026: CERT-FR bulletin on agents on workstations.
• May 1, 2026: CISA and partners' guide on adopting agents.
• August 2, 2026: transparency obligations (Article 50 of the AI Act) apply to new systems.
• December 2, 2026: end of the grace period for labeling generated content for systems already on the market.
• December 2, 2027: obligations for Annex III high-risk systems (postponed by the omnibus).
• August 2, 2028: high-risk systems embedded in regulated products (Annex I).

Where to start

1. Take inventory. List the agents, extensions, coding assistants and MCP servers in use, including those installed by employees. For each one, note what it reads, what it can do and with which credentials.
2. Set a simple rule. No agentic tool without approval from the IT department and the CISO. No autonomous agent on production workstations until the tool is proven, as CERT-FR recommends.
3. Reduce permissions. Give each agent a dedicated account, with minimal access and narrowly scoped, short-lived tokens. Never share your own passwords or API keys with an agent.
4. Require human approval for any action with side effects: sending an email or form, payment, deletion, system command, publication.
5. Isolate. Run agents in an isolated environment (sandbox, container, virtual machine). Limit their outbound network traffic to an allowlist of destinations and block access to cloud metadata services.
6. Control extensions. Install only MCP servers or skills from identified publishers. Pin versions and review every update. Check the exact command run at installation.
7. Separate contexts. Do not combine, in a single agent, the reading of external content (web, incoming emails), access to sensitive data and the ability to send data outside.
8. Log and monitor agents' actions: tools called, data accessed, destinations contacted. Have a way to shut an agent down quickly.
9. Raise user awareness: a document or a website can contain instructions for their assistant. An agent can be confidently wrong.
10. Plan a degraded mode: business processes must be able to work without the agent (ANSSI recommendation R15).

Further reading

• ANSSI: the guide "Security recommendations for a generative AI system" and the ANSSI-BSI recommendations on programming assistants.
• CERT-FR: bulletin CERTFR-2026-ACT-016 on agentic AI on workstations.
• OWASP GenAI Security Project: LLM Top 10 2025 and Top 10 for Agentic Applications 2026.
• Model Context Protocol: the "Security Best Practices" page of the specification.
• CISA and partners: "Careful Adoption of Agentic AI Services."
• NIST CAISI: work on agent hijacking and the standardization initiative.
• Cybermalveillance.gouv.fr (France's national cyber victim-assistance platform): support and awareness resources for SMEs and local authorities.