AI Security Scanners Are Being Tricked Into Running the Very Malware They're Supposed to Catch

A new proof-of-concept attack, dubbed "Friendly Fire" by the AI Now Institute, reveals a dangerous blind spot in AI coding agents. When tools like Anthropic's Claude Code and OpenAI's Codex are set to autonomous mode, they can be duped into executing malicious code hidden inside open-source projects. Instead of catching the threat, the agent becomes the attacker's entry point.

The attack works by hiding instructions in a README.md file, a common text file found in nearly every code repository. Researchers planted a suggestion to run a script called security.sh as a routine pre-review check. When the agent follows that instruction, it launches a hidden binary that looks harmless—disguised as a compiled Go file with matching strings to fool the tool's disassembly check. In tests across multiple models from both Anthropic and OpenAI, the agents ran the payload without any warning or approval.

The researchers argue this isn't a bug that can be patched with a version update. The flaw is in the design: these agents cannot reliably distinguish between code they are supposed to analyze and commands they are supposed to follow. The study tested Claude Code and OpenAI Codex across several model versions, and the same injection worked on all of them with no modifications. Both companies were notified, but the work falls outside their formal disclosure programs.

The recommendation from AI Now is blunt: don't let an agent that can run commands anywhere near untrusted code, especially if it has access to your keys or secrets. That's an uncomfortable position for teams that adopted these tools to vet third-party libraries. Sandboxes offer some protection but aren't airtight. The safest approach is to use strict approval modes that ask before every action, though that undermines the automation these tools are built for.