Anthropic Warns Small AI Models Could Power Self-Replicating IoT Malware

Anthropic is warning that advances in small AI models could soon lead to a new kind of self-replicating malware capable of spreading across smart fridges, robot vacuum cleaners, and other Internet-of-things devices, and proving harder to remove than conventional threats.

According to the company, smaller AI models may be able to autonomously self-replicate on IoT devices within the next year. That capability could eventually give rise to computer worms, which spread without human interaction and can infect unfamiliar devices. Anthropic acknowledged that its own recent experiment showed significant obstacles to building such an AI-driven replicator, so the results are tempered. Even so, the findings raise concerns about new forms of computer worms.

Anthropic also cautioned that such malware would be difficult to stop. In a blog post, the company said a replicator of this kind would be "challenging to shut down, since there's no centralized point to shut off," though it noted the process may be slow, with a doubling time of days. Unlike non-AI malware, it could also work out how to operate unfamiliar devices on its own.

Smaller AI models need far less powerful hardware than frontier models. Anthropic ran small open-weight models on real and simulated low-cost, off-brand Android TV boxes and other IoT hardware. Researchers found that by using quantization, representing models with fewer bits, and splitting workloads into "shards" across devices, the models could reach inference: their weights loaded and they could generate output.

One lightweight model needed just 15 boxes with 4GB of memory each to achieve inference. Another model was able to configure other large language models to run on eight of nine hardware types tested. Unlike traditional malware, which must be written for specific hardware or software, the model used its weights to make configurations on the fly for the devices it encountered.