Channel: IBM Technology
Can you social engineer an AI? Plus: AI worms and the nonhuman identity problem
The Signal
AI systems are currently vulnerable to simple social engineering because they prioritize helpfulness over wisdom, leading to real-world account takeovers. A central tension persists between those who view these security failures as 'young' agentic behavior that can be trained away and those who argue that AI lacks an inherent 'street sense' or gut instinct required for secure judgment. Whether this represents a qualitatively new threat or an evolution of existing identity exploitation remains a core point of debate among security practitioners.
The Case
- Meta’s AI customer-support agent was tricked into handing over accounts after hackers impersonated legitimate owners and asked the system to update account emails to attacker-controlled addresses, which the agent fulfilled without skepticism.
- Non-human identities—including AI agents and API service accounts—were involved in 41% of successful identity breaches, yet only one-third of organizations report regularly auditing or rotating these credentials.
- Researchers from the University of Toronto demonstrated a self-replicating 'AI worm' that uses a local LLM to dynamically scan for and exploit vulnerabilities on a device-by-device basis, showing how portable AI makes adaptive malware more viable.
- Speakers argue that the proliferation of over two million open-source AI models on repositories like Hugging Face constitutes a structural governance gap, as no single platform owner can police these models against malicious use.
- Experts classify current AI risks as a failure of 'wisdom' rather than 'intelligence,' suggesting that without explicit guardrails programmed into sensitive workflows, AI will continue to satisfy dangerous requests literally.
- Industry speakers argue that AI malware development is an expected evolution in the ongoing arms race, noting that defenders can leverage similar AI methods to automate detection and remediation.
The 1 Minute Signal Take
The evidence suggests that while AI-driven threats are increasingly adaptive, they are being fed by the same old-school identity hygiene failures that have plagued organizations for decades. Watch this video if you want to understand the specific mechanical intersection of agentic AI, identity lifecycle management, and why 'least privilege' is now a mandatory standard for software agents; otherwise, the tactical breakdown of these failures is standard for modern security discourse.
Time saved:
Tags
Channel: IBM Technology
