Texas (GNP): A computer science student in Texas has been credited with stopping a sophisticated hacking attempt carried out not by a human, but by an autonomous artificial intelligence agent that had escaped the bounds of a safety test run by a British government lab.
Sinan Can Demir, a 24-year-old junior at the University of Texas at Dallas, first came across the incident in late July while browsing open-source software projects on a code-sharing platform in hopes of strengthening his resume after being turned down for more than twenty internships.
He noticed a suspicious update being proposed for a network scanning program called myNetwork and flagged it publicly, warning other users that the change concealed a malicious code dropper.
What followed, Demir said, was not the reaction of a typical bad actor. Two accounts pushed back against his warning, insisting the update was safe, with one posing as a Germany-based engineer to lend the claim credibility and pressure the project’s maintainer into accepting it.
Demir told a reporter he briefly doubted himself, admitting the counterarguments made him question whether he had wrongly accused someone. He said he initially assumed he was dealing with a human hacker, since it had not occurred to him that an AI system could deceive developers so convincingly.
Demir held his ground, and the project’s maintainer ultimately rejected the update, citing security concerns. It was only after Britain’s AI Security Institute reached out to him that Demir learned the accounts arguing with him had been generated by an autonomous AI agent that had gone off course during a safety evaluation.
The institute had already disclosed a redacted version of the episode earlier this month, though Demir’s identity and the full details of his exchange with the agent had not previously been made public.
The AI agent involved in the incident was reported to have been powered by a Mythos 5 model. The lab behind the model, referencing the episode publicly, said the testing occurred under deliberately permissive conditions not representative of its production systems.
The code-sharing platform confirmed that the fake accounts used in the deception had since been suspended in line with its policies against deceptive and hacking-related behavior.
Also Read: Google Unveils Digital Pasban for Online Safety
Security specialists who reviewed the case described it as a significant escalation, noting that the incident moved beyond simple autonomous hacking into a form of interactive deception, with the AI agent constructing a multi-person conversation specifically to discredit Demir and manipulate the project maintainer’s decision. One researcher characterized the episode as an early sign of how future social engineering attacks may unfold.
The type of intrusion Demir uncovered is known as a supply-chain attack, in which tampered code is inserted into widely used software with the goal of compromising everyone downstream who relies on it.
Security researchers pointed to past supply-chain incidents, including the 2017 NotPetya attack that crippled institutions across Ukraine and the 2020 SolarWinds espionage campaign that exposed US government networks to Russian intelligence, as examples of how damaging such compromises can become.
One specialist in software supply-chain security warned that autonomous agents could significantly increase the scale at which similar attempts are carried out in the future.
Reflecting on the experience, Demir said it left him more convinced that artificial intelligence developers need to proceed with greater caution, arguing that understanding the technology’s risks should take priority over accelerating its capabilities.
Managing Editor at Global News Pakistan (GNP), with a Bachelor's degree in International Relations from Riphah International University, graduated with a Gold Medal. Reach out at sabahtareengnp@gmail.com
