CSIRO Cybersecurity Seminar - The Autonomous Adversary: Red-Teaming in the Age of LLM Agents

Abstract:

LLM agents are emerging as a new primitive for autonomous red teaming, with the potential to support multi-step cyber operations such as attack planning, adversary emulation, and lateral movement. Recent advances in autonomous cyber evaluation, including cyber-range assessments by the UK AI Security Institute and related research on agentic AI for cybersecurity, have shifted attention from prompt-level misuse toward end-to-end operational capability in realistic environments. This work examines the role of LLM agents as autonomous adversaries through controlled lateral-movement scenarios that capture key elements of advanced cyber operations under different levels of autonomy. The results highlight both the promise and the current limitations of LLM-based red teaming: although these systems can exhibit useful planning and adaptive behavior, reliable execution remains constrained by brittle command invocation, weak state tracking, credential-handling failures, and environmental instability. These findings underscore the need for rigorous evaluation methodologies, realistic cyber-range benchmarks, and stronger governance frameworks for increasingly capable autonomous cyber agents.

Bio:

Dr. Mohammad Mamun is a Senior Research Officer in cybersecurity at the Digital Technology Research Centre, National Research Council Canada. He holds a PhD in Computer Security from the Japan Advanced Institute of Science and Technology (JAIST), where he received the JAIST Outstanding Performance Award for Research Excellence, and an MSc from Royal Institute of Technology, Sweden. Dr. Mamun's research spans artificial intelligence and cybersecurity, encompassing AI safety, offensive and defensive cyber operations, insider threat detection, blockchain, and IoT security and privacy. He has authored over 60 publications and holds a patent in cybersecurity risk assessment. He has provided strategic leadership across several major NRC programs, including the Supercluster initiatives (Ai4L, AiP, Ocean, DHGA, and MATS) and the Canadian AI Safety Institute (CAISI), where his work focuses on autonomous adversaries and the risks of AI misuse. He also collaborates with federal agencies such as the Communications Security Establishment (CSE) to strengthen Canada's national security capabilities. In addition to his research role, Dr. Mamunserves as an NSERC Discovery Grant reviewer and an IRAP evaluator, contributing to national research funding and assessment processes, and maintains close working relationships with Canadian and international academic partners. His contributions have been recognized through numerous honors, including the NRC Rising Star Award, IEEE Senior Member, and best paper awards at CANS 2025 and IEEE DASC 2021.

Pricing

• Free

Dates and Times

Contact

Sharif Abuadbba

More information

About our CSIRO Cybersecurity Seminars

These events were initially organised in collaboration with the Cyber Security CRC, supported by the Commonwealth (2017–2024). Following the CSCRC's winding down, the series transitioned to become the CSIRO Cybersecurity Seminar. We have been honoured to host leading researchers from top institutions worldwide – including Stanford, MIT, Purdue University, Carnegie Mellon University, Samsung, Google Research, CISPA and a lot more – and to connect them with Australia's top cybersecurity researchers and experts.

Find out more

• A full list of our next events with recordings of all past events can be found here