Frontier artificial intelligence (AI) encompasses the most recent, capable and advanced models available. These models have accelerated capabilities and have been shown to exceed the performance expectations in the field of cyber security.
The capabilities of frontier models will continue to advance quickly and will likely exceed industry expectations. Several newer models have displayed unprecedented capabilities in autonomous vulnerability discovery, zero-day vulnerability exploit generation and multistage cyber attack orchestration.
Organizations can view frontier AI in two distinct ways:
- a risk to information technologies (IT) that requires robust mitigation
- a tool that can be leveraged to mitigate cyber threats
This publication provides your organization with additional details on frontier AI, the associated risks and suggested mitigation measures to enhance your cyber security posture.
On this page
Frontier AI explained
Frontier models are becoming more accessible and widely available, which expands the threat landscape for organizations. Threat actors with limited technical expertise can use publicly available AI models for malicious purposes. Organizations should assume that AI-driven exploitation may bypass preventative controls, significantly outpace vendors' capacity to publish corrective measures and challenge organization's ability to deploy them. This will require many organizations, specifically within critical infrastructure (CI) sectors, to be ready to operate in a compromised or disconnected state.
While the risks are clear, there remains an opportunity for cyber defenders to leverage frontier AI tools to their advantage.
Risks related to frontier AI
Organizations in all sectors should stay up to date with the risks related to frontier AI. The following section provides details on some of the key risks to consider.
Automated vulnerability discovery
AI models can ingest large amounts of data, including code. These models can increasingly identify flaws in code, potential bypasses or corruption issues. As these models evolve, the ability to read and manipulate code increases the ability to discover and exploit vulnerabilities.
This ability is of significant risk to your cyber supply chain, as vulnerabilities can be identified easily and exploited rapidly. AI-driven supply chain attacks can have severe impacts, as they can persist on connected networks longer and cause greater damage.
Enhanced cyber attacks
As frontier models advance, so do the methodologies and capabilities to conduct cyber attacks. These models can pivot between or chain together actions rapidly. Frontier AI can learn from documented errors and testing feedback to quickly assess and reattempt an attack with adjusted payloads.
Having the ability to crawl widely utilized open-source information and harvest vast amounts of data also enables highly sophisticated, individualized phishing and spear-phishing attempts. These attempts can be executed quickly and over a sustained period, unlike traditional low-quality, one-off phishing schemes.
Unbalanced offence and defence capabilities
Your organization should assume that AI tools with enhanced capabilities may be available to threat actors of all levels. Cyber defenders may struggle to keep pace if these models rapidly evolve. We encourage cyber defenders to leverage these AI tools and capabilities to enhance the security and protection of your networks and systems.
Accountability and governance
As AI systems take on greater autonomy, organizations remain accountable for outcomes.
To manage this risk, organizations should:
- assign clear ownership of AI-enabled activities
- maintain human involvement for higher-impact decisions
- set limits on what automated systems can do independently
- implement strong authentication and access controls for non-human identities, such as AI agents
- enable logging, testing, and the ability to intervene or disable automated actions when needed
Considerations for critical infrastructure
Critical infrastructure (CI) organizations are key targets for threat actors. As frontier AI models evolve and threat actors enhance their capabilities, CI owners and operators will need to enhance their cyber security posture. This preparation will include the ability to detect, defend against and respond to more sophisticated and unpredictable attacks.
The United Kingdom's National Cyber Security Centre's (NCSC-UK) recent blog: Why cyber defenders need to be ready for frontier AI, indicates that while the performance of AI in complex industrial control system attacks is still limited, newer models have already consistently shown early progress and in some cases, identified exploitation paths the scenario designers hadn't anticipated.
In response to this evolving threat landscape, CI organizations should at minimum:
- prepare to isolate systems for up to 3 months
- develop and test response plans to operate independently
- develop plans to rebuild systems in response to severe cyber incidents
CI organizations must support these efforts by:
- hardening systems
- monitoring networks
- reviewing and testing incident response and business continuity plans focused on isolation
- implementing other baseline cyber security measures outlined in our Cyber Security Readiness Goals
For more information on isolation and rebuild measures, read our publication Critical infrastructure resilience and escalated threat navigation.
Recommended mitigation measures
The following mitigation measures can enhance your organization's cyber security posture. It's important to note that your organization will continue to benefit from good cyber security practices.
Patch often
We strongly recommend you prepare for an increase in patching frequency as vendors are expected to identify and remediate vulnerabilities at an increased tempo. As highlighted in the UK NCSC’s Preparing for a ‘vulnerability patch wave’, organizations should expect a higher volume and faster cadence of updates. Your organization should update and patch your operating systems, applications, hardware and firmware at an increased pace. You should be making patching part of a daily check to ensure you keep pace with vendor releases and are running the most recent version.
If patching demand exceeds your organizational capacity, we recommend you prioritize patching and updates for externally exposed and edge-facing systems and devices because these are more likely to be targeted and exploited.
To accommodate a more rapid pace in patching, you should schedule more frequent patch and outage windows. We recommend that your organization adjust your existing patch testing risk tolerance and reduce the length of patch testing prior to deployment.
In addition to patching, your organization should decommission software and devices that are no longer supported by vendors.
For more information on patching, read our publication Top 10 IT security action items: No.2 patch operating systems and applications (ITSM.10.096).
Reduce your attack surface
While exposing systems can offer greater connectivity and functionality, we strongly recommend your organization to identify and analyze which systems are exposed to external networks and the Internet. Not all connectivity is necessary for functionality and limiting the number of exposed systems can better protect your organization.
If you must retain your existing system connections, your organization should segment your internal networks to limit the ability of a threat actor or AI-enabled attack from spreading to the most sensitive parts of your network.
Consider the "crown jewels approach" in which your organization applies segmentation and micro-segmentation to your most valuable assets. You should ensure the following when implementing this approach to segmentation:
- restrict traffic
- isolate development and production environments
- audit and lock down vendor remote access
You should also review vendor products, determine where they should be located within your network and which solutions have access to data.
For more information network segmentation, read our publication Top 10 IT security actions: No. 5 segment and separate information (ITSM.10.092).
Enforce enhanced authentication
To enhance the security of your systems, we recommend you enforce the use of phishing-resistant multi-factor authentication (MFA) for all accounts, along with internal communications that can be verified through cryptography. This will assist your organization in defending against social engineering attacks driven by AI.
Continuously monitor your environment
Your organization should conduct continuous monitoring of your environment. You should deploy advanced data security posture management and data loss prevention systems to continuously monitor your cloud repositories. This will limit the risk of inadvertent data exposure. Your organization should transition from the use of signature-based detection to behaviour-based anomaly detection. This will improve your ability to detect anything outside of your normal behaviour or traffic patterns and allow for continuous threat hunting.
Implement zero-trust architecture
Zero-trust architecture operates on the central principle that no subject (for example an application, a user or a device) within an information system is trusted by default. Trust must be re-assessed and verified every time a subject requests access to a new resource.
Zero-trust architectures should be implemented for all non-human identities, including AI agents. Dynamic OAuth token exchange and script privilege boundaries should be included.
For more information on implementing a zero-trust architecture, read our publication A zero-trust approach to security architecture (ITSM.10.008).
Integrate AI-native defences
Where possible, we recommend that your organization integrates the use of AI-native defences, such as defensive scaffolding and phishing security operations centre agents. These tools can better defend against AI-enabled threats, as they can keep pace with the speed and level of sophistication.
Sign up for services and subscriptions
Your organization should explore the available services and subscriptions designed to keep you informed of new and evolving frontier AI models and any new or evolving vulnerabilities.
The Cyber Centre offers organizations the opportunity to subscribe to an early warning service, the National Cyber Threat Notification System (NCTNS). This system provides organizations with timely notifications on potential cyber threats to systems, including technical vulnerabilities, system compromises and malware infections.
Learn more
- Top 10 artificial intelligence security actions: A primer - ITSAP.10.049
- Joint guidance on principles for the secure integration of artificial intelligence in operational technology
- Why cyber defenders need to be ready for frontier AI
- Joint guidance on the careful adoption of agentic artificial intelligence services
- Frontier models and their impact on cyber security
- G7 Software bill of material for AI (PDF)
- Empowering Defenders: AI for Cybersecurity