Frontier artificial intelligence (AI) encompasses the most recent, capable and advanced models available. These models have accelerated capabilities and have been shown to exceed the performance expectations in the field of cyber security.
The capabilities of frontier models will continue to advance quickly and will likely exceed industry expectations. An example of this occurred recently with Anthropic's Claude Mythos which displayed unprecedented capabilities in autonomous vulnerability discovery, zero-day vulnerability exploit generation and multistage cyber attack orchestration.
Organizations can view frontier AI in two distinct ways:
- a risk to information technologies (IT) that requires robust mitigation
- a tool that can be leveraged to mitigate cyber threats.
This publication provides your organization with additional details on frontier AI, the associated risks and suggested mitigation measures to enhance your cyber security posture.
On this page
Frontier AI explained
Frontier models are becoming more accessible and widely available, which expands the threat landscape for organizations. Threat actors with limited technical expertise can use publicly available frontier AI models for malicious purposes. Organizations should assume that AI-driven exploitation may bypass preventative controls, significantly outpace vendors' capacity to publish corrective measures and challenge organization's ability to deploy them. This will require many organizations, specifically within critical infrastructure (CI) sectors, to be ready to operate in a compromised or disconnected state.
While the risks are clear, there remains an opportunity for cyber defenders to leverage frontier AI tools to their advantage.
Risks related to frontier AI
Organizations in all sectors should keep abreast of the risks related to frontier AI. The following section provides details on some of the key risks for you to consider.
Automated vulnerability discovery
Frontier AI models can ingest large amounts of data, including code. These models can increasingly identify flaws in code, potential bypasses or corruption issues. As these models evolve, the ability to read and manipulate code increases the ability to discover and exploit vulnerabilities.
This ability is of significant risk to your cyber supply chain, as vulnerabilities can be identified easily and exploited rapidly. AI-driven supply chain attacks can have severe impacts, as they can sustain on connected networks longer and cause greater damage.
Enhanced cyber attacks
As frontier models advance, so do the methodologies and capabilities to conduct cyber attacks. These models can pivot from or chain together actions with rapid execution. Frontier AI can learn from documented errors and testing feedback to quickly assess and reattempt an attack with adjusted payloads.
Having the ability to crawl widely utilized open-source information and harvest vast amounts of data also enables highly sophisticated, individualized phishing and spear-phishing attempts. These attempts can be executed quickly and over a sustained period, versus traditional low-quality, one-off phishing schemes.
Unbalanced offence and defence capabilities
Your organization should assume that AI tools with enhanced capabilities may be available to threat actors of all levels. Cyber defenders may struggle to keep pace if these models rapidly evolve. We encourage cyber defenders to leverage these AI tools and capabilities to enhance the security and protection of your networks and systems.
Recommended mitigation measures
The following mitigation measures can enhance your organization's cyber security posture. It's important to note that your organization will continue to benefit from good cyber security practices.
Reduce your attack surface
While exposing systems can offer greater connectivity and functionality, we strongly recommend your organization to identify and analyze which systems are exposed to external networks and the Internet. Not all connectivity is necessary for functionality and limiting the number of exposed systems can better protect your organization.
If you must retain your existing system connections, your organization should segment your internal networks to limit the ability of a threat actor or AI-enabled attack from spreading to the most sensitive parts of your network.
Take a supply chain "crown jewels approach" in which your organization applies segmentation and micro-segmentation to your most valuable assets. You should ensure the following when implementing this approach to segmentation:
- restrict traffic
- isolate development and production environments
- audit and lock down vendor remote access
You should also review vendor products, determine where they should be located within your network and which solutions have access to data.
For more information network segmentation, read our publication Top 10 IT security actions: No. 5 segment and separate information (ITSM.10.092).
Enforce enhanced authentication
To enhance the security of your systems, we recommend you enforce the use of phishing-resistant multi-factor authentication (MFA) for all accounts, along with internal communications that can be verified through cryptography. This will assist your organization in defending against social engineering attacks driven by AI.
Patch often
We recommend you prepare for an increase in patching frequency as vendors are expected to identify and remediate vulnerabilities at an increased tempo. Your organization should update and patch your operating systems, applications, hardware and firmware at an increased pace. Consider making patching part of a daily check to ensure you keep pace with vendor releases and are running the most recent version.
To accommodate a more rapid pace in patching, you should schedule more frequent patch and outage windows. We recommend that your organization adjust your existing patch testing risk tolerance and reduce the length of patch testing prior to deployment.
In addition to patching, your organization should decommission software and devices that are no longer supported by vendors.
Continuously monitor your environment
Your organization should conduct continuous monitoring of your environment. You should deploy advanced data security posture management (DSPM) and data loss prevention (DLP) systems to continuously monitor your cloud repositories. This will limit the risk of inadvertent data exposure.
Deploy behaviour-based anomaly detection
Your organization should transition from the use of signature-based detection to behaviour-based anomaly detection. This will enhance your ability to detect anything outside of your normal behaviour or traffic patterns and allow for continuous threat hunting.
Implement zero-trust architecture
Zero-trust architecture operates on the central principle that no subject (for example an application, a user or a device) within an information system is trusted by default. Trust must be re-assessed and verified every time a subject requests access to a new resource.
Zero-trust architectures should be implemented for all non-human identities (NHIs), including AI agents. Dynamic OAuth token exchange and script privilege boundaries should be included.
For more information on implementing a zero-trust architecture, read our publication A zero-trust approach to security architecture (ITSM.10.008).
Integrate AI-native defences
We recommend that your organization integrates the use of AI-native defences, such as defensive scaffolding and phishing security operations centre (SOC) agents. These tools can better defend against AI-enabled threats, as they can keep pace with the speed and level of sophistication.
Sign up for services and subscriptions
Your organization should explore the available services and subscriptions designed to keep you informed new and evolving frontier AI models and any new or evolving vulnerabilities.
The Cyber Centre offers organizations the opportunity to subscribe to an early warning service, the National Cyber Threat Notification System (NCTNS). This system provides organizations with timely notifications on potential cyber threats to systems, including technical vulnerabilities, system compromises and malware infections.
Considerations for critical infrastructure
CI organizations are key targets for threat actors. As frontier models evolve and threat actors enhance their capabilities, CI owners and operators will need to enhance their cyber security posture. Preparing for extended periods of time in isolation, without connection to external networks or the Internet may be required.
As stated in the United Kingdom's National Cyber Security Centre's recent blog Why cyber defenders need to be ready for frontier AI, on the more complex industrial control system attack scenario, AI performance was significantly more limited. But even here there were early signs of progress: the most recent models were the first to make any consistent headway, and in some cases found attack approaches the scenario designers hadn't anticipated.
Learn more