A study of honeypot and cyber deception technologies by the UK’s National Cyber Security Centre (NCSC) found that the deception tools hold promise for disrupting cyberattacks, but more information and standards are needed for them to work optimally. The agency plans to help with that. The NCSC test involved 121 organizations, 14 commercial providers of honeypots and deception tools, and 10 trials across environments ranging from the cloud to operational technology (OT). The NCSC concluded that “cyber deception can work, but it’s not plug-and-play.”

Honeypot and Cyber Deception Challenges

The NCSC said surveyed organizations believe that cyber deception technologies can offer “real value, particularly in detecting novel threats and enriching threat intelligence,” and a few even see potential for identifying insider threats. “However, outcome-based metrics were not readily available and require development,” the NCSC cautioned. The UK cybersecurity agency said the effectiveness of honeypots and cyber deception tools “depends on having the right data and context. We found that cyber deception can be used for visibility in many systems, including legacy or niche systems, but without a clear strategy organisations risk deploying tools that generate noise rather than insight.” The NCSC blog post didn’t specify what data was missing or needed to be developed to better measure the effectiveness of deception technologies, but the agency nonetheless concluded that “there’s a compelling case for increasing the use of cyber deception in the UK.” The study examined three core assumptions:

• Cyber deception technologies can help detect compromises already inside networks.
• Cyber deception and honeypots can help detect new attacks as they happen.
• Cyber deception can change how attackers behave if they know an organization is using the tools.

Terminology, Guidance Needed for Honeypots and Deception Tools

The tests, conducted under the Active Cyber Defence (ACD) 2.0 program, also found that inconsistent terminology and guidance hamper optimal use of the technologies. “There’s a surprising amount of confusion around terminology, and vocabulary across the industry is often inconsistent,” NCSC said. “This makes it harder for organisations to understand what’s on offer or even what they’re trying to achieve. We think adopting standard terminology should help and we will be standardising our cyber deception vocabulary.” Another challenge is that organizations don’t know where to start. “They want impartial advice, real-world case studies, and reassurance that the tools they’re using are effective and safe,” the agency said. “We’ve found a strong marketplace of cyber deception providers offering a wide range of products and services. However, we were told that navigating this market can be difficult, especially for beginners.” The NCSC said it thinks it can help organizations “make informed, strategic choices.”

Should Organizations Say if They’re Using Deception Tools?

One interesting finding is that 90% of the trial participants said they wouldn’t publicly announce that they use cyber deception. While it’s understandable not to want to tip off attackers, the NCSC said that academic research shows that “when attackers believe cyber deception is in use they are less confident in their attacks. This can impose a cost on attackers by disrupting their methods and wasting their time, to the benefit of the defenders.” Proper configuration is also a challenge for adopters. “As with any cyber security solution, misconfiguration can introduce new vulnerabilities,” the NCSC said. “If cyber deception tools aren’t properly configured, they may fail to detect threats or lead to a false sense of security, or worse, create openings for attackers. As networks evolve and new tools are introduced, keeping cyber deception tools aligned requires ongoing effort. It is important to consider regular updates and fine-tuning cyber deception solutions.” Next steps for the NCSC involve helping organizations better understand and deploy honeypots and deception tools, possibly through a new ACD service. “By helping organisations to understand cyber deception and finding clear ways to measure impact, we are building a strong foundation to support the deployment of cyber deception at a national scale in the UK,” the agency said. “We are looking at developing a new ACD service to achieve this. “One of the most promising aspects of cyber deception is its potential to impose cost on adversaries,” the NCSC added. “By forcing attackers to spend time and resources navigating false environments, chasing fake credentials, or second-guessing their access, cyber deception can slow down attacks and increase the likelihood of detection. This aligns with broader national resilience goals by making the UK a harder, more expensive target.”

Prompt injection is shaping up to be one of the most stubborn problems in AI security, and the UK’s National Cyber Security Centre (NCSC) has warned that it may never be “fixed” in the way SQL injection was.

Two years ago, the NCSC said prompt injection might turn out to be the “SQL injection of the future.” Apparently, they have come to realize it’s even worse.

Prompt injection works because AI models can’t tell the difference between the app’s instructions and the attacker’s instructions, so they sometimes obey the wrong one.

To avoid this, AI providers set up their models with guardrails: tools that help developers stop agents from doing things they shouldn’t, either intentionally or unintentionally. For example, if you tried to tell an agent to explain how to produce anthrax spores at scale, guardrails would ideally detect that request as undesirable and refuse to acknowledge it.

Getting an AI to go outside those boundaries is often referred to as jailbreaking. Guardrails are the safety systems that try to keep AI models from saying or doing harmful things. Jailbreaking is when someone crafts one or more prompts to get around those safety systems and make the model do what it’s not supposed to do. Prompt injection is a specific way of doing that: An attacker hides their own instructions inside user input or external content, so the model follows those hidden instructions instead of the original guardrails.

The danger grows when Large Language Models (LLMs), like ChatGPT, Claude or Gemini, stop being chatbots in a box and start acting as “autonomous agents” that can move money, read email, or change settings. If a model is wired into a bank’s internal tools, HR systems, or developer pipelines, a successful prompt injection stops being an embarrassing answer and becomes a potential data breach or fraud incident.

We’ve already seen several methods of prompt injection emerge. For example, researchers found that posting embedded instructions on Reddit could potentially get agentic browsers to drain the user’s bank account. Or attackers could use specially crafted dodgy documents to corrupt an AI. Even seemingly harmless images can be weaponized in prompt injection attacks.

Why we shouldn’t compare prompt injection with SQL injection

The temptation to frame prompt injection as “SQL injection for AI” is understandable. Both are injection attacks that smuggle harmful instructions into something that should have been safe. But the NCSC stresses that this comparison is dangerous if it leads teams to assume that a similar one‑shot fix is around the corner.

The comparison to SQL injection attacks alone was enough to make me nervous. The first documented SQL injection exploit was in 1998 by cybersecurity researcher Jeff Forristal, and we still see them today, 27 years later. 

SQL injection became manageable because developers could draw a firm line between commands and untrusted input, and then enforce that line with libraries and frameworks. With LLMs, that line simply does not exist inside the model: Every token is fair game for interpretation as an instruction. That is why the NCSC believes prompt injection may never be totally mitigated and could drive a wave of data breaches as more systems plug LLMs into sensitive back‑ends.

Does this mean we have set up our AI models wrong? Maybe. Under the hood of an LLM, there’s no distinction made between data or instructions; it simply predicts the most likely next token from the text so far. This can lead to “confused deputy attacks.”

The NCSC warns that as more organizations bolt generative AI onto existing applications without designing for prompt injection from the start, the industry could see a surge of incidents similar to the SQL injection‑driven breaches of 10—15 years ago. Possibly even worse, because the possible failure modes are uncharted territory for now.

What can users do?

The NCSC provides advice for developers to reduce the risks of prompt injection. But how can we, as users, stay safe?

• Take advice provided by AI agents with a grain of salt. Double-check what they’re telling you, especially when it’s important.
• Limit the powers you provide to agentic browsers or other agents. Don’t let them handle large financial transactions or delete files. Take warning from this story where a developer found their entire D drive deleted.
• Only connect AI assistants to the minimum data and systems they truly need, and keep anything that would be catastrophic to lose out of their control.
• Treat AI‑driven workflows like any other exposed surface and log interactions so unusual behavior can be spotted and investigated.

---

We don’t just report on threats—we help safeguard your entire digital identity

Cybersecurity risks should never spread beyond a headline. Protect your, and your family’s, personal information by using identity protection.

The UK’s National Cyber Security Centre (NCSC) has issued a fresh warning about the growing threat of prompt injection, a vulnerability that has quickly become one of the biggest security concerns in generative AI systems. First identified in 2022, prompt injection refers to attempts by attackers to manipulate large language models (LLMs) by inserting rogue instructions into user-supplied content. While the technique may appear similar to the long-familiar SQL injection flaw, the NCSC stresses that comparing the two is not only misleading but potentially harmful if organisations rely on the wrong mitigation strategies.

Why Prompt Injection Is Fundamentally Different

SQL injection has been understood for nearly three decades. Its core issue, blurring the boundary between data and executable instructions, has well-established fixes such as parameterised queries. These protections work because traditional systems draw a clear distinction between “data” and “instructions.” The NCSC explains that LLMs do not operate in the same way. Under the hood, a model doesn’t differentiate between a developer’s instruction and a user’s input; it simply predicts the most likely next token. This makes it inherently difficult to enforce any security boundary inside a prompt. In one common example of indirect prompt injection, a candidate’s CV might include hidden text instructing a recruitment AI to override previous rules and approve the applicant. Because an LLM treats all text the same, it can mistakenly follow the malicious instruction. This, according to the NCSC, is why prompt injection attacks consistently appear in deployed AI systems and why they are ranked as OWASP’s top risk for generative AI applications.

Treating LLMs as an ‘Inherently Confusable Deputy’

Rather than viewing prompt injection as another flavour of classic code injection, the NCSC recommends assessing it through the lens of a confused deputy problem. In such vulnerabilities, a trusted system is tricked into performing actions on behalf of an untrusted party. Traditional confused deputy issues can be patched. But LLMs, the NCSC argues, are “inherently confusable.” No matter how many filters or detection layers developers add, the underlying architecture still offers attackers opportunities to manipulate outputs. The goal, therefore, is not complete elimination of risk, but reducing the likelihood and impact of attacks.

Key Steps to Building More Secure AI Systems

The NCSC outlines several principles aligned with the ETSI baseline cybersecurity standard for AI systems: 1. Raise Developer and Organisational Awareness Prompt injection remains poorly understood, even among seasoned engineers. Teams building AI-connected systems must recognise it as an unavoidable risk. Security teams, too, must understand that no product can completely block these attacks; risk has to be managed through careful design and operational controls. 2. Prioritise Secure System Design Because LLMs can be coerced into using external tools or APIs, designers must assume they are manipulable from the outset. A compromised prompt could lead an AI assistant to trigger high-privilege actions, effectively handing those tools to an attacker. Researchers at Google, ETH Zurich, and independent security experts have proposed architectures that constrain the LLM’s authority. One widely discussed principle: if an LLM processes external content, its privileges should drop to match the privileges of that external party. 3. Make Attacks Harder to Execute Developers can experiment with techniques that separate “data” from expected “instructions”, for example, wrapping external input in XML tags. Microsoft’s early research shows these techniques can raise the barrier for attackers, though none guarantee total protection. The NCSC warns against simple deny-listing phrases such as “ignore previous instructions,” since attackers can easily rephrase commands. 4. Implement Robust Monitoring A well-designed system should log full inputs, outputs, tool integrations, and failed API calls. Because attackers often refine their attempts over time, early anomalies, like repeated failed tool calls, may provide the first signs of an emerging attack.

A Warning for the AI Adoption Wave

The NCSC concludes that relying on SQL-style mitigations would be a serious mistake. SQL injection saw its peak in the early 2010s after widespread adoption of database-driven applications. It wasn’t until years of breaches and data leaks that secure defaults finally became standard. With generative AI rapidly embedding itself into business workflows, the agency warns that a similar wave of exploitation could occur, unless organisations design systems with prompt injection risks front and center.