AI governance, security & compliance
AI security risks are threats that compromise the confidentiality, integrity, availability, or control of AI systems. They include deliberate attacks — such as prompt injection, data poisoning, model theft, and abuse of over-privileged agents — as well as insecure configurations that expose company data.
In IBM's 2025 breach study, 97% of organizations that reported an AI-related security incident lacked proper AI access controls, while 63% either had no AI governance policy or were still developing one. Organizations with high levels of shadow AI recorded breach costs USD 670,000 higher on average than organizations with little or no shadow AI. In one of the costliest documented cases, UK engineering firm Arup lost about USD 25.6 million after an employee joined a video call where the other "colleagues," including the CFO, were AI-generated deepfakes.
This gap between how fast businesses adopt AI and how well they secure it defines the current moment. This guide breaks down the ten risks every business should know, the incidents that made them real, the frameworks that structure a defense, and a practical action plan to close the gap.
AI security risks and AI hallucinations get lumped together, but they describe different failures. Security risks affect confidentiality, integrity, availability, or system control. Many involve an attacker, while others start with excessive permissions, accidental disclosure, or insecure system design. Hallucinations are reliability failures: the model produces false or fabricated output without needing malicious input. A hallucination can still create a security incident when a person or automated system acts on it. For a broader guide to deploying generative AI with appropriate governance and review controls, see our generative AI for business guide.
The two categories do overlap in one specific way: a hallucination becomes a security risk the moment an attacker can act on it. When an AI coding assistant invents a software package that doesn't exist, an attacker can register that exact package name and load it with malware — a pattern researchers call "slopsquatting." A 2025 USENIX Security study analyzing 576,000 AI-generated code samples across 16 models found hallucinated-package rates of at least 5.2% for commercial models and 21.7% for open-source models tested, with more than 205,000 unique fake package names identified (Spracklen et al., USENIX Security 2025). The hallucination is the entry point; the attacker's action is what turns it into a breach.
The OWASP Top 10 for LLM Applications (2025 edition) is a widely used taxonomy for structuring AI security risk, published by the OWASP GenAI Security Project. Here's what each risk means for a business running AI systems.
1. Prompt Injection. The top risk two editions running. LLMs process instructions and untrusted data in the same channel, so attackers can craft input the model interprets as a new command. Direct injection means typing malicious instructions straight into a chat. Indirect injection hides instructions inside documents, emails, web pages, or even images the AI processes automatically — the attacker never touches the AI directly. The UK's National Cyber Security Centre warned in December 2025 that prompt injection may never be fully patched, since it stems from how language models fundamentally interpret text.
2. Sensitive Information Disclosure. LLMs can memorize and reproduce fragments of the data they were trained or fine-tuned on — proprietary code, contracts, PII. This risk moved from sixth place in the previous OWASP edition to second place in 2025. It also appears through careless prompts, RAG systems, and exposed vector databases.
3. Supply Chain. Vulnerabilities inherited from third-party models, plugins, APIs, open-source components, and compromised development tools. In a 2026 vendor survey of 500 US CISOs, 99% expressed concern about a SaaS or AI supply-chain breach (Vorlon 2026 CISO Report).
4. Data and Model Poisoning. Attackers introduce malicious data during training, fine-tuning, or embedding to plant backdoors, degrade accuracy, or bias outputs. The 2025 OWASP update expanded this beyond pre-training data to cover fine-tuning and embedding manipulation too.
5. Improper Output Handling. When an AI's output feeds directly into another system — a shell command, a database query, a web page — without validation, the result can be remote code execution, SQL injection, or cross-site scripting. The LangChain vulnerability CVE-2023-29374 (CVSS 9.8, Critical) let prompt injection trigger arbitrary code execution through Python's exec() method in affected versions through 0.0.131. Treat every AI output as untrusted input, the same way you'd treat a form submission from an anonymous user.
6. Excessive Agency. This is the risk that determines how much damage every other risk on this list can do — and it's the reason AI agents raise the stakes so sharply. Excessive agency comes from giving an AI system more tools, more permissions, or more autonomy than the task requires. One in three respondents to Vorlon's survey of 500 US CISOs reported an AI-agent-related security incident in 2025. The fix is architectural, not a prompt tweak: least privilege, scoped credentials, and human approval for high-impact actions, enforced outside the model — never left to the AI to decide for itself.
7. System Prompt Leakage. New to the 2025 list. System prompts sometimes contain business logic or embedded secrets; if an attacker extracts them, that logic is exposed. Keep secrets out of prompts entirely — a vault, not a prompt, is the right place for them.
8. Vector and Embedding Weaknesses. New in 2025 and specific to RAG (retrieval-augmented generation) systems. Poor isolation can expose one customer's data in another customer's results, while attacks against embeddings may reveal information about source data. Attackers can also poison the documents an AI retrieves. The ConfusedPilot research demonstrated how malicious content placed in documents used by RAG systems could manipulate answers and undermine confidentiality and integrity.
9. Misinformation. Blind trust in AI-generated falsehoods — the seam where reliability and security meet, discussed above.
10. Unbounded Consumption. Uncontrolled resource use that leads to denial of service or "denial of wallet" — attackers inflating your AI cloud costs through excessive queries. Rate limiting and usage monitoring are the standard defense.
The OWASP Top 10 for Agentic Applications, released in December 2025 as the 2026 edition, extends this taxonomy to systems that can act, not just answer.
Arup's USD 25.6 million deepfake fraud (Hong Kong, 2024). A finance employee at the UK engineering firm received a spear-phishing email, then joined a video call where the "CFO" and several "colleagues" were all AI-generated deepfakes. Believing the instructions were legitimate, the employee transferred about HKD 200 million through 15 transactions. No internal system was compromised — this was social engineering amplified by AI. Arup's then-CIO Rob Greig said the number and sophistication of such attacks had risen sharply.
Samsung's reported ChatGPT source-code leak (2023). Within roughly 20 days of lifting an internal ban, Samsung engineers reportedly pasted proprietary semiconductor source code and confidential meeting notes into ChatGPT on three separate occasions. Samsung restricted employee use of generative AI and began preparing internal AI tools. The incident remains a widely cited example of shadow-AI data leakage.
EchoLeak, Microsoft 365 Copilot (CVE-2025-32711, disclosed 2025). Security researchers demonstrated a zero-click indirect prompt-injection path that could disclose data through a single crafted email without user interaction. The research bypassed Microsoft's prompt-injection classifier. Microsoft patched the hosted service before public disclosure, and the CVE record reports no known exploitation against customers.
The Chevrolet chatbot (2023). A dealership's ChatGPT-powered bot was talked into agreeing to sell a 2024 Chevy Tahoe for one dollar, describing the offer as "legally binding." No sale occurred and no data was breached, but screenshots spread widely online — a clear example of how prompt injection can become a public incident.
These cases share a pattern worth remembering: AI security is multiplicative. An attack rarely relies on breaking the model alone — it chains a prompt injection with an over-privileged agent, or a leaked token with a poisoned dependency. Evaluating each risk in isolation tends to underestimate what a determined attacker can actually do by combining several small weaknesses.
No single framework covers everything. Most mature security programs combine several:
| Framework | What it covers | Best for |
|---|---|---|
| OWASP Top 10 for LLM Applications | Application-level taxonomy: prompt injection, data leakage, supply chain, and more | AppSec teams building or reviewing LLM applications |
| NIST AI Risk Management Framework | Four functions — Govern, Map, Measure, Manage | Structuring an ongoing AI risk program, sector-agnostic |
| MITRE ATLAS | A living knowledge base of adversary tactics and techniques against AI, modeled on MITRE ATT&CK | Threat modeling and red-teaming; expanded by 45+ techniques in a May 2026 update |
| ISO/IEC 42001 + 27001 | Certifiable AI management system (42001) paired with information security management (27001) | Organizations wanting formal, auditable certification — already held by AWS, Anthropic, and Microsoft |
| EU AI Act, Article 15 | Legal requirement for high-risk AI systems to be accurate, robust, and resilient against manipulation | Any business deploying high-risk AI systems in the EU |
| ENISA FAICP | Three-layer framework connecting ICT foundations, AI-specific ML threats, and sector guidance | EU organizations seeking layered cybersecurity practices for AI |
One correction worth making explicitly: the EU AI Act is not a "future" law in any simple sense, and not all of its high-risk obligations are equally in force yet. General application began 2 August 2026, and obligations for general-purpose AI models started even earlier, in August 2025. But the AI Act Omnibus, in force since 27 July 2026, pushed the deadline for certain Annex III high-risk systems to 2 December 2027, and for high-risk systems embedded in regulated products to 2 August 2028. Before buying or building a high-risk AI system, classify it and your role under the Act, and ask any vendor for evidence proportional to their current obligations — not a blanket assumption that "the AI Act doesn't apply yet."
Security maturity for AI follows a sequence. Skipping ahead to sophisticated defenses before the basics are in place tends to leave the highest-impact gaps wide open.
You can't secure what you don't know exists. Inventory every AI system, model, and agent in use, including the ones nobody officially approved. A Kiteworks report says 86% of organizations lack visibility into AI data flows; it separately cites an average of around 1,200 unsanctioned applications of all types per organization. In a WalkMe survey of 1,000 US workers, 78% reported using AI tools not provided by their employer. Start by asking every team what they're actually using, not just what's on the approved list.
In a 2025 global study led by the University of Melbourne with KPMG, 34% of employees reported that their organization had guidance for generative AI use, 19% reported a ban, 41% reported no policy or guidance, and 6% did not know. Write a policy people can follow, since an impractical ban can push usage further into the shadows. Assign clear ownership: security accepts and manages cyber risk, an AI governance lead sets policy and classifies use cases, the DPO owns personal-data handling, and MLOps owns the technical pipeline. Adopt a framework — NIST AI RMF or ISO/IEC 42001 both work — rather than building AI governance as a parallel, disconnected process from the rest of your risk management.
Give AI agents the minimum privilege the task requires, never more. Validate every output before it touches a downstream system. Build RAG retrieval so it respects the same access permissions a human user would have — a document being indexed doesn't mean everyone should be able to retrieve it. And where you can, turn on your AI vendor's stricter data-handling settings: zero-data-retention options, no-training defaults, and enterprise admin controls exist specifically to reduce this exposure.
Test your defenses instead of assuming they hold. Red-team your AI systems the way you'd red-team any other critical infrastructure. Log enough to reconstruct what happened after an incident — which identity, which model version, which documents were retrieved, which actions an agent took autonomously. And write an AI-specific incident response plan before you need one; the questions a generic playbook won't think to ask ("what exact model and data version was active?", "what did the system do on its own after the malicious input?") are often the ones that determine whether you can even reconstruct an incident.
The legal position differs sharply by market: the EU already has a binding, risk-based framework; El Salvador combines a promotion law with registration rules; Colombia mixes policy guidance with targeted criminal penalties; and the United States remains a state-by-state compliance exercise.
| Jurisdiction | Current legal position | What businesses should do |
|---|---|---|
| Spain + EU | The EU AI Act is binding, although obligations phase in by role and system type. AESIA is a Spanish market-surveillance authority; Spain's wider national governance bill remained pending in August 2026. | Classify your role and each AI system's risk. Document cybersecurity for high-risk AI and coordinate AI Act controls with GDPR, NIS2 and, where applicable, DORA or the Cyber Resilience Act. |
| El Salvador | Decreto 234 created ANIA and made security and privacy core principles. ANIA Resolution 0001/2025 adds registration for covered consequential-decision AI systems. | Identify systems used for consequential decisions and register those covered in areas such as credit, insurance or real-time biometric identification in public spaces. |
| Colombia | CONPES 4144 is a non-binding AI roadmap through 2030. Ley 2502 of 2025 is a binding criminal-law amendment covering AI-enabled identity fraud; broader AI legislation remains in development. | Use the roadmap's risk-mitigation approach, strengthen identity and deepfake controls, and monitor the progress of broader binding rules. |
| United States | There is no single comprehensive federal AI security law. Requirements are fragmented across state laws, with different rules for developers, deployers and automated decisions. | Map obligations by state and use case. Track Texas TRAIGA defenses, California transparency, frontier-model and automated-decision rules, and Colorado's requirements beginning in 2027. |
Spain and the EU. Spain designated AESIA (Agencia Española de Supervisión de la Inteligencia Artificial) as an AI Act market-surveillance authority. AESIA supervises applicable AI Act rules, while Spain's national bill allocating the wider governance and sanctioning regime was still before Parliament in August 2026. Article 15 of the EU AI Act requires high-risk systems to reach "an appropriate level of accuracy, robustness and cybersecurity" and explicitly addresses data poisoning, model poisoning, adversarial examples, and confidentiality attacks — see the framework section above for the current Omnibus-adjusted timeline. For most EU businesses, AI security sits alongside GDPR, which requires notification of qualifying personal-data breaches within 72 hours where feasible, NIS2 for essential and important entities, and, in regulated sectors, DORA or the Cyber Resilience Act. Our EU AI Act guide covers the full compliance picture.
El Salvador. Decreto 234 (the Ley de Fomento a la Inteligencia Artificial y Tecnologías, enacted February 2025) names "security and privacy" as a core principle and created ANIA, the national AI agency. ANIA Resolution 0001/2025 requires registration for AI systems that make "decisiones consecuenciales" — consequential decisions — in areas like credit, insurance, or real-time biometric identification in public spaces.
Colombia. CONPES 4144 (February 2025) sets a national AI roadmap through 2030 with an explicit risk-mitigation axis, though it's policy guidance rather than binding law. Ley 2502 of 2025 already has teeth: it amended the criminal code to add an aggravating factor — up to a one-third penalty increase — for identity fraud committed using AI, including deepfakes. Broader binding AI legislation remains in the pipeline.
United States. As of August 2026, no single federal AI security law replaces the growing set of state statutes. Texas's TRAIGA, effective 1 January 2026, provides several defenses against liability. These include discovering a violation through adversarial or red-team testing, or substantially complying with the NIST Generative AI Profile or another recognized framework and maintaining an internal review process; the statute presents these as alternatives, not one combined checklist. Colorado repealed its original AI Act before it took effect and replaced it with SB 26-189, a narrower automated-decision-making law that takes effect 1 January 2027 alongside the state's Chatbot Safety Act. California's AI Transparency Act (SB 942) became operative on 1 January 2026. A separate law, the Transparency in Frontier Artificial Intelligence Act (SB 53), governs large frontier-model developers. California's CCPA automated-decision-making requirements follow in January 2027. At the federal level, Executive Order 14365 directed federal action against state AI laws the administration viewed as onerous, and the March 2026 National AI Legislative Framework called for congressional action; neither document itself replaces state statutes. The financial stakes remain high: IBM reported a USD 10.22 million average US breach cost in 2025, while Deloitte projected that US generative-AI-enabled fraud losses could reach USD 40 billion by 2027.
The comparison below covers business and enterprise products. Consumer plans can have different training, retention, and human-review settings, so confirm the terms for the exact product your organization buys.
| Platform | Business data handling | Security and admin controls |
|---|---|---|
| Commercial-product inputs and outputs are not used for training by default. Explicit feedback or opt-in are exceptions. Approved Enterprise API customers can negotiate Zero Data Retention, but its scope does not automatically include Claude for Work or other stateful products. | SOC 2 Type II, ISO/IEC 27001, and ISO/IEC 42001. Organization owners can disable member feedback submission; retention and control coverage still need to be checked per product. | |
| Organization data is not used to train models by default. Qualifying organizations can configure retention, including Zero Data Retention for supported API use cases. | AES-256 encryption at rest, TLS 1.2+ in transit, optional Enterprise Key Management, SSO, role-based access, admin and audit-log APIs, plus SOC 2 Type 2 and ISO certifications. | |
| Customer data is not used to train foundation models. Copilot operates inside the Microsoft 365 service boundary, while interaction retention and deletion follow configured Microsoft Purview policies. | Existing permissions, sensitivity labels, encryption, DLP, and SharePoint or OneDrive access controls carry into Copilot. Prompts, responses, and referenced content can be audited through Purview. | |
| Workspace business data is not used to train Gemini models or target ads. Admins can control Gemini conversation history and choose 3-, 18-, or 36-month retention; when history is off, chats can still be retained for up to 72 hours to provide the service. | Workspace permissions, DLP, and data-region controls apply. Gemini also uses content classifiers, model hardening, URL redaction, and user confirmation for sensitive actions, with SOC and ISO certifications. |
The AI platforms businesses already use offer enterprise controls, but the exact terms depend on the product and contract. Anthropic does not train on commercial-product inputs or outputs by default, and approved enterprise API customers can negotiate Zero Data Retention; that arrangement does not automatically cover Claude for Work or other stateful products. Anthropic holds SOC 2 Type II, ISO/IEC 27001, and ISO/IEC 42001 certifications. OpenAI excludes business data from model training by default and offers custom retention controls for qualifying products and organizations. Microsoft 365 Copilot does not use customer data to train foundation models and lets administrators govern stored interactions through Microsoft Purview retention policies. Google applies layered defenses against indirect prompt injection in Gemini, including content classifiers, model hardening, URL redaction, and user confirmation for sensitive actions.
The gap most businesses face isn't the absence of these controls — it's that they're rarely configured correctly, monitored, or connected to the rest of the security stack. That's the part that needs deliberate engineering, not a vendor default.
AI security risks affect the confidentiality, integrity, availability, or control of an AI system. Many are adversarial, but insecure configuration and accidental disclosure can also create them. Hallucinations are false outputs generated without malicious input. The two intersect when a person or automated system acts on a hallucination in a way that exposes data, executes unsafe code, or causes another security impact.
Because it exploits how LLMs process language — they don't reliably separate instructions from data in the same input stream. The UK's NCSC has described it as a problem that may never be fully patched, which is why defense-in-depth (input validation, output filtering, least privilege, human review) matters more than any single fix.
Yes. WalkMe found that 78% of 1,000 US workers used AI tools not provided by their employer. IBM found that organizations with high shadow-AI use recorded breach costs USD 670,000 higher on average than organizations with little or no shadow AI. The fix isn't banning AI outright — it's offering approved tools that are secure and useful enough for people to adopt.
Start with the OWASP Top 10 for LLM Applications if you're building AI applications, and NIST AI RMF if you need a broader governance structure. Both are free and widely referenced. ISO/IEC 42001 certification is worth pursuing once your program matures and you need to demonstrate compliance formally to customers or regulators.
Obligations depend on your role and the risk classification of your system. General-purpose AI model obligations began in August 2025; general application of the Act started 2 August 2026; but the AI Act Omnibus pushed several high-risk obligations to December 2027 and August 2028. Classify your system and role before assuming any specific deadline applies — or doesn't.
Start with a free 30-minute AI discovery session. We identify your highest-value automation opportunity and explain exactly how Liorant can help — no slides, no pitch.
Book your session