Generative AI Risks & Vulnerabilities

Employees use dozens of different GenAI tools in their daily operations, most of them unbeknownst to their IT teams. Key concerns are limited visibility, absence of governance, compliance risk, and data exposure.

The risk of sensitive information disclosure has become increasingly significant in the era of Generative AI: whether it's employees exfiltrating company data to GenAI tools or LLM-based applications revealing sensitive data.

Prompt Injection is a cybersecurity threat where attackers manipulate a large language model (LLM) through carefully crafted inputs.

Indirect Prompt Injection occurs when a LLM processes input from external sources that are under the control of an attacker.

A jailbroken LLM behaving unpredictably can pose significant risks, potentially endangering an organization, its employees, or customers if it outputs toxic, biased or harmful content.

Denial of Wallet attacks, alongside Denial of Service, are critical security concerns where an attacker excessively engages with an LLM-based apps leading to substantial resource consumption.

Jailbreaking represents a category of prompt injection where an attacker overrides the original instructions of the LLM, deviating it from its intended behavior and established guidelines.

The emergence of GenAI technologies and the accompanying regulatory frameworks is raising substantial legal concerns within organizations.

Prompt Leak is a specific form of prompt injection where a Large Language Model (LLM) inadvertently reveals its system instructions or internal logic.

The non-deterministic nature of LLMs poses significant risks to your brand reputation when exposing users to your GenAI apps.

A potential attacker can construct a malicious request to an LLM plugin, which could result in a wide range of undesired behaviors, up to and including remote code execution.‍

As organizations integrate LLMs with more and more tools within the organization, like databases, APIs, and code interpreters, the risk of privilege escalation increases.