Thread by @JorisTechTalk

AI and Safety:

@owasp has released a list of the top 10 most critical vulnerabilities found in artificial intelligence applications based on large language models (LLMs).

These vulnerabilities include prompt injections, data leakage, and unauthorized code execution.

A 🧵

#AI

1. Prompt injections:

This involves bypassing filters or manipulating the LLM using carefully crafted prompts that make the model ignore previous instructions or perform unintended actions.

2. Data Leakage:

Data leakage occurs when an LLM accidentally reveals sensitive information through its responses. #cybersecurity

3. Inadequate sanboxing:

Inadequate sandboxing can lead to potential exploitation, unauthorized access, or unintended actions by the LLM.

4. Unauthorized code execution:

Unauthorized code execution occurs when an attacker exploits an LLM to execute malicious code, commands, or actions on the underlying system through natural language prompts.

5.Server-side request forgery vulnerabilities:

Server-side request forgery (SSRF) vulnerabilities occur when an attacker exploits an LLM to perform unintended requests or access restricted resources such as internal services, APIs, or data stores.

6. Overreliance:

Overreliance on LLM-generated content can lead to the propagation of misleading or incorrect information.

7. Inadequate AI alignment:

Inadequate AI alignment occurs when the LLM’s objectives and behavior do not align with the intended use case, leading to undesired consequences or vulnerabilities.

8. Insufficient access controls:

Insufficient access controls occur when access controls or authentication mechanisms are not properly implemented.

9. Improper error handling:

Improper error handling occurs when error messages or debugging information are exposed in a way that could reveal sensitive information or potential attack vectors to a threat actor.

10. Tranining data poisoning:

Training data poisoning is when an attacker manipulates the training data or fine-tuning procedures of an LLM to introduce vulnerabilities, backdoors, or biases.

Security leaders/teams and their organizations are responsible for ensuring the secure use of generative AI chat interfaces.

Regular updates and human oversight are essential to ensure LLMs function correctly and catch any security issues.