Top 10 Vulnerabilities for Large Language Models

When considering the efficacy of large language models (LLMs) for AI training, there are a lot of factors to bear in mind. You’ll likely be familiar with insecure output handling in LLMs, which was highlighted as part of OWASP’s top 10 for LLM applications. Here, let’s discuss this and other vulnerabilities of LLMs.

Prompt Injections

Prompt injections are a method of carefully wording the prompt for an AI, and in so doing, getting it to output something undesirable. Perhaps a famous version of this is the ‘grandmother’ technique, which was used to manipulate chat GPT into revealing a method of making napalm.

Insecure Output Handling

This is a fairly self-explanatory vulnerability: the output of the LLM may not be properly checked and validated before being passed along through the program. This could result in confused data, or even improper data types getting inputted into variables.

Training Data Poisoning

Another self-explanatory issue of LLMs, training data poisoning occurs when a bad actor deliberately influences the training data. This could be done to introduce bias or backdoors into the final iteration of the LLM.

Model Denial of Service

This is perhaps slightly more difficult to achieve than other items on the list. Model denial of service is the practice of a user deliberately asking an LLM a question that would take up a vast amount of computing resources. This may be done to deliberately reduce the overall quality of service for users.

Supply Chain Vulnerabilities

Many LLMs are used as part of a long chain of hardware and software. For instance, a company may use an open-source AI as a customer service bot on its website without knowing what the training data has been or where it may have come from. This obfuscation of the supply chain could lead to biased outcomes and security breaches.

Sensitive Information Disclosure

During the process of training the LLM in question and ensuring that it has access to all the relevant data it may need for an organization’s highest-level employees, there’s a chance it may interact with sensitive data. If this is the case, the LLM may store this sensitive data and could share it with unsuitable users in the future. Data sanitization is important to avoid this.

Insecure Plugin Design

Generally speaking, the plugins used for LLMs are designed and maintained entirely by third parties. Furthermore, these plugins are typically called and accessed by the AI itself rather than by the user or any other element of the application.

Because there’s no room for human intervention in that decision chain, there’s a chance that the AI could be manipulated into calling an insecure plugin. This is bad enough if it’s handing over sensitive data, but it could be as simple as the user calling a malware function that they have designed.

Excessive Agency

As LLMs develop more and more with time, companies want to use this valuable resource to do an increasing number of things. While the concept of a near-free employee with a lot of skills is very exciting to many people, they run the risk of the AI being given too much agency. If this is the case, it’s somewhat akin to building a house of cards. It may be stable for quite a while, but a bad actor could manipulate the LLM and bring the whole company down.

Overreliance

Overreliance on AI output is an issue when an LLM is capable of creating and distributing fictional information. While it may be fun to ask an LLM to write a story, it can lead to the model generating false or misleading text in response to prompts. If there’s no validation on the data, and this data is used, then a great deal of issues can arise, from simple confusion to legal issues.

Model Theft

Through conversing with an LLM, it’s theoretically possible for a bad actor to learn enough information to recreate the model. This could lead to a business losing a competitive advantage and possibly even to the user getting access to the sensitive information contained within the model.

While LLMs have a lot of potentially great applications, they also have some vulnerabilities. Ensuring that we’re aware of these potential issues allows us to create more robust solutions.