Jump to contentJump to page navigation: previous page [access key p]/next page [access key n]
documentation.suse.com / Maintaining Data Integrity of AI Applications

Maintaining Data Integrity of AI Applications

Publication Date: 19 Dec 2024
WHAT?

Maintaining the security of the entire AI stack environment.

WHY?

To learn how to prevent data leaks and maintain the correct functioning of your AI application.

EFFORT

Understanding the security threats and safety measures for running AI services requires less than 30 minutes of your time.

GOAL

Understand how attackers can exploit your AI stack to access sensitive data and learn safety techniques to prevent such attacks.

1 Data integrity for AI applications

 

This topic describes how to avoid compromising AI applications and keep sensitive data secure.

1.1 Why care about the security of AI applications?

 

AI applications use AI-driven chatbots to interact with users. These chatbots are powered by large language models (LLMs) and can process external data sources (RAGs). Such applications are prone to cyber attacks as any other software solutions. Attackers may impersonate users and apply a series of techniques to steal data and to corrupt the responses provided by AI models.

1.2 Which SUSE AI components are prone to attacks

 

Users interact with SUSE AI via the Open WebUI user interface. With Open WebUI, you can manage users, permissions, AI models, knowledge bases, and chat interactions. The following SUSE AI components are the most susceptible to security attacks:

Open WebUI

Open WebUI enables you to specify external data sources to improve responses. On a user level, you can append documents directly to the chat input field. With administrator privileges, you can upload documents to create a knowledge base that enhances the AI model. The knowledge base acts as a domain-specific augmentation tool for the LLM. It prevents chatbot hallucination and improves the model's responses with accurate and up-to-date information.

Tip
Tip

Actions performed by users—both the administrators and guests—are recorded in an audit log. With the audit log, it is possible to map all actions that took the system to its current state.

Milvus

It is possible to input documents directly into Milvus—the vector database responsible for the low-level implementation of the knowledge base concept. Although the user interaction normally takes place via Open WebUI, attackers may bypass Open WebUI to interact with Milvus. This can happen if no identity access management (IAM) policy is controlling database access.

Ollama

Ollama manages the interactions with several LLMs. It can search, download, start and manage models within a unified interface. Ollama does not offer authentication and authorization by default. Therefore, Ollama's API should not be exposed without a an element providing IAM capabilities. In SUSE AI, user interacts with Ollama via Open WebUI, which is able to configure and secure Ollama.

1.3 What are common attacks and security risks?

 

This section lists several attacks and security risks related to AI applications.

RAG poisoning

A common exploit when a knowledge base—often a vector database—that provides a context for AI model responses is corrupted by the addition of misleading, false or even harmful content. The malicious documents tend to be crafted specifically to provide wrong answers for a set of user prompts. This kind of attack usually requires access to a user with privileges to configure the whole platform or the vector databases that support the platform.

Facilitated data exploits

RAG-powered models can search knowledge bases, summarize content and provide references. Attackers may use these characteristics to discover and retrieve organizational data with simple prompts instead of relying on more refined data-exploitation techniques.

Prompt leaks

User prompts may contain sensitive data, so chat caches and system logs need to be protected against attackers.

1.4 What safety measures should my organization follow?

 

To avoid having your system corrupted, there are a few security measures that need to be properly implemented. Open WebUI and Milvus allow high level user access configurations. Besides these high level configurations, provide access management with low level network configurations. To verify that your whole AI stack is secure, consider the following points:

Adopt strong IAM policies.

  • At the authentication level, limit the creation of guest users.

  • At the authorization level, never allow new users to be automatically set as system administrators.

  • Limit the number of users with privileges for adding documents to your knowledge base.

  • Keep in mind that the same policies set for Open WebUI need to be propagated in all systems composing the AI stack.

  • Limit the exposure of internal services (such as Milvus and Ollama) to the Internet.

  • Configure authentication and authorization for all components of the AI stack.

Adopt an audit log review policy.

Periodically check the audit logs provided in the Web interface, from both the chatbot and the vector databases. Look for abnormal behavior from one or more users.

Set up retention policies.

Avoid saving chat prompts and system logs.

Train users to avoid LLM overreliance.

Encourage users to approach the answers from the RAG-based models with a critical mindset. Make sure they are able to verify the references provided by the AI chatbot. Files used in the context of a user prompt are appended to the AI model's answer.

Facilitate incident reports.

Educate your users about how to report problems with the AI model's answers. Assign responsibilities for the system support.

Ensure fast action against attacks.

Remember that when a security breach happens, the sooner the system is restored to a trusted state, the less damage your organization takes.

Set up a secure environment for your applications.

Make sure that there are components enforcing authentication and authorization rules over the whole installation of the AI Stack. We do not recommend exposing Milvus and Ollama without proper network configurations.

1.5 For more information