大語言模型中的提示注入攻擊分析與緩解策略

Analysis and Mitigation Strategies for Prompt Injection Attacks

李昆積(Kun-Ji Li)；陳勝舢(Sheng-Shan Chen)；孫勤昱(Chin-Yu Sun)

大語言模型 ； 提示注入 ； Large Language Model ； Prompt Injection

資訊安全通訊

30卷3期（2024 / 08 / 01）

1 - 19

繁體中文;英文

近年來，大語言模型憑藉著強大的自然語言處理能力，受到了廣泛關注，並迅速的被應用於眾多領域，對日常生活與工作流程帶來了巨大的改變。隨著應用範圍的增加，相關的風險也隨之增加。提示注入（Prompt Injection, PI）攻擊正是其中一種威脅。著名的非營利組織OWASP已將PI攻擊列為大語言模型中最具威脅的攻擊之一。攻擊者利用精心設計的輸入，讓包含大語言模型的系統執行開發者預料之外操作，進而達成危害。這類攻擊不僅會增加提供服務方的運營成本，還可能對用戶安全構成威脅。包括但不限於洩露敏感資料、執行未經授權操作與生成有害內容等。本研究分析現有PI攻擊方式及其對大語言模型應用系統的危害，並探討常見的緩解策略。我們整理了一些攻擊與緩解策略，並選擇了一個串接GPT 3.5 API的大語言模型系統來進行驗證，為開發者提供PI攻擊與緩解的實踐參考。對大語言模型相關應用系統的安全設計提供了寶貴的實踐經驗，有助於提升系統的安全性與穩定性。

In recent years, Large Language Models (LLM) have garnered significant attention due to their powerful natural language processing capabilities and have rapidly been applied across various fields, bringing substantial changes to daily life and workflows. However, as their application scope expands, so do the associated risks. Prompt injection (PI) attacks have emerged as one of the most critical threats in this context. The prominent nonprofit organization, Open Web Application Security Project (OWASP), has identified PI attacks as one of the most dangerous attack vectors in LLM systems. Attackers craft carefully designed prompts that cause systems incorporating LLM to execute unintended operations, leading to potential harm. These attacks not only increase the operational costs for service providers but may also threaten user security, resulting in unauthorized data disclosure, execution of unauthorized actions, and generation of harmful content. This study analyzes existing PI attack techniques and their impacts on LLM systems while exploring common mitigation strategies. We compile various attacks and defenses, selecting an LLM system that integrates the GPT-3.5 API for validation, providing developers with practical insights into PI attacks and mitigation measures. This research offers valuable practical experience for the secure design of LLM applications, helping enhance system security and stability.