Prompt injection
Prompt injection is a critical vulnerability where attackers manipulate the input prompts to alter the chatbot's behavior, potentially exposing sensitive data or executing unauthorized actions. This can lead to severe security breaches and operational disruptions.
Context leakage
detectable with Probe
Context leakage in AI chatbots refers to the unintentional exposure of sensitive internal documents, intellectual property, and the system prompt of the chatbot itself. The system prompt is crucial as it guides the chatbot’s behavior and responses. If leaked, it can provide detailed insights into the chatbot’s operational parameters and proprietary algorithms. Such leaks are critical because they can allow competitors to replicate proprietary solutions, leading to significant competitive disadvantages. Additionally, context leakage can serve as a gateway for further adversarial activities, amplifying the overall risks to the organization and compromising the security and integrity of the chatbot system.
Fake news
detectable with Probe
Fake news generation through chatbots poses a medium-level risk, primarily impacting brand reputation and user trust. When a chatbot disseminates false information, it can be used to manipulate public opinion against local authorities or the brand itself. This not only degrades the user experience but also potentially tarnishes the brand's image, leading to long-term reputational damage and loss of customer loyalty.
Breaking Prompt Length Limit
detectable with Probe
Exceeding the prompt length limit is a high-risk issue that can serve as a catalyst for various adversarial activities. It can lead to denial of service (DoS) and denial of wallet (DoW) attacks, where legitimate users are unable to access the chatbot due to resource exhaustion. Additionally, it can result in increased operational costs as the system struggles to handle the excessive input, thereby draining the budget allocated for customer service operations.
Jailbreak
detectable with Probe
Jailbreaking involves manipulating the chatbot to bypass its preset operational constraints, posing a high risk. This vulnerability can open the door to various malicious activities, allowing attackers to exploit the chatbot for unintended purposes. Once the chatbot is compromised, it can be used to disseminate harmful information or perform unauthorized actions, significantly jeopardizing the security and integrity of the system.
Model leakage
detectable with Probe
Model leakage is a critical security threat in AI chatbots, involving the unintended exposure of the underlying model’s architecture, parameters, training data, personal user data as well as proprietary company data. This can occur through sophisticated prompt injection attacks where malicious actors manipulate the chatbot to reveal sensitive details about its construction and functioning. Model leakage can lead to significant competitive disadvantages, as adversarials might replicate or manipulate the exposed model for their own purposes and increase the risk for further adversarial attacks.
Off-topic
Off-topic conversations occur when a chatbot strays from its intended use, engaging in irrelevant or inappropriate discussions. This not only degrades user experience but can also introduce security vulnerabilities.
Off-topic discussion
detectable with Probe
Off-topic discussions can steer the conversation away from the user's original intent, leading to a medium risk of poor user experience. When the chatbot engages in irrelevant dialogue, it fails to address the user's needs effectively, resulting in frustration and decreased satisfaction with the service.
Intentional Misuse
detectable with Probe
Intentional misuse of chatbots by users can lead to a medium-level risk involving unexpected behaviors and security threats. Such misuse can strain resources, causing denial of service for legitimate users. Additionally, it introduces security risks as unforeseen prompts may lead to unintended and potentially harmful responses from the chatbot.
Competition infiltration
detectable with Probe
Competition infiltration occurs when users are redirected to a competitor's services, representing a medium risk. This can result in direct revenue loss as potential customers are diverted away. The risk extends to potential leaks of competitive intelligence, where sensitive business strategies might be inadvertently exposed.
Comparison
detectable with Probe
Comparisons made by the chatbot between different countries or entities can lead to high-risk discussions on sensitive topics. Such comparisons might provoke negative sentiments or controversies, potentially damaging the brand's reputation and causing friction with local authorities or other stakeholders.
Exploiting rail aggression limit
detectable with Probe
Exploiting the chatbot's aggression limits is a high-risk issue. Over-aggressiveness can result in disabled features and a decrease in user engagement, leading to potential revenue loss. Conversely, insufficient aggressiveness can make the chatbot vulnerable to adversarial attacks and misuse, compromising its effectiveness and security.
Toxicity
detectable with Probe
Toxicity in chatbots is a high risk that involves the generation of harmful, abusive, or offensive content. Toxic responses can severely damage the user experience and brand reputation, leading to user distress and disengagement. This issue is particularly critical because it can escalate into broader public relations issues and legal challenges, if the chatbot’s toxic behavior is widely reported or causes significant harm.
Bias
detectable with Probe
Bias in AI chatbots represents a high risk issue, as it can lead to the dissemination of prejudiced or discriminatory information. When a chatbot exhibits bias, it can inadvertently reflect and amplify societal prejudices, impacting the user experience negatively. This can result in user alienation and loss of trust, as well as potential legal repercussions for discriminatory behavior. Additionally, biased response can tarnish the brand’s reputation and lead to public backlash, emphasizing the need for vigilant monitoring and mitigation strategies to ensure fair and unbiased interactions.
Hallucination
AI hallucinations refer to instances where a chatbot generates responses that are factually incorrect or nonsensical. These errors can mislead users and undermine the reliability of the AI system.
Relevancy
detectable with Probe
Relevancy issues, where the chatbot provides information that is not pertinent to the user's query, pose a medium risk. This can lead to a subpar user experience, as users may become frustrated with irrelevant responses, reducing their overall satisfaction and trust in the service.
Domain-specific errors
detectable with Probe
Domain-specific errors occur when the chatbot provides incorrect information within specialized fields, posing a high risk. Such inaccuracies can cause significant harm to users who rely on the chatbot for precise and reliable information, leading to potential legal liabilities and loss of credibility.
Model-language precision
detectable with Probe
Imprecise language models in chatbots present a medium risk, as they increase the likelihood of legal disputes arising from incorrect offers or misunderstandings. Users may seek refunds or compensation for misleading information, leading to financial and reputational damage for the business.
Generating non-existing info
detectable with Probe
The generation of non-existing information by chatbots is a medium-risk issue that primarily affects user experience. When users receive fictitious or fabricated details, it erodes their trust and satisfaction with the service, ultimately diminishing the chatbot's effectiveness and reliability.
Citation/URL/Title Check
detectable with Probe
This risk refers to the chatbot providing inaccurate or fabricated references, URLs, or titles, posing a medium risk. When a chatbot generates false citations or links, it can mislead users and spread misinformation. This not only deteriorates the user experience but also can have serious legal and reputational consequences, if the misinformation leads to significant harm or public backlash.
Social engineering
detectable with Probe
Social engineering through chatbots is a high-risk threat, as it exploits the trust and naivety of average users. Attackers can manipulate the chatbot to deceive users into divulging personal or sensitive information. This method is particularly dangerous because it leverages human psychology, making it one of the easiest yet most effective ways to harm users and compromise their security.