Data Poisoning Attacks on LLM leveraged Product Support Bots: Attack, Risk, Prevention

Background: Because industry started leveraging AI capabilities for routine cases like general customer support, a new threat has emerged: data poisoning.

Core attack category: Based on MITRE ATLAS, it is categorized as AML.T0020 (Poison Training Data). This type of data poisoning can have a significant impact on the product side because models may perform inadequately as a result.

Simple Attack Demo: Let’s imagine that our company has an AI-powered chatbot for customer support. Based on data provided by customers, we train our model.

Where is the risk in this case? At minimum, an attacker could inject information claiming that product ratings or prices are lower than they really are, or provide other misleading product-related information.

What the attacker did was flood the chat with misleading information, which creates a chance that the model could learn from it. Since all user input can become a source of training data, this can corrupt the model’s behavior.

Risk: 1. Fake information spreading 2. Malicious link injection 3. Reputation damage 4. Compliance issues

Prevention: 1. Put Access Control on production model training scope 2. Add rate limit on public exposed components 3. Sanitize data to detect offensive language from beeing used . 4. Filter data before train production model 5. Always use version control for the LLM model , to be able to revert back to old one version .

Conclusion: Data poisoning is a serious risk in AI-powered support systems because untrusted user input can corrupt training data and degrade model behavior. To reduce exposure, organizations should tightly control who can influence training, rate-limit public inputs, sanitize and filter data before training, and keep model versions under strict version control so they can quickly roll back if poisoning is detected.