Adversarial machine learning studies how AI models can be misled by deliberately manipulated inputs or training data – and how to protect them. Well-known examples are barely visible image changes that flip a classification, or poisoned training data. The topic matters for organisations as soon as AI supports security-relevant decisions. Countermeasures include robust training, input checks, monitoring and regular security testing.
Model robustness describes how reliably an AI model performs when inputs are noisy, incomplete, unusual or deliberately manipulated. A robust model still delivers stable results, such as image recognition in poor lighting or a language model handling typos. Robustness is assessed with stress tests, diverse training data and tests against deliberate deception attempts, and it is particularly important in safety-critical applications.
AI red teaming is the structured examination of an AI system from the perspective of potential attackers or misuse. Specialists test in a controlled way whether a chatbot reveals confidential instructions, can be led into unwanted behaviour, provides false facts or exposes data. The findings feed into better instructions, filters and permissions. A chatbot security audit is a lighter form: it assesses configuration and sample conversations against known risk areas such as the OWASP Top 10 for LLM Applications.
LLM security covers all measures that protect applications built on large language models from misuse, data leakage and unwanted behaviour. Typical risks include prompt injection, disclosure of confidential instructions or data, and agents with excessive permissions. Protection comes from several layers: clear instructions, restricted rights, checks on inputs and outputs, AI firewalls, red teaming and regular audits.
Training data are the examples from which an AI model learns patterns: large text collections for language models, labelled images for image models. Their quality, diversity and recency largely determine how reliable and fair a model is and how up to date its knowledge will be. For organisations it also matters whether a provider uses the customer data they enter for further training.
