Article Series: An Introduction to AI in context of business

  1. What are AI Agents ↗
  2. AI 'Brains': Large Language Models ↗
  3. Prompt Engineering for Business ↗
  4. Tool-Enabled AI Systems ↗
  5. Retrieval-Augmented Generation (RAG) ↗
  6. AI Memory Systems ↗
  7. Context Engineering ↗
  8. AI Orchestration and Workflows â†—
  9. Multi-Agent Systems ↗
  10. AI Quality Assurance

Traditional software either works or it doesn't. Enter your password incorrectly and access is denied, every single time. The behaviour is deterministic and predictable.

AI systems, particularly those built on Large Language Models (LLMs), operate differently. Ask the same question twice and you will receive two different answers, both of which could be plausible, partially correct, or entirely fabricated. This non-deterministic behaviour creates a fundamental challenge: how do you test something that doesn't behave the same way twice?

For businesses deploying AI agents, customer service chatbots, or automated content generation systems, this question isn't academic. It's the difference between a system that builds trust and one that erodes it, between efficiency gains and expensive mistakes, between competitive advantage and reputational damage.

AI Quality Assurance (QA) isn't simply traditional testing with a new name. It requires different thinking, different tools, and an acceptance that 'perfect' might not be achievable, but 'reliably good enough' absolutely must be.

Why It Matters

The consequences of poor AI quality extend beyond embarrassing chatbot responses or slightly odd email suggestions. When AI systems make decisions about customer queries, financial recommendations, medical information, or operational processes, errors compound rapidly.

A traditional software bug affects everyone the same way and can be reproduced, diagnosed, and fixed. An AI system might work perfectly for 999 queries and catastrophically fail on the 1000th, with no obvious pattern to explain why. It might perform brilliantly during testing and deteriorate over time as the world changes around it. 

Without robust QA processes, businesses face several risks:

  • Hallucinations: When an AI confidently generates plausible but false information. This happens because language models predict the most likely next word rather than fact-check the previous one.
  • Inconsistency: Models can give different answers to the same question, depending on context or phrasing.
  • Drift: Performance decline over time as data changes or user patterns evolve. What worked in January may not hold by July.
  • Bias & Fairness issues: Subtle data or design biases can creep in, affecting business decisions or customer outcomes.

The challenge is compounded because AI failures often fail 'gracefully'. The system doesn't crash or throw error messages; it simply produces plausible-sounding rubbish. Users might not even realise they've received incorrect information until the consequences emerge later.

Business Context

For traditional software, testing is straightforward: given a set of inputs, the outputs are either right or wrong. AI systems are fuzzier. ‘Right’ often lives on a sliding scale of relevance, clarity and accuracy. This means QA must evolve. Leading organisations are now treating AI QA as a continuous process, incorporating evaluation datasets, human review pipelines, benchmark scoring, and performance dashboards into their product lifecycle.

Consider a customer service chatbot. Traditional testing might verify that it responds to queries, integrates with your knowledge base, and escalates appropriately. But does it maintain consistent tone across conversations? Does it handle ambiguous questions appropriately? Does it avoid making promises your business can't keep?
Does it recognise when it's uncertain versus when it's confident?

The reality is that AI QA requires investment in three areas:

  • Pre-deployment testing - catches obvious problems and establishes baseline performance.
  • Ongoing monitoring - detects when performance degrades or if unexpected patterns emerge.
  • Continuous evaluation - measures whether the system still meets business objectives as circumstances change.

Testing Strategies for AI Outputs

Traditional software testing relies on expected outputs: given input X, the system should produce output Y. AI testing requires a different approach because the 'correct' output often isn't singular or precisely definable.

Effective AI testing typically employs several complementary strategies:

  • Unit Testing: Even AI systems have deterministic components. Database queries, API integrations, and data processing logic can be tested conventionally. This catches mechanical failures separate from AI-specific issues.
  • Output Quality Evaluation: Rather than expecting specific text, evaluate whether outputs meet quality criteria. Is the response relevant? Is it appropriately formatted? Does it address the question asked? This often requires automated scoring systems or human evaluation against rubrics.
  • Golden Dataset Testing: Create a curated set of test cases representing typical, edge-case, and problematic scenarios. Regularly run your AI system against this dataset and track performance over time. This provides a consistent benchmark for detecting degradation.
  • Adversarial Testing: Deliberately try to break your system. Test with ambiguous queries, contradictory information, inappropriate requests, and edge cases. See what happens when users try to manipulate the system or cause it to behave inappropriately.
  • Red Teaming: Have a dedicated team attempt to find failures, security vulnerabilities, or ways to extract sensitive information or cause inappropriate behaviour. This adversarial approach often uncovers issues that normal testing misses.
  • A/B Testing: Compare different prompts, models, or configurations against each other using real or realistic scenarios. Quantify which approach performs better according to your business metrics.

The key difference from traditional testing is accepting that you're evaluating statistical performance across many examples rather than verifying deterministic behaviour in individual cases.

Consistency and Reproducibility

AI systems face a peculiar challenge: they're designed to be somewhat random. Temperature settings and sampling methods deliberately introduce variability to make outputs more natural and creative. But businesses often need consistent behaviour.

Consider a chatbot answering policy questions. You want the core information to remain consistent even if the exact phrasing varies. You don't want the system to say refunds are available within 30 days on Monday and 14 days on Tuesday.

Managing this requires several approaches. Setting temperature to zero (or very low values) makes outputs more deterministic, though potentially more stilted. Using fixed random seeds for testing allows reproducible outputs during evaluation even if production uses variability. Implementing response templates or guardrails ensures critical information remains consistent regardless of phrasing variations.

For high-stakes applications, some organisations run the same query multiple times and compare outputs to ensure consistency in key information. If running the same query five times produces five different answers, that's a quality problem requiring investigation.

Version control becomes critical. AI models themselves change, but so do prompts, system instructions, and integration logic. Tracking exactly what configuration produced specific outputs allows reproducibility and rollback when problems emerge.

Model Drift and Monitoring

AI systems don't age gracefully. Even if you change nothing, performance can degrade over time through a phenomenon called model drift.

Data drift occurs when the real-world data your AI encounters changes from the patterns it was trained on or evaluated against. A customer service chatbot trained on pre-pandemic queries might struggle with post-pandemic customer concerns. A system trained on British English might encounter unexpected slang or regional variations.

Concept drift happens when the relationships between inputs and desired outputs change. Product names change, policies update, business processes evolve. If your AI system isn't updated accordingly, it provides increasingly outdated information.

Model drift is particularly insidious because the system appears to function correctly - it still generates responses, completes tasks, and integrates with other systems. Performance simply becomes less appropriate over time.

Effective monitoring tracks several metrics: task completion rates, escalation frequencies, user satisfaction scores, error rates, and output quality metrics. Establishing baselines during initial deployment allows you to detect when performance degrades significantly.

For critical systems, some organisations implement canary deployments where a small percentage of traffic uses updated configurations while most users continue with the proven version. This allows early detection of problems before full rollout.

AI Evaluations and Benchmarking

How do you measure whether an AI system is 'good enough' for your business needs? This requires establishing evaluation frameworks appropriate to your use case.

Accuracy metrics measure how often the system produces correct outputs. For classification tasks (routing customer queries to appropriate departments) this is relatively straightforward. For generation tasks (writing email responses), it's more nuanced and often requires human evaluation against rubrics.

Relevance measures whether outputs address the actual query or task. An AI might generate perfectly grammatical, factually accurate text that completely misses the point of what was asked.

Completeness evaluates whether responses contain all necessary information. Partial answers might technically be correct but practically useless.

Appropriateness assesses tone, formality, and alignment with brand voice and business values. A factually correct response delivered in inappropriate tone damages customer relationships.

Robustness measures how well the system handles edge cases, ambiguous inputs, and adversarial queries. Real users won't always provide perfectly formed queries.

Benchmarking against industry standards provides context, but most business-specific AI applications require custom evaluation frameworks aligned to specific business objectives. A legal research AI and a marketing content generator require entirely different quality metrics.

Many organisations implement tiered evaluation:

  • Automated metrics provide continuous monitoring of basic quality indicators.
  • Regular sampling and human evaluation assess nuanced quality aspects.
  • Periodic comprehensive assessments measure alignment with evolving business objectives.

Quality Assurance Frameworks

Effective AI QA requires systematic approaches rather than ad-hoc testing. Several frameworks have emerged:

  • Pre-deployment Validation: Comprehensive testing before release including unit tests, integration tests, golden dataset evaluation, security testing, bias assessment, and performance benchmarking. This establishes baseline capability and identifies obvious problems.
  • Shadow Mode Deployment: Running the AI system alongside existing processes without affecting outcomes. Comparing AI recommendations or outputs against current practice identifies gaps and builds confidence before full deployment.
  • Gradual Rollout: Starting with limited use cases, user groups, or geographic regions allows controlled evaluation of real-world performance. Problems can be identified and addressed before full-scale deployment.
  • Continuous Monitoring: Automated tracking of key metrics, alerting when performance degrades beyond acceptable thresholds, and regular manual review of sample outputs.
  • Feedback Loops: Systematic collection and analysis of user feedback, escalation patterns, and correction requirements. This qualitative information often reveals problems that quantitative metrics miss.
  • Regular Re-evaluation: Scheduled comprehensive assessments ensure the system still meets business needs as requirements evolve. What was 'good enough' six months ago might no longer be acceptable.
  • Incident Response: Clear procedures for when problems occur including rapid rollback capabilities, root cause analysis, and systematic improvements to prevent recurrence.

The framework must be proportionate to risk. A marketing content generator and a medical diagnosis support system require vastly different QA rigour.

AI Reliability Frameworks

As the field matures, governance frameworks are emerging. Examples include:

  • NIST AI Risk Management Framework (US): A structured approach to identifying and mitigating AI system risks.
  • ISO 42001 (expected 2025): The forthcoming international management standard for AI oversight and trustworthiness.
  • UK’s AI Assurance Roadmap: Focused on standards and testing methodologies for trustworthy AI adoption.

Benefits of Robust AI QA

  • Risk Mitigation: Catching problems before customers encounter them prevents reputational damage, legal liability, and costly remediation.
  • Trust Building: Consistently reliable AI systems build user confidence, increasing adoption and value realisation.
  • Cost Efficiency: Early detection of issues is invariably cheaper than fixing problems in production or dealing with consequences of AI failures.
  • Performance Optimisation: Systematic evaluation identifies opportunities for improvement and quantifies the impact of changes.
  • Regulatory Compliance: Demonstrable QA processes increasingly matter for regulatory requirements around AI systems, particularly in regulated industries.
  • Competitive Advantage: Reliable AI systems deliver better customer experiences and operational outcomes than unreliable competitors.

Challenges and Considerations

  • Subjectivity: Many AI outputs cannot be objectively scored as correct or incorrect. Evaluation often requires human judgement, which is expensive, potentially inconsistent, and doesn't scale well.
  • Coverage: The input space is effectively infinite. You cannot test every possible query, scenario, or edge case. Strategic sampling and risk-based testing become necessary.
  • Explainability: When an AI system produces an incorrect or inappropriate output, understanding why is often difficult. LLMs are notoriously opaque, making root cause analysis challenging.
  • Cost: Comprehensive AI testing requires significant resources. API calls to commercial LLMs cost money. Human evaluation is expensive. Maintaining test datasets and evaluation frameworks requires ongoing investment.
  • Pace of Change: AI technology evolves rapidly. Models improve, new capabilities emerge, and best practices evolve. QA frameworks must adapt accordingly.
  • Context Dependency: AI behaviour often depends on subtle context that's difficult to standardise in testing. The same query in different conversation contexts might appropriately produce different responses.

How Companies Can Incorporate AI QA

Practical implementation of AI QA doesn't require massive investment from day one. Start proportionate to your use case and scale as needed:

  • Begin with Risk Assessment: Understand what could go wrong and what the consequences would be. Customer-facing systems handling sensitive information require more rigorous QA than internal productivity tools.
  • Establish Baselines: Before deploying AI systems, measure their performance against representative scenarios. Document what 'good' looks like and track changes over time.
  • Implement Basic Monitoring: Even simple logging and periodic review of AI outputs can catch many problems. You don't need sophisticated observability platforms on day one.
  • Create Feedback Mechanisms: Make it easy for users to flag problems and systematically review this feedback. Users often identify issues that automated testing misses.
  • Develop Golden Datasets: Curate test cases representing typical, edge-case, and problematic scenarios. Regularly evaluate your AI against these benchmarks.
  • Plan for Iteration: Treat AI deployment as an ongoing process rather than a one-time project. Budget for continuous evaluation and improvement.
  • Consider Hybrid Approaches: For high-stakes outputs, implement human review or multi-model verification. Don't rely solely on a single AI system for critical decisions.
  • Document Everything: Maintain clear records of configurations, evaluation results, and changes. This supports debugging, compliance, and continuous improvement.

Examples

A UK financial services firm implemented AI-powered customer query routing. Initial testing showed 85% accuracy, acceptable for their needs. However, ongoing monitoring revealed accuracy dropped to 72% within three months. Investigation identified that seasonal query patterns (tax year-end questions) differed significantly from training data. They implemented quarterly re-evaluation against seasonal datasets and adaptive retraining, stabilising performance.

A healthcare provider deployed an AI system to help schedule appointments based on symptom descriptions. Adversarial testing revealed that certain phrasings caused the system to incorrectly categorise urgent symptoms as routine appointments. Implementation of a separate urgency detection system and mandatory human review for high-risk categories caught potentially dangerous misclassifications.

An e-commerce company used AI to generate product descriptions. Systematic sampling and review discovered that approximately 3% of outputs contained subtle hallucinations (claiming features products didn't have). They implemented automated fact-checking against structured product data and human review before publication, eliminating customer complaints about misleading descriptions.

Summary and Next Steps

AI Quality Assurance represents a fundamental shift from traditional software testing. It requires accepting uncertainty, evaluating statistical performance rather than deterministic behaviour, and maintaining ongoing vigilance rather than one-time validation.

The core principle is this: you cannot eliminate all AI failures, but you can systematically reduce them to acceptable levels and catch most problems before they reach customers.
Remember that AI systems change over time even if you don't actively modify them. Model drift, data drift, and evolving business needs require continuous evaluation, not just pre-deployment testing.

Invest proportionately. A chatbot answering simple FAQs requires less elaborate QA than an AI making financial recommendations or medical decisions. Scale your approach to match your risk and business value.

Most importantly, recognise that AI QA isn't a purely technical challenge. It requires business judgement about acceptable trade-offs between capability, reliability, cost, and risk.
Technology can measure and monitor, but humans must decide what 'good enough' means for your specific context.

The next article in this series examines AI Monitoring and Observability, exploring how to maintain visibility into AI system behaviour in production and detect problems as they emerge.

Links

** Notes

  • The article was drafted by an AI Agent.
  • Then reviewed by a human and published - (Us here at Siris!)
  • The topic research and guidance on style, tone and audience were built into the agent.