AI Testing
SPONSOR TRACK TALK
Orchestrating LLM Pipelines with Metamorphic Testing for QA Automation
Continuous testing has driven demand for scalable, self-maintaining QA automation, but traditional methods often fail with evolving requirements, brittle scripts and the need for manual output verification. While generative AI aids test creation, the industry still needs easier pipeline validation. Metamorphic testing, a novel method which mutates inputs to ensure consistent outputs, offers a robust way to validate input reliability. This talk explores how these and other converging technologies provide fast, thorough QA coverage and functional requirement checks beyond standard automation.
This talk will demonstrate an end-to-end LLM pipeline that: Translates technical specifications into Gherkin scenarios using LLMs (e.g., GPT-4) as well as highlighting prompt engineering techniques used for parameterised feature files. We will also demonstrate how to convert Gherkin scenarios to JavaScript test scripts using the same automatic approach while validating outputs with metamorphic testing and applying input transformations (e.g., data permutation) to verify consistency via predefined relation. A demo will showcase the entire pipeline using real specification documents from customers.
What you’ll learn
From this talk you will learn:
Session details
A short note from Victor on his talk
Victor Saenger
Victor Saenger currently serves as the R&D AI Lead at Capitole. He has more than 8 years of experience developing and deploying AI pipelines that tackle real world problems. He holds a PhD in Computer Science and has worked in various industries, from medical to water management and from organisational psychology to information extraction.
Víctor Trujillo
QA Practice Lead at Capitole, has over 8 years experience as QA Engineer, has multiple backgrounds in different industries, from QA analyst in Finance to QA solution architect in Retail sector. He has worked with some of the most popular tools in QA like K6 for Performance, Cypress, Appium or Newman for Automation and other tools like Sonar, GH Actions or Allure for Continuous Integration and Reporting.