MASTERCLASS

The AI Oracle: Building and Using Golden Datasets for Quality

How do you test something that gives a different, “correct” answer every time? This is the fundamental question in AI quality, and most teams don’t have a good answer yet. We’re trying to apply old-world testing rules to a new-world problem, leaving us with a critical blind spot. This masterclass provides a solution: a disciplined approach to creating a “golden dataset.” This isn’t just test data; it’s the source of truth, the oracle, and the stable foundation that makes repeatable, automated AI validation possible. And it needs to be managed, adapted and modified as the feature evolves.

In this 90-minute masterclass, Gil will cover: The Core Problem: Why traditional testing methods fail when faced with non-deterministic AI systems and how to establish a reliable “source of truth.” The Golden Dataset: A disciplined, hands-on approach to creating and curating a “golden dataset” to act as your test oracle. From Manual to Automated: A practical, two-part exercise where attendees first manually build a golden set, then use it to power simple, automated checks in a zero-setup environment. The Living Asset: How to manage, adapt, and modify your golden dataset over time as your AI feature evolves, turning it into a sustainable quality practice.

What you’ll learn

What you’ll need

Session details

Gil Zilberfeld

For over 25 years, Gil Zilberfeld has been in the software trenches, helping teams solve real-world quality problems. He teaches pragmatic testing frameworks that apply timeless principles to today’s most complex challenges, including web automation and ensuring AI quality. As a consultant and trainer, his core philosophy is that quality is a team sport, with testers as key players. Learn more at testingil.com. He also shoots zombies, for fun.