Potato: AI Model A/B Testing

This talk covers Potato, a platform with a custom container orchestrator for running A/B experiments to scientifically test and improve AI model changes.

Overview

Potato is a platform for running experiments, to make AI models better. Under the hood I’ve built a custom container orchestrator designed specifically for AI development workloads. Potato is useful when you’re iterating on model development, and want to be scientific about testing and measuring changes.