SotA on Function Calling with every LLM

We present SAP, an algorithmic approach that improves function‑calling reliability, achieving state‑of‑the‑art results on the Berkeley benchmark across all LLMs.

Overview

We recently achieved state of the art results on the Berkley function calling benchmark with every model using a novel technique called SAP. Instead of improving the model, we worked on algorithms the refine the response of the model. The results: we can now beat naive function calling on Sonnet-3.5 or GPT4/4o with models like Haiku and GPT3.5.

Links

https://github.com/boundaryml/baml
BAML engineers type-safe, multi-language LLM prompts with reliable tool-calling via Rust.

Tech stack