Beeline

From intent to proof

Company
Biene Club
Role
Methodology, Product Design, Vibe coding
Year
2026
Tags
Methodology, Product Design, Vibe coding
Tech
Claude API, FastAPI, React, Braintrust, OpenRouter, Docker

Try it

Summary

A demonstration of eval-driven development for AI features. Describe a feature in one sentence; Beeline builds it, derives a testable definition of "good" (the seed), generates a dataset and scorers from it, grades the result, rewrites until every scorer passes, deploys it behind a live URL, and keeps scoring real production calls.

Problem

Teams ship AI features on vibes — a few inputs tried, the outputs looked fine. You can't check every output by hand, and a machine can't check them for you until someone writes down what "good" means. So evals land on whoever built the feature, late, written against what they already shipped.

Process

The method came out of consulting with teams building AI features. Framed the method, designed the app around it, built the backend orchestrator and React frontend, deployed it, and monitored production. The central artifact is the seed: a structured, testable definition of good — discrete criteria grouped by coverage, balance, alignment, rot, and safety, each traceable to a stated intent and specific enough that two people would grade it the same. The dataset and scorers are generated from the seed, so they stay pinned to the use cases. The improve loop rewrites the feature until every scorer clears threshold (not the average), and the seed and scorers are never edited to fit.

Outcome

Live at beeline.biene.club and still scoring production calls. Running it in production found two classes of broken scorer: alignment scorers that had never graded a single live call, and refusals scoring as passes because a model that declines still returns something clean. Both failures reshaped the method — meta-evals now grade Beeline's own build-time calls, and CI gates on an offline eval self-test. Daniel built the whole thing: method, product design, orchestrator, frontend, deployment and production monitoring.

MVP

One sentence in, a deployed and continuously scored feature out — with the seed, dataset, scorers, baseline, improve-loop rounds, and production traces all visible.

Best for