Blogs
Notes on building, connecting, and deploying agents.
Engineering deep-dives, product updates, and field notes from teams shipping agents in production.
Back Now
We Raised Our Series B
We Raised Our Series B
We Raised Our Series B
We Raised Our Series B
We Raised Our Series B
Every team that ships agents eventually builds a dashboard. Latency, cost, success rate a wall of charts that looks reassuring right up until a customer reports something the charts never caught. The problem isn’t the dashboard. It’s when it tells you.
Dashboards are a trailing indicator. By the time a regression shows up as a dip in your success-rate chart, it has already reached production and, worse, real users. The fix is to move quality checks to the same place we catch every other regression: continuous integration.
The case for evals-as-tests
An eval is just a test with a fuzzier assertion. Instead of expect(x).toBe(2), you assert that an agent’s output satisfies a rubric: did it resolve the ticket, stay in policy, and avoid hallucinating an order number? Run that assertion on every pull request and quality stops being a vibe.
We Raised Our Series B
Every team that ships agents eventually builds a dashboard. Latency, cost, success rate a wall of charts that looks reassuring right up until a customer reports something the charts never caught. The problem isn’t the dashboard. It’s when it tells you.
Dashboards are a trailing indicator. By the time a regression shows up as a dip in your success-rate chart, it has already reached production and, worse, real users. The fix is to move quality checks to the same place we catch every other regression: continuous integration.
The case for evals-as-tests
An eval is just a test with a fuzzier assertion. Instead of expect(x).toBe(2), you assert that an agent’s output satisfies a rubric: did it resolve the ticket, stay in policy, and avoid hallucinating an order number? Run that assertion on every pull request and quality stops being a vibe.
Every team that ships agents eventually builds a dashboard. Latency, cost, success rate a wall of charts that looks reassuring right up until a customer reports something the charts never caught. The problem isn’t the dashboard. It’s when it tells you.
Dashboards are a trailing indicator. By the time a regression shows up as a dip in your success-rate chart, it has already reached production and, worse, real users. The fix is to move quality checks to the same place we catch every other regression: continuous integration.
The case for evals-as-tests
An eval is just a test with a fuzzier assertion. Instead of expect(x).toBe(2), you assert that an agent’s output satisfies a rubric: did it resolve the ticket, stay in policy, and avoid hallucinating an order number? Run that assertion on every pull request and quality stops being a vibe.
