Selected Work
Three systems, each built on the same pattern: the agent does the routine work, the rules are code, a person keeps the exceptions. All three are live below. The records are sample data; the rules, the code, and the approval step are the real thing.
Refund Approvals · Online Store
Approval Desk
- The founder was approving every refund by hand. Most were routine.
- We wrote the refund policy as code: a limit, a return window, a duplicate check, a frequent-refunder hold.
- Routine requests clear themselves. Exceptions land in an approval inbox with the order and the customer attached.
- Any change to an approved amount reopens the decision. Every step is in the audit log.
Churn & LTV · Subscriptions
Customer Lifecycle Analysis
- The owner knew the monthly revenue number and nothing about the customers behind it.
- We measured how long each kind of customer stays and what each is worth over a year, from invoices, the CRM and the onboarding checklist.
- Cut by contract, channel, industry, size and onboarding, the best customer had an address and the worst had a fix.
- Measured facts and hypotheses stay in separate columns. The brief is drafted from the numbers.
Invoice Follow-Up · Services Firm
Operations Assistant
- Every morning someone read the receivables list and decided who to chase.
- We built an assistant with read-only tools over the invoices. It finds the disputed one, ranks the rest by customer history, and drafts each follow-up in the right tone.
- Drafts wait for approval. Nothing is sent without a person.
- Every answer lists what it checked. Every approval is logged.
Transaction Review · Payments
Transaction Review
- A payments team could not review every card transaction by hand.
- We built a model that scores each transaction for fraud, trained and tested on real payment data with a time-ordered holdout.
- Reviewing the riskiest 1% of transactions catches over a quarter of the fraud at 88.7% precision; the riskiest 2% catches over 40% at 70.4% precision, both measured on transactions the model never trained on.
- Every score is calibrated against the observed rate in its bin, and the review rate is a knob the team sets, not a guess.
Reorder Ranking · Online Grocery
Reorder Ranking
- An online grocer's app has to guess which of a shopper's past products to show first.
- We built a ranking model that orders each shopper's past products by how likely they are to be bought again, trained on the full Instacart dataset.
- It beats a buy-it-again-by-frequency baseline on every ranking metric, evaluated on shoppers the model never trained on.
- Every number on this page is copied from the pipeline's own metrics file, not typed by hand.
SignOff App
- When an agent workflow hits an exception, it escalates to your phone.
- The request, the evidence, two buttons. Decide anywhere.
- Every decision logged. The routine work never waits for you.
- Time back, on the go.
Concept preview. The screen below is a working mock: open a request, read the evidence, approve it, and watch the decision get logged. The rules behind it are the ones running in the Approval Desk build.
SignOff3 agent requestsEvery Agent System Is Tuned on Three Dials
Each dial is a real configuration choice: model tier, retry budget, retrieval depth. Each maps to a number your team will live with. Move one dial and the other two move too.
Seconds per decision, p50 and p95. Set by model tier and retry budget.
Verdict and citation validity against an agreed golden set.
Dollars per case, itemized by step. You set the ceiling; this dial reports what the system spent against it.
How a Badly Tuned System Fails
These are failure modes, not certainties. Each one shows up when one dial gets set without measuring the other two.
The cheapest model, no review passes. A decision can go out citing a policy section that does not apply. Fast, cheap, and wrong. Where the work is regulated, wrong is measured in customer harm.
Top-tier models and unlimited review loops. A routine approval can take ninety seconds and forty times the tokens. At volume the queue backs up and staff start overriding the system.
The verification step is the first expense cut, and it is the check that catches invented criteria. The savings can hold until the first audit finding.
The engagement work is choosing the setting deliberately, with the trade-offs on paper, before production.
Which Model Wins the Quality Dial?
A bench that runs the same banking questions through several hosted models and reports accuracy, cost, and speed side by side, on real data. Full Banking77 test set, 3,080 messages, three hosted Bedrock models.
Open Model Bench