All systemsOperations AssistantApproval DeskTransaction ReviewCustomer LifecycleResearch AgentModel BenchReorder Ranking
built by giggit · research agent

Ask the Payments Record Get a Cited Answer

AREA (Automated Research Evaluation Assistant) answers questions about industry payments to doctors from the public CMS Open Payments record, with every number cited to a real row. A customized AREA system was delivered for a fintech platform. The public version below shows the research approach on a separate public dataset.

Every number in an answer is checked against the row it's cited to before the answer is shown Real CMS Open Payments records, loaded year by year
Where It Runs
ComputeA Vercel function on this site, /api/area-ask. Two calls to AWS Bedrock (Amazon Nova Lite, us-east-1): one writes the SQL, one writes the answer.
DataCMS Open Payments rows in Postgres on Neon. Read-only, allow-listed SQL, one query per question. The citation checker runs before the answer is returned.
DeployA merge to main deploys the site on Vercel. The AREA repository runs its tests against a real Postgres service container on every push.
ChecksEvery six hours a job asks 8 fixed questions with hand-written reference SQL and writes the score to a public ledger. Live Checks. Every answer here carries its own receipt.

Run It

How It Works

  1. The question goes to a hosted model, which writes one read-only SQL query against the loaded payments table.
  2. The query is checked against an allow-listed schema, then run for real: retrieval, not a guess, over the rows that are actually loaded.
  3. A second call to the hosted model drafts a short answer from the real rows returned, with a citation marker on every number.
  4. A citation checker matches every number in the answer to a cited row and flags any number without one as unverified, before the answer is shown.

Results

Rows Loaded
loading…
Top Manufacturer
loading…
Top Specialty
loading…
Total Dollars, Years Loaded
loading…

Loading years covered…

Ask a Question

Ask about industry payments to doctors, by manufacturer, specialty, state, or quarter.

Reference suite: eight questions with hand-written SQL, the ones the scheduled check re-asks

Eight questions have a reference query written by hand when the build shipped. Ask one and the receipt shows whether the agent's answer agrees with it.

Every question runs a live, read-only query over the real, loaded rows. An answer with an uncited number is flagged.

Latest scheduled check: loading…

Open: three share questions differ from their reference query, and the first-quarter 2021 question returns an error. History on Live Checks.

Code

The retrieval, the guardrail, and the citation checker: github.com/alphan-ml/area

What It Does

Retrieval, Not a Guess

Each question becomes a real SQL query against the payments table that's actually loaded. Nothing is answered from memory or estimated.

Every Number Is Checked

A citation checker walks the answer and matches every number to the row it names. A number without a backing row is flagged as unverified. A cited answer can still differ from the reference query; the receipt shows both checks separately.

Coverage Is Stated, Not Assumed

The record is loaded year by year. The page always says which years are in and which aren't, read from the database, not typed by hand.

How It Was Built

1 INPUT A Question Plain English, about payments to doctors 2 GIGGIT BUILT Guarded Retrieval One SELECT, checked against an allow-list, run on real rows 3 GIGGIT BUILT Citation Checker Every number matched to the row it names, or flagged 4 OWNER KEEPS The Cited Answer Read the citation, decide what to do with it
Client sideA questionPlain English, about payments to doctors
Giggit builtGuarded retrievalOne SELECT, checked against an allow-list, run on real rows
Giggit builtCitation checkerEvery number matched to the row it names, or flagged
Stays with the ownerThe cited answerRead the citation, decide what to do with it