Architecture
Eight decisions, and what each one costs
Written for someone deciding whether they would have built it the same way. Every choice lists the trade-off it accepted.
The layers
A request passes down through five layers, and each one narrows what the next may do.
Web app
Next.js, TypeScript, Tailwind
Chat, approvals, run traces, evaluations
API
FastAPI, Pydantic
Sessions, roles, rate limits; the only way in
Agent
LangGraph, checkpoints in Postgres
A fixed graph of steps that can pause for a person
Tools
Typed, allowlisted, also served over MCP
Every read and write re-checks tenant, role and scope
Database
PostgreSQL 16, pgvector, full-text search
App data, billing data, vectors, checkpoints, audit log
The decisions
Decision 01
Code decides refunds; the model handles language
- What
- The model classifies the message, proposes which read-only lookups to run and rewrites a draft reply. Duplicate detection, the 30-day window, approval limits and every write are plain Python.
- Why
- A refund is a financial write. Its rules must be testable, repeatable and explainable to an auditor, and a prompt change must never be able to alter them.
- What it costs
- The agent cannot handle a request the code has no branch for; it escalates instead. A new kind of action means writing code, not a prompt.
Decision 02
A person approves every action, bound to exact parameters
- What
- The run pauses and its state is checkpointed. The approval stores the target, amount, currency, required role and an idempotency key; the tool re-checks all of them and only runs in the approving reviewer’s session.
- Why
- “Approved” must mean “approved this”. Re-verifying at execution time closes the gap between what the reviewer saw and what is executed.
- What it costs
- No refund is instant, and the pause-and-resume path has to be tested, including a restart in between.
Decision 03
Every failure has a designed outcome
- What
- Timeout before commit: check by idempotency key, then retry with the same key. Reply lost after commit: the check finds the refund, so no retry. Decline: escalate. Model down or unsure: fall back to rules or hand to a person.
- Why
- The dangerous failures are the ambiguous ones. Idempotency keys make “did it happen?” answerable, so a retry can never refund twice.
- What it costs
- One retry only, by design. After that a person takes over.
Decision 04
Hybrid search inside Postgres, weighted by measurement
- What
- Vector search and keyword search run in the same database as the application data and are fused with reciprocal rank fusion. With a learned embedding model the vector ranking counts three times the keyword ranking.
- Why
- On 40 labelled questions, keyword-only put the right document first 63% of the time, vector-only 93%, and the original equal-weight design only 75%. Weighting the vector side raised it to 83%, with the right document in the top four 98% of the time.
- What it costs
- Vector-only still scores at least as well on this question set. Hybrid is kept for exact terms and for the offline mode. The weight was tuned and reported on the same small set.
Decision 05
Tenant isolation in the schema, not only in queries
- What
- Tenant-owned tables use composite keys, so a row cannot reference another company’s row even if a query has a bug. Identity comes from the session; ids typed into a message are never trusted.
- Why
- A filter that lives only in application code is one forgotten WHERE clause away from a leak.
- What it costs
- Wider keys and more verbose SQL. Row-level security is not enabled, so a missing filter is caught by tests, not by the database.
Decision 06
Stripe is synced into a read model
- What
- A company’s Stripe test data is copied into the billing tables on demand. The agent reads the copy; an approved refund is created in Stripe using the approval’s idempotency key.
- Why
- Every existing tool, permission check and refund rule works unchanged on Stripe data, and the agent’s speed does not depend on Stripe’s API.
- What it costs
- Data is only as fresh as the last sync. Test mode only: live keys are refused.
Decision 07
A fallback path instead of a hard dependency on one model
- What
- Claude or a local Ollama model, with a rule-based path used when no key is set or the model fails. Each fallback is recorded on the run.
- Why
- The demo and the test suite run offline and for free, and an outage degrades wording, not correctness.
- What it costs
- The rule-based path understands far fewer phrasings.
Decision 08
Runs happen inside the request
- What
- The agent runs inside the HTTP request and streams its steps back. Rate limits and the daily message cap are enforced per process and per company.
- Why
- Simple to reason about and enough for a single-instance demo.
- What it costs
- A second instance needs shared rate limiting and a job queue. This is the first thing to change under real load.