Skip to content

Architecture

Eight decisions, and what each one costs

Written for someone deciding whether they would have built it the same way. Every choice lists the trade-off it accepted.

The layers

A request passes down through five layers, and each one narrows what the next may do.

  1. Web app

    Next.js, TypeScript, Tailwind

    Chat, approvals, run traces, evaluations

  2. API

    FastAPI, Pydantic

    Sessions, roles, rate limits; the only way in

  3. Agent

    LangGraph, checkpoints in Postgres

    A fixed graph of steps that can pause for a person

  4. Tools

    Typed, allowlisted, also served over MCP

    Every read and write re-checks tenant, role and scope

  5. Database

    PostgreSQL 16, pgvector, full-text search

    App data, billing data, vectors, checkpoints, audit log

The decisions

  1. Decision 01

    Code decides refunds; the model handles language

    What
    The model classifies the message, proposes which read-only lookups to run and rewrites a draft reply. Duplicate detection, the 30-day window, approval limits and every write are plain Python.
    Why
    A refund is a financial write. Its rules must be testable, repeatable and explainable to an auditor, and a prompt change must never be able to alter them.
    What it costs
    The agent cannot handle a request the code has no branch for; it escalates instead. A new kind of action means writing code, not a prompt.
  2. Decision 02

    A person approves every action, bound to exact parameters

    What
    The run pauses and its state is checkpointed. The approval stores the target, amount, currency, required role and an idempotency key; the tool re-checks all of them and only runs in the approving reviewer’s session.
    Why
    “Approved” must mean “approved this”. Re-verifying at execution time closes the gap between what the reviewer saw and what is executed.
    What it costs
    No refund is instant, and the pause-and-resume path has to be tested, including a restart in between.
  3. Decision 03

    Every failure has a designed outcome

    What
    Timeout before commit: check by idempotency key, then retry with the same key. Reply lost after commit: the check finds the refund, so no retry. Decline: escalate. Model down or unsure: fall back to rules or hand to a person.
    Why
    The dangerous failures are the ambiguous ones. Idempotency keys make “did it happen?” answerable, so a retry can never refund twice.
    What it costs
    One retry only, by design. After that a person takes over.
  4. Decision 04

    Hybrid search inside Postgres, weighted by measurement

    What
    Vector search and keyword search run in the same database as the application data and are fused with reciprocal rank fusion. With a learned embedding model the vector ranking counts three times the keyword ranking.
    Why
    On 40 labelled questions, keyword-only put the right document first 63% of the time, vector-only 93%, and the original equal-weight design only 75%. Weighting the vector side raised it to 83%, with the right document in the top four 98% of the time.
    What it costs
    Vector-only still scores at least as well on this question set. Hybrid is kept for exact terms and for the offline mode. The weight was tuned and reported on the same small set.
  5. Decision 05

    Tenant isolation in the schema, not only in queries

    What
    Tenant-owned tables use composite keys, so a row cannot reference another company’s row even if a query has a bug. Identity comes from the session; ids typed into a message are never trusted.
    Why
    A filter that lives only in application code is one forgotten WHERE clause away from a leak.
    What it costs
    Wider keys and more verbose SQL. Row-level security is not enabled, so a missing filter is caught by tests, not by the database.
  6. Decision 06

    Stripe is synced into a read model

    What
    A company’s Stripe test data is copied into the billing tables on demand. The agent reads the copy; an approved refund is created in Stripe using the approval’s idempotency key.
    Why
    Every existing tool, permission check and refund rule works unchanged on Stripe data, and the agent’s speed does not depend on Stripe’s API.
    What it costs
    Data is only as fresh as the last sync. Test mode only: live keys are refused.
  7. Decision 07

    A fallback path instead of a hard dependency on one model

    What
    Claude or a local Ollama model, with a rule-based path used when no key is set or the model fails. Each fallback is recorded on the run.
    Why
    The demo and the test suite run offline and for free, and an outage degrades wording, not correctness.
    What it costs
    The rule-based path understands far fewer phrasings.
  8. Decision 08

    Runs happen inside the request

    What
    The agent runs inside the HTTP request and streams its steps back. Rate limits and the daily message cap are enforced per process and per company.
    Why
    Simple to reason about and enough for a single-instance demo.
    What it costs
    A second instance needs shared rate limiting and a job queue. This is the first thing to change under real load.