Practical reference library
Practical specifications and release checks for AI-built applications.
Guide 01
Write acceptance criteria as observable outcomes for a named user, record and action. For a client portal, define who can view a project, submit a file, approve work and recover from an error. A generated screen is ready for review when those outcomes work with persistent data and the intended permissions.
Read the guide →Guide 02
Test tenant isolation with two accounts in different organizations and an explicit list of protected actions. Use Account A to create a record, then use Account B to request that record through the application's normal API. The server must apply the intended access policy even when the interface hides the record.
Read the guide →Guide 03
Test a duplicate webhook by delivering the same synthetic event more than once and checking the business outcome. The receiver should authenticate the event, record a stable deduplication key and apply the intended change once. Also test a crash between recording receipt and completing the action.
Read the guide →Guide 04
A restore rehearsal proves that a backup can recreate a usable application in an isolated environment. Restore a known dataset, connect a compatible app version, verify important records and permissions, and measure the time required. Record what data would be lost between the backup and the incident.
Read the guide →Guide 05
Before inviting a paying customer, assemble a small release evidence pack: the customer workflow, test results, permission checks, billing behavior, recovery procedure and named support owner. Record what was actually tested on the release build. Separate demonstrated behavior from features that are still assumptions.
Read the guide →Guide 06
Score each builder 1-5 on eight weighted criteria — client-billable output quality, revision speed, code ownership, white-label/domain control, multi-project management, pricing at agency volume, handoff quality, and support responsiveness — using the same one real client brief for every tool. The weights are yours; the discipline of one fixed brief is not negotiable.
Read the guide →Guide 07
Write one fixed project brief, then compute for each builder: subscription cost for the months to ship, plus metered usage consumed by the brief (credits, tokens, messages), plus any per-seat or per-app charges to run it for a year. Compare those totals — never headline prices, which are denominated in different units across tools.
Read the guide →Guide 08
An export passes acceptance when a developer who has never seen the project can clone it, install dependencies from the lockfile, build without errors, run it locally against their own database, and deploy it somewhere the vendor doesn't control — inside one working day. Anything less is a code viewer wearing an ownership claim.
Read the guide →Guide 09
Ask eight questions in writing: Where does my database live and can I dump it? Who owns the domain binding? Where do uploaded files live? Can auth/user records leave with password hashes? Whose keys are my integration secrets? What does deletion actually delete? What breaks the day I stop paying? Is any runtime dependency vendor-only? Score the answers, not the marketing page.
Read the guide →Guide 10
During trials, deliberately produce one small, genuinely reproducible bug in each builder, file it through the standard support channel with a clean reproduction, and score what returns: time to first human response, whether the reproduction was actually run, time to workaround, and time to fix. Support is a product feature; test it like one.
Read the guide →Guide 11
Model five entities: Client (the organization), User (people, each belonging to one client or to your team), Project (belongs to one client), Deliverable (belongs to one project, carries a status), and Approval (belongs to one deliverable, records who, when, and which version). Files attach to deliverables, not projects. Every query filters by the requesting user's client id — that single discipline is what keeps client A from ever seeing client B.
Read the guide →Guide 12
Specify approvals as a state machine on deliverables: draft → submitted → in_review → approved | changes_requested, with changes_requested looping to draft. Only client users with the approver role can approve; approvals bind to a specific version; resubmission after changes creates a new version and a new review cycle; reminders fire on days 3 and 7 of in_review. Write the spec before building — approval ambiguity is where agency-client trust actually erodes.
Read the guide →Guide 13
Specify four components before building: the routing matrix (amount bands × categories → approver chains, e.g. <$500 auto-approve with log; $500-5k department head; $5k-25k +finance; >$25k +CEO; IT always +security), request states (draft → pending → approved/rejected → ordered → received), delegation rules (time-boxed, self-service, logged), and the audit row every action writes (actor, timestamp, state change, comment). The matrix is the business logic — get it signed off by finance before any build, because the app merely enforces what this table says.
Read the guide →Guide 14
Replace an ops spreadsheet in four moves: mine the sheet first (its columns, color codes and margin notes are the real requirements doc — including the ones nobody will say out loud), build the app to preserve the sheet's virtues (visible density, sortability, bulk edit — the things apps usually lose), parallel-run for two weeks with the sheet frozen read-only rather than deleted, and cut over on a date the team picked. The spreadsheet isn't the enemy; it's the incumbent that won an election. The app has to win the next one.
Read the guide →Guide 15
An entitlement model is one table that turns pricing-page promises into enforceable rules: each row is a capability (projects, seats, exports, API calls), each column a plan, each cell a limit, boolean, or quota-with-period — plus two columns pricing pages never show: what happens AT the limit (block, soft-warn, or meter) and what lapsed/downgraded accounts keep. Code should check entitlements ('can this account export?'), never plan names ('is this account Pro?') — the indirection is what makes plan changes, grandfathering and one-off exceptions survivable. Write the table before the pricing page, not after the refund requests.
Read the guide →Guide 16
Test eight cases before launch: expiry with no card (what locks, what stays readable), expiry with card on file (charge fires once, not twice), conversion mid-trial (no double charge at term end), cancellation mid-trial (access until expiry, then clean lapse), expiry during an active session (in-app state changes without data loss), the day-before warning email (fires once, links to payment), re-registration after expiry (old data policy enforced), and timezone edges (expiry at UTC midnight vs the user's midnight). Make them testable by injecting a clock the test can shift — 'set account trial to expire in 60 seconds' beats waiting 14 days or hand-editing production dates.
Read the guide →Guide 17
Write five things down before any integration is built: the operations you actually call (not the whole API), the data crossing the boundary in each direction, the failure policy per call (retry, queue, degrade, or surface), where credentials live and who can rotate them, and the observable signal that tells you the integration is healthy. Describing an integration to a builder without these produces a happy-path call and nothing else — the code works in the demo and fails silently the first time the vendor returns 503. The contract is also the test plan: every row is something you can verify before launch.
Read the guide →Guide 18
Simulate four failure modes rather than waiting to meet them: hard failure (connection refused or 503), slow failure (responses at or beyond your timeout — the one that actually takes apps down), partial failure (200 with a malformed or empty body), and rate limiting (429 with a Retry-After header). Force each by pointing the integration at a local stub you control rather than by unplugging anything: a stub that can be told to hang for 30 seconds, return 503, or return `{}` gives you all four on demand. Then assert on user-visible behaviour, not logs — the question is whether the page still renders, the queue still drains, and the retry does not become a storm.
Read the guide →Guide 19
Run eight checks before launch: fetch the page with JavaScript disabled and confirm the copy is in the HTML; confirm one canonical tag per page pointing at the URL you want indexed; confirm robots.txt does not block what you want crawled and that no page carries an accidental noindex; confirm the sitemap exists, lists only 200-returning canonical URLs, and is referenced from robots.txt; confirm every internal link resolves without a redirect chain; confirm titles and descriptions are unique per page; confirm the site answers on one hostname with the others redirecting; and confirm a 404 returns 404 rather than 200 with a friendly page. Generated sites fail the first and the last most often, and both are invisible in a browser.
Read the guide →Guide 20
Build the inventory before you launch, from three sources rather than one: your old sitemap, your analytics top pages for the last twelve months, and Search Console's indexed URLs. Map every old URL to exactly one new URL — or deliberately to a 410 if nothing replaces it. Then test six cases: a mapped URL redirects once with 301 to a 200; a URL with a trailing slash variant behaves the same; a query-string URL keeps or drops parameters as intended; a deliberately retired URL returns 410 rather than redirecting to the homepage; no redirect points at another redirect; and the new URL is the one in your sitemap and canonical tag. Redirecting everything unmapped to the homepage is the mistake that looks tidy and reads to a crawler as a soft 404.
Read the guide →Guide 21
Send `X-Robots-Tag: noindex, nofollow` on every response from any hostname that is not production, driven by an environment check rather than a file someone must remember to edit — and **allow crawling of those hosts in robots.txt**, because a crawler that obeys a disallow never fetches the page and therefore never sees the noindex. Disallow plus noindex is the combination that fails: the crawler honours the block and can still index the URL it was told about from a link. If you want the content genuinely unreachable, authenticate it — that is an access-control decision, and then robots.txt does not matter. Add three supports: canonical tags on preview pages pointing at production, no sitemap served from preview hostnames, and a launch check confirming production does not carry the same noindex. That last one is the failure that costs the most.
Read the guide →Guide 22
Specify routing as four explicit decisions, in order: intake (what channels arrive, what fields are mandatory before a ticket exists), classification (the small closed list of categories, each with one named owner), assignment (the rule that maps category plus tier to a person, and the named fallback when that person is away), and escalation (the condition and the clock that moves a ticket to someone else). Write the fallback and the clock down explicitly — those two are what teams leave implicit, and they are the reason tickets sit unanswered for four days with nobody feeling responsible.
Read the guide →Guide 23
Write three things down before you write any matching code: the match rule (which fields, at what confidence, and the threshold above which a match is automatic versus queued for review), the survivorship table (for every field, which record's value wins — newest, non-empty, primary record, or manual), and the merge record (what is kept so the merge can be explained later, and whether it can be reversed). Most CRM duplicate problems are actually missing survivorship rules: the merge ran, it picked a value by an implicit rule nobody chose, and a correct email address was silently replaced by a stale one.
Read the guide →Guide 24
Specify moderation as trust tiers rather than a single approve/reject queue. New employers are reviewed on every listing; employers with a clean history post directly with post-publish sampling; flagged employers are blocked at submission. Attach the pre-publish checks to the tier, not to every listing — required fields, salary disclosure, location honesty, duplicate detection and the keyword list. Then write the expiry policy down: how long a listing runs, what happens at expiry, and what happens to live listings when an employer account lapses. Boards do not fail on spam — they fail on stale listings that make the whole board look abandoned.
Read the guide →Guide 25
Write a contract per event before you instrument anything: the event name in a fixed convention, the precise moment it fires, the properties it carries with their types, whether it can fire more than once per user, and a named owner. For activation specifically, add the sentence that matters most — what the user has actually experienced by the time it fires. 'Completed onboarding' is a system state; 'sent their first message to a real recipient' is an experience. Then treat the contract as versioned: changing what an event means without renaming it silently rewrites your own history, and every chart built on it becomes a comparison between two different definitions.
Read the guide →Guide 26
Use a separate analytics project per environment, keyed by an environment variable, and never let a non-production environment hold a production write key. That single change prevents the usual accident. Then pick a testing method by what you are checking: a local debug sink for whether events fire at all, a staging analytics project for end-to-end checks with synthetic users, contract assertions in your test suite for whether properties match the spec, and the vendor's own debug view for verifying delivery. Where a test must run against production, use a synthetic account you own and can delete, and exclude it from every report by a property rather than by memory.
Read the guide →Guide 27
Check eight things before anyone acts on a generated dashboard: the date boundary (inclusive or exclusive, and in whose timezone), whether the current period is partial and labelled as such, the denominator of every rate, whether test and internal users are excluded, how nulls are treated in averages, whether the axis starts at zero, whether a percentage change is relative or absolute, and whether the numbers reconcile with one number you already trust. A generated dashboard is confident and complete-looking on first render, which removes the friction that normally makes people ask these questions. The friction was doing useful work.
Read the guide →Guide 28
Specify seven things before changing a price, starting with the randomisation unit — account or company for B2B, person for B2C — because that one choice decides whether two colleagues can see different numbers. Then: who is eligible (almost always new visitors only — never existing customers mid-term), what each variant actually shows including on every surface where the price appears, how long a shown price is honoured once someone has seen it, the primary metric and the guardrail metrics, the duration and minimum sample, and the grandfathering rule for what happens to test participants when the test ends. That last field is the one teams skip and the one that produces a support queue: someone signed up at a tested price and now needs to be told, in writing, what they will pay at renewal.
Read the guide →Guide 29
Seven fields: the exact steps from a known starting state, what you expected, what happened instead, the error text verbatim rather than described, where it happened (URL or screen, environment, account state), how often it reproduces out of how many attempts, and — the field people omit — what must not change as a result of the fix. That last one is what prevents a plausible repair that quietly breaks a working feature. The single most damaging line in a bug report is a guess at the cause stated as fact, because an agent will take it as a constraint and fix the thing you named rather than the thing that is broken.
Read the guide →Guide 30
Bound every change three ways before asking for it: name the specific behaviour to change, name the flows that must keep working unchanged, and name the files or areas that are in scope if you know them. Then review the diff by size before reading it — a request to change a button's validation that returns a forty-file diff is telling you something regardless of how good the code is. The instruction that causes the most damage is a broad improvement request ('clean this up', 'tighten up the auth') because it has no boundary, so any change is technically compliant and you have no basis to reject one.
Read the guide →Guide 31
Capture nine things during the incident rather than after: when it started and how you found out, what users experienced, what you have confirmed is true, what you have ruled out, what you changed (each action, timestamped), what is still unknown, what is currently mitigating it and whether that is temporary, what must be undone later, and the next thing you were about to try. Written as you go, in a single file. The field that pays for the whole exercise is what you ruled out — without it, the second pass at 2am repeats the first pass's investigation, because tired-you does not remember which theories were already eliminated.
Read the guide →Guide 32
Validate nine things before importing, in this order: row count, uniqueness of whatever will be the key, date formats and ambiguity, number formats including thousands separators and currency symbols, leading zeros that Excel has already eaten, text encoding, empty versus null versus the string 'NULL', values that will not fit the destination type, and referential integrity between sheets. Then run the import into a staging table first and reconcile counts and checksums before touching anything real. The dangerous failures are not the rows that get rejected — they are the ones silently coerced into a valid value that is wrong.
Read the guide →Guide 33
Run both systems on the same real work for at least one full business cycle — a month for anything with monthly reporting, a quarter if quarterly processes exist. Pick a named set of workflows to compare, keep the old system authoritative throughout, and record every divergence rather than fixing it silently. The trial ends on a written criterion decided in advance, not on a feeling: every named workflow completed in the new system, all divergences either resolved or accepted with a reason, and one full cycle-end process run successfully in parallel. Most replacements that fail do so because the trial was shorter than the cycle that exposes the problem.
Read the guide →Guide 34
Move accounts in four separable pieces, because each has different rules: identity (the user record and its identifier), credentials (password hashes, which migrate only if the algorithms are compatible), authorisation (roles and permissions, which usually need remapping rather than copying), and state (sessions, consent, preferences and anything with a legal basis). Never re-hash passwords you cannot verify and never store them in a recoverable form during the move. If hashes cannot transfer, the fallback is a rehash-on-next-login shim, not a mass password reset — a reset email to your entire user base looks exactly like a breach notification and will be treated as one.
Read the guide →Guide 35
Accept a generated dashboard for keyboard use only when one tester, with no mouse or trackpad, can complete every task the dashboard exists for: get past the sidebar to the main content, change the date range and filters, sort and page the table, open a row's actions, edit a record in its dialog, and read the numbers a chart shows. At every step the focus indicator must be visible and never entirely hidden behind a sticky header, focus must move in an order that matches the layout, and nothing may trap it. Those checks produce evidence for WCAG 2.2 success criteria 2.1.1 Keyboard, 2.1.2 No Keyboard Trap, 2.4.3 Focus Order, 2.4.7 Focus Visible and 2.4.11 Focus Not Obscured (Minimum). A pass is evidence for those rows on that build, not a conformance claim.
Read the guide →Guide 36
Write the error behaviour down before the form is generated, one row per error: the field and rule, where it is detected (browser, app code, API or database), the exact message, where it appears and how it is announced. Every error must identify the field and describe the problem in text (WCAG 2.2 3.3.1 Error Identification, Level A), and should say how to fix it when that is known and safe (3.3.3 Error Suggestion, AA). Each message is tied to its field in markup, and the spec picks one focus behaviour for a failed submit. Spell out server-side errors most carefully, such as an email rejected by a unique constraint: a prompt that describes only the happy path gives the builder nothing to map them to.
Read the guide →Guide 37
Test every modal in four moves. Open it from the keyboard and check that focus lands inside it. Tab and Shift+Tab past both ends and check that focus wraps inside rather than escaping to the page behind. Press Escape and check that it closes. Then check where focus goes: back to the control that opened it, or, when that control no longer exists because the dialog deleted its row, to a sensible next element you have specified. The WAI-ARIA Authoring Practices dialog pattern describes this behaviour, and it supports WCAG 2.2 criteria including 2.1.2 No Keyboard Trap, 2.4.3 Focus Order and 4.1.2 Name, Role, Value. Run the test once per kind of dialog the app contains, not once for 'the modal'.
Read the guide →Guide 38
Treat motion on a generated marketing site as two separate jobs. First, anything that moves automatically for more than five seconds alongside other content (a logo marquee, an auto-advancing testimonial carousel, a looping background video) needs a mechanism to pause, stop or hide it, which in practice means a control on the page. That is WCAG 2.2 success criterion 2.2.2 Pause, Stop, Hide, Level A, and it applies whatever the visitor's settings. Second, honour the prefers-reduced-motion CSS media feature for motion triggered by interaction or scrolling, replacing slides, zooms and parallax with a fade or an instant change. The media feature is not itself a WCAG requirement: W3C lists it as a sufficient technique for 2.3.3 Animation from Interactions, which is Level AAA. Then test with the preference on and off, and confirm no content stays hidden.
Read the guide →Guide 39
Pick the flow that matters most in the generated app (sign up and create the first record, say), turn on a screen reader you can run (VoiceOver with Command-F5 on a Mac, or the free NVDA on Windows), and walk the flow by ear. Check seven things: each screen's title says where you are, headings and landmarks let you jump to the main content, every control is announced with a role and a name that matches its visible label, every field announces its label, images are described or silent, confirmations such as 'Saved' are spoken without you hunting for them, and data tables announce their headers. It's a smoke test: it finds failures that block the flow, not everything an audit would.
Read the guide →Guide 40
Write one brief small enough to build and check in a day, and specific enough that two builders can't both claim success while doing different things: a one-sentence purpose, three named roles, a permissions table, five entities with their rules, five workflows written as start, action and end state (one of them a failure), one integration with a sandbox, a synthetic seed dataset, the acceptance tests that decide pass or fail, and an out-of-scope list. Give every builder the identical text, log every clarification you make and add it to the next version. The brief is the control in your evaluation: a cost comparison or a scorecard is only as fair as the brief behind it.
Read the guide →Guide 41
Estimate complexity by counting what multiplies work, not screens: roles, workflows, the states each workflow's main record passes through, the role-and-workflow pairs where permissions differ, each direction of each integration, workflows that move money, time-dependent rules, and jobs that run without a user. Give each driver a weight, total them, and compare the total with projects you have already built rather than with a universal scale. The weights here are an illustrative starting point, not a benchmark. The value is in the counting, which makes a 'simple app' visibly not simple before anyone commits to a date or a plan.
Read the guide →Guide 42
Review a generated schema table by table against a fixed list before customer data goes in. Every tenant-owned table carries the tenant key and every query filters by it; relationships are declared as foreign keys with a delete rule someone chose; the rules from the brief exist as unique, not-null and check constraints rather than only in app code; money uses an exact type; moments in time are stored with their time zone; the columns that filters and joins use are indexed; deletion works one documented way per table; and personal-data columns are listed. The review is cheapest now: once customers' data is in the tables, every fix becomes a migration.
Read the guide →Guide 43
A traceability matrix has one row per requirement in the brief, linking it to where it was built, the acceptance test that checks it, the evidence that the test passed, and the build that evidence came from. For an AI-built app the last column matters most: a later change can alter code that satisfied an earlier requirement, so 'verified' only means something next to a build identifier. Keep requirement IDs stable, record evidence as artifacts rather than ticks, re-test the rows a change could touch, and treat any row without evidence on the current build as unverified, whatever its status once said.
Read the guide →Guide 44
Script the demo so several beats show the app handling something going wrong, because anyone who has bought software knows the happy path is the easy part. Choose failures the app genuinely recovers from on the build you are showing: an input error that keeps the user's work, a permission refusal, a double submit that creates one record, a declined payment in the provider's test mode, and a notification that reaches the right people. For each beat, write what you do, what the audience should see, the fixture it needs and your fallback if the environment misbehaves. Never stage a failure the app doesn't really handle.
Read the guide →Guide 45
Decide on evidence about where the problems live, not on how the code looks. Improve in place when the data model still fits the product, the defects are local (a screen, a rule, a message) and fixes stay fixed. Rebuild when a foundation is wrong in a way every feature inherits: no tenant key in a product that now has many customers, a data model shaped around a workflow you abandoned, or regressions that keep landing on the same invariants. Between the two sits a staged rebuild that replaces one workflow at a time behind the running app. Score the signals in the worksheet, and before any rebuild, list what the current app does that nobody wrote down, because a rebuild drops it unless someone carries it over.
Read the guide →Guide 46
Run the first month after launch as four weekly passes, each task with an owner and written evidence. Before day one: a backup restored successfully, error tracking and an uptime check on the production URL, a support inbox someone reads, and a named person who can roll back a release. Week one: read every error and every support message daily, and ship fixes only. Week two: rehearse a restore and re-run the permission and payment tests on the live build. Week three: watch trials expire, renewals charge and scheduled jobs run for the first time. Week four: hold the first monthly release review and decide what month two allows. The point is to find what pre-launch testing missed while the user count is still small.
Read the guide →Guide 47
Treat acquiring an AI-built app as a transfer of control over every account the app depends on, verified by the buyer, rather than a download of code. List every asset: the builder project, the code export or repository, the domain and DNS, the database and its backups, file storage, the email-sending domain, the payment processor, OAuth app registrations, every third-party API key, analytics, and the admin accounts inside the app. For each one the seller starts the transfer, the buyer proves control by making a harmless change, and only then is every secret the seller knew rotated. Remove the seller's access at close and agree a support window in which they answer questions without holding access.
Read the guide →Guide 48
Write a limits register before the prototype reaches anyone who might treat it as a product. Each entry names an area, the prototype's actual behaviour there, its status (missing, stubbed, built but unverified, or verified on a named build), the risk if someone assumes more, and what would lift the limit. Cover the areas people over-assume: data and backups, users and permissions, integrations, payments, scale, offline use, security, accessibility, devices, and who can change it. Put the three limits that matter most on the prototype's first screen, keep the full register next to its link, and update it with every build you share.
Read the guide →Guide 49
Once a month, review every release the app shipped as a list, not a feeling: what was requested, what was delivered, what broke afterwards, what was reverted or hotfixed, and which requirements were re-verified. Then decide four things: which kinds of change caused rework and what rule or test prevents it, which invariants broke more than once, what carries into next month with an owner, and how many changes next month allows. The output is decisions with owners, not a changelog. For a team changing an app through an AI builder, the most useful line is requested versus delivered, because a change that did more than it was asked is the one whose side effects nobody tested.
Read the guide →