Baseline skips recall; the memory boundary result tests exclusion after retaining decisions. Those checks overlap and have different denominators. They do not add up to a general accuracy or security score. The await regression remains part of the result.

Hindsight stores and retrieves the facts. The application checks visitor identity, repository and file scope around that retrieval. The billing contracts are authored fixtures. Reusing their wrapper-verification rule for a direct HTTP endpoint would erase the condition that justified it.

Give each workspace a separate bank

Two visitors can both enter acme/billing-service; their saved decisions must remain separate.

The evaluated guest path creates a random workspace identifier and derives a Hindsight bank from it. This excerpt comes from guest session creation:

const id = randomUUID();
const value = `v1.${id}.${Math.floor(now / 1000) + SESSION_SECONDS}`;
return { token: `${value}.${signature(value, secret)}`, bank: `rm-${id}` };

In that guest path, the browser receives a signed cookie; it cannot choose an arbitrary bank in an API payload. The server verifies the HMAC signature and expiration before using the identifier. The cookie is HTTPOnly, uses SameSite Strict, and is Secure in production. Its lifetime is thirty days.

The fixed suite used anonymous workspaces: Hindsight held decisions; browser storage held session history. The current account flow adds PostgreSQL-backed identity, workspace settings and review history. After sign-in, the server resolves the account’s persistent Hindsight bank; client-supplied IDs cannot select another user’s bank. Hindsight still stores decision facts. Ten PostgreSQL integration tests cover account persistence, isolation and errors; they do not rerun the eight-case review evaluation.

Guest workspace reset has a specific meaning: it creates a new identity. It does not delete the previous Hindsight bank. Clearing a cookie can remove access from that browser while leaving provider-side records stored. A deletion feature would need to handle those records through the provider.

Require the right repository and file scope

Within a bank, every saved decision receives a repository tag and a review-decision tag. Recall requires both, using this actual request configuration:

types: ['world', 'experience'],
tags: [repositoryTag(input.repository), 'review-decision'],
tags_match: 'all_strict',
budget: 'mid',

Repository matching is only the first filter. Each retained document also carries a scope: tag. For the example wrapper decisions, its value is src/webhooks/**. The application supports relative file paths, component wildcards, and a terminal /** for descendant paths; it validates that limited grammar.

The server examines returned tags and rejects a response containing evidence from a different repository. Before assembling the model prompt, it requires a single usable scope that matches a changed path:

if (paths) {
  if (
    scopes.length !== 1 ||
    !paths.some((path) => scopeMatches(scopes[0], path))
  ) {
    excludedByScope++;
    continue;
  }
}

This check removes an inapplicable fact before the model can read it. The review trace reports how many facts the scope filter excluded, so an empty evidence panel has an inspectable explanation.

A mixed-file diff needs another check. It could change both a webhook handler and a direct route, allowing a webhook fact into the prompt for a legitimate reason. ReviewMemory validates each finding's memory references against that finding's file. It rejects a response that cites the wrapper convention on a file outside the convention's scope.

That validation checks declared references. It cannot prove that the model never generalized from evidence without citing it. The prompt also states the scope rule, but semantic correctness still requires inspection.

The ledger after retaining three example decisions. The account-routing fact includes its file scope and document identifier.

Test both inclusion and exclusion

The fixed evaluation used eight synthetic scenarios: four defect cases, two handlers satisfying the stated contracts, and two boundary controls. Each ran once in baseline mode and once with retained decisions, using the same model and prompt. All 16/16 requests completed in the paced run on September 28, 2026.

The 2/2 boundaries test different conditions. One changes src/routes/refund-callback.ts, outside the saved scope; its trace records three exclusions and zero supplied facts. The other uses a different repository. We check prompt evidence and declared references, without grading their route findings as correct.

The 3/3 exact connectedAccount: event.account suggestions each cited the corresponding decision. For the primary check, the scorer required every expected literal fix in each of four defective handlers. That passed 1/4 baseline and 3/4 with memory; it is a narrower claim than overall review quality.

Exact await-call syntax fell 3/3 to 2/3. One response recommended awaiting in prose but omitted the literal call; the frozen scorer kept that failure. Zero-findings controls passed 1/2 and 2/2. We did not execute generated patches or use an LLM judge.

An earlier integration check also verified fresh-visitor isolation and later retrieval. The paired evaluation adds repeatable contract checks, but one billing domain and one repeat cannot establish broad accuracy. Its baseline lacks prior decisions by design; it does not compare against another tool given the same context directly.

The interactive direct-route example receives zero facts. Its trace records three facts excluded by file scope; this capture is separate from the fixed evaluation.

Replace a decision without accumulating old versions

Remembering feedback also creates an update problem. If a maintainer changes a convention, retaining both versions as current guidance can make the next review less reliable.

ReviewMemory derives a stable document identifier from the normalized repository name, decision title, and scope. Saving an edited rationale under that same identity sends a replacement. The retained item includes these fields:

document_id: documentId,
update_mode: 'replace',
tags: [
  repositoryTag(decision.repository),
  'review-decision',
  `scope:${decision.scope}`,
],

The Hindsight implementation provides document replacement behavior. The application's choice of stable identity makes an edit target the earlier source instead of creating an unrelated new document. Recall requests use source facts of the world and experience types rather than derived observations.

The identity rule has an important limit: changing the title or scope creates a different document identifier. It is not general contradiction resolution. Two decisions with different names can still disagree. The reviewer instructions ask the model to acknowledge unresolved conflicts; a maintainer must inspect and reconcile them.

A provider write can finish after a local timeout. The backend reports uncertainty without retrying on its own. An explicit retry uses the stable identifier; a successful response triggers a document-specific list request to verify extracted facts.

Separate valid references from correct reasoning

The review generator must return structured JSON. The server validates its shape, checks that cited identifiers came from the supplied evidence, and checks file locations against the diff. Malformed output or invented references produce errors rather than replacement findings.

Those checks do not validate meaning. The qualitative audit records memory responses that supplied correct fixes but invented duplicate processing after a premature commit or unsupported validation failures. Extracted facts also compressed details from the original decisions. Inspect the source rationale as well as the recalled text; a passing identifier check cannot establish that either was interpreted correctly.

The reviewer cannot inspect unseen wrapper code. Examine the saved rationale, recalled fact, file scope and patch before accepting a change.

Operational failure is part of the record too. The unpaced pilot produced 26/32 PROVIDER_BUSY errors. Those outputs remain available; they were not pooled into an effectiveness score. Before the paced run, an amendment reduced repeats from two to one and introduced 65-second gaps. The sanitized errors did not identify the exact upstream quota. No semantic failure triggered a retry.

Use non-confidential samples on the public app. Recall sends a diff excerpt to Hindsight, and the review request sends code to Groq. Neither a scoped convention nor valid citations establish that unseen code is safe.

Vectorize's agent memory overview offers background. Here, inspect the boundary result alongside the behavior result: 2/2 boundaries excluded facts, 3/3 account mappings matched, and one of three await-syntax checks failed. Scope and citations make evidence inspectable; they cannot certify the reasoning.

Related developer tooling: Code.in.