CUSTOMER ACCOUNT / ATTRIBUTED EVIDENCE
What this case establishes
This is a commercial publisher’s customer account. Roseram checked the source and wrote the analysis; it did not interview the customer, run the product, audit the figures or establish that AI caused the reported business result. The pilot below is a proposal, not a completed experiment.
Primary material reviewed 2026-10-12. Analysis and proposed evaluation by Roseram Editorial.- 01Diff question
- 02Adaptive source reads
- 03Reproducible finding
Greptile: the documented workflow
Anthropic’s Greptile story describes review agents following leads across source, similar functions and Git history. It reports approximately 90% prompt-cache hits through the Agent SDK. That is a cache-efficiency claim in the published deployment, not a 90% bug-detection rate or a guarantee of equivalent savings in another repository.
The mechanism worth examining
A review can require several connected questions: what changed, where is the function called, when was the behavior introduced, and does another implementation reveal an invariant? Adaptive retrieval lets the next source read depend on what was found. The engineering challenge is to keep that flexibility bounded. An investigative agent needs a stopping rule and a record of evidence, otherwise its tool activity can resemble progress without ever reaching a reviewable conclusion.
The implementation decision
Test adaptive investigation on defects that require more than reading the diff. Choose an issue involving a caller, a configuration assumption or a historical compatibility contract. Compare it with a simple diff-only task so the added retrieval cost has somewhere to justify itself. Give the agent read-only tools initially. Decide how many reads, how much latency and which repositories belong in scope, then evaluate the resulting finding rather than the apparent sophistication of the agent trace.
Where the workflow can fail
Prompt caching can make repeated context cheaper, but it cannot make irrelevant context correct. A high hit rate might reflect many similar reads rather than useful completed reviews. Repository permissions also matter when an investigation follows references into another project. Require missing or unauthorized evidence to be reported explicitly. A reviewer should not infer private implementation details from a public dependency or continue issuing the same search after the relevant file was unavailable.
A practical evaluation, designed for this use case
Use an instrumented fixture with one defect whose cause sits outside the changed file. Log each requested path, result, revision and tool duration. Check whether the final explanation connects the actual caller to a reproducible failure. Include a missing-file variant to test stopping behavior. Calculate cost per confirmed useful finding with and without caching; keep cache hits, model spend, retrieval spend and reviewer time as different measurements.
Primary sources
Figures and product details above are dated and attributed; they are not current guarantees.
- Anthropic: Greptile primary account ↗The attributed workflow and reported result. Proposed pilots and decisions below are original Roseram analysis.
The decision matrix
Original evaluation criteria for this workflow. These are questions to test, not results measured by Roseram.
| Decision | Evidence to collect | Stop or revise when |
|---|---|---|
| Investigation scope | Each read has a task-related purpose | Search repeats without new evidence |
| Cache benefit | Cost per useful completed review | Cache percentage is used as correctness |
| Source authority | Retrieved revision and permissions | Another repository is silently consulted |
YOUR EVALUATION / NOT A REPORTED RESULT
A small pilot you can inspect
Use these steps as a planning checklist. Checking a box records your progress in this view; it does not run or validate an experiment.
Questions this case answers
Does a 90% cache-hit rate mean 90% cheaper reviews?
Not necessarily. Cache pricing, uncached work, retrieval and output all contribute to total cost.
Why test a missing file?
It shows whether the reviewer can admit insufficient evidence instead of producing a confident explanation from an incomplete investigation.
Tools & capabilities referenced
Operators studying this story often evaluate adjacent AI products. Explore Roseram guides where available:
PRACTICAL NEXT STEPS
Take the evidence into your own project.
Calculate the whole project cost
Compare subscriptions, usage, launch work and renewals.
Test an existing-project workflow
Check import, source loading, preview and publication.
Verify the output you buy
Require evidence tied to a real customer task.
Wiki brief
Greptile is examined here for multi-step code-review investigation. The article answers “What makes an AI reviewer investigative rather than checklist-driven?” using linked primary material and original evaluation design. It is not a customer testimonial for Roseram or an endorsement by Greptile.
Operator takeaways
- Bound the investigation loop.
- Measure caching separately from correctness.
- Require a traceable final finding.
Related case studies
Plan → build → review with routed models and transparent usage.