A Run That Cannot Account for Every Fact Does Not Read as Clean
A scanner's own bookkeeping can fail silently, and the result looks exactly like a clean project. TheAuditor now ends every analysis with a verdict on its own work: only a measured match passes, a step that measured nothing must prove it had nothing to measure, and an unverified run exits as incomplete.
We have written before that the most dangerous output a scanner can produce is a silent zero: a clean result that only means nothing ran. There is a quieter version of the same problem one layer down. A scanner reads your files, turns them into facts, and answers from those facts. If a fact goes missing on the way, nothing in the answer shows it. The finding that would have been there is simply not there, and the report looks like a project with one less problem.
So TheAuditor checks its own work, and reports on that check separately from your findings.
A verdict on the analysis, not only on the code
Every analysis now ends with two answers. One is about your code: the findings. The other is about the analysis itself: whether it can account for what it read, from the files on disk to the stored facts it answers from.
The second answer is held to a strict rule. A pass must mean the project was measured and matched. Only two outcomes count: verified, or verified as having nothing to check. Anything else fails the verdict, including a step that was scheduled and did not run, and any status nobody anticipated when the check was written. It is an allow-list, so a new way to go wrong cannot slip through as a pass.
Empty has to be earned
A step that finds nothing is the hardest case, because “nothing” is a legitimate answer for some projects and a broken one for others. A project with no web routes should report no web routes. A project whose routes were never read should not.
So an empty result now has to prove it is empty. A step that measured nothing counts as a verified absence only when the steps it depends on were themselves verified. If they were not, the result is a gap, and the gap names what it was waiting on.
Two concrete cases moved because of this rule. A taint step that found no sources and no sinks used to read as fine; it now reads as empty only when the analysis it rests on checks out, and as a gap otherwise. And a run over an empty index used to finish as a success with no verdict at all; it now writes its verdict and exit code like any other run.
Incomplete is its own exit code
The verdict reaches your pipeline. TheAuditor’s exit codes, described in The First Hour Is Part of the Product, reserve 3 for incomplete: a step failed or stopped, or the results could not be verified. The verdict on the analysis is read before the findings, so a run that cannot vouch for itself exits 3 even when it also found problems. A build gate never mistakes an unverified run for a clean one, or for a complete list of findings.
Honest scope note
This is a check that the analysis kept what it read. It is not a claim that the analysis understands everything it read: what TheAuditor does not yet model is reported as a gap, as it always was. And the check is held to its own standard. A failure must mean a fact was lost or invented, so when the check itself was wrong about a file, the check was fixed, not the verdict softened.
Where this sits
An answer you can trust needs an analysis that can account for itself. That is the ground-truth discipline behind Code Reality Labs: a result that cannot be verified says so, out loud, in a code your pipeline can act on. TheAuditor is in final commercial release preparation and ships when its hardening checks pass. Subscribe on the main site for launch news.
Was this useful?