What a scan is

A scan takes one exact version of a codebase and returns a set of scores, the findings behind them, and a report. Run it again on the same code under the same rubric and you get the same numbers.

One repository at one commit. Not "the project" in general — a specific, identified state of it. Everything a scan reports is a statement about that commit, and nothing else.

This matters more than it sounds. Code changes daily; a number that cannot name what it measured cannot be checked, compared, or defended later. Every Watchdog result carries the commit it read.

• Scores. Each thing we measure gets a score from 0 to 10. Related measurements are grouped into lenses — broad areas such as code health, architecture, or security and compliance — and the lenses roll up into a headline score.
• Findings. The individual observations behind the scores: what was found, where, and why it counted.
• A report. The above, rendered for a human to read, in several formats depending on who is going to read it.

Some measurements are advisory: they are reported and explained, but deliberately do not move the score. These are usually observations we believe are useful but cannot yet stand behind as strictly as the rest — a signal offered without being charged for.

The same commit, measured under the same rubric version, produces the same result. That is the central promise, and everything else in the system is arranged to keep it true.

A rubric is the frozen definition of how scoring works at a point in time. When we change anything that could move a score — what we look for, how we weigh it, how we detect it — we do not quietly adjust the existing rubric. We publish a new version, and old results stay attached to the version they were measured under. The rubric contract explains this in full, including how anyone can fetch a rubric and check a result themselves.

Why not just always use the newest rules? Because then no two reports would be comparable, and a score could drop overnight without a line of code changing. Freezing the rubric is what makes "we improved from 6.4 to 7.1" a real statement rather than an artefact of us changing our minds.

It is not a penetration test: nothing is attacked, and no running production system is touched. It is not a replacement for human review — it measures properties that can be measured consistently across many codebases, which is a different job from knowing what a product is supposed to do.

A score is a statement about the code at one commit under a stated rubric, with the findings attached — not a verdict on the people who wrote it.

Sometimes we cannot measure something honestly. A repository may not build in our environment; a property may be one where finding nothing tells you nothing. In those cases we report that we could not establish it, rather than scoring it zero or awarding a perfect ten by default.

This distinction is important enough that it has its own section — see How we measure.

• From GitHub App to your first baseline — walks through connecting a repository and reading the report that comes back.