Watchdog · System documentation
How a survey works
What we produce, and for whom
- Products, portfolios and deliveries
- Quality bars and acceptance profiles
Quality bars and acceptance profiles
A prototype and a payment system should not be held to the same standard, and a score does not decide anything on its own. This page covers the two declarations that surround a measurement: the producer's quality bar, which states which kind of code is being measured, and the buyer's acceptance profile, which states what "passing" means for a delivery.
What a quality bar does
A quality bar is declared by the owner of a repository and travels into the analysis. It does not change what is measured, because the same properties are examined either way. It changes how the resulting bands are interpreted: what counts as adequate for a throwaway template is not what counts as adequate for production.
The tiers run from early-stage work through to production code. A repository with no declared bar is measured against the ordinary production expectation, which is the safe default: unstated code is assumed to matter.
This is interpretation, not exemption. A lower bar does not switch measurements off or hide findings. Everything is still measured and still reported; the bar affects how the result should be read. A prototype with exposed credentials is still a prototype with exposed credentials.
The declared bar travels with the evidence
Because the bar changes how a result reads, it is recorded alongside the result and carried in the signed evidence. Anyone assessing that evidence later can see which standard was claimed, and judge whether it was the right one for what they are buying.
That visibility is what keeps the mechanism honest. Declaring a low bar is allowed; declaring it invisibly is not.
Not the same thing as an acceptance profile
Two different concepts, easily confused. A quality bar is declared by the producer and shifts how measurement bands are interpreted. An acceptance profile is a buyer's binding criteria, meaning what must be true for a delivery to pass. They coexist and neither replaces the other. In Assay, the buyer's standard is authored as a bar in its own sense, which compiles into a profile: the same word, a different thing.
Acceptance profiles
A score does not decide anything on its own. An acceptance profile is the written-down definition of what "passing" means for a delivery, and every deviation from the standard baseline is disclosed rather than hidden.
What a profile contains
A profile is an account-level, versioned definition of binding criteria:
- which lenses and dimensions are in scope for the decision;
- the minimums that must be met;
- critical floors, meaning the things that fail a delivery outright regardless of the overall picture;
- how change over time is treated, where a delivery is judged partly on movement rather than absolute position.
A repository or a sponsorship is bound to a specific published version of a profile, so it is always determinable which criteria a given delivery was judged against.
Disclosure is the anti-gaming mechanism
Profiles derive from standard bases, and every deviation is disclosed in the generated documents. Nobody is prevented from writing a lenient profile. What cannot be done is write one quietly, because where a profile departs from the baseline it derives from, the departure is stated in the document a reader receives.
Restriction would have been the obvious design: forbid weak profiles. Disclosure is stronger, because it keeps the criteria honest without pretending we can anticipate every legitimate reason an organisation needs something different.
Versioning
Profiles are versioned and a binding names a version. Changing acceptance criteria means publishing a new version, and a delivery already judged under an earlier version stays judged under it. Criteria that could be edited after the fact would make every past pass or fail uncheckable, because nobody could establish what was actually required at the time.
The buyer's side of the same idea
A profile is the compiled, binding form of a standard. In Assay, a buyer authors that standard as a bar, publishes it, and binds it to what it applies to, and publishing compiles it into a profile of the kind described here. Producer and buyer are looking at the same object from two ends.
Related guide
- Which gate to add first covers choosing the first gate to enforce, and in what order.