Skip to content
Watchdog
Sign inSurvey a repo free

Watchdog · System documentation

How a survey works

What we produce, and for whom

Products, portfolios and deliveries

A survey measures one repository. Almost nothing anyone cares about is one repository. Products and portfolios are how a measurement becomes a statement about a thing that is actually bought, sold or owned, and contracts and deliveries are how that statement is owed, signed and shared. This page covers the whole chain.

A product

A product is a curated set of repositories that together constitute a delivered thing: the specific "X of Y repositories" a contract attests to. A product of a single repository is perfectly ordinary.

Membership is deliberate and stated, not inferred. Nothing joins a product because it looks related, shares a naming convention, or lives beside it. That is the entire point: a product whose membership drifts cannot be the subject of a claim, because the thing being claimed about keeps changing.

Why curation is the important property. If membership were automatic, a supplier could weaken a product's result by adding repositories, or strengthen it by having some quietly fall out of scope. Declared membership makes the scope of every claim explicit and stable.

A portfolio

A portfolio is the roll-up across products, meaning the aggregate view for an organisation or a scope within it. It answers a different question from a product report: not "is this thing good enough" but "where does our estate stand, and which way is it moving".

Roll-ups are weighted by size rather than counting repositories equally, so a large system is not outvoted by several small ones. Alongside the aggregate line, the spread is shown, covering the best, the worst and the middle, because an average alone hides whether an estate is uniformly acceptable or contains something serious.

Which to use

  • Assessing something specific, such as a supplier's deliverable, an acquisition target or a contractual obligation: use a product. It has defined boundaries and can carry a verdict.
  • Steering, meaning where to invest, whether last quarter's effort worked and which teams need support: use the portfolio. It shows direction and distribution rather than a pass or a fail.

Where this leads

A product is the unit a contract attests to, and a buyer assessing that product does so against a standard they set themselves. See quality bars and acceptance profiles.

Contracts

A contract is an obligation to keep proving something: it records that a supplier owes a buyer a fresh attestation about a product, and how often. It moves through a simple life: set up as a draft, active while in force, and ended when the relationship closes.

Ending a contract does not invalidate its history. Deliveries minted under a contract stay valid and checkable after it ends. A buyer who received an attestation keeps something that can still be checked years later, because otherwise the attestation would only have been worth as much as the commercial relationship, which is not worth much.

Deliveries

A delivery is the shareable, signed artifact produced from a measurement, and it is the artifact the rest of the platform, including the buyer's side, is built on. It goes through three steps:

  1. Minted. The evidence and its provenance are captured: the verdict, the exact commit measured, the rubric version and the declared quality bar.
  2. Signed. The verdict is recomputed and the package frozen and signed, so any later alteration is detectable.
  3. Published. Registered with the registry, from which an authorised buyer can fetch it.

Its whole history is reconstructible, because an artifact that is shared externally has to be auditable and tamper-evident long after the fact.

Who may see one

Publishing a delivery does not make it public. Access is granted to specific buyers by the supplier, and can be withdrawn. Sharing a result with one customer does not release it to the market.

What the buyer does with it

A buyer fetches a granted delivery, checks the signature, re-derives the verdict from the evidence to confirm it follows, and assesses it against their own acceptance criteria. None of that recomputes the underlying measurements, and the numbers in a delivery are echoed rather than adjusted.