Conformance report

Result

By area

Coverage

A conformance claim is only as good as its coverage, so an operation that was never exercised is named, not quietly left out of the totals.

Specifications this run validated against

Fetched from their authoritative repository at the start of this run and stored alongside it in spec/. Nothing is vendored into the suite: a claim about a specification should be checkable against the specification as published.

• publisher: version 0.4.0 publisher specification proposed in CycloneDX/transparency-exchange-api PR 147
• insights: not yet upstream; proposed in CycloneDX/transparency-exchange-api and fetched here from the proposing fork

Areas

• discovery: DNS, the .well-known/tea document, endpoint selection and TEI resolution
• consumer: every operation of the consumption specification, including its error paths
• purl: package-URL identifiers, filters and purl-typed TEIs
• cyclonedx: the published BOM documents, validated against the version each declares
• spdx: the licence identifiers those documents carry
• insights: the Insights API and the CycloneDX documents it answers with
• cel: the query language, cross-checked against the reference CEL implementation
• provenance: build provenance, attestations, signatures and checksum coverage
• performance: cold and cached latency, reported separately
• provider: the publication specification, as a create-read-update-delete round-trip

Method

The suite is a black-box HTTP client. It knows nothing about any implementation, and every judgement here comes from comparing a response against the specifications named above, fetched from their authoritative repository at the start of this run.

OpenAPI 3.1 schema objects are JSON Schema 2020-12, so the specification’s own bytes are what was enforced. There is no intermediate model that could drift from it.

Some of what the specification requires cannot be written as a schema. Collection and release UUID identity, lifecycle event ordering, pagination-token consistency and additionalProperties:false on error bodies were each checked separately against the normative prose, and are reported as case failures in the same way as schema violations.

Every request and response is stored under responses/, named deterministically, so this report can be regenerated offline from that directory and checked against the bytes it was derived from.