Training Corpus Governance: Verifiable Lineage From Training Data to Model

Mechanism

Training corpus governance is implemented as a three-layer construction: a per-item lineage record affixed to each candidate artifact, a signed corpus policy that defines admissibility predicates over those records, and a tamper-evident corpus manifest that commits the admitted set into a Merkle structure referenced by the trained model's identity. Each layer is cryptographically dependent on the layer beneath it, so that a model's identity, by construction, transitively commits to the lineage of every artifact that contributed to its weights.

The per-item lineage record travels with the artifact rather than being stored alongside it in a separate registry. It carries, at minimum, a content-anchored identifier derived from the artifact's structural variance signature, the chain of custody hashes describing how the artifact was acquired and transformed prior to candidacy, the rights declarations attached at each transformation boundary, and the signing keys of the parties asserting those declarations. Because the identifier is anchored to the structural variance of the artifact itself, the lineage cannot be detached, swapped, or laundered through re-encoding: any structural change either preserves the variance signature, in which case the lineage continues to apply, or alters it, in which case the artifact has become a different artifact and must acquire its own lineage de novo.

The signed corpus policy is a declarative document, itself content-anchored and signed by the corpus authority, that specifies the predicates a candidate artifact's lineage must satisfy to be admitted. Predicates may reference rights flags, jurisdictional constraints, age of the lineage record, the identity of upstream signers, structural variance class, or any other field exposed by the lineage schema. The policy is evaluated deterministically: given a candidate lineage record and a policy document, every conforming evaluator produces the same admit-or-refuse outcome and the same diagnostic in the case of refusal.

Refusal is terminal at the corpus boundary. An artifact whose lineage does not satisfy the active policy is not quarantined, queued for human review, or admitted under a downstream filter — it is simply not present in the corpus manifest, and the manifest commits to the refusal by recording the rejected identifier and the predicate that failed. This is structurally important: it means a downstream operator cannot recover an ineligible artifact by editing a configuration flag or by replaying the ingestion pipeline with a different policy, because the manifest under which the model was trained has already committed to the rejection.

The tamper-evident corpus manifest is the artifact that closes the loop. It is a Merkle commitment over the ordered set of admitted lineage records, signed by the corpus authority and bound by reference into the trained model's identity record. Any party in possession of the model identity can request the manifest, verify its signature, walk the Merkle structure to confirm membership of any specific artifact, and re-evaluate the policy against each admitted lineage to confirm that the corpus, as committed, is consistent with the policy under which it was assembled.

Operating Parameters

The mechanism is parameterized along several axes that operators tune to deployment needs without altering the structural guarantees. The lineage record schema is versioned; new fields may be added to accommodate evolving rights regimes or jurisdictional metadata, and policies may require minimum schema versions, but no schema migration may rewrite an existing record without producing a new content-anchored identifier and thereby invalidating downstream commitments.

Policy expressiveness ranges from simple allowlist predicates over signing keys to compound predicates combining rights state, jurisdiction, structural variance class, and time bounds. The reference implementation uses a deterministic, side-effect-free predicate language so that policy evaluation is reproducible across heterogeneous evaluators; non-deterministic predicates, including those that consult external services, are rejected at policy compile time. Policies themselves are versioned and content-anchored, and the manifest records both the policy identifier and the policy hash under which each admission decision was made.

Manifest construction is incremental and append-friendly. New artifacts may be added to an in-flight corpus by extending the Merkle structure and re-signing the root, but admitted artifacts may not be retracted: a retraction is recorded as a new manifest version that excludes the retracted identifier, and the model identity binds to a specific manifest version rather than to the corpus as an abstract collection. This preserves the property that a model's training set is fixed at training time and that subsequent governance actions on the corpus do not retroactively alter the audit surface of already-trained models.

Verification cost scales logarithmically with corpus size for membership proofs and linearly with the size of the inspected subset for policy re-evaluation. In practice, full re-evaluation of a multi-billion-item corpus is tractable on commodity infrastructure because policy predicates are pure functions of lineage records and may be evaluated in parallel without coordination.

Alternative Embodiments

The mechanism admits several embodiments that preserve the structural guarantees while accommodating operational and regulatory variation. In a federated embodiment, multiple corpus authorities each maintain manifests over disjoint or overlapping artifact sets and a meta-manifest commits to the federation's union; a model trained on the federation binds to the meta-manifest, and verifiers walk one additional Merkle level to reach individual artifact lineages. In a confidential embodiment, lineage records are encrypted under keys held by a designated audit party and the manifest commits to ciphertext identifiers; verifiers can confirm membership without learning the underlying lineage, and the audit party can produce decryption proofs on demand.

A streaming embodiment supports continuous training regimes by issuing manifest checkpoints at fixed intervals or after fixed admission counts, with the trained model identity binding to a specific checkpoint. A retraction embodiment treats rights revocation as a first-class operation: a revocation record signed by an authorized party causes future manifest versions to exclude the affected artifact and signals downstream consumers that any model bound to a manifest version still containing the artifact should be re-evaluated against the revoked rights state.

A cross-organizational embodiment supports pooled training where each contributor admits artifacts under its own corpus authority and policy, and a coordinating authority assembles the union manifest under a meta-policy that constrains the per-contributor policies. This permits collaborative training without requiring contributors to surrender their lineage data or to harmonize their rights regimes, while still producing a model whose training set is provable end-to-end.

Composition With Adjacent Mechanisms

Training corpus governance composes with the broader content anchoring stack along several well-defined seams. The structural variance signature that anchors each artifact's identity is produced by the same anchoring mechanism used elsewhere in the system for content identity and provenance, so a single artifact carries one identifier across ingestion, training, distribution, and downstream attribution. This eliminates the seam between provenance systems and training systems that, in conventional architectures, requires an explicit join across two registries with independent failure modes.

Composition with the rights and licensing layer is direct: rights declarations are fields on the lineage record, and policy predicates may reference them in the same evaluation pass as structural and jurisdictional predicates. Composition with the audit layer is via the manifest: any audit query that asks "what was this model trained on?" is answered by the manifest, and any query that asks "was this artifact eligible under the policy in force at training time?" is answered by re-evaluating the recorded policy against the recorded lineage. Composition with model distribution is via the model identity record, which transitively commits to the manifest and therefore to every admitted lineage.

Because the mechanism is implemented at the content identity layer, it is independent of the training framework, the model architecture, and the substrate on which training occurs. A corpus assembled under the mechanism may be consumed by any trainer that respects the manifest's ordering and admission set, and a model trained under the mechanism may be distributed through any channel that preserves the binding between model identity and manifest reference.

Prior-Art Distinction

Conventional approaches to training data governance fall into three families, each of which the mechanism distinguishes itself from along structural rather than incremental lines. The first family is metadata-based registries, in which artifacts are tagged with rights and provenance metadata stored in an external database that the training pipeline consults. These systems fail open when the metadata is stripped, when the registry is unavailable, or when the artifact is re-ingested through a path that bypasses the registry; the mechanism here fails closed by anchoring identity to structural variance and by binding admission to a signed manifest that the model identity transitively commits to.

The second family is policy-as-code pipelines, in which admission rules are expressed as code executed during ingestion. These systems produce no durable artifact attesting to the policy under which a given training run admitted its data, and policy changes between runs are recoverable only by inspecting source control and ingestion logs whose integrity is itself unattested. The mechanism here records the policy hash and version directly in the manifest, so that the policy under which a model was trained is recoverable from the model identity alone.

The third family is centralized provenance ledgers, which assert lineage but do not bind it to model training events. The mechanism here closes that gap by making the manifest a structural input to model identity rather than an adjacent record, so that asserting a model's identity is logically equivalent to asserting its training set.

Disclosure Scope

The disclosure encompasses the construction of per-item lineage records anchored by structural variance, the signed corpus policy as a deterministic predicate over those records, the tamper-evident manifest binding admitted lineages into a Merkle commitment, and the structural binding of that manifest into model identity. It encompasses the federated, confidential, streaming, retraction, and cross-organizational embodiments described above, as well as additional embodiments differing in cryptographic primitive selection, manifest layout, or policy language without departing from the structural construction.

The disclosure is intended to cover any implementation in which training corpus admission is governed by a deterministic predicate over per-item lineage, the admitted set is committed in a tamper-evident manifest, and the resulting model identity transitively binds to that manifest. It is not limited to specific hash functions, signature schemes, or Merkle layouts, and it is not limited to any particular training framework or model architecture.

The disclosure further encompasses the operational practices that the construction enables: the re-derivation of a model's training set from its identity alone, the verification of admission decisions against the policy hash recorded in the manifest, the production of membership and non-membership proofs against arbitrary subsets of the corpus, and the retraction workflow by which rights revocations are propagated forward into successor manifest versions without altering the audit surface of prior model identities. These practices are properties of the construction rather than separable applications, and any system implementing the construction will exhibit them by virtue of the structural commitments described above.