Deduplicate a video library while keeping provenance

Define what counts as a duplicate

Two files can have identical bytes, show the same recording with different encoding or contain separate useful takes of the same action. Those relationships need different handling. Establish which relationship you are checking before merging records, then preserve source and derivative links for the files you retain.

In our production in October 2026, source IDs, hashes and media matching help trace clips, while removal proposals preserve originals and provenance. ai ad workflow small business provides the small-business context. This guide focuses on reducing confusion in active selection without treating every similar thumbnail as permission to discard a source.

A hypothetical lamp library contains an exact copy, a smaller encode and another take from a similar angle. Only the first relationship is established by identical bytes. The encode needs a source link, while the alternate take may contain useful action differences. Review those relationships before merging records. Similar appearance can identify candidates for inspection, but it should not authorize removing a take or losing the explanation of a derivative’s origin.

A content hash can group files with the same bytes. Keep path, size and source location beside each group. The hash identifies content equality under the chosen hashing method; it does not explain which copy is the authoritative source or which path an edit uses.

For a hypothetical exact group, one copy may be the original download and another a staging copy used by a renderer. Before consolidating, inspect references and permissions. Name video assets with stable IDs and readable labels covers asset IDs and names. Choose a canonical catalog record deliberately, and preserve aliases or source locations where they matter. A matching hash should simplify the evidence review, not skip it.

Source: Turntake editorial checklist (proposed), October 2026.

1. Clip A
2. Clip B

Treat perceptual similarity as a candidate signal

Visual matching can suggest related footage even when files differ in encoding, resolution or crop. It can also group separate takes that happen to look alike. Use the signal to create a review list rather than automatically declare identity.

Compare the beginning, action and ending. A useful take may contain the only unobstructed moment of a product demonstration, while a similar-looking take hides the control. In our October 2026 source-matching method, image fingerprints, length and hashes are combined rather than trusting filenames. Keep the reason for the proposed relationship so a reviewer knows whether the group was found by exact bytes, shared source or visual resemblance.

1. 01Group exact bytesPreserve paths and source metadata
2. 02Review similar footageDistinguish takes and derivatives
3. 03Check usagesProtect recipe references
4. 04Archive reviewed copiesValidate catalog relationships

Preserve trims and technical derivatives

A trim, proxy, constant-rate intermediate and final export can share visible content while serving different roles. Record parent source, interval and transformation. Do not force them into one catalog entry merely because the scene looks the same.

1. Original
2. Mirrored copy

Video review proxies: small files, traceable notes covers review proxies and their timeline mapping. A proxy might be expendable after review under the project’s retention policy, but that decision is different from declaring it a duplicate original. Keep the approved output and its source relationship intact. If a derivative becomes the active editing source, retain enough conversion information to reproduce it and enough provenance to reach the original recording.

Check usage before proposing consolidation

Search recipes, project files, review notes and manifests for each candidate ID or path. A file that appears unused in the current folder may be referenced by an older approved ad. Keep those usage records in the proposal.

If two catalog IDs resolve to identical bytes, decide whether to retain aliases or update references to one canonical ID. Make that update reviewable and validate affected recipes. Do not assume a filesystem move updates project references automatically. The purpose is to simplify selection without breaking the ability to reconstruct an approved cut or locate the asset a reviewer meant.

RelationshipHandling

• RelationshipIdentical bytesHandlingReview canonical location and aliases
• RelationshipSame source, new trimHandlingPreserve interval relationship
• RelationshipSame scene, another takeHandlingInspect action before grouping
• RelationshipTechnical derivativeHandlingPreserve role and parent source

Archive through a reversible plan

Prepare a manifest of candidate paths, chosen canonical records, relationship evidence, usages and proposed destination. Move copies out of active selection only within the owner’s authorized process. Keep restoration information.

In our production in October 2026, “delete” decisions become archive proposals rather than hard deletion of the owner’s material. Archive video ad assets without losing the next edit covers archiving ad assets. A reversible plan is useful because duplicate detection can reveal relationships without fully explaining rights, provenance or future editorial value. Keep ambiguous groups active until someone resolves them. A smaller library is not automatically a better library if its useful alternatives have disappeared.

Validate the library after the change

Resolve catalog entries, open affected recipes and inspect a sample of moved or aliased assets. Confirm that review notes still point to the intended material. Record what changed and which candidates remain unresolved.

Turntake’s content library approach describes Turntake’s content-library approach. This method is a catalog maintenance workflow, not a promise of an available automatic cleanup feature.

For a hypothetical group containing an original, a trimmed clip and a smaller review copy, keep three role records even if the preview images look identical. The original preserves capture evidence, the trim defines the usable action and the proxy supports review. The maintenance decision may remove an unnecessary physical copy, but it should preserve those distinct relationships in the catalog. That is what allows the next editor to find the right ingredient without restoring an entire archive.

Review two almost identical demonstrations

Suppose two takes show the same product opening, but one includes a clear view of the latch while the other hides it behind a hand. Keep the first as a proof candidate and record the second’s limitation. Their perceptual similarity is useful for grouping, but their editorial value differs. A reviewer should see the relevant action before proposing consolidation. This is a suggested catalog check that protects useful coverage while still allowing unnecessary byte-identical staging copies to be managed through the archive plan.

Who wrote this We made the ads first, for SmartSolar in Denmark, Sweden and Ireland. Then we built Turntake. Where Turntake comes from