How to find duplicate tracks in a DJ library.

How to find duplicate tracks in a DJ library

In a library growing for years, the same track appears several times: a repeated download, a backup copy inside the music folder, a reimport after changing software, or a file renamed with a version suffix. Duplicates do more than take up space: they fragment playlists, clutter analysis, and put a good copy alongside a worse one under the same name. Detection has two different levels, and only one is mathematical proof.

---

1. Why libraries acquire duplicates

Duplicates arrive through dull, predictable routes. Repeated downloads because the browser saved the second copy under another name. Backup folders inside the library, indexed again by the software. Software migrations importing the collection twice. Format conversions retaining both original WAV and derived MP3. And different edits of one track: original, radio edit, and extended versions sharing a title.

Before using any tool, recognise three coexisting cases: identical files, versions sounding different, and the same track at different qualities. Cleanup only makes sense when you know which case you are dealing with, so the method has two passes: exact matches first, suspicious ones second.

---

2. The exact method: SHA-256 hash

An exact duplicate is detected through the complete file’s fingerprint: the SHA-256 hash, defined in NIST FIPS 180-4, summarises a file’s bytes in a unique string. Two files with the same hash are byte-for-byte identical; different fingerprints mean the files differ in some way.

The advantage is proof rather than suspicion: no listening or guessing is needed. The cost is that it only works for identical files. The same track as a 320 kbps MP3 and a WAV never shares a hash, even if it is the same recording; neither does a file with a corrected tag, despite sounding identical.

The hash pass also comes first because it is cheapest: it runs on files as they stand, without opening them in a player or listening, and returns closed groups you can resolve in one sitting. Group by fingerprint and choose which copy to keep within each group. Anything outside that pass enters the second.

---

3. Similarity matching and its limits

The second pass seeks candidates: tracks whose durations differ by one second or less and whose normalised title, artist, or filename reach similarity of 0.85 or greater, measured by combining Jaccard and Levenshtein distances on the strings.

Honesty matters here. Equal duration proves nothing: many tracks fall within that margin without being the same recording. The method does not listen to audio: there is no acoustic fingerprint, no Chromaprint, or other audio-fingerprinting system. If two recordings sound identical but come from different masters, or the same take is re-encoded with codec-related duration changes, this pass may not group them. Second-level results are candidates for listening review, not verdicts.

---

4. Which copy to keep

When a group contains multiple copies and keeping one makes sense, the recommendation ranks technical criteria: file quality, penalised when the spectral ceiling suggests transcoding; bit depth; sample rate; clipping; direct-current offset; and tag completeness, including ISRC where present.

The logic is to keep the copy with the strongest technical foundation and discard the already degraded one. If copies are equivalent, structure decides: retain the one best fitting your physical folder organisation, as explained in library organisation. Tag-comparison fields come from the ID3 standard, the same ones covered in bulk metadata editing.

---

5. A cleanup workflow without blind deletion

1. Hash pass. Detect exact SHA-256 duplicates and resolve each group by choosing the copy to retain.
2. Similarity pass. Generate candidates with close durations and similar tags; confirm them by listening, one by one.
3. Choose the copy. Apply the quality ranking or, for ties, keep the one best fitting your structure.
4. Relink or replace. Update the library so playlists and analysis point to the retained copy.
5. Controlled deletion. Nothing is deleted blindly: use a PowerShell or Bash script dry-run, an audit CSV showing what would be deleted, and undo guidance. Review and confirm yourself. A dry-run is the script version that only displays what it would do without deleting anything: read it carefully, correct the list if an entry is wrongly excluded, and execute for real when the groups make sense.
6. Second run. Run detection again: the goal is an empty hash pass and no similarity candidates still awaiting review.

If a group has no clearly better copy, leaving it alone is fine: keeping both and reviewing by ear later is better than blind deletion. Library cleanup is a repeated process, not a one-off event.

Before touching the library, have an up-to-date backup: if cleanup goes wrong, restoration is cheaper than rebuilding. The rule and files to back up are in 3-2-1 backups, and cleanup results are checked in the series’ next guide, the audio quality checklist.