dedupe

Fast, secure, efficient backup program

🆔 A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.

Deduplication tool for yarn.lock files

Scalable identity resolution, entity resolution, data mastering and deduplication using ML

A powerful and modular toolkit for record linkage and duplicate detection in Python

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

Best-Effort Extent-Same, a btrfs dedupe agent

Make CSS easier and more maintainable by using JavaScript

🆔 Command line tool for deduplicating CSV files

🆔 Examples for using the dedupe library

Identifying and removing near-duplicate images using perceptual hashing.

A fast file deduplicator

Fast block-level out-of-band BTRFS deduplication tool.

📧 CLI to deduplicate mails from mail boxes

Check your files for data corruption and run quick file deduplication

Daxus is a server state management library for React that provides full control over data, leading to a better user experience.

Yet Another Dupes Finder

Curated list of awesome software and resources for Senzing, The First Real-Time AI for Entity Resolution.

A simple command line interface to the datamade/dedupe library.

Self-contained C# library for data deduplication using Sqlite