Back to blog

How I Built a Duplicate Finder That Sees What Your Files Look Like

Every duplicate finder on the Mac App Store works the same way: compare file names and sizes. If two files aren't identical down to the last byte, they're "not duplicates."

But that's not how you and I think about duplicates.

I took a photo on my iPhone, airdropped it to my Mac, then imported it into Photos. Three copies of the same image. Different filenames. Different formats. Different file sizes. Every duplicate finder on the market said: "No duplicates found."

🤯

I had 47 GB of invisible duplicates. Every duplicate finder on the market missed them. So I built a tool that finds them.

The Problem: Same Photo, "Not a Duplicate"

Here's what most people don't realize about their photo and video libraries:

Other apps compare raw file data. Change anything — different name, different format, different resolution — and they see two completely different files. They'll never match.

I needed a tool that could look at what's inside the file and decide: "These look like the same photo" or "These look like the same video."

The Idea: Compare What Files Look Like, Not Their Data

The solution is surprisingly simple in concept. Instead of comparing the file's raw data, compare what the file looks like to a human.

For photos, think of it like this:

  1. Shrink the image way down to a tiny thumbnail
  2. Strip away the color
  3. Create a compact "visual fingerprint" from what's left

Two photos of the same scene will produce nearly identical fingerprints — even if one is HEIC and the other is JPG, or one is full resolution and the other is a compressed thumbnail.

For videos, it's similar but more involved. You sample frames throughout the video and build a fingerprint that represents the entire video's visual story. Two videos of the same footage match — even at different resolutions or in different formats.

Why I Built It in Rust

I needed raw speed. Scanning thousands of photos and videos is heavy work. Rust gave me native performance with numbers that speak for themselves:

500
images per second
<15 MB
app size
0
files uploaded

For the desktop app, I used Tauri 2 — it lets you build native Mac apps that are fast, small, and feel native. The UI is built with Svelte 5.

🔒

100% private. The whole app runs locally. No files are uploaded anywhere. No internet needed. Your photos never leave your Mac.

The Scanning Experience

I obsessed over making the scanning feel fast and transparent. The UI shows three clear phases:

1

Finding your files — A file count ticks up as Dup discovers all your photos and videos

2

Analyzing content — A progress ring shows each file being fingerprinted

3

Matching duplicates — A verification step double-checks every match

Each phase shows rotating tips ("Did you know? The average Mac user has 10-50 GB of duplicate files") to keep you engaged during longer scans.

When duplicates are found, they're grouped visually. For videos, you see side-by-side thumbnail storyboards so you can confirm the match yourself. Smart Select auto-picks the best file to keep based on quality, date, and size.

What I Learned from Real Users

People have way more duplicates than they think. Beta testers consistently found 5-15 GB of duplicates in their first scan. One tester found 47 duplicate videos that three other apps completely missed.

"Does this upload my files?"

The #1 question during beta. When people learned everything runs locally, they were twice as likely to try it.

🎬

Video duplicates are the hidden goldmine. Most duplicate finders only handle photos. But finding a 2 GB duplicate video is worth more than finding fifty 3 MB duplicate photos. Videos are where the real storage savings hide.

Try It Free

Dup is free on the Mac App Store. The free version finds all duplicates and lets you clean up to 500 MB. Pro ($14.99/year) unlocks unlimited cleanup, Smart Select, and cloud drive support.

Find the duplicates your current tools miss

Dup finds similar photos and videos — not just exact copies. Free on the Mac App Store.