Skip to content

JSON Duplicate Remover

Remove duplicate elements from a JSON array using deep structural equality, or dedupe objects by a chosen key. Pretty output.

What this does

Paste a JSON array, get the same array back with the repeats gone. That’s the whole job.

The interesting part is how it decides what counts as a repeat. Two values match when they’re structurally equal, not just when their text happens to line up. So {"id": 1, "name": "Ada"} and {"name": "Ada", "id": 1} are treated as the same record even though the keys appear in a different order. Most naive dedupe scripts miss that and leave you with phantom duplicates.

Numbers, strings, booleans, null, nested objects, arrays inside arrays: all of it gets compared down to the leaf. If two entries describe the same thing, only the first one survives.

Two ways to dedupe

There’s a deep-equality mode and a by-key mode, and they answer different questions.

Deep equality keeps an element only if no earlier element is identical all the way through. Use this when a row is a duplicate only if every single field matches.

Dedupe by key is for arrays of objects where one property is your real identity. Pick a key like id or email, and the tool keeps the first object it sees for each distinct value of that key, dropping the rest. Two users with the same id but slightly different name? By-key mode collapses them to one. Deep mode would keep both, because the names differ.

The key dropdown only fills in when your array actually holds objects, so it picks up id, email, sku, whatever your data carries. Items that don’t have the chosen key, or aren’t objects at all, are left alone rather than silently deleted.

How to use it

Drop your array into the left box. The right box updates as you type, so there’s no Generate button to hunt for. Up top you’ll see three numbers: how many elements came in, how many remain, and how many got removed. Quick sanity check before you trust the output.

Switch to “Dedupe by key” when you want identity to hinge on a single field, then choose that field from the dropdown. Hit Copy to grab the cleaned JSON, or Download to save it as deduped.json. The output is always pretty-printed with two-space indentation, ready to commit or paste somewhere.

Made a mess? Sample loads a tiny example you can poke at, and Clear wipes the slate.

Good to know

Everything runs in your browser. Your JSON never gets uploaded, logged, or sent to an API, which matters when the array is a database export full of real emails.

Order is preserved. The first occurrence of any value stays put in its original position, so the result isn’t reshuffled. If parsing fails, you get the actual error message from the JSON parser in red, pointing at roughly where things went sideways, instead of a vague “something broke.”

One limit worth naming: the input has to be a JSON array at the top level. A bare object or a number won’t work, and the tool tells you so. Wrap your data in [ ] if you’ve only got a single object and you’re testing.

Common questions

Does key order in objects matter? Nope. {"a":1,"b":2} and {"b":2,"a":1} are the same element here. Keys get sorted internally before comparison.

What happens to objects missing the dedupe key? They’re kept as-is. The tool won’t drop a record just because it lacks the field you picked, so you never lose data by accident.

Which copy survives when there are duplicates? The first one in the array. Later matches are the ones removed.

Can it handle nested arrays and deeply nested objects? Yep. Comparison goes all the way down, so two complex objects only match if every nested value matches too.

Is there a size limit? No hard cap, but since it runs client-side, a multi-megabyte array will lean on your device’s memory. A few thousand records is no sweat.

json duplicates dedupe array cleanup

Related Tools

More in Data Tools