How are duplicates identified?

Two lines are considered duplicates if they match after optional whitespace trimming. The comparison is case-insensitive, so 'Hello' and 'hello' are treated as the same line and only the first is kept.

Does the order of unique lines change?

On this page the output keeps the original order of the first occurrence of each line, because Keep original order is selected. Pick another mode in the side panel, such as A → Z, to sort the unique lines after duplicates are removed.

Can I remove nearly-duplicate lines?

Lines are matched whole, ignoring letter case (and surrounding whitespace when Trim whitespace is on), so lines that are similar but not identical are all kept. To merge near-duplicates, first use the regex replace tool with a pattern that normalises the variations, then remove duplicates here.

How do I see how many duplicates were removed?

The count above the output shows lines in and lines out, for example 120 lines → 87 lines. The difference is the number of duplicates removed, plus any empty lines if Remove empty lines is on.

Remove duplicate lines

Paste your list and remove all duplicate lines in one click. The tool keeps the first occurrence of each unique line and discards the rest. Optionally trim whitespace before comparing, sort the result alphabetically, and remove empty lines. All processing runs locally in your browser.

Input

Output

SORT OPTIONS

Remove duplicatesTrim whitespaceRemove empty lines

How it works

The input is split on newlines to produce an array of lines; a final newline does not add an empty line. Trim whitespace applies String.trim() to each line. Remove duplicates uses a Set keyed on the lowercased line to preserve the first occurrence. Keep original order skips sorting. The other sort modes use Array.sort() with an Intl.Collator for alphabetical ordering, the number each line starts with for numeric ordering, or a character count from Intl.Segmenter for length-based modes. Shuffle uses the Fisher-Yates algorithm with Math.random(). The result array is joined back with newlines. All processing runs in JavaScript in your browser.

Processing runs in your browser

All processing happens locally in your browser tab. Our servers are not involved at any point.

Related operations

For swapping recurring substrings across the list, try find and replace. To compare two sorted lists side by side, use text diff. For checking item counts after deduplication, see the word counter.

Frequently asked questions

How are duplicates identified?: Two lines are considered duplicates if they match after optional whitespace trimming. The comparison is case-insensitive, so 'Hello' and 'hello' are treated as the same line and only the first is kept.
Does the order of unique lines change?: On this page the output keeps the original order of the first occurrence of each line, because Keep original order is selected. Pick another mode in the side panel, such as A → Z, to sort the unique lines after duplicates are removed.
Can I remove nearly-duplicate lines?: Lines are matched whole, ignoring letter case (and surrounding whitespace when Trim whitespace is on), so lines that are similar but not identical are all kept. To merge near-duplicates, first use the regex replace tool with a pattern that normalises the variations, then remove duplicates here.
How do I see how many duplicates were removed?: The count above the output shows lines in and lines out, for example 120 lines → 87 lines. The difference is the number of duplicates removed, plus any empty lines if Remove empty lines is on.

Last reviewed September 30, 2026

Other pages for this tool

Related tools

Background reading: Complete guide to text tools