Cómo usar: Remove Duplicate Lines
- Paste your list into Input (one item per line) or open a
.txt/.csvfile. - Choose how lines are compared: Ignore case treats “Apple” and “apple” as the same, Trim spaces ignores leading and trailing whitespace.
- Optionally pick a Sort order — alphabetical, natural numeric, by length, reversed or shuffled.
- Copy or download the result. The counters show how many duplicates and empty lines were removed.
What does a duplicate line remover do?
It goes through text line by line and keeps only the first (or last) occurrence of each line, deleting the repeats while preserving the original order. It's the quickest way to clean up email lists, keyword lists, URLs from a crawl, log entries, CSV columns, tag lists or any export that contains the same value more than once — without opening a spreadsheet or writing a script. Everything runs in your browser, so even lists with hundreds of thousands of lines are processed instantly and privately.
Comparison options
| Option | Effect | Example |
|---|---|---|
| Ignore case | Lines that differ only in capitalization count as duplicates; the first spelling is kept. | Apple, apple, APPLE → Apple |
| Trim spaces | Leading and trailing spaces and tabs are removed before comparing and in the output. | " red " and "red" → red |
| Remove empty lines | Blank or whitespace-only lines are dropped. | — |
| Keep only lines that never repeat | Removes every copy of a duplicated line, leaving values that occur exactly once. | a b a c → b c |
| Keep last occurrence | Each value stays at the position of its last appearance — useful for logs where the latest entry wins. | a b a → b a |
Sorting lines
- A → Z / Z → A sorts alphabetically using your browser's locale-aware collation, so accented letters sort next to their base letters (
éwithe). - Natural sorts numbers inside text by value:
file2,file10,file100instead of the plain alphabeticalfile10,file100,file2. - By length orders lines from shortest to longest (or the reverse), handy for finding outliers.
- Shuffle randomizes the order with an unbiased Fisher–Yates shuffle — for raffles, sampling or randomized test data.
Equivalent command-line recipes
On Linux or macOS, sort -u file.txt removes duplicates but also sorts; awk '!seen[$0]++' file.txt keeps the original order like this tool's default; sort file.txt | uniq -u keeps only lines that never repeat; sort -V is a natural sort. This page gives the same results without a terminal and shows exactly which lines were duplicated and how often.
Need to compare two lists instead? Use the Diff Checker. To change capitalization of the whole list first, run it through the Case Converter.
También se busca como: dedupe lines, remove duplicates, unique lines, sort lines, delete duplicate lines, distinct, sort alphabetically, shuffle lines.
Herramientas que combinan con: Remove Duplicate Lines
- Contador de palabras — Pega o escribe tu texto para contar palabras y caracteres en tiempo real, además de calcular el tiempo de lectura y de pronunciación, la densidad de palabras clave y comprobar la longitud de los títulos SEO, las metadescripciones y las publicaciones en redes sociales.
- CSV to JSON — Paste CSV or TSV and get a JSON array of objects.
- Slug Generator — Convert any title into a clean, lowercase URL slug — with proper transliteration of accents, umlauts and Cyrillic, optional stop-word removal and a bulk mode for whole lists.
- Markdown Table Generator — Build a table in a spreadsheet-like editor — or paste cells from Excel, Google Sheets or CSV — set column alignment and get a neatly aligned Markdown table (GitHub-flavored) or clean HTML.