Transforming text
This topic covers the commands that reshape text: putting lines in order, dropping duplicates, swapping characters, reflowing paragraphs to a width, counting what is there. Its companion, Viewing and joining text, covers printing and combining files; this topic is about changing the text itself.
This page is the map.
The commands
Ordering lines
| Command | Purpose |
|---|---|
sort |
Sort lines — alphabetically, numerically, and many other ways. |
shuf |
The opposite of sorting: put lines into a random order. |
tsort |
Topological sort — order items so that each comes after the things it depends on. |
Filtering lines
| Command | Purpose |
|---|---|
uniq |
Collapse or report adjacent repeated lines. |
Substituting characters
| Command | Purpose |
|---|---|
tr |
Translate, squeeze, or delete individual characters. |
Whitespace
Reflowing and paginating
| Command | Purpose |
|---|---|
fmt |
Reflow paragraphs to a target width. |
fold |
Hard-wrap long lines at a fixed width. |
pr |
Paginate and columnate text for printing. |
Indexing and counting
| Command | Purpose |
|---|---|
ptx |
Produce a permuted index of the words in a file. |
wc |
Count the lines, words, and bytes in a file. |
Two commands that need sorted input
A theme worth knowing before you start: uniq only collapses adjacent duplicate lines, so duplicates scattered through a file are not caught unless the file is sorted first. sort and uniq are almost always used together — and sort -u does both jobs in one step.
Where to start
sort is the workhorse of the topic and the one with the most depth — start there.