Peiosutils

This topic covers the commands that change the file system: creating files and directories, copying and moving them, removing them, and adjusting their size. If a command brings a file into existence, takes one out of it, or duplicates one, it is here.

This page names every command, and then explains the one theme that runs through the whole topic on Peios: what happens to a file's security descriptor when the file is created or copied.

The commands #

Every file has a security descriptor #

On Peios, every file and directory carries a security descriptor — the record of who owns it and who may do what to it. Access to a file is decided from its descriptor; see Security descriptors for the full picture.

That fact shapes two groups of commands in this topic.

Creating a file gives it a descriptor #

When mkdir, mkfifo, mknod, or touch creates a new object, that object needs a security descriptor. By default it gets one inherited from the directory it is created in — the new object picks up the access rules its parent directory hands down.

All four of these commands share one set of options — the creation flags — that let you set the new object's descriptor explicitly instead of taking the inherited default: name an owner, set a different group, lock the object so it does not inherit, give it an integrity label, or supply a complete descriptor. The flags are documented in full on the mkdir page; mkfifo, mknod, and touch each link back to it.

Copying a file makes a fresh descriptor — unless you preserve #

cp creates new files, and a cross-file-system mv does too. A brand-new copy gets a brand-new descriptor inherited from its destination directory — it does not carry the source file's security across by default.

When you do want the copy to keep the source's owner, access rules, timestamps, or other attributes, that is the job of the --preserve family of options. cp and mv share an identical --preserve surface; it is documented on the cp page.

Removing a file checks whether you may write it #

rm and shred both make a decision based on whether you can write a file. rm prompts before removing a file you cannot write to; shred's --force overrides a file's protection so it can be destroyed. Both judge "can you write this?" by a live access check against the file's security descriptor — not by any decorative metadata on the inode. Each command's page explains exactly where that check is made.

Where to start #

For copying — and for the whole --preserve model — read cp.

For creating files and directories, and the shared creation flags, read mkdir.

For removing things safely, read rm.

cp copies files and directories.

cp [options] source dest
cp [options] source... directory
cp [options] -t directory source...

In the first form, cp copies one source to one dest. In the other two, it copies any number of sources into a directory, keeping their names.

$ cp report.txt report-backup.txt
$ cp report.txt notes.txt /home/jack/archive/

Copying directories #

cp refuses to copy a directory unless you ask it to descend into one:

-a is the option to use when you want the copy to be as close to the original as possible — same structure, same security, same timestamps. A plain -r copies the contents but, as the next section explains, gives the copies fresh security of their own.

What a copy's security descriptor is #

This is the part of cp that is specific to Peios, so it is worth being precise.

Every file carries a security descriptor — the record of who owns it and who may do what to it. When cp copies a file, it creates a new file, and that new file needs a descriptor.

By default, a copy gets a fresh descriptor — it does not inherit the source's. Specifically, a plain cp:

• gives the copy a descriptor inherited from the destination directory, exactly as if you had created a new file there;
• makes you the owner of the copy;
• gives the copy fresh timestamps — the copy's modification time is "now".

This is usually what you want. A file copied into your home directory should be governed by your home directory's rules and owned by you, regardless of where it came from. But when you need the copy to keep something from the source, you ask for it explicitly — with the --preserve family.

The --preserve family #

--preserve carries chosen attributes from the source onto the copy. The attributes you can preserve:

You rarely name those individually. These options select sensible sets:

--sd versus --sd-explicit #

Both carry the source's security across; they differ in how they treat inherited rules.

• --sd copies the descriptor exactly — inherited rules and all. The copy ends up with precisely the access rules the source had, even if it lands in a directory that would have handed down something different. Use it when the copy must be a security-exact duplicate.
• --sd-explicit copies only the rules that were set explicitly on the source, and lets the destination directory supply inheritance for the rest. Use it when copying a file into a different directory and you want the file's own tailored rules to come along while still picking up the new location's inherited rules.

For what "inherited" versus "explicit" means, see Inheritance.

When a preserve cannot be honoured #

A requested preserve is a firm instruction. If cp cannot carry an attribute across — for example, writing the copy's SACL requires the SeSecurityPrivilege privilege, and the caller does not have it — cp treats that as a hard error and the copy fails. It does not quietly produce a copy that is missing what you asked to preserve.

There is one exception, and it is a matter of definition rather than a softening of the rule. security and xattrs name extended attributes, and some filesystems have no extended attributes at all — FAT, 9p, and a squashfs image built without an attribute table are the ones you will meet. Copying from such a filesystem carries nothing, because there was nothing there: the preserve is satisfied vacuously, and the copy succeeds.

That is narrow on purpose. It applies only to reading the source. If the source does have extended attributes and the destination filesystem cannot hold them, the preserve genuinely failed and cp still stops:

# cp -a /home/jack/notes /mnt/usb-fat/
cp: failed to set extended attributes on '/mnt/usb-fat/notes': Operation not supported

The distinction matters most for -a, which requests both attribute classes on every file it touches. Without it, one xattr-less filesystem anywhere in a tree would abort the whole copy partway through.

Overwriting an existing destination #

By default, if the destination already exists, cp overwrites it. These options change that.

Symbolic links in the source #

When a source is a symbolic link, cp can copy the link itself or the file it points to.

Other ways to copy #

cp can also produce something other than a plain duplicate.

Other options #

Exit status #

mv moves files and directories from one place to another. Moving a file within the same directory, under a new name, is how you rename it.

mv [options] source dest
mv [options] source... directory
mv [options] -t directory source...

$ mv draft.txt report.txt                 # rename
$ mv report.txt notes.txt /home/jack/done/ # move into a directory

A move is either a rename or a copy-and-delete #

mv does its job in one of two ways, and which one decides what happens to the file's security.

Within one file system, a move is a rename. Nothing is copied: the file stays exactly where it physically is, and only its name and directory entry change. The file keeps its inode, and so it keeps its security descriptor unchanged — every owner, access rule, audit rule, and timestamp is exactly as before. A rename does not re-evaluate inheritance; a file does not pick up new rules just because it landed in a new directory.

Across file systems, a move is a copy followed by a delete. When the destination is on a different file system, mv cannot just rename — it copies the data to a new file on the destination, then removes the original. That new file is a genuinely new object, so, exactly as with cp, it gets a fresh security descriptor inherited from the destination directory unless you ask otherwise.

So: rename a file and its security is untouched; move it to another file system and the copy starts fresh.

Preserving security on a cross-file-system move #

For the copy-and-delete case, mv carries the same --preserve family that cp does — an identical set of options, behaving the same way. They let a cross-file-system move carry the source's owner, access rules, timestamps, and other attributes onto the new file instead of taking the destination directory's inherited default.

The full --preserve reference — the attribute list, --sd, --sd-explicit, --preserve-all, --no-preserve, and what happens when a preserve cannot be honoured — is on the cp page. Everything there applies to mv unchanged.

On a same-file-system move the --preserve options have nothing to do: a rename already keeps every attribute, because it is the same file.

Overwriting an existing destination #

By default mv overwrites an existing destination. These options change that. When more than one of -i, -f, -n is given, the last one wins.

Other options #

Exit status #

rm removes files. With -r, it removes directories and everything inside them.

rm [options] file...

$ rm draft.txt old-notes.txt
$ rm -r build/

Removal is not reversible — once rm removes a file, its directory entry is gone. rm has a few safety behaviours, below, but the basic rule stands: think before you confirm.

Removing directories #

rm will not remove a directory unless you say so:

The write-protected prompt #

By default — when you have not passed -f and rm is running interactively — rm prompts before removing a file you cannot write to:

$ rm policy.db
rm: remove write-protected regular file 'policy.db'?

The point of that prompt is to catch mistakes: a file you cannot write is one you most likely did not mean to delete.

"Write-protected" here means a real access check. rm decides it by asking the kernel whether your token may write the file — a live check against the file's security descriptor. It is not read off any decorative metadata on the inode; it is the same authority question that any write to the file would face. So the prompt is honest: if rm says a file is write-protected, it is a file the access model would genuinely stop you writing.

(The check is about write access to the file's contents. Whether you may actually remove the file is a separate question, decided by its own access right — so you can be prompted about a write-protected file and still be allowed to delete it. The prompt is a courtesy check, not the authorisation.)

Controlling the prompts #

The root failsafe #

rm refuses to recursively remove / — the root of the file system — because doing so by accident would be catastrophic.

Other options #

rm and shred #

rm unlinks a file — it removes the name, and the space becomes free. The file's data may still be physically present on the device until something else overwrites it. When you need the contents to be genuinely unrecoverable, use shred, which overwrites the data before removing the file.

Exit status #

rmdir removes directories — but only empty ones. If a directory still has anything in it, rmdir leaves it alone and reports the failure.

rmdir [options] directory...

$ rmdir old-cache/

That refusal to touch a non-empty directory is the whole point of rmdir: it is the safe way to remove a directory you believe is empty. If it is not empty, you find out instead of losing the contents. To remove a directory together with everything inside it, use rm -r.

Options #

Exit status #

mkdir creates directories.

mkdir [options] directory...

$ mkdir reports
$ mkdir -p projects/2026/q2

Creating parent directories #

By default mkdir creates exactly the directories you name, and fails if a parent is missing. -p removes that restriction:

The creation flags #

A new directory needs a security descriptor — the record of who owns it and who may do what to it. By default, a new directory inherits its descriptor from the directory it is created in: it picks up the access rules its parent hands down. For most directories that is exactly right, and you pass no extra options at all.

When you need the new directory to have a different descriptor, the creation flags set it explicitly. These five flags are shared, unchanged, by mkdir, mkfifo, mknod, and touch — whenever one of those commands creates a new object, it accepts this same set. This page is the full reference for them.

A SID argument is either a literal S-1-… value or a well-known alias — BA for the built-in administrators, for example.

--label takes one of these levels, lowest to highest: untrusted, low, medium, medium-plus, high, system, protected. The label applies the standard rule for files and directories — a principal below the object's level cannot write to it.

Inherited, or explicit #

The default and --no-inherit are the two ends of a choice.

• Inherited (the default) — the new directory tracks its parent. Rules the parent hands down apply to it, and as the parent's rules change the directory follows along. This keeps a tree of directories consistent without per-directory effort.
• --no-inherit — the new directory takes nothing from its parent. Its access is locked to its owner, and it does not follow the parent's rules. Use it to carve out a directory whose security stands apart from the tree around it.

See Inheritance for the full model.

--sddl for a complete descriptor #

The four flags above are shortcuts for common adjustments. When you need to specify the whole descriptor — a particular set of access rules, audit rules, owner, and group all at once — --sddl takes it as a single string in the security-descriptor definition language:

$ mkdir --sddl 'O:BAG:BAD:(A;OICI;FA;;;BA)' secure-area

--sddl is mutually exclusive with --owner, --group, --label, and --no-inherit — it already says everything those flags would say, so combining them is rejected.

How the descriptor is applied #

mkdir creates the directory first and then applies the descriptor to it. The two steps are not separately visible — mkdir either finishes with the directory created and secured as asked, or fails as a whole. If applying the descriptor fails, the half-created directory is not left behind.

Exit status #

mkfifo creates named pipes, also called FIFOs.

mkfifo [options] name...

$ mkfifo events

A named pipe is a file that two programs use to pass a stream of data: one writes into it, the other reads from it, and the data flows through in order — first in, first out, which is what "FIFO" stands for. Unlike an ordinary pipe between two commands, a named pipe has a name on the file system, so the two programs do not have to be started together or be related to each other.

Setting the new pipe's security #

A FIFO is a file, and a new file needs a security descriptor. By default a new FIFO inherits its descriptor from the directory it is created in.

mkfifo accepts the shared creation flags — --owner, --group, --label, --no-inherit, and --sddl — to set that descriptor explicitly instead. They behave exactly as they do for mkdir; the full reference is on the mkdir page.

Exit status #

mknod creates special files — files that are not data but an interface to something else: a device, or a named pipe.

mknod [options] name type [major minor]

$ mknod /dev/mydisk b 8 0
$ mknod backlog p

The type argument #

The type says what kind of special file to create:

For a block or character device (b, c, u) you must also give a major and a minor number. The major number selects which driver handles the device; the minor number tells that driver which specific device is meant. For a pipe (p), the major and minor numbers are omitted — a pipe has no driver behind it.

A major or minor number is read as hexadecimal if it begins with 0x, as octal if it begins with 0, and as decimal otherwise.

mknod NAME p and mkfifo NAME do the same thing; mkfifo exists as the clearer way to ask for just a pipe.

Setting the new file's security #

A special file needs a security descriptor like any other file, and by default a new one inherits its descriptor from the directory it is created in.

mknod accepts the shared creation flags — --owner, --group, --label, --no-inherit, and --sddl — to set that descriptor explicitly. They behave exactly as they do for mkdir; the full reference is on the mkdir page.

Exit status #

touch does one of two things, depending on whether the file already exists:

• if the file does not exist, touch creates it, empty;
• if the file does exist, touch updates its timestamps to the current time.

touch [options] [file...]

$ touch notes.txt        # create notes.txt if absent, else bump its times

The "create an empty file" behaviour is the everyday use. The "update timestamps" behaviour matters to tools that decide what work to do by comparing file times — touch is how you tell such a tool that a file should be considered freshly changed.

Which timestamps, and to what #

By default touch sets both the access time and the modification time to the current moment. These options narrow or redirect that.

Not creating a file #

To use touch purely to update timestamps, and never to create anything:

Setting a created file's security #

When touch creates a file, that new file needs a security descriptor, and by default it inherits one from the directory it is created in.

touch accepts the shared creation flags — --owner, --group, --label, --no-inherit, and --sddl — to set that descriptor explicitly. They behave exactly as they do for mkdir; the full reference is on the mkdir page.

These flags apply only on the create path. If touch is updating the timestamps of a file that already exists, there is no new descriptor to set, and the creation flags have nothing to do — they do not alter an existing file's security. Pair them with -c and they never take effect at all, since -c means nothing is ever created.

Exit status #

ln creates links — extra ways to reach a file.

ln [options] target link_name
ln [options] target
ln [options] target... directory
ln [options] -t directory target...

$ ln -s /opt/app/bin/app /lcl/bin/app   # a symbolic link
$ ln data.csv data-archive.csv                # a hard link

Hard links and symbolic links #

There are two kinds of link, and they are genuinely different things.

A hard link is another name for the same file. The file's data exists once; a hard link is an additional directory entry pointing at it. Every hard link to a file is equal — none is "the original" — and the file's data stays as long as at least one hard link remains. Hard links cannot span file systems, and cannot be made to directories.

A symbolic link is a small file that simply contains a path. When something opens a symbolic link, it is redirected to whatever that path names. The symbolic link and its target are separate files: if the target is moved or removed, the link still exists but now points at nothing — it "dangles". Symbolic links can point anywhere, across file systems and at directories.

ln creates hard links by default, and symbolic links with -s. When in doubt, -s — a symbolic link's behaviour is the easier one to reason about.

The two also differ in how they are secured. A hard link is not a new file, so creating one involves no new security descriptor — every name for the file shares the file's existing one. A symbolic link is a new file, and like any new file it inherits its own descriptor from the directory it is created in.

The forms #

• ln target link_name — create one link named link_name.
• ln target — create a link to target in the current directory, with the same final name.
• ln target... directory — create a link to each target inside directory.
• ln -t directory target... — the same, with the directory named first.

Options #

Inspecting links afterwards #

readlink prints where a symbolic link points. The low-level link command makes a single hard link with no options, for scripts that want exactly that and nothing else.

Exit status #

link and unlink are the bare, single-purpose versions of two operations that ln and rm also perform. Each does exactly one thing, takes exactly its operands, and has no options. They exist for scripts that want one precise operation with no surrounding behaviour — no prompting, no link kinds to choose, no recursion.

link #

link file1 file2

link creates one hard link: a new name, file2, for the existing file file1. file1 must exist; file2 must not.

$ link data.csv data-archive.csv

That is the whole command. It performs the link operation directly and reports whether it succeeded. For anything more — symbolic links, replacing an existing destination, linking into a directory — use ln.

unlink #

unlink file

unlink removes one file: it deletes the directory entry file. It takes a single name and removes it directly.

$ unlink stale.lock

unlink does not prompt, does not recurse, and does not remove directories. For removing several files, removing directories, or any of the safety prompts, use rm.

Why they exist #

ln and rm are the commands to use day to day — they are flexible and have the safety behaviours. link and unlink are deliberately rigid: a script that calls unlink will only ever remove a single file, and a reviewer can see that at a glance. The narrowness is the feature.

Exit status #

Both commands:

shred destroys a file's contents. It does not just unlink the file — it overwrites the data, repeatedly, so that what was there cannot be read back even with effort and specialised equipment.

shred [options] file...

$ shred -u -v secret-keys.txt

rm removes a file's name; the data may sit on the device, recoverable, until something else happens to overwrite it. shred overwrites the data on purpose. Use it when a file held something that must not be recoverable.

By default shred overwrites a file three times and then leaves the file in place — empty of its old contents but still present. That default exists because shred is often pointed at device files, which usually should not be removed. To remove the file after shredding it, pass -u.

When shredding actually works #

shred relies on one assumption: that writing to the file overwrites the same physical storage the old data occupied. On a traditional file system that holds. Several common file system designs break it — anything that writes new data to a fresh location instead of in place, keeps snapshots, journals file data, caches copies elsewhere, or compresses. On those, shred cannot guarantee the old data is gone.

Backups and remote mirrors are a separate matter entirely: shred cannot reach a copy of the file that lives somewhere else.

Treat shred as effective on a plain local file system and uncertain otherwise.

Options #

--force and a file's protection #

To overwrite a file, shred has to be able to write it. If the file's security descriptor does not grant you write access, an ordinary shred cannot proceed.

-f (--force) handles that case. When --force is given and a live access check shows you cannot currently write the file, shred rewrites the file's security descriptor to one that grants full access, and then overwrites the file. The reasoning is deliberate: the file is about to be destroyed anyway, and --force is you saying "override this file's protection."

--force can do this only when you are allowed to change the file's security in the first place — that is, when you own the file or hold the right to rewrite its access rules. If you cannot change the descriptor, --force cannot grant itself write access, and the shred still fails. --force overrides a file's protection; it does not manufacture authority you do not have.

Exit status #

dd copies data from one place to another, a block at a time, and can transform the data as it goes. It is the tool for low-level copying — writing a disk image to a device, extracting an exact byte range out of a file, copying from or to a raw device.

dd [operand]...

$ dd if=disk.img of=/dev/mydisk bs=4M status=progress

How dd is told what to do #

dd does not take options in the usual -x form. It takes operands, each written name=value, in any order. With no if= operand it reads from standard input; with no of= operand it writes to standard output.

The operands #

A size value accepts a unit suffix (K, M, G, …). So bs=4M is four mebibytes per block.

Conversions (conv=) #

When dd creates the output file — an of= file that does not already exist — the new file needs a security descriptor, and it inherits one from the directory it is created in.

Input and output flags (iflag=, oflag=) #

What dd prints #

Unless status=none is set, dd prints a summary when it finishes:

16+0 records in
16+0 records out
67108864 bytes (67 MB, 64 MiB) copied, 1.234 s, 54.4 MB/s

The records in / records out counts are written complete+partial — full-sized blocks plus any short final block. status=progress prints the last line periodically during a long copy.

Exit status #

df — "disk free" — reports, for each file system, how much space it has and how much is still available.

df [options] [file...]

With no arguments, df reports on every mounted file system. Given a file, it reports on the one file system that file lives on.

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda2       456G  198G  235G  46% /
/dev/sda1       511M   12M  499M   3% /boot

The default columns #

Each line is one file system:

• Filesystem — the device or source backing it.
• Size — its total capacity.
• Used — space in use.
• Avail — space still available.
• Use% — used space as a percentage.
• Mounted on — where it is attached in the directory tree.

Choosing the units #

By default sizes are shown in blocks. These options change that.

Choosing what to report #

Changing the output #

Sync behaviour #

Exit status #

du — "disk usage" — reports how much space files and directories occupy. Where df tells you about a whole file system, du tells you where the space has actually gone.

du [options] [file...]

$ du -sh /home/jack/projects
4.2G    /home/jack/projects

With no arguments, du works on the current directory. By default it walks the directory tree, prints the cumulative size of each directory (a directory's size includes everything under it), and ends with the total for the argument itself. Individual files are not listed unless you ask for them.

What to count and how deep #

What "size" means #

Following symbolic links #

By default du does not follow symbolic links.

Limiting the output #

Exit status #

truncate sets a file's size to an exact figure — shrinking it, or extending it.

truncate [options] file...

$ truncate -s 0 logfile        # empty the file
$ truncate -s 1G disk.img      # make a 1 GiB file

Shrinking a file discards everything past the new size. Extending it adds space that reads back as zero bytes; that added space is a hole — it costs no actual storage until something writes real data into it, which is how truncate -s 1G can create a "1 GiB file" instantly.

By default, truncate creates a file that does not yet exist (as an empty file, then sized as asked). A file created this way needs a security descriptor, and it inherits one from the directory it is created in.

Specifying the size #

-s (--size) takes the size. A plain number sets the size outright. A unit suffix scales it: K, M, G, T, … where the bare letter is a power of 1024 (K = 1024) and the letter with B is a power of 1000 (KB = 1000).

A size may also begin with a prefix, which makes it relative to the file's current size:

$ truncate -s +4K notes.txt    # 4 KiB larger than it is now
$ truncate -s '<1M' notes.txt  # cap it at 1 MiB, leave smaller files alone

Options #

Exit status #

mktemp creates a temporary file — or, with -d, a temporary directory — gives it a name that is not already taken, and prints that name.

mktemp [options] [template]

$ mktemp
/tmp/tmp.QV8x3kZ1aR
$ mktemp -d
/tmp/tmp.7bNc0wPe2L

The reason to use mktemp rather than inventing a name yourself is safety. A script that picks a fixed name like /tmp/work has two problems: two copies of the script collide, and a name an attacker can predict is a name an attacker can interfere with before the script gets to it. mktemp chooses an unpredictable name and creates the file in the same step, so nothing can slip in between. It prints the name it used, for the script to capture:

work=$(mktemp)
# ...use "$work"...
rm "$work"

The temporary file or directory mktemp creates is a new object, and a new object needs a security descriptor; it inherits one from the directory it is created in.

Templates #

The name comes from a template — a name with a run of trailing X characters, each of which mktemp replaces with a random character. More Xs means a wider range of possible names.

$ mktemp report-XXXXXX.txt
report-a8Kp2Q.txt

With no template, mktemp uses a built-in default that places the file in the temporary directory. A template needs at least three Xs.

Options #

Exit status #

sd is the command-line tool for working with security descriptors on files. Everything this topic describes — owners, DACLs, ACEs, the SACL, integrity labels, inheritance — sd is how you read it and change it from a shell.

sd subcommand path [arguments]

$ sd show ./report.txt
$ sd allow ./report.txt alice:read
$ sd owner ./report.txt BA

Where ls -l shows a file's owner and a summary, and cp --preserve carries a descriptor across, sd is the tool that edits the descriptor directly.

The subcommands #

Naming a principal #

Wherever a subcommand takes a PRINCIPAL, it accepts any of:

See SIDs for what these are.

Naming permissions #

Wherever a subcommand takes PERMS, several notations are accepted, and may be mixed:

Run letters together (rwx) or separate names with commas (read,write,execute).

Inspecting #

sd show #

Prints the descriptor on a path — the owner, the group, the DACL, the SACL.

$ sd show ./report.txt

sd check #

Simulates an access decision: "would this access be allowed?" — without performing it.

$ sd check ./report.txt write
$ sd check ./report.txt read --pid 4821 --explain

sd check is the first thing to reach for when an access is denied and you do not know why. --explain walks the descriptor the same way the kernel does.

Changing the DACL #

The DACL is the list of allow and deny rules. These three subcommands edit it.

sd allow and sd deny #

Add an allow (or deny) rule for one or more PRINCIPAL:PERMS pairs.

$ sd allow ./report.txt alice:read bob:rw
$ sd deny  ./report.txt Everyone:write

Remember that a deny rule, when it matches, wins over any allow — see DACL evaluation.

sd remove #

Drops every DACL rule — allow and deny — for the named principals.

$ sd remove ./report.txt bob carol

Auditing #

sd audit and sd unaudit #

The SACL holds audit rules — see The SACL. sd audit adds one; sd unaudit drops every SACL rule for the named principals.

$ sd audit ./secrets.db Everyone:write:failure

An audit spec is PRINCIPAL:PERMS:WHEN, where WHEN is success, failure, or both — which outcomes to log. sd audit takes the same --flags, --if, --replace, and -r options as sd allow.

Ownership #

sd owner and sd group #

Set the descriptor's owner or group SID.

$ sd owner ./report.txt alice
$ sd group ./report.txt Administrators

Both take a single PRINCIPAL and accept -r. Changing an owner is itself an access-controlled act — see Ownership.

Integrity #

sd integrity #

Sets the file's mandatory integrity label.

$ sd integrity ./report.txt high

The level is one of untrusted, low, medium, medium-plus, high, system, protected. The standard catalog is five levels (untrusted, low, medium, high, system); medium-plus (RID 8448) and protected (RID 20480) are non-standard Windows-compatibility levels — the kernel compares any S-1-16-<rid> numerically.

For what the label does, see Mandatory integrity control.

Inheritance #

sd inherit #

Turns inheritance protection on or off — the + mark ls -l shows.

$ sd inherit off ./report.txt    # lock the file; stop inheriting
$ sd inherit on  ./report.txt    # let it inherit from its parent again

inherit on lets the file inherit rules from its parent directory. inherit off protects the file — it keeps its current rules and stops tracking the parent.

sd reset #

Drops the file's own explicit rules and rebuilds its DACL purely from what the parent directory hands down — returning the file to "inherits everything".

sd propagate #

Pushes this directory's inheritable rules down into its descendants, refreshing what they inherit. Use it after changing a directory's rules so the children pick up the change.

See Inheritance for the full model.

Replacing the whole descriptor #

sd set #

Replaces the entire descriptor in one step.

$ sd set ./report.txt 'O:BAG:BAD:P(A;;FA;;;BA)(A;;0x1200a9;;;BU)'

Common flags #

These apply across the subcommands:

Exit status #

Before you can do anything with a file you usually have to find it — see what a directory holds, learn one file's details, turn a relative name into an absolute one. This topic covers the commands for that: listing directories, inspecting a single file, and resolving path names.

This page names every command in the topic, says in one line what each is for, and points you at the page that covers it in full.

The commands #

These commands and file security #

Most commands in this topic work purely on path strings — they never touch the file the path names. basename, dirname, realpath, and pathchk are all string operations.

Two report on real files, and they relate to file security differently:

• ls -l surfaces security-descriptor information directly. On Peios a file's access is governed by its security descriptor — the record of who owns the file and who may do what to it. The long listing shows the owner and the parts of the descriptor that fit in a one-line-per-file format.
• stat is a low-level diagnostic. It dumps the raw contents of a file's inode. The inode carries some numeric fields — an owner id, a group id, a mode value — that the Peios access model does not consult. stat reports them because it reports the inode verbatim, but they are not the file's access policy.

Neither command prints the full security descriptor; inspecting that is a job for the dedicated security tooling. If a line of ls -l looks unfamiliar, or you want to know why stat shows fields that "don't count", the explanation is in Security descriptors.

Where to start #

If you want the everyday "what's in this directory" command and the meaning of every column it prints, read ls.

If you need the full detail of one specific file, read stat.

If you are writing a script and need to take a path apart or canonicalise it, the string commands — basename, dirname, realpath — are what you want.

ls lists the contents of a directory. It is the command you use to answer "what is in here?" — and, with the right options, "who owns these files, how big are they, and when were they last changed?".

ls [options] [file...]

Each file is a directory to list the contents of, or a single file to report on. With no file argument, ls lists the directory you are currently in.

$ ls
build.sh  notes.txt  projects

The default listing #

Run on its own, ls prints the names of the entries in a directory and nothing else. Three rules shape that default:

• Hidden entries are skipped. A name beginning with a dot (.config, .cache) is hidden. ls leaves hidden entries out unless you ask for them with -a or -A.
• Entries are sorted by name. Alphabetical order, case-sensitive. Other sort orders are available.
• The layout adapts to where output goes. When ls is writing to a terminal it arranges names into columns sized to the terminal width. When output is redirected to a file or a pipe it prints one name per line, so the result is easy for another program to read.

Everything past this default is opt-in. The rest of this page is the options, grouped by what they do.

The long format #

-l (or --long) switches ls into the long format: one entry per line, each with its mode, owner, size, modification time, and name.

$ ls -l
total 28
d-- S-1-5-21-9f3a-1c4e-7b20-1001   4096 May 14 09:12 projects
-x- S-1-5-21-9f3a-1c4e-7b20-1001   8344 May 16 17:40 build.sh
--- S-1-5-21-9f3a-1c4e-7b20-1001    219 May 15 11:03 notes.txt
--+ S-1-5-18                      12048 May 10 22:15 policy.db

The total line at the top reports the combined disk space used by the listed files, measured in blocks.

Each entry line has five columns: mode, owner, size, modification time, name. There is no permission-bits column and no group column — the reason is below.

The mode column #

The mode column is three characters: [type][executable][protected].

The first character is the file's type:

The second character is the executable mark. It is x when the entry is a regular file that is marked as an executable, and - otherwise (it is always - for directories, links, and other non-regular entries).

The executable mark describes the file, not your relationship to it. x means "this file is runnable code." It does not mean that you, specifically, are allowed to run it — whether a given principal may execute a file is a separate access decision made against the file's security descriptor. A file can show x and still be one you cannot run, and the reverse. See Access decisions for how execution is actually authorised.

The third character is the protected mark. It is + when the file's access control is inheritance-protected, and - otherwise.

A - here means the file's access control is inherited from the directory it lives in: it tracks its parent. A + means the file's access control has been set deliberately and locked, so it no longer tracks the parent directory. The + is the signal that someone has tailored this file's security on purpose. See Inheritance.

So in the example above: projects is a directory (d--); build.sh is an executable file (-x-); notes.txt is an ordinary file inheriting its security from its directory (---); and policy.db is an ordinary file whose access control has been protected (--+).

Why there are no permission bits #

A long listing does not carry a fixed block of permission characters. On Peios, what a principal may do to a file is decided by the file's security descriptor — a record that can contain any number of entries granting or denying specific rights to specific principals. There is no small, fixed set of bits that captures that, so ls -l does not pretend there is.

The mode column tells you the file's type and two facts about it (is it executable, is its security protected). To see who may do what, look at the security descriptor itself with stat or the dedicated security tooling. The concepts are in Security descriptors.

The owner column #

The owner column is the SID of the file's owner — shown in full, in the S-1-… form. ls does not translate SIDs into account names; it prints the identifier exactly as the security descriptor stores it. A well-known owner such as the system itself appears as its fixed SID (S-1-5-18); an ordinary account appears as a longer domain-style SID.

For what a SID is and how to read one, see SIDs. For what "owner" means and what it grants, see Ownership.

If ls cannot read a file's descriptor — for a dangling symlink, say — the owner column shows ? and the mode column shows the type character followed by ??.

Choosing what to list #

By default ls shows non-hidden entries, and for a directory argument it shows the directory's contents. These options change that.

Sorting #

ls sorts by name by default. These options change the sort key; -r reverses whatever order is in effect.

Output layout #

When several entries fit on a line, ls has to decide how to arrange them.

File-type indicators and colour #

These options make a listing easier to scan by marking entries with a symbol or a colour.

The colours ls uses are configurable. dircolors generates the colour settings and explains how to install them.

File sizes #

In the long format, and with -s, sizes are printed in bytes by default. These options rescale them.

Timestamps #

The long format shows the modification time by default. These options change which timestamp is shown — and, when sorting by time, which one is sorted on.

Symbolic links #

By default ls reports on a symbolic link itself — its type is l, its size is the length of the link text. These options make it report on the link's target instead.

readlink and realpath are the dedicated tools for inspecting and resolving links.

Other options #

The long-tail options, each in one line.

Exit status #

dir and vdir list the contents of a directory. They are ls with one decision already made for you: the output style is fixed instead of adapting to where the output is going.

dir [options] [file...]
vdir [options] [file...]

What they fix #

ls chooses its layout based on whether it is writing to a terminal — columns for a terminal, one name per line for a pipe or file. dir and vdir each pick one layout and always use it:

Both also default to a quoting style that escapes unusual characters in names without wrapping every name in quotes.

That is the only difference. The fixed style is just a default — pass an explicit format option (-1, -m, -l, -C, …) and it overrides the built-in choice, so dir -l produces a long listing and vdir -C produces columns.

Everything else is ls #

dir and vdir accept every option ls accepts and behave identically in every other respect — sorting, filtering, the long-format columns, colour, indicators, sizes, timestamps. vdir's long format is the same Peios long format documented for ls, with the [type][executable][protected] mode column and the owner SID.

For the full option set and the meaning of every long-format column, see ls. This page exists only to explain how dir and vdir differ from it — and the answer is "they pin the output style."

Exit status #

The same as ls:

stat displays the detailed status of a file. Where ls gives you a line per file, stat gives you everything the system records about one file: its type, its size, its timestamps, and the low-level metadata stored in its inode.

stat [options] file...

stat is a diagnostic tool. It dumps what a file's inode physically contains, verbatim. That makes it the right command for "tell me exactly what is recorded about this file" — and it means some of what it prints needs a word of explanation, below.

The default output #

With no options, stat prints a labelled, multi-line block per file:

$ stat notes.txt
  File: notes.txt
  Size: 219             Blocks: 8          IO Block: 4096   regular file
Device: 8,2     Inode: 1572931     Links: 1
Access: (0644/-rw-r--r--)  Uid: ( 1000/    jack)   Gid: ( 1000/    jack)
Access: 2026-05-15 11:03:42.000000000 +0000
Modify: 2026-05-15 11:03:42.000000000 +0000
Change: 2026-05-15 11:03:18.000000000 +0000
 Birth: 2026-05-15 11:03:18.000000000 +0000

Reading it line by line:

• File — the name, and for a symbolic link, what it points to.
• Size / Blocks / IO Block / type — the size in bytes, the number of allocated disk blocks, the preferred I/O block size, and the file type in words.
• Device / Inode / Links — the device the file lives on, its inode number (its unique index within that device), and the number of hard links to it.
• Access / Uid / Gid — a numeric owner id, a group id, and a mode value. See the caveat below — these are not the access policy.
• Access / Modify / Change / Birth — four timestamps: when the file was last read, last written, last had its metadata changed, and first created.

The Uid, Gid, and mode fields are decorative #

The Access: (0644/-rw-r--r--) Uid: (…) Gid: (…) line needs care.

A file's inode carries a numeric owner id, a numeric group id, and a mode value, and stat reports them because stat reports the inode verbatim. But the Peios access model does not consult these fields. What a principal may do to a file is decided by the file's security descriptor — see Security descriptors. The numbers on the Uid/Gid/mode line are inert metadata: they are stored, they are reported, and they have no effect on any access decision.

Do not read that line as a permission summary. A file showing 0644 is not "world-readable" in any meaningful sense — whether anyone can read it depends entirely on its security descriptor. stat shows these fields for completeness as a low-level diagnostic; it does not claim they mean anything.

To see the parts of a file's security that do count, use ls -l, which shows the owner SID and the mode column built from the security descriptor.

Custom output formats #

Two options replace the default block with output you control.

$ stat --format='%n is %s bytes' notes.txt
notes.txt is 219 bytes

File directives #

Used in the format string when stating files (the default mode):

The directives %a and %A (the mode in octal and in symbolic form) and %u, %U, %g, %G (the owner and group, numeric and by name) report the decorative inode fields described above. They are available for completeness; they are not the file's access policy.

The %C directive (security context) is inactive — it produces no meaningful value on a standard Peios system.

File-system directives #

With -f, stat reports on the file system a file lives on rather than the file itself, and the format string uses a different directive set:

Options #

Exit status #

pwd — "print working directory" — prints the absolute path of the directory you are currently in.

pwd [options]

$ pwd
/home/jack/projects

Every process has a working directory: the directory that relative path names are interpreted against. pwd tells you what yours is.

Physical and logical paths #

There are two honest answers to "where am I", and they differ only when symbolic links are involved.

Suppose /home/jack/work is a symbolic link pointing at /data/projects/jack, and you move into it. The physical path — the real location, with every link resolved — is /data/projects/jack. The logical path — the route you took to get there — is /home/jack/work.

By default pwd prints the physical path — -P is the default, and naming it explicitly just makes that choice visible. Use -L when you want the path as you navigated it, links and all.

If -L is requested but the recorded PWD value is missing or does not actually match the current directory, pwd falls back to the physical path rather than print something wrong.

A note on shells #

Many command shells provide their own built-in pwd, and the shell's version is what runs when you type pwd at a prompt. The two behave the same for everyday use. To be certain you are running this command rather than the shell built-in, invoke it by its full path.

Exit status #

readlink prints where a symbolic link points.

readlink [options] file...

A symbolic link is a file whose contents are another path. readlink reads that path and prints it:

$ readlink /home/jack/work
/data/projects/jack

By default readlink resolves exactly one level: it prints the link's immediate target, whether or not that target is itself a link, and whether or not it exists. If the named file is not a symbolic link, readlink prints nothing and reports failure.

Canonicalising a whole path #

The canonicalise options change readlink from "read this one link" into "resolve this entire path to its real, absolute form" — following every symbolic link in every component, collapsing . and ... The three differ only in how strict they are about components existing:

With any of these, readlink succeeds on an ordinary (non-link) file too — it simply returns the canonical path. realpath is the dedicated command for this canonicalising job and has more options for it; the canonicalise flags here exist so readlink can do it without a second tool.

Output and error control #

Exit status #

realpath resolves a path to its canonical form: absolute, with every symbolic link followed and every . and .. component removed. Given any path, it tells you the one true location it refers to.

realpath [options] file...

$ realpath ../work/./notes.txt
/data/projects/jack/notes.txt

A single file can be named many ways — through links, through relative paths, with redundant components. realpath reduces all of them to the single canonical path, which makes it the right tool for comparing two paths, or for turning a path a user typed into one a script can rely on.

How much must exist #

By default realpath requires every component of the path except the last to exist. These options change that requirement:

How symbolic links are handled #

Output options #

realpath and readlink #

readlink with -f/-e/-m does the same canonicalising job. The difference is focus: readlink is primarily for reading a single link's target, with canonicalising as an extra; realpath is built for canonicalising and carries the richer option set — relative output, the symlink-handling modes, the strip-only mode. Use realpath when canonicalising is the actual goal.

Exit status #

basename takes a path and prints just its final component — the file name, with all the leading directories removed.

basename name [suffix]
basename [options] name...

$ basename /home/jack/projects/notes.txt
notes.txt

It is a pure text operation: basename never looks at the file system. It works on the string you give it, so the path need not exist.

Removing a suffix #

Give a second argument and basename also strips that suffix from the end of the name — handy for turning a file name into a bare stem:

$ basename /home/jack/projects/notes.txt .txt
notes

The suffix is only removed if it is actually there, and never if it would leave an empty string.

Options #

basename takes one name by default. The options let it process several at once.

$ basename -s .txt notes.txt readme.txt changelog.txt
notes
readme
changelog

Exit status #

dirname takes a path and prints everything except its final component — the directory the name lives in. It is the counterpart of basename.

dirname [options] name...

$ dirname /home/jack/projects/notes.txt
/home/jack/projects

Like basename, it is a pure text operation — dirname never touches the file system, so the path need not exist.

How it handles edge cases #

• If the name has no / in it at all, there is no directory part, so dirname prints . — the current directory.
• Trailing slashes on the name are ignored before the final component is removed.

$ dirname notes.txt
.
$ dirname /home/jack/projects/
/home

Multiple names #

dirname accepts any number of names and prints the directory part of each on its own line:

$ dirname /etc/hosts /var/log/messages report.txt
/etc
/var/log
.

Options #

Exit status #

pathchk checks whether a path name is valid and usable. It tells you, before you try to create a file, whether the name would be rejected — because it is too long, contains an unusable component, or could not be reached.

pathchk [options] name...

$ pathchk /home/jack/projects/notes.txt
$ echo $?
0

pathchk prints nothing when a name is fine; it prints a diagnostic and fails when a name is not. It is meant for scripts that build up path names and want to fail early, with a clear message, instead of discovering the problem halfway through an operation.

Like the other name commands in this topic, pathchk does not require the file to exist — it is checking the name, not looking for the file.

What the default check covers #

With no options, for each name pathchk checks:

• that the name is not empty;
• that no part of the name exceeds the length limits of the file systems the path would actually touch;
• that the leading directories of the name can be searched.

Stricter checks #

The options replace "valid on this system" with "valid across a wide range of systems" — useful when a name has to work somewhere other than where you are checking it.

A name with a leading - on a component is worth catching: many commands would read it as an option rather than a file name. -P flags exactly that class of trouble.

Exit status #

When ls colours its output, it reads the colours from an environment variable named LS_COLORS. dircolors is the command that produces the value of that variable.

dircolors [options] [file]

dircolors does not set the variable itself — a command cannot change its parent shell's environment. Instead it prints shell commands that, when run by the shell, set LS_COLORS. The usual way to use it is to have the shell evaluate that output:

eval "$(dircolors)"

Put that line in a shell startup file and every ls --color afterwards picks up the colours. With no arguments, dircolors uses a built-in default colour scheme.

Choosing the shell syntax #

The commands dircolors prints have to match the shell that will run them. By default it guesses the syntax from the SHELL environment variable; these options state it outright.

If no shell option is given and SHELL is not set, dircolors cannot guess and reports an error.

Customising the colours #

To change the colours, you start from the default scheme, edit it, and feed it back.

The workflow is:

dircolors -p > ~/.dircolors      # save the default scheme to a file
# ...edit ~/.dircolors to taste...
eval "$(dircolors ~/.dircolors)" # load the edited scheme

When dircolors is given a file argument, it reads the colour definitions from that file instead of using the built-in database. The file maps file types and name extensions to colours; dircolors -p shows the format, with each line commented.

Exit status #

Much of the work at a command line is reading text — looking at a file, checking the start or end of one, pulling a column out, stitching two files together. This topic covers the commands for that: viewing files, showing parts of them, and joining or splitting text by line and by column.

This page names every command in the topic. The companion topic, Transforming text, covers the commands that reshape text — sorting, de-duplicating, substituting characters.

The commands #

Viewing a whole file

Viewing part of a file

Cutting and combining

Splitting a file into files

A note on input #

Almost every command here follows the same convention for where its text comes from. Name one or more files and it reads those; name none and it reads from standard input, so it can sit in the middle of a pipeline. Where a command accepts files, the name - stands for standard input, so you can mix files and piped input in one invocation.

Where to start #

To simply print a file, or join a few together, read cat — the most-used command in the topic.

cat prints the contents of files. Its name is short for "concatenate" — given several files, it prints them one after another, as a single stream.

cat [options] [file...]

$ cat notes.txt
$ cat part1.txt part2.txt part3.txt > whole.txt

With no file — or with the name - — cat reads standard input, which is what makes it useful in a pipeline or for capturing typed input into a file.

Plain use #

Most of the time cat is run with no options at all: it copies its input to its output, byte for byte, unchanged. The options exist for the times you want to see something about the text that is normally invisible, or to adjust spacing and numbering.

Numbering lines #

Making invisible characters visible #

These options reveal characters that normally print as nothing, or as whitespace — useful when a file is not behaving and you suspect a stray tab or control character.

Adjusting spacing #

Exit status #

tac prints a file with its lines in reverse — the last line first, the first line last. The name is cat spelled backwards, which is exactly what it does.

tac [options] [file...]

$ tac events.log

Reversing a log file so the newest entries come first is the everyday use. With no file, tac reads standard input.

Reversing on something other than lines #

tac does not have to split on newlines. You can tell it to treat some other string as the separator, and it will reverse the order of the pieces between those separators.

$ tac -s ', ' -r names.csv

Exit status #

head prints the beginning of a file — by default, the first 10 lines.

head [options] [file...]

$ head config.toml

It is the quick way to glance at the top of a file without printing the whole thing — a header, the first few records, the start of a log. With no file, head reads standard input.

Choosing how much #

NUM accepts a unit suffix (K, M, …) when counting bytes.

Counting from the end #

If NUM is given a leading -, the meaning flips: head prints everything except the last NUM.

$ head -n -5 report.txt    # all but the final 5 lines

Headers for multiple files #

When given more than one file, head prints a header line before each one so you can tell them apart. Two options override that:

Other options #

Exit status #

tail prints the end of a file — by default, the last 10 lines.

tail [options] [file...]

$ tail server.log

It is the counterpart of head: the quick way to see the most recent lines of a file. With no file, tail reads standard input.

Choosing how much #

If NUM is given a leading +, the meaning flips: tail prints everything from line NUM onward.

$ tail -n +20 report.txt    # from line 20 to the end

Following a growing file #

tail's most useful trick is -f. Instead of printing the end and exiting, it stays running and prints new lines as they are appended — the standard way to watch a log file live.

Press Ctrl-C to stop a tail -f.

Headers for multiple files #

As with head, when given more than one file tail prints a header before each.

Other options #

Exit status #

more displays a file one screenful at a time. Where cat prints a whole file at once — and a long file scrolls straight past — more shows the first screen, then waits for you before showing the next.

more [options] file...

$ more long-report.txt

Moving through the file #

While more is paused, it is waiting for a keypress:

more moves forward through a file. It is the simple, always-available pager — enough for reading something through once.

Options #

Exit status #

nl prints a file with its lines numbered. cat -n also numbers lines; nl is the command for when you need control over the numbering — which lines count, how the number looks, where it restarts.

nl [options] [file...]

$ nl chapter.txt
     1  Once the system is installed,
     2  the first boot brings up
       
     3  the configuration service.

Which lines get a number #

By default nl numbers only the non-empty lines. -b sets the rule:

How the number looks #

Logical pages #

nl can treat one file as a sequence of logical pages, each with a header, a body, and a footer, and number the three parts by different rules. Pages are separated by special delimiter lines in the input.

The styles for -h and -f are the same a / t / n / pBRE set as -b. For an ordinary file with no delimiter lines, the whole file is one body and only -b matters.

Exit status #

cut extracts columns from text. For each line of input it prints only the parts you select, dropping the rest.

cut option... [file...]

$ cut -d , -f 1,3 people.csv     # the 1st and 3rd comma-separated field
$ cut -c 1-8 log.txt             # the first 8 characters of each line

Every run of cut needs two decisions: what counts as a column, and which columns to keep.

What counts as a column #

Choose exactly one mode:

-b and -c cut by position. -f cuts by field, which is what you want for tabular data — a CSV, a table, the columns of a report.

Which columns to keep #

The mode option takes a sequence: numbers and inclusive ranges, separated by commas.

Field mode: the delimiter #

In field mode, cut needs to know what separates the fields.

Other options #

Exit status #

paste joins files side by side. It takes the first line of each file and prints them on one line, then the second line of each, and so on — merging files into columns.

paste [options] [file...]

$ paste names.txt ages.txt
Ada      36
Grace    41
Linus    29

By default the merged pieces are separated by a tab. paste is the column-wise counterpart of cat, which joins files end to end.

Pasting one file at a time #

$ paste -s -d , names.txt
Ada,Grace,Linus

Choosing the separator #

paste and join #

paste merges by position — line 1 with line 1, line 2 with line 2, blind to content. When you want to merge by a matching value — pairing rows that share a key — that is join.

Exit status #

join merges two files on a shared field. For every pair of lines — one from each file — that have the same value in their join field, it prints a combined line. It is the command-line form of a database join.

join [options] file1 file2

$ join employees.txt salaries.txt

If employees.txt has 101 Ada and salaries.txt has 101 80000, join pairs them on the shared 101 and prints 101 Ada 80000.

The input must be sorted #

join works by stepping through both files together, so both files must be sorted on the join field. If they are not, join will miss matches. Sort them first with sort, on the same field join will use.

join checks the ordering as it goes and reports a file that is out of order; --nocheck-order turns that check off, and --check-order forces it on.

Choosing the join field #

By default join joins on the first field of each file, and fields are separated by whitespace.

Unmatched lines #

By default join prints only the lines that paired. A line with no match on the other side is dropped. These options bring unmatched lines back:

Shaping the output #

Exit status #

comm compares two sorted files and tells you which lines are unique to each and which are common to both.

comm [options] file1 file2

By default it prints three columns:

$ comm list-a.txt list-b.txt
            apple
banana
                        cherry
date

• Column 1 — lines only in file1.
• Column 2 — lines only in file2.
• Column 3 — lines in both.

Each column is indented past the one before, so a line's column position tells you where it was found.

The input must be sorted #

comm steps through both files together, so both must be sorted. On unsorted input the comparison is meaningless. Sort the files first with sort.

comm checks the ordering and reports a file that is out of order. --check-order forces the check; --nocheck-order disables it.

Showing only the columns you want #

Each column can be switched off. The remaining columns close up to fill the gap.

These combine to answer specific questions. comm -12 shows only the common lines — the intersection of the two files. comm -23 shows lines in file1 that are not in file2 — the difference.

Other options #

Exit status #

split breaks one file into several smaller files of roughly equal size.

split [options] [input [prefix]]

$ split -l 1000 big.log
$ ls
xaa  xab  xac  xad

By default split writes 1000 lines per piece, names the pieces xaa, xab, xac, … — a prefix of x followed by a two-letter suffix — and reads its input from a named file or from standard input.

To put the file back together, concatenate the pieces in order: cat xaa xab xac … > original.

Each piece is a newly created file, and a new file needs a security descriptor; each piece inherits one from the directory it is created in.

How big each piece is #

Choose one of these to set the piece size:

A SIZE accepts a unit suffix: K, M, G, … as powers of 1024, or KB, MB, … as powers of 1000.

Fixed number of chunks (-n) #

-n divides the input into a set number of pieces rather than fixing each piece's size. CHUNKS may be:

Naming the pieces #

Other options #

Exit status #

csplit — "context split" — breaks a file into pieces at points you choose: a line number, or a line that matches a pattern. Where split cuts a file into equal sizes, csplit cuts it at meaningful boundaries.

csplit [options] file pattern...

$ csplit report.txt '/^Chapter/' '{*}'

That splits report.txt into one piece per chapter, cutting at every line beginning with Chapter. The pieces are written to xx00, xx01, xx02, … and csplit prints the byte size of each.

Each piece is a newly created file, and a new file needs a security descriptor; each piece inherits one from the directory it is created in.

Patterns #

Each pattern argument marks a cut point. csplit copies everything up to that point into the next output file, then continues.

A pattern that finds no match is an error, and by default csplit removes the pieces it had already written. -k keeps them.

Naming the pieces #

Other options #

Exit status #

od — short for "octal dump" — prints the raw bytes of a file. Where cat shows you a file as text, od shows you the actual bytes underneath: each one rendered as a number, a character, or a floating-point value, in whatever base you ask for. It is the tool for looking at binary files, checking exactly which bytes a text file contains, or finding a stray control character that isn't visible on screen.

od [options] [file...]

$ od hello.txt
0000000 062510 066154 020157 067527 066162 020144 000012
0000015

With no file — or with the name - — od reads standard input, so it can sit at the end of a pipeline. Given several files, it reads them one after another as a single continuous stream, and the byte offsets run straight through from one file into the next.

How the output is laid out #

Every line of output has two parts:

• An offset on the left: the position, counted in bytes from the start of the input, of the first byte on that line. By default it is printed in octal (see -A to change the base).
• One or more format columns on the right: the bytes of that line rendered in the format(s) you chose.

With no format option, od prints the input as octal, two bytes at a time — the traditional default. The very last line of output is a lone offset marking the total length of the input.

If you ask for more than one format at once, each format is printed on its own line, stacked under a single shared offset. Only the first line of the group carries the offset; the rest are indented to line up beneath it.

Choosing what the bytes look like #

There are two ways to say how bytes should be rendered: a set of single-letter named shortcuts for the common cases, and the general -t / --format option for full control. You can give several at once, and they are applied in the order they appear on the command line.

Named format shortcuts #

Each of these selects one output format. Combine them freely (od -cx prints characters and hex together).

Type specifications (-t, --format) #

-t takes a type specification and gives you every combination the named shortcuts cover and more. Repeat -t (or list several specs in one argument) to print multiple formats.

A type specification is a type letter, an optional size, and an optional z suffix:

For the numeric types (d, u, o, x, f) a size says how many bytes make up one unit. It can be a number, or a letter:

If you leave the size off, integer types default to 4 bytes and floating point defaults to 8 bytes. For the character types a and c, a size is not allowed.

Add a z at the end of a spec to append an ASCII dump — the bytes of that line shown as text, . for anything non-printable — to the right of the numbers. For example, od -t x1z gives the familiar hex-plus-text layout:

$ od -A x -t x1z file.bin
000000 48 65 6c 6c 6f 2c 20 77 6f 72 6c 64 21 0a           >Hello, world!.<

Examples: od -t d2 is signed decimal in 2-byte units; od -t x1 is hex byte-by-byte; od -t fD is double-precision floats; od -t c is characters. od -t x1 -t c prints hex and characters as two stacked lines.

Byte order (--endian) #

For multi-byte numeric formats, --endian sets the byte order used to assemble each unit. Without it, od uses the machine's native byte order. Use --endian=big or --endian=little to force one regardless of the host.

The offset column #

Only the first character of the value is examined, so od -A n and od -Anone mean the same thing.

Which bytes to read #

By default od dumps the whole input. Two options narrow that to a window:

In both, BYTES is decimal by default, octal if it starts with 0, or hexadecimal if it starts with 0x. A suffix scales it: b = ×512, KB/K = ×1000 / ×1024, MB/M = ×1000² / ×1024², GB/G = ×1000³ / ×1024³.

Collapsing repeated lines #

When several output lines in a row would be identical, od prints the first one, then a single line containing just * to stand for the run, and resumes at the next line that differs. This keeps a dump of a large block of identical bytes (a file full of zeros, say) short.

Output width #

The width must be a whole multiple of the size of the largest format unit in play; if it isn't, od warns and rounds it to a usable value.

Finding printable strings #

This turns od into a quick way to pull the human-readable text out of a binary file. The offset in front of each string honours -A (and is dropped entirely under -A n). -j and -N still apply, so you can restrict the scan to a window.

The traditional offset form #

od also accepts the historical calling convention, where an offset (and, in --traditional mode, a label) is given as a bare operand rather than an option:

od [named-format-options] [file] [[+]offset[.][b]]
od --traditional [options] [file] [[+]offset[.][b] [[+]label[.][b]]]

The offset says where in the input to begin, exactly like -j. It is read as octal by default, hexadecimal if prefixed with 0x, or decimal if it carries a trailing .; a trailing b multiplies it by 512. The optional leading + is allowed everywhere and is required when there is no filename (so od +20 reads standard input starting at octal offset 20).

The label (only in --traditional mode) is a second such number: the dump still begins at offset, but the offset column is displayed as if it started at label — useful for making a partial dump's offsets match the original file.

od only reads an operand as an offset in the plain form when none of -A, -j, -N, -t, -v, or -w are present. So if a real filename happens to look like a number, add a harmless option such as -j0 to force it to be treated as a filename.

A note on reading files #

od has no access rules of its own — it simply opens a file and reads its bytes. That read is subject to the same check as every read on Peios: whether you may open the file for reading is decided by the file's security descriptor (see Security descriptors). od never reveals bytes you could not already read another way; it only presents the bytes you can read in a different form.

Exit status #

This topic covers the commands that reshape text: putting lines in order, dropping duplicates, swapping characters, reflowing paragraphs to a width, counting what is there. Its companion, Viewing and joining text, covers printing and combining files; this topic is about changing the text itself.

The commands #

Ordering lines

Filtering lines

Substituting characters

Whitespace

Reflowing and paginating

Indexing and counting

Two commands that need sorted input #

A theme worth knowing before you start: uniq only collapses adjacent duplicate lines, so duplicates scattered through a file are not caught unless the file is sorted first. sort and uniq are almost always used together — and sort -u does both jobs in one step.

Where to start #

sort is the workhorse of the topic and the one with the most depth — start there.

sort reads lines and writes them out in order.

sort [options] [file...]

$ sort names.txt

With no file, sort reads standard input. Given several files, it sorts all of their lines together as one stream. By default it compares lines as plain text, character by character.

How lines are compared #

The default text comparison is often not the order you want — 10 sorts before 9, and case matters. These options change the comparison:

And these adjust what is compared:

Sort order options #

Sorting by a field #

By default sort compares whole lines. -k sorts on a key — a chosen part of each line — which is what you need for tabular data.

A line is split into fields (by default at whitespace). A key is written FIELD[.CHAR][OPTIONS][,FIELD[.CHAR]] — a start field, an optional start character within it, and an optional end. The single-letter comparison options above can be appended to a key to apply only to it.

$ sort -k 2 -n scores.txt        # sort on the 2nd field, numerically
$ sort -t , -k 3 people.csv      # sort a CSV on the 3rd field

Checking and merging #

Output and resources #

Exit status #

uniq deals with repeated lines: it can collapse a run of identical lines into one, count them, or print only the ones that repeated.

uniq [options] [input [output]]

$ uniq events.txt

input and output may be given as operands; with neither, uniq reads standard input and writes standard output.

uniq only sees adjacent duplicates #

This is the one thing to know about uniq: it only compares each line with the one before it. It collapses adjacent repeats. Duplicate lines scattered through a file, with other lines between them, are not caught.

So uniq is almost always paired with sort, which brings every copy of a line together first:

$ sort events.txt | uniq

If collapsing duplicates is all you need, sort -u does both jobs in one command. Use uniq itself when you want what sort -u cannot do — counting repeats, or printing only the duplicated lines.

What to output #

By default uniq prints every line, with each run of adjacent duplicates reduced to a single copy. These options change that:

What counts as "the same" #

Other options #

Exit status #

tr — "translate" — works on text one character at a time. It can replace characters with other characters, remove characters, or collapse runs of a character into one.

tr [options] set1 [set2]

tr reads standard input and writes standard output — it has no file arguments, so it always sits in a pipeline.

$ tr 'a-z' 'A-Z' < notes.txt        # lower-case to upper-case
$ tr -d ' ' < spaced.txt            # delete every space
$ tr -s ' ' < padded.txt            # collapse runs of spaces to one

The three jobs #

What tr does depends on the options and how many sets you give it.

• Translate — give two sets. Each character in set1 is replaced by the character at the same position in set2. tr 'abc' 'xyz' turns every a into x, b into y, c into z.
• Delete (-d) — give one set. Every character in set1 is removed.
• Squeeze (-s) — each run of a repeated character that appears in the relevant set is collapsed to a single occurrence.

-d and -s can be combined: delete one set of characters, then squeeze another.

Writing a set #

A set is a string of characters, with a few shorthands:

Options #

Exit status #

expand converts tab characters into spaces, so that text lines up the same way no matter what tab width something is viewed with.

expand [options] [file...]

$ expand source.txt > source-spaces.txt

A tab is not a fixed number of spaces — it jumps to the next tab stop, and where those stops are is a display setting. expand removes the ambiguity: it replaces each tab with exactly the number of spaces needed to reach the same column, baking the layout in. With no file, it reads standard input.

unexpand is the reverse operation.

Options #

Exit status #

unexpand is the reverse of expand: it converts runs of spaces back into tab characters.

unexpand [options] [file...]

$ unexpand spaced.txt > tabbed.txt

Where a run of spaces reaches a tab stop, unexpand replaces it with a tab. By default it only converts the leading spaces on each line — the indentation — and leaves spacing within the text alone. With no file, it reads standard input.

Options #

Exit status #

fmt reflows paragraphs: it rejoins the lines of a paragraph and re-breaks them so each comes out close to a target width. Words move freely between lines to make the result fit.

fmt [options] [file...]

$ fmt -w 72 draft.txt

fmt works on paragraphs — runs of non-blank lines separated by blank lines — and treats word boundaries as places it may break. The result reads naturally. This is what distinguishes it from fold, which simply cuts every line at a fixed column without regard to words.

With no file, fmt reads standard input.

Setting the width #

Controlling the reflow #

Selecting which lines to format #

Exit status #

fold breaks long lines so that none exceeds a fixed width. Every line longer than the limit is cut into pieces.

fold [options] [file...]

$ fold -w 80 wide.txt

By default fold wraps at 80 columns, and it cuts at exactly that column — even in the middle of a word. That hard cut is the difference between fold and fmt: fmt reflows paragraphs and breaks between words; fold just enforces a width, mechanically. Use fold when you need a guarantee that no line is wider than N; use fmt when you want the result to read well.

With no file, fold reads standard input.

Options #

Exit status #

pr prepares text for printing on paper. It breaks a file into pages, puts a header and a trailer on each, and can arrange the text into columns.

pr [options] [file...]