Hashing and encoding

This topic covers two related jobs. Hashing reduces a file to a short fixed-size value — a checksum — that can be used to tell whether the file has changed. Encoding rewrites binary data as plain text so it can travel safely through channels that only handle text.

The commands #

Checksums and hashes

Encoding

What these commands are not #

It is worth being clear up front, because both jobs are easy to mistake for something they are not.

A hash is not encryption. Hashing is one-way: a checksum tells you whether data has changed, but the original cannot be recovered from it. It protects against corruption and detects tampering — it does not keep anything secret.

Encoding is not encryption either. base64 rewrites data into a text-safe form, but anyone can decode it straight back — it is a change of representation, not of secrecy. Encoding something does not protect it.

Neither hashing nor encoding hides data. They are about integrity and transport, not confidentiality.

Where to start #

To verify a file you downloaded against a published checksum, read the checksum commands. To turn binary data into something safe to paste into text, read base32 and base64.

This page covers a family of commands that all work the same way and differ only in which hash algorithm they use:

sha256sum [options] [file...]

Everything below is written with sha256sum, but applies to every command in the table.

Computing a checksum #

Run plainly, the command prints the hash of each file, followed by the file name:

$ sha256sum installer.iso
9f86d0818884...b1a5  installer.iso

With no file, it reads standard input. The hash is a fingerprint of the file's contents: change a single byte and the hash changes completely.

Verifying with a checksum #

The everyday use is verification — confirming a file is exactly what it should be, usually a download.

First, the file's publisher computes a checksum and publishes it, often in a file:

$ sha256sum installer.iso > installer.iso.sha256

Then anyone with the file and that checksum file can verify:

$ sha256sum -c installer.iso.sha256
installer.iso: OK

-c reads each hash filename line, recomputes the hash, and reports OK or FAILED. A FAILED means the file is not the one the checksum was made from — corrupted in transit, or altered.

Output format #

b2sum additionally accepts -l, --length=BITS to produce a shorter BLAKE2b digest.

A note on choosing an algorithm #

These commands detect change — but not all of them resist a deliberate attempt to forge a match. MD5 and SHA-1 can be defeated by an attacker who wants two different files to share a checksum. They are still fine for catching accidental corruption, but for verifying that a file has not been tampered with, use a SHA-2 command (sha256sum and up) or b2sum.

Exit status #

cksum is the general checksum command. Where each of the checksum commands is fixed to one algorithm, cksum does any of them — the algorithm is an option.

cksum [options] [file...]

$ cksum installer.iso
3915528286 372736000 installer.iso
$ cksum -a sha256 installer.iso
9f86d0818884...b1a5  installer.iso

With no -a, cksum computes a CRC checksum and prints the CRC value, the file's byte count, and the name. With -a, it behaves like the corresponding dedicated command.

Choosing the algorithm #

NAME may be:

crc, crc32b, sha3, and sm3 are available only through cksum — there is no dedicated command for them.

Verifying #

cksum checks files the same way the dedicated commands do:

Output format #

Exit status #

sum computes a small, old-style checksum of a file and reports it along with the file's size in blocks.

sum [options] [file...]

$ sum archive.tar
12345 84 archive.tar

The output is the checksum, the block count, and the file name. With no file, sum reads standard input.

Which algorithm #

sum predates the modern hashes and offers two historical algorithms:

When to use it #

sum exists for compatibility — for reading checksums produced by old tools, and for scripts that still expect its output. Its checksum is short and weak: it catches some accidental corruption, but it is easily fooled and gives no protection against tampering.

For anything new, do not use sum. Use a checksum command such as sha256sum, or cksum — both of which cksum can still produce, via cksum -a bsd and cksum -a sysv, if you need the legacy values.

Exit status #

base32 and base64 encode binary data as plain text — and decode it back. They are the same command with one difference: the alphabet they use.

base32 [options] [file]
base64 [options] [file]

$ echo Peios | base64
UGVpb3MK
$ echo UGVpb3MK | base64 -d
Peios

What encoding is for #

Some channels only carry text safely — they mangle or reject arbitrary bytes. Encoding rewrites binary data using a small, safe set of characters, so it can pass through unharmed; decoding reverses it exactly. base64 uses 64 characters and is the more compact; base32 uses 32 and is more robust where case might not survive. With no file, both read standard input.

Encoding is not encryption — anyone can decode the result. See the overview.

Options #

By default these commands encode. The options below apply identically to both.

When decoding, newlines in the input are always tolerated; -i is for other stray characters.

Exit status #

basenc is the general encoding command. Where base32 and base64 each do one encoding, basenc does many — the encoding is chosen by an option.

basenc encoding [options] [file]

$ echo Peios | basenc --base16
5065696F730A

Every run of basenc must name an encoding. With no file, it reads standard input.

The encodings #

Options #

By default basenc encodes. These options apply to whichever encoding you chose:

basenc and the dedicated commands #

For plain base64 or base32, base64 and base32 are the shorter way to ask. Use basenc when you need an encoding the dedicated commands do not offer — hex, URL-safe base64, the bit-string forms, z85, or base58.

Exit status #