MD5SHA1SUM

Wed 15 Sep 2021

These two tools are great to check the integrity of the file. Transport of files, when for example streaming a files content, can change the integrity and make them corrupt. The other party can check the integrity of the file when you create the md5 or sha1 signature and provide this.

This is a basic article but found the subject worthwhile. It is important to know why these are made and how to check them. Modern package managers have these checks built-in

md5, sha1 or something else?

You might ask, what to use? Is there a difference? md5sum and sha1sum implement two different hashing algorithms, MD5 and SHA-1 respectively. However, you should really consider using a stronger algorithm.

I would argue that you should not use these anymore to create file integrity signatures that you share to other people. They are both regarded as algorithms that are cryptographically insecure. A good read on this is shattered.io, proving a SHA-1 collision is possible. You should take into consideration that shattered is years old and that collision attacks are becoming more common.

That aside, these are still commonly used. There are plenty of sum files out there that use either MD5 or SHA-1. I’ve named the article md5sha1sum because that’s how the package is named when using homebrew on Mac OS. I needed a quick way to tell if my file corrupted itself during transfer.

You can check on your distro what kind of sum binaries are installed. You probably got something like what I’ve got sha1sum, sha224sum, sha256sum, sha384sum, sha512sum installed.

Performance is a thing on large files. Try checking a small file, then try stepping up like 1G - 10G - 20G. You will notice the performance impact it has. Still, it is worth the wait for larger files.

Basic usage

You can generate a hash by pointing the commando to a file. It will output a hash of the file, followed by the name. These are commonly saved to a checksums file. Checksum files can contain multiple files and this way you can easily check multiple files.

# Generate a SHA-512 sum of a test file
# Sample output:
# cf83e1357eefb8bdf1542850d66d8007d620e4050b5715dc83f4a921d36ce9ce47d0d13c5d85f2b0ff8318d2877eec2f63b931bd47417a81a538327af927da3e  test.txt
rob@Rathalos ~ $ sha512sum test.txt >> checksums

# Run it again if you want a clear example of how the -c option works
rob@Rathalos ~ $ sha512sum test.txt >> checksums

# Check a sum file (checksums is a general name for such file)
rob@Rathalos ~ $ sha512sum -c checksums

That is pretty much it. Make it an exercise to check the ISO’s you download. Supply-chain attacks are becoming more common. Be weary when you see these types of scripts curl https://some.url.here | sudo bash as you have no real way of telling the code you are about to execute. Always opt for downloading the file first, checking the integrity and then go from there.