Finding Corrupt Files on a Raspberry SD-Card

Every second — somewhere — a SD-Card in a Raspberry Pi dies.¹ In my experience SD-Cards fail after about 2 years of heavy usage (constant database writes). At first, you might not notice the symptoms as non-essential files could be corrupted. The corruption becomes apparent when binaries or shared libraries become corrupted, as this can impact the system behavior.

I don’t think it’s necessary to mention the only reliable method to fix this problem — backups. If you notice a faulty SD-Card, you should swap it for a new one and restore your backups.

Also, if you don’t have a backup and you need the data on the SD-Card, remove it immediately and never write to it again.

Finding the Corrupted Files

In case you want to find files which are corrupted, you can try to find them by scanning every file and looking for a repeating pattern. All my corrupt files typically consisted of a never ending stream of NUL-bytes (^@, ␀, NUL, \0). cat would not terminate when reading them to stdout. Therefore we have to restrict the amount of bytes read from each file by using head, in order not to be stuck.

I came up with the following script:

find / -type f -size +0 -readable -print0 \
  | LC_ALL=C xargs -n1 -I {} -r0 sh -c \
  "head -c 1M < '{}' | grep -qP \"[^\x00]\" || echo {}" \
  | tee files.txt

• Look at all readable (-readable) files (-type f) in the whole file system (find /), which are larger than zero bytes (-size +0)
• For each of those files do (LC_ALL=C xargs -n1 -I {} -r0 sh -c)
  • Look at the first MiB of data (head -c 1M < '{}')
  • Search for a non-NUL character grep -qP "[^\x00]"
    • Print the filename to stdout if a non-NUL character was not found (|| echo {})
• Write the results to stdout and a file (| tee files.txt)

After completion, you have a list of all possibly corrupted files. In my case I deleted them (xargs rm < files.txt) so that my backups would still manage to finish, without being trapped in the infinite stream of NUL-bytes.