Disk Checks for Large Arrays

If you have a large array of disks attached to your server, which is obviously going to be running FreeBSD or OpenBSD if you care about security, stability, and scalability; there are some tricks for dealing with large numbers of disks (like having 227 4TB disks attached to a single host).

Using Bash (yes there are security issues, but it is powerful)

# for i in `seq 0 227`; do smartctl -t short /dev/da$i; sleep 15; done¹Thanks Jared

executes a short smart test on all disks. Smartctl seems to max out at 32 concurrent tests, so sleep 15 ensures the 3 minute tests are finishing before new ones are executed. If you’re in a hurry, sleep 5 should do the trick and ensure all of them execute.

to get results try something like:

# for i in `seq 0 227`; do echo "/dev/da$i"; smartctl -a /dev/da$i; sleep .5; done

Bulk Fixes

Problem with the disks – need to clear existing formatting?

unmount each disk

# for i in `seq 0 227`; do umount -f /dev/da$i; done

unlock (if needed)

# sysctl kern.geom.debugflags=0x10

Overwrite the start of each disk

# for i in `seq 0 227`; do dd if=/dev/zero of=/dev/da$i bs=1k count=100; done

Overwrite the end of each disk

# for i in `seq 0 227`; do dd if=/dev/zero of=/dev/da$i bs=1m oseek=`diskinfo da$i | awk '{print int($3 / (1024*1024)) - 4;}'`; done

Recreate GPT (for ZFS)

# for i in `seq 0 227`; do gpart create -s gpt /dev/da$i; sleep .5; done

Destroy multipaths

# for i in `seq 1 114`; do gmultipath destroy disk$i; done

Disable multipath completely

# for i in `seq 1 114`; do gmultipath destroy disk$i; done
# gmultipath unload
# mv /boot/kernel-debug/geom_multipath.ko /boot/kernel-debug/geom_multipath.ko.bad
# mv /boot/kernel/geom_multipath.ko /boot/kernel/geom_multipath.ko.bad