Aww, shucks.

After 7 years, one of the two 6TB Western Digital Red drives that forms half of my RAID1 mirror failed to come up after a server reboot:

[   19.762832] ata2.00: exception Emask 0x0 SAct 0x100 SErr 0x0 action 0x0
[   19.762838] ata2.00: irq_stat 0x40000008
[   19.762841] ata2.00: failed command: READ FPDMA QUEUED
[   19.762847] ata2.00: cmd 60/08:40:08:00:00/00:00:00:00:00/40 tag 8 ncq dma 4096 in
                        res 41/40:00:09:00:00/00:00:00:00:00/00 Emask 0x409 (media error) <F>
[   19.762850] ata2.00: status: { DRDY ERR }
[   19.762852] ata2.00: error: { UNC }
[   19.764196] ata2.00: configured for UDMA/133
[   19.764212] sd 3:0:0:0: [sdd] tag#8 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
[   19.764215] sd 3:0:0:0: [sdd] tag#8 Sense Key : Medium Error [current]
[   19.764218] sd 3:0:0:0: [sdd] tag#8 Add. Sense: Unrecovered read error - auto reallocate failed
[   19.764222] sd 3:0:0:0: [sdd] tag#8 CDB: Read(16) 88 00 00 00 00 00 00 00 00 08 00 00 00 08 00 00
[   19.764225] blk_update_request: I/O error, dev sdd, sector 9 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0
[   19.764230] Buffer I/O error on dev sdd, logical block 1, async page read
[   19.764248] ata2: EH complete

Fun times. Best Buy was running a Black Friday sale on the 14TB EasyStore external drives, which I discovered were shuckable, so I picked two up ($199/ea) and they came apart easily. Tossed them in the microserver and configured them into a RAID1 array, using parted to label the drives (GPT) and create a full disk partition, and then mdadm to setup a RAID:

mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 \
/dev/disk/by-id/scsi-1ATA_WDC_WD140EDGZ-11B1PA0_SERIAL1-part1 \
/dev/disk/by-id/scsi-1ATA_WDC_WD140EDGZ-11B1PA0_SERIAL2-part1

It took about 26 hours for the resync to complete (watching cat /proc/mdstat), after which I formatted it with the ext4 filesystem. I’m now transferring all the contents from the 6TB array to the 14TB array; it’s running about 1GB every 20 seconds. There’s 5.1TB to transfer, total. Probably 29 hours or so for that process to complete.