paulgorman.org/technical

Linux md RAID

See md(4) for “multiple Device driver aka Linux Software RAID”.

The admin tool for interacting with the software raid is mdadm. The mdadm(8) page provides good usage examples.

The configuration file (on Debian) is /etc/mdadm/mdadm.conf. We may never need to directly touch the config file, but check the MAILADDR to verify that alerts go to a mail account someone actually reads. (This file may have been created with mdadm --detail --scan >> /etc/mdadm/mdadm.conf.)

$  cat /proc/mdstat
Personalities : [raid10] [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4]
md0 : active raid10 sdf2[5](F) sda2[0] sde2[4] sdd2[3] sdc2[2] sdb2[1]
	  2927827968 blocks super 1.2 512K chunks 2 near-copies [6/5] [UUUUU_]
	  bitmap: 3/22 pages [12KB], 65536KB chunk

unused devices: <none>
$  sudo mdadm --detail /dev/md0
/dev/md0:
		Version : 1.2
  Creation Time : Thu May 11 10:59:08 2017
	 Raid Level : raid10
	 Array Size : 2927827968 (2792.19 GiB 2998.10 GB)
  Used Dev Size : 975942656 (930.73 GiB 999.37 GB)
   Raid Devices : 6
  Total Devices : 6
	Persistence : Superblock is persistent

  Intent Bitmap : Internal

	Update Time : Tue Jan  2 10:21:49 2018
		  State : active, degraded
 Active Devices : 5
Working Devices : 5
 Failed Devices : 1
  Spare Devices : 0

		 Layout : near=2
	 Chunk Size : 512K

		   Name : tiger:0  (local to host tiger)
		   UUID : 8313c081:eeeb68f7:010303cf:7e88554e
		 Events : 19306

	Number   Major   Minor   RaidDevice State
	   0       8        2        0      active sync set-A   /dev/sda2
	   1       8       18        1      active sync set-B   /dev/sdb2
	   2       8       34        2      active sync set-A   /dev/sdc2
	   3       8       50        3      active sync set-B   /dev/sdd2
	   4       8       66        4      active sync set-A   /dev/sde2
	   -       0        0        5      removed

	   5       8       82        -      faulty   /dev/sdf2

How do we identify a physical drive? With either smartctl or hdparm.

$  sudo smartctl -i /dev/sda
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-3-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Device Model:     Crucial_CT1050MX300SSD1
Serial Number:    1652152BD46C
LU WWN Device Id: 5 00a075 1152bd46c
Firmware Version: M0CR040
User Capacity:    1,050,214,588,416 bytes [1.05 TB]
Sector Size:      512 bytes logical/physical
Rotation Rate:    Solid State Device
Form Factor:      2.5 inches
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   ACS-3 T13/2161-D revision 5
SATA Version is:  SATA 3.2, 6.0 Gb/s (current: 6.0 Gb/s)
Local Time is:    Fri Aug 11 10:44:36 2017 EDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled
$  sudo hdparm -i /dev/sda

/dev/sda:

 Model=Crucial_CT1050MX300SSD1, FwRev=M0CR040, SerialNo=1652152BD46C
 Config={ Fixed DTR>10Mbs }
 RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=0
 BuffType=unknown, BuffSize=unknown, MaxMultSect=16, MultSect=off
 CurCHS=16383/16/63, CurSects=16514064, LBA=yes, LBAsects=2051200368
 IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
 PIO modes:  pio0 pio3 pio4
 DMA modes:  mdma0 mdma1 mdma2
 UDMA modes: udma0 udma1 udma2 udma3 udma4 udma5 *udma6
 AdvancedPM=yes: unknown setting WriteCache=enabled
 Drive conforms to: unknown:  ATA/ATAPI-3,4,5,6,7

 * signifies the current active mode

Replacing a Failed Drive

We want to replace a failed disk in a mirror.

# mdadm --manage /dev/md0 --fail /dev/sdb1
# mdadm --manage /dev/md0 --remove /dev/sdb1

Physically replaced the drive. gdisk makes it slightly easier to partition the replacement drive by copying the partition scheme from the matching good drive. After copying the partitions, -G generates a new GUUID, and running -p lets us verify that the tables match. (Being a bit more cavalier, we could just sfdisk -d /dev/sda | sfdisk /dev/sdb.)

# sgdisk --backup=/root/sda.partitiontable /dev/sda
# sgdisk --replicate=/dev/sdb /dev/sda
# sgdisk -G /dev/sdb
# sgdisk -p /dev/sda
# sgdisk -p /dev/sdb
# mdadm --manage /dev/md0 --add /dev/sdb1

Tue Jan 2 10:24:42 EST 2018