Replacing a hot swap drive in a gmirror mirror under FreeBSD

Situation: there is a RAID array of two SATA disks in the mirror created with gmirror under FreeBSD.

You need to replace one drive without stopping the server.

Drop of theory

There are many articles on the topic of actually creating a mirror on GEOM .

When creating a mirror, gmirror synchronizes all data, including the MBR, reads http://people.freebsd.org/~rse/mirror/ (section Summary -> GEOM mirror on whole disk). If the server died during a disk failure, you can boot from the remaining disk, regardless of which disk crashes. It is important only at boot to choose which disk to boot from.

When the array is running, the command

# gmirror list

almost constantly shows Flags: DIRTY. This is normal : the flag is set when information is written to the disk, and at this moment the state of the data on the disks of the array does not match. If the disc is constantly being recorded, the DIRTY flag will be constantly set.

Replacement procedure

Suppose there are two disks in the gm0 array: da0 and da1. Replace da0.

In the described situation, camcontrol was used. Depending on the controller through which the screws are connected, atacontrol may be needed instead.

1. We bring the disk out of the mirror:
   
   

   # gmirror remove gm0 da0
   

   
   
2. We determine which disk physically needs to be replaced - the activity indicator will not work for the disk that is output from the mirror.
3. Without turning off the server, we take out the disk .
4. Here the following is possible: the server does not understand anything for several seconds, and then falls into a coma for half a minute . You need to wait a bit .
5. After this command
   
   

   # geom disk list
   # gmirror list

   
   help to find out that there is no one disk.
6. If item # 1 has not been completed, we ask gmirror to forget about all the drives that are currently inactive in the mirror:
   
   

   # gmirror forget gm0

   
   gmirror detects that da0 is gone and forgets about it.
   
7. We insert a new disk (preferably identical to the one with which it will work together, up to the model).
8. Scan the bus so that the system detects a new mapped drive:
   
   

   # camcontrol devlist # what do we have now?
   # camcontrol rescan all # scanned
   # camcontrol devlist # what happened?
   

   
   
9. Add the new da0 to the array :
   
   

   # gmirror insert gm0 da0

   
   
10. gmirror detects a new disk and starts data synchronization . We look at the composition of the array:
    
    

    # gmirror list

    
    
11. We are not afraid, if it seems to us that synchronization is going in the wrong direction :) If da0 was replaced, then now it is in the AFTER da1 list , and not BEFORE, as it was before.
12. We experience happiness.