[root@centos7 Bash]$ fdisk /dev/sdbWelcome to fdisk (util-linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0x93d380cf.Command (m for help): nPartition type: p primary (0 primary, 0 extended, 4 free) e extendedSelect (default p): pPartition number (1-4, default 1): First sector (2048-209715199, default 2048): Using default value 2048Last sector, +sectors or +size{K,M,G} (2048-209715199, default 209715199): +1GPartition 1 of type Linux and of size 1 GiB is setCommand (m for help): tSelected partition 1Hex code (type L to list all codes): fdChanged type of partition 'Linux' to 'Linux raid autodetect'Command (m for help): pDisk /dev/sdb: 107.4 GB, 107374182400 bytes, 209715200 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk label type: dosDisk identifier: 0x93d380cf Device Boot Start  End Blocks Id System/dev/sdb1  2048 2099199 1048576 fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@centos7 Bash]$ fdisk /dev/sdcWelcome to fdisk (util-linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0xc56b90d8.Command (m for help): nPartition type: p primary (0 primary, 0 extended, 4 free) e extendedSelect (default p): pPartition number (1-4, default 1): First sector (2048-41943039, default 2048): Using default value 2048Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): +1GPartition 1 of type Linux and of size 1 GiB is setCommand (m for help): tSelected partition 1Hex code (type L to list all codes): fdChanged type of partition 'Linux' to 'Linux raid autodetect'Command (m for help): pDisk /dev/sdc: 21.5 GB, 21474836480 bytes, 41943040 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk label type: dosDisk identifier: 0xc56b90d8 Device Boot Start  End Blocks Id System/dev/sdc1  2048 2099199 1048576 fd Linux raid autodetectCommand (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.[root@centos7 Bash]$ fdisk /dev/sddWelcome to fdisk (util-linux 2.23.2).Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition tableBuilding a new DOS disklabel with disk identifier 0x7e0900d8.Command (m for help): nPartition type: p primary (0 primary, 0 extended, 4 free) e extendedSelect (default p): pPartition number (1-4, default 1): First sector (2048-41943039, default 2048): Using default value 2048Last sector, +sectors or +size{K,M,G} (2048-41943039, default 41943039): +1GPartition 1 of type Linux and of size 1 GiB is setCommand (m for help): pDisk /dev/sdd: 21.5 GB, 21474836480 bytes, 41943040 sectorsUnits = sectors of 1 * 512 = 512 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk label type: dosDisk identifier: 0x7e0900d8 Device Boot Start  End Blocks Id System/dev/sdd1  2048 2099199 1048576 83 LinuxCommand (m for help): tSelected partition 1Hex code (type L to list all codes): fdChanged type of partition 'Linux' to 'Linux raid autodetect'Command (m for help): wThe partition table has been altered!Calling ioctl() to re-read partition table.Syncing disks.

[root@centos7 Bash]$ mdadm -C /dev/md5 -a yes -l 5 -n 3 /dev/sd{b1,c1,d1} -c 256 # -C指定創(chuàng)建， -a yes 自動(dòng)創(chuàng)建設(shè)備 ， -l 設(shè)定level , -n 設(shè)定磁盤個(gè)數(shù)， -c chunk大小Continue creating array? ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md5 started.[root@centos7 Bash]$ mdadm -Ds        # 查看信息ARRAY /dev/md5 metadata=1.2 name=centos7.magedu.com:5 UUID=2c8ae60d:a799fcb7:9008a046:ae6ea430[root@centos7 Bash]$ mdadm -Ds >/etc/mdadm.conf      # 將軟raid信息寫入到配置文件中去[root@centos7 Bash]$ mkdir /mnt/md5       # 創(chuàng)建掛載點(diǎn)目錄 [root@centos7 Bash]$ mkfs.ext4 /dev/md5       # 創(chuàng)建文件系統(tǒng)mke2fs 1.42.9 (28-Dec-2013)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=64 blocks, Stripe width=128 blocks131072 inodes, 523776 blocks26188 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=53687091216 block groups32768 blocks per group, 32768 fragments per group8192 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912Allocating group tables: done    Writing inode tables: done    Creating journal (8192 blocks): doneWriting superblocks and filesystem accounting information: done [root@centos7 Bash]$ mount /dev/md5 /mnt/md5      # 掛載設(shè)備 [root@centos7 Bash]$ tail -n 1 /etc/mtab/dev/md5 /mnt/md5 ext4 rw,seclabel,relatime,stripe=128,data=ordered 0 0  # 查看掛載信息[root@centos7 Bash]$ tail -n 1 /etc/mtab >>/etc/fstab     #添加到fstab文件中，確保開機(jī)啟動(dòng)，這里建議使用uuid

[root@centos7 md5]$ mdadm -D /dev/md5       #查看詳細(xì)raid5詳細(xì)信息，可以發(fā)現(xiàn)有3個(gè)都是working狀態(tài)的/dev/md5:Version : 1.2Creation Time : Wed Dec 6 19:28:22 2017Raid Level : raid5Array Size : 2095104 (2046.00 MiB 2145.39 MB)Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)Raid Devices : 3Total Devices : 3Persistence : Superblock is persistentUpdate Time : Wed Dec 6 19:39:06 2017State : clean Active Devices : 3Working Devices : 3Failed Devices : 0Spare Devices : 0Layout : left-symmetricChunk Size : 256KConsistency Policy : resyncName : centos7.magedu.com:5 (local to host centos7.magedu.com)UUID : 2c8ae60d:a799fcb7:9008a046:ae6ea430Events : 18Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 3 8 49 2 active sync /dev/sdd1[root@centos7 md5]$ man mdadm[root@centos7 md5]$ mdadm /dev/md5 -f /dev/sdc1       # -f 設(shè)定指定設(shè)備故障， 將/dev/sdc1 這個(gè)盤標(biāo)記失敗， 看是否數(shù)據(jù)能訪問，我這里使用-f標(biāo)記失敗，工作中可以根據(jù)硬盤指示燈判斷磁盤狀態(tài)mdadm: set /dev/sdc1 faulty in /dev/md5  [root@centos7 md5]$ mdadm -D /dev/md5        #在次查看信息，發(fā)現(xiàn)工作的是2個(gè)， 一個(gè)失敗的設(shè)備 /dev/md5:Version : 1.2Creation Time : Wed Dec 6 19:28:22 2017Raid Level : raid5Array Size : 2095104 (2046.00 MiB 2145.39 MB)Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)Raid Devices : 3Total Devices : 3Persistence : Superblock is persistentUpdate Time : Wed Dec 6 19:41:08 2017State : clean, degraded        # 這里注意了。 我們的一個(gè)盤壞掉了。 raid5狀態(tài)為降級(jí)使用了。Active Devices : 2Working Devices : 2Failed Devices : 1Spare Devices : 0Layout : left-symmetricChunk Size : 256KConsistency Policy : resyncName : centos7.magedu.com:5 (local to host centos7.magedu.com)UUID : 2c8ae60d:a799fcb7:9008a046:ae6ea430Events : 20Number Major Minor RaidDevice State0 8 17 0 active sync /dev/sdb1- 0 0 1 removed3 8 49 2 active sync /dev/sdd11 8 33 - faulty /dev/sdc1[root@centos7 md5]$ cat a.txt      # 發(fā)現(xiàn)我們的數(shù)據(jù)還是能訪問的。沒有問題。

[root@centos7 md5]$ mdadm /dev/md5 -a /dev/sde1   # 上面我們的sdc1數(shù)據(jù)損壞，我們需要更換新的磁盤來頂替他的位置。這里添加一個(gè)sde1的磁盤， fdisk操作這里省去了。mdadm: added /dev/sde1[root@centos7 md5]$ mdadm -Ds      # 查看詳細(xì)信息ARRAY /dev/md5 metadata=1.2 name=centos7.magedu.com:5 UUID=2c8ae60d:a799fcb7:9008a046:ae6ea430[root@centos7 md5]$ mdadm -D /dev/md5     # 查看詳細(xì)信息/dev/md5:Version : 1.2Creation Time : Wed Dec 6 19:28:22 2017Raid Level : raid5Array Size : 2095104 (2046.00 MiB 2145.39 MB)Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)Raid Devices : 3Total Devices : 4Persistence : Superblock is persistentUpdate Time : Wed Dec 6 19:50:01 2017State : clean       # 狀態(tài)恢復(fù)正常了。沒有問題Active Devices : 3Working Devices : 3Failed Devices : 1Spare Devices : 0Layout : left-symmetricChunk Size : 256KConsistency Policy : resyncName : centos7.magedu.com:5 (local to host centos7.magedu.com)UUID : 2c8ae60d:a799fcb7:9008a046:ae6ea430Events : 43Number Major Minor RaidDevice State0 8 17 0 active sync /dev/sdb14 8 65 1 active sync /dev/sde13 8 49 2 active sync /dev/sdd11 8 33 - faulty /dev/sdc1  # 這個(gè)盤是壞掉的，我們已經(jīng)加入了新的磁盤， 這個(gè)盤可以干掉了[root@centos7 md5]$ man mdadm[root@centos7 md5]$ mdadm /dev/md5 --remove /dev/sdc1   # 這個(gè)盤我們從raid5中移除去。 mdadm: hot removed /dev/sdc1 from /dev/md5

[root@centos7 mnt]$ mkadm -G -r /dev/md5 -n 4 -a /dev/sdxx   # 這里我就不測(cè)試了。使用/dev/sdxx代替一個(gè)設(shè)備。-G 是Grown增長(zhǎng)的意思，-r 是resizefs的意思，

[root@centos7 mnt]$ umount /dev/md5     # 卸載設(shè)備[root@centos7 mnt]$ mdadm -S /dev/md5     # 停止raid5 mdadm: stopped /dev/md5[root@centos7 mnt]$ sed -i '$d' /etc/fstab    # 刪除fstab中關(guān)于raid5掛載的行[root@centos7 mnt]$ cat /etc/fstab     # 確保fstab沒有大問題## /etc/fstab# Created by anaconda on Tue Nov 7 16:07:01 2017## Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info#UUID=59ccea87-3c4e-4bbc-9e2f-3fadb1dcf2e6 /   ext4 defaults 1 1UUID=f4e867e8-bcde-43a2-adc7-c80b0948e85f /app   ext4 noatime,usrquota,grpquota 1 2UUID=1d6cbe88-ffb4-4adf-bacf-76be1fa75708 /boot   ext4 defaults 1 2#UUID=b2c064f5-1ee5-4b5c-9e75-ed41cb99c5aa swap   swap defaults 0 0#UUID=a0516c4f-40e6-4919-905a-8b44db12ff7b swap  swap defaults,pri=0 0 0 #/dev/sdb2 /test ext4 rw,seclabel,relatime,data=ordered 0 0#/dev/sdb1 /home xfs rw,seclabel,relatime,attr2,inode64,usrquota,grpquota 0 0[root@centos7 mnt]$ rm -rf /etc/mdadm.conf     # 刪除raid默認(rèn)配置文件

[root@centos7 mnt]$ mdadm --zero-superblock /dev/sd{b1,e1,d1,c1}  # 清空設(shè)置上的超級(jí)塊信息

[root@centos7 mnt]$ lsblk    # 就是使用fdisk 創(chuàng)建的設(shè)備， 具體這里就不寫了。 最終使用lsblk顯示，我們可以看到sdb1,sdb2,sdd1,sde1一共6個(gè)磁盤NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTsda 8:0 0 200G 0 disk ├─sda1 8:1 0 1G 0 part /boot├─sda2 8:2 0 128G 0 part ├─sda3 8:3 0 48.8G 0 part /├─sda4 8:4 0 512B 0 part └─sda5 8:5 0 19.5G 0 part /appsdb 8:16 0 100G 0 disk ├─sdb1 8:17 0 1G 0 part └─sdb2 8:18 0 1G 0 part sdc 8:32 0 20G 0 disk ├─sdc1 8:33 0 1G 0 part └─sdc2 8:34 0 1G 0 part sdd 8:48 0 20G 0 disk └─sdd1 8:49 0 1G 0 part sde 8:64 0 20G 0 disk └─sde1 8:65 0 1G 0 part sdf 8:80 0 20G 0 disk sr0 11:0 1 8.1G 0 rom /run/media/root/CentOS 7 x86_64

[root@centos7 mnt]$ mdadm -C /dev/md11 -a yes -l 1 -n 2 /dev/sd{b1,c1}    # 創(chuàng)建第一個(gè)raid1mdadm: /dev/sdb1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Dec 6 19:28:22 2017mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90mdadm: /dev/sdc1 appears to be part of a raid array: level=raid5 devices=3 ctime=Wed Dec 6 19:28:22 2017Continue creating array? ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md11 started.[root@centos7 mnt]$ mdadm -C /dev/md12 -a yes -l 1 -n 2 /dev/sd{b2,c2}    #創(chuàng)建第二個(gè)raid1mdadm: Note: this array has metadata at the start and may not be suitable as a boot device. If you plan to store '/boot' on this device please ensure that your boot-loader understands md/v1.x metadata, or use --metadata=0.90Continue creating array? ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md12 started.[root@centos7 mnt]$ mdadm -C /dev/md13 -a yes -l 1 -n 2 /dev/sd{d1,e1}     # 創(chuàng)建第三個(gè)raid1mdadm: /dev/sdd1 appears to be part of a raid array:level=raid5 devices=3 ctime=Wed Dec 6 19:28:22 2017mdadm: Note: this array has metadata at the start andmay not be suitable as a boot device. If you plan tostore '/boot' on this device please ensure thatyour boot-loader understands md/v1.x metadata, or use--metadata=0.90mdadm: /dev/sde1 appears to be part of a raid array:level=raid5 devices=3 ctime=Wed Dec 6 19:28:22 2017Continue creating array? ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md13 started.[root@centos7 mnt]$ mdadm -C /dev/md10 -a yes -l 0 -n 3 /dev/md{11,12,13}    # 將3個(gè)raid1 合并為一個(gè)raid0 mdadm: /dev/md11 appears to contain an ext2fs file systemsize=2095104K mtime=Wed Dec 6 19:29:45 2017mdadm: /dev/md13 appears to contain an ext2fs file systemsize=2095104K mtime=Wed Dec 6 19:29:45 2017Continue creating array? ymdadm: Defaulting to version 1.2 metadatamdadm: array /dev/md10 started.[root@centos7 mnt]$ mkfs.extmkfs.ext2 mkfs.ext3 mkfs.ext4  [root@centos7 mnt]$ mkfs.ext4 /dev/md10         # 創(chuàng)建文件系統(tǒng)            mke2fs 1.42.9 (28-Dec-2013)Filesystem label=OS type: LinuxBlock size=4096 (log=2)Fragment size=4096 (log=2)Stride=128 blocks, Stripe width=384 blocks196224 inodes, 784896 blocks39244 blocks (5.00%) reserved for the super userFirst data block=0Maximum filesystem blocks=80530636824 block groups32768 blocks per group, 32768 fragments per group8176 inodes per groupSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912Allocating group tables: done    Writing inode tables: done    Creating journal (16384 blocks): doneWriting superblocks and filesystem accounting information: done [root@centos7 mnt]$ mdadm -Ds          # 查看配置信息ARRAY /dev/md11 metadata=1.2 name=centos7.magedu.com:11 UUID=0ce2cd6c:cd21fab6:3e65cfb5:64bd86f3ARRAY /dev/md12 metadata=1.2 name=centos7.magedu.com:12 UUID=8af31dff:efab06ed:48e2613b:a599c774ARRAY /dev/md13 metadata=1.2 name=centos7.magedu.com:13 UUID=a8c99d60:2d0c61e7:97a76809:9396c020ARRAY /dev/md10 metadata=1.2 name=centos7.magedu.com:10 UUID=50b2fa58:4ce65d67:8c50c853:fa175a28[root@centos7 mnt]$ mdadm -Ds >> /etc/mdadm.conf        # 寫配置文件到mdadm的配置文件中[root@centos7 mnt]$ mkdir /mnt/md10         # 創(chuàng)建掛載目錄[root@centos7 mnt]$ mount /dev/md10 /mnt/md10        # 掛載文件系統(tǒng)[root@centos7 mnt]$ tail -n 1 /etc/mtab         # 查看mtab文件中的最后一行， 也就是我們的md10掛載信息/dev/md10 /mnt/md10 ext4 rw,seclabel,relatime,stripe=384,data=ordered 0 0[root@centos7 mnt]$ tail -n 1 /etc/mtab >> /etc/fstab       #添加到開機(jī)啟動(dòng)

[root@centos7 mnt]$ umount /dev/md10         # 取消掛載[root@centos7 mnt]$ rm -rf /etc/mdadm.conf        # 刪除mdadm的默認(rèn)配置[root@centos7 mnt]$ mdadm -S /dev/md10         # 停止raid0設(shè)置mdadm: stopped /dev/md10[root@centos7 mnt]$ mdadm -S /dev/md11         # 停止raid1設(shè)置mdadm: stopped /dev/md11[root@centos7 mnt]$ mdadm -S /dev/md12         # 停止radi1 設(shè)置mdadm: stopped /dev/md12 [root@centos7 mnt]$ mdadm -S /dev/md13         # 停止raid 1 設(shè)置mdadm: stopped /dev/md13[root@centos7 mnt]$ sed -i '$d' /etc/fstab        # 刪除fstab的掛載 [root@centos7 mnt]$ cat /etc/fstab         # 確保正確## /etc/fstab# Created by anaconda on Tue Nov 7 16:07:01 2017## Accessible filesystems, by reference, are maintained under '/dev/disk'# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info#UUID=59ccea87-3c4e-4bbc-9e2f-3fadb1dcf2e6 /   ext4 defaults 1 1UUID=f4e867e8-bcde-43a2-adc7-c80b0948e85f /app   ext4 noatime,usrquota,grpquota 1 2UUID=1d6cbe88-ffb4-4adf-bacf-76be1fa75708 /boot   ext4 defaults 1 2#UUID=b2c064f5-1ee5-4b5c-9e75-ed41cb99c5aa swap   swap defaults 0 0#UUID=a0516c4f-40e6-4919-905a-8b44db12ff7b swap  swap defaults,pri=0 0 0 #/dev/sdb2 /test ext4 rw,seclabel,relatime,data=ordered 0 0#/dev/sdb1 /home xfs rw,seclabel,relatime,attr2,inode64,usrquota,grpquota 0 0[root@centos7 mnt]$ mdadm -D           # 再次查看下mdadm信息，確保沒有了mdadm: No devices given.[root@centos7 mnt]$ mdadm --zero-superblock /dev/sd{b1,b2,c1,c2,d1,e1}     # 請(qǐng)求md的元數(shù)據(jù)信息