前言
为了搞NoKVM重装了好几次了，每次都需要提前配置好RaidZ阵列才行。因为之前一直没有做笔记，只能一次次的查教程，这次做一个笔记吧！
参考文档：https://github.com/zfsonlinux/zfs/wiki/RHEL-and-CentOS
安装ZFS
版本是CentOS7.6，请注意自己的具体版本
yum install epel-release -y && yum localinstall --nogpgcheck http://download.zfsonlinux.org/epel/zfs-release.el7.noarch.rpm -y && yum update -y
按照下面的提示修改文件：/etc/yum.repos.d/zfs.repo，将repo版本改为kmod，也就是标准发行版的kernel
[zfs]
name=ZFS on Linux for EL 7 - dkms
baseurl=http://download.zfsonlinux.org/epel/7/$basearch/
-enabled=1
+enabled=0
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
@@ -9,7 +9,7 @@
[zfs-kmod]
name=ZFS on Linux for EL 7 - kmod
baseurl=http://download.zfsonlinux.org/epel/7/kmod/$basearch/
-enabled=0
+enabled=1
metadata_expire=7d
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-zfsonlinux
然后再安装zfs，为了保险起见先删除可能会被安装的错误依赖：
yum remove zfs zfs-kmod spl spl-kmod libzfs2 libnvpair1 libuutil1 libzpool2 zfs-release -y && yum install http://download.zfsonlinux.org/epel/zfs-release.el7_6.noarch.rpm -y && yum update -y && yum install kernel-devel zfs -y
安装成功后重启，注意看可能会有报错，出现报错需要进一步看是不是repo选错了。重启后查看zfs是否启动了，没启动的话启动一下：
[root@NoKVM-EU-1 ~]# lsmod | grep zfs
zfs 3564425 3
zunicode 331170 1 zfs
zavl 15236 1 zfs
icp 270148 1 zfs
zcommon 73440 1 zfs
znvpair 89131 2 zfs,zcommon
spl 102412 4 icp,zfs,zcommon,znvpair
#如果没有上面的内容就手动开启
modprobe zfs
设置ZFS的内存限制
ZFS默认把一半的内存作为缓存使用，在大机器中其实不需要那么多，因此设置ZFS缓存大小很重要。
vi /etc/modprobe.d/zfs.conf
#写入以下信息：
# Min 5GB / Max 16GB Limit
options zfs zfs_arc_min=5368709120
options zfs zfs_arc_max=17179869184
然后重启生效，验证一下：
[root@NoKVM-EU-1 ~]# cat /proc/spl/kstat/zfs/arcstats |grep c_
c_min 4 5368709120
c_max 4 17179869184
arc_no_grow 4 0
arc_tempreserve 4 0
arc_loaned_bytes 4 0
arc_prune 4 0
arc_meta_used 4 0
arc_meta_limit 4 12884901888
arc_dnode_limit 4 1288490188
arc_meta_max 4 0
arc_meta_min 4 16777216
sync_wait_for_async 4 0
arc_need_free 4 0
arc_sys_free 4 2109268096
建立zpool冗余阵列RaidZ2
我这里有14块盘，做RaidZ2，有效存储量为12块盘，容错率为2块盘。
zpool create home /dev/sd{b,c,d,e,f,g,h,i,j,k,l,n,m,o}
但是可能会出现报错，这是因为之前有将磁盘添加到zpool中，所以需要先删除无用分区，用fdisk
[root@NoKVM-EU-1 ~]# zpool create home /dev/sd{b,c,d,e,f,g,h,i,j,k,l,n,m,o}
invalid vdev specification
use '-f' to override the following errors:
/dev/sdb1 is part of potentially active pool 'tank'
/dev/sdc1 is part of potentially active pool 'tank'
/dev/sdd1 is part of potentially active pool 'tank'
/dev/sde1 is part of potentially active pool 'tank'
/dev/sdf1 is part of potentially active pool 'tank'
/dev/sdg1 is part of potentially active pool 'tank'
/dev/sdh1 is part of potentially active pool 'tank'
/dev/sdi1 is part of potentially active pool 'tank'
/dev/sdj1 is part of potentially active pool 'tank'
/dev/sdk1 is part of potentially active pool 'tank'
/dev/sdl1 is part of potentially active pool 'tank'
/dev/sdn1 is part of potentially active pool 'tank'
/dev/sdm1 is part of potentially active pool 'tank'
/dev/sdo1 is part of potentially active pool 'tank'
[root@NoKVM-EU-1 ~]# fdisk /dev/sdb
WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion.
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Command (m for help): d
Partition number (1,9, default 9):
Partition 9 is deleted
Command (m for help): d
Selected partition 1
Partition 1 is deleted
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
然后查看可用的磁盘设备，有没有多出来的盘符，没有就可以重新添加到zpool了
[root@NoKVM-EU-1 ~]# ls -l /dev/sd*
brw-rw---- 1 root disk 8, 0 Jan 13 10:56 /dev/sda
brw-rw---- 1 root disk 8, 1 Jan 13 10:56 /dev/sda1
brw-rw---- 1 root disk 8, 2 Jan 13 10:56 /dev/sda2
brw-rw---- 1 root disk 8, 3 Jan 13 10:56 /dev/sda3
brw-rw---- 1 root disk 8, 4 Jan 13 10:56 /dev/sda4
brw-rw---- 1 root disk 8, 5 Jan 13 10:56 /dev/sda5
brw-rw---- 1 root disk 8, 16 Jan 13 11:02 /dev/sdb
brw-rw---- 1 root disk 8, 32 Jan 13 11:02 /dev/sdc
brw-rw---- 1 root disk 8, 48 Jan 13 11:02 /dev/sdd
brw-rw---- 1 root disk 8, 64 Jan 13 11:03 /dev/sde
brw-rw---- 1 root disk 8, 80 Jan 13 11:03 /dev/sdf
brw-rw---- 1 root disk 8, 96 Jan 13 11:03 /dev/sdg
brw-rw---- 1 root disk 8, 112 Jan 13 11:03 /dev/sdh
brw-rw---- 1 root disk 8, 128 Jan 13 11:03 /dev/sdi
brw-rw---- 1 root disk 8, 144 Jan 13 11:03 /dev/sdj
brw-rw---- 1 root disk 8, 160 Jan 13 11:03 /dev/sdk
brw-rw---- 1 root disk 8, 176 Jan 13 11:03 /dev/sdl
brw-rw---- 1 root disk 8, 192 Jan 13 11:03 /dev/sdm
brw-rw---- 1 root disk 8, 208 Jan 13 11:03 /dev/sdn
brw-rw---- 1 root disk 8, 224 Jan 13 11:04 /dev/sdo
使用zpool status查看一下效果：
[root@NoKVM-EU-1 ~]# zpool status
pool: home
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
home ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
sdg ONLINE 0 0 0
sdh ONLINE 0 0 0
sdi ONLINE 0 0 0
sdj ONLINE 0 0 0
sdk ONLINE 0 0 0
sdl ONLINE 0 0 0
sdn ONLINE 0 0 0
sdm ONLINE 0 0 0
sdo ONLINE 0 0 0
errors: No known data errors
[root@NoKVM-EU-1 ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 63G 0 63G 0% /dev
tmpfs 63G 0 63G 0% /dev/shm
tmpfs 63G 900K 63G 1% /run
tmpfs 63G 0 63G 0% /sys/fs/cgroup
/dev/sda4 5.3T 1.5G 5.1T 1% /
/dev/sda2 488M 219M 244M 48% /boot
/dev/sda3 64G 33M 64G 1% /tmp
tmpfs 13G 0 13G 0% /run/user/0
home 74T 0 74T 0% /home
Please quote the original link：https://www.liujason.com/article/474.html