部署longhorn

longhorn参考：https://longhorn.io/docs/1.4.1/deploy/important-notes/

github：https://github.com/longhorn/longhorn

安装Longhorn的Kubernetes集群中的每个节点必须满足以下要求：

• 与Kubernetes兼容的容器运行时（Docker v1.13+，containerd v1.3.7+等）

• Kubernetes >= v1.21

• open-iscsi 已安装，并且 iscsid 守护进程正在所有节点上运行。这是必要的，因为Longhorn依赖于主机上的 iscsiadm 来为Kubernetes提供持久卷

• RWX支持要求每个节点都安装了NFSv4客户端。

• 主机文件系统支持 file extents 功能来存储数据。目前支持：ext4、XFS

• 必须安装 bash 、 curl 、 findmnt 、 grep 、 awk 、 blkid 、 lsblk

cd longhorn
# 安装open-iscsi/nfs-utils依赖
# centos
yum install -y iscsi-initiator-utils
yum install -y nfs-utils

# debian
apt-get install open-iscsi -y
apt-get install nfs-common -y
apt-get install jq -y

## 也可以使用官方提供的yaml安装方式（需要网络）
cd prerequisite
kubectl apply -f longhorn-iscsi-installation.yaml -nlonghorn-system
kubectl apply -f longhorn-nfs-installation.yaml -nlonghorn-system
## yaml安装验证
kubectl get pod | grep longhorn-iscsi-installation
kubectl logs longhorn-iscsi-installation-xxxx -c iscsi-installation
kubectl get pod | grep longhorn-nfs-installation
kubectl logs longhorn-nfs-installation-xxxx -c nfs-installation

# 检测安装环境
yum -y install jq
sh check.sh 
[INFO]  Required dependencies 'kubectl jq mktemp' are installed.
[INFO]  Hostname uniqueness check is passed.
[INFO]  Waiting for longhorn-environment-check pods to become ready (0/0)...
[INFO]  Waiting for longhorn-environment-check pods to become ready (2/3)...
[INFO]  All longhorn-environment-check pods are ready (3/3).
[INFO]  Required packages are installed.
[INFO]  MountPropagation is enabled.
[INFO]  Cleaning up longhorn-environment-check pods...
[INFO]  Cleanup completed.

#如果全部通过，执行下一步安装
kubectl apply -f longhorn.yaml

# 验证
kubectl get pods -nlonghorn-system
kubectl get sc

longhorn.yaml需要关注的配置

# 修改longhorn-ui通过nodeport方式暴露
# Source: longhorn/templates/deployment-ui.yaml
kind: Service
...
spec:
type: NodePort
...
nodePort: 30011
...

# 修改fstype为本地文件系统格式
# Source: longhorn/templates/storageclass.yaml
apiVersion: v1
kind: ConfigMap
...
	  numberOfReplicas: "3"		# 卷副本数，一般和node节点数一致
fsType: "ext4"
...

# 修改longhorn数据存储路径
# Source: longhorn/templates/default-setting.yaml
apiVersion: v1
kind: ConfigMap
...
data:
default-setting.yaml: |-
default-data-path: /longhorn

longhorn卸载

kubectl -n longhorn-system patch -p '{"value": "true"}' --type=merge lhs deleting-confirmation-flag
kubectl apply -f uninstall.yaml -nlonghorn-system # 等待卸载job执行完成
kubectl get job/longhorn-uninstall -n longhorn-system -w

kubectl delete -f longhorn.yaml -nlonghorn-system
kubectl delete -f uninstall.yaml -nlonghorn-system