安装IBM Cloud Object Storage集群附加组件
IBM Cloud Object Storage集群插件的测试版仅适用于允许列表账户。 要添加到允许列表,请联系支持部门。 有关更多信息,请参阅 请求访问允许列表功能。
- 先决条件
- IBM Cloud Object Storage 插件至少需要 0.2、vCPU 和128 MB内存。
了解水桶的创建和移除
- 您可以通过在 PVC 中指定水桶名称来使用现有的水桶。
- 如果您提供了一个水桶名称,而该水桶并不存在,那么就会创建一个具有该名称的水桶。
- 如果不提供存储桶名称,则会创建一个命名为
temp-xxx
的存储桶。 - 根据存储类中定义的回收策略删除存储桶。
- 如果设置了
reclaimPolicy: Delete
,则在删除 PVC 时删除水桶。 - 如果设置了
reclaimPolicy: Retain
,则即使删除了 PVC,也会保留水桶。
- 如果设置了
启用IBM Cloud Object Storage附加组件
开始之前 访问你的Red Hat OpenShift集群。
- 列出附加组件并找到要安装的版本。
示例输出ibmcloud oc cluster addon versions
OK Name Version Supported Kubernetes Range Supported OpenShift Range Kubernetes Default OpenShift Default ibm-object-csi-driver 0.1 (default) >=1.30.0 >=4.15.0 - -
- 安装插件。
ibmcloud oc cluster addon enable ibm-object-csi-driver --cluster CLUSTER [--version VERSION]
- 验证安装。
ibmcloud oc cluster addon ls --cluster CLUSTER
OK Name Version Health State Health Status ibm-object-csi-driver 0.1 normal Addon Ready. For more info: http://ibm.biz/addon-state (H1500)
- 列出可用的存储类别。
oc get sc | grep object
ibm-object-storage-smart-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-smart-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-smart-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-rclone cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-rclone-retain cos.s3.csi.ibm.io Retain Immediate false 17h ibm-object-storage-standard-s3fs cos.s3.csi.ibm.io Delete Immediate false 17h ibm-object-storage-standard-s3fs-retain cos.s3.csi.ibm.io Retain Immediate false 17h
部署使用IBM Cloud Object Storage的应用程序
创建一个包含 COS 凭据的Kubernetes秘密。
-
将以下配置保存为名为
secret.yaml
.apiVersion: v1 kind: Secret type: cos-s3-csi-driver metadata: name: cos-secret-1 # Name your secret. This same name is used for the PVC in the following steps. namespace: <namespace> # Specify the namespace where you want to create the secret. data: bucketName: <base64-encoded-bucket-name> apiKey: <base64-encoded-COS-Service-Instance-API-key> accessKey: <base64-encoded-HMAC-access-key> secretKey: <base64-encoded-HMAC-secret-key> stringData: # uid: "3000" # Optional: Provide a uid to run as non root user. This must match runAsUser in SecurityContext of pod spec. mountOptions: |
-
将上一节中获取的凭据编码为base64。 对每个参数重复此命令。
echo -n "<value>" | base64
-
用base64编码值更新配置文件。
-
创建私钥。
oc apply -f secret.yaml
创建 PVC
-
将以下配置保存到名为
pvc.yaml
.apiVersion: v1 kind: PersistentVolumeClaim metadata: name: cos-secret-1 # Give your PVC the same name as the secret you created in the previous step. namespace: <namespace> # The namespace where you want to create the PVC. spec: accessModes: - ReadWriteMany resources: requests: storage: 10Gi storageClassName: <storage_class_name> # The storage class you want to use.
-
编辑配置文件值。 确保与创建秘密的命名空间相同。 有关存储类的列表,请参阅 存储类参考。
-
创建 PVC。
oc apply -f pvc.yaml
创建部署
-
将以下配置保存到名为
dep.yaml
.apiVersion: apps/v1 kind: Deployment metadata: name: <name> labels: app: <name> spec: replicas: 1 selector: matchLabels: app: <name> template: metadata: labels: app: <name> spec: containers: - name: app-frontend image: <image> # Enter your app image. imagePullPolicy: IfNotPresent volumeMounts: - mountPath: <path_you_want_to_mount_the_volume_on> # For example `/dev` name: cos-csi-volume volumes: - name: cos-csi-volume persistentVolumeClaim: claimName: <pvc_name> # Enter the name of the PVC you created earlier.
-
创建部署。
oc apply -f dep.yaml
为陈旧卷设置自动恢复功能
当 ibm-object-csi-driver
节点服务器 pod 和应用程序 pod 之间的连接丢失时,可能会出现 TransportEndpoint
连接错误。 出现这种错误的一种可能情况是应用了补丁更新。 为避免出现连接错误,请通过完成以下步骤设置过期卷的自动恢复。
-
复制以下 yaml 并将其保存为名为
stale.yaml
的文件apiVersion: objectdriver.csi.ibm.com/v1alpha1 kind: RecoverStaleVolume metadata: labels: app.kubernetes.io/name: recoverstalevolume app.kubernetes.io/instance: recoverstalevolume-sample name: recoverstalevolume-sample namespace: default spec: logHistory: 200 data: - namespace: default # The namesapce where your app is deployed deployments: [<A comma separated list of all the apps you want to recover>]
-
在群集中创建
RecoverStaleVolume
资源。oc create -f stale.yaml
示例输出
recoverstalevolume.objectdriver.csi.ibm.com/recoverstalevolume-sample created
-
确认资源已创建。
oc get recoverstalevolume
示例输出
NAME AGE recoverstalevolume-sample 41s
-
如果问题仍然存在,请联系支持团队。 打开 支持案例。 在案例详细信息中,请务必包含任何相关日志文件、错误信息或命令输出。
通过模拟错误验证恢复
-
列出您的部署。
oc get deploy -o wide
示例输出
NAME READY UP-TO-DATE AVAILABLE AGE CONTAINERS IMAGES SELECTOR cos-csi-test-app 1/1 1 1 7h24m app-frontend rabbitmq app=cos-csi-test-app
-
列出应用程序 pod。
oc get pods -o wide
示例输出
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES cos-csi-test-app-6b99bd8bf4-5lt7p 1/1 Running 0 7h24m 172.30.69.21 10.73.114.86 <none> <none>
-
列出
ibm-object-csi-operator
命名空间中的 pod。oc get pods -n ibm-object-csi-operator -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ibm-object-csi-controller-d64df8f57-l6grj 3/3 Running 0 7h31m 172.30.69.19 10.73.114.86 <none> <none> ibm-object-csi-node-6d4x4 3/3 Running 0 7h31m 172.30.64.24 10.48.3.149 <none> <none> ibm-object-csi-node-gg5pj 3/3 Running 0 7h31m 172.30.116.13 10.93.120.14 <none> <none> ibm-object-csi-node-vk8jf 3/3 Running 0 7h31m 172.30.69.20 10.73.114.86 <none> <none> ibm-object-csi-operator-controller-manager-8544d4f798-llbf8 1/1 Running 0 7h37m 172.30.69.18 10.73.114.86 <none> <none>
-
Delete the
ibm-object-csi-node-xxx
pod in theibm-object-csi-operator
namespace.oc delete pod ibm-object-csi-node-vk8jf -n ibm-object-csi-operator
示例输出
pod "ibm-object-csi-node-vk8jf" deleted
-
列出
ibm-object-csi-operator
命名空间中的 pod。oc get pods -n ibm-object-csi-operator -o wide
示例输出
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES ibm-object-csi-controller-d64df8f57-l6grj 3/3 Running 0 7h37m 172.30.69.19 10.73.114.86 <none> <none> ibm-object-csi-node-6d4x4 3/3 Running 0 7h37m 172.30.64.24 10.48.3.149 <none> <none> ibm-object-csi-node-gg5pj 3/3 Running 0 7h37m 172.30.116.13 10.93.120.14 <none> <none> ibm-object-csi-node-kmn94 3/3 Running 0 8s 172.30.69.23 10.73.114.86 <none> <none> ibm-object-csi-operator-controller-manager-8544d4f798-llbf8 1/1 Running 0 7h43m 172.30.69.18 10.73.114.86 <none> <none>
-
获取
ibm-object-csi-operator-controller-manager
的日志,跟踪应用程序 pod 的恢复情况。 请注意,操作员会删除应用程序的 pod,以便重新启动它们。2024-07-10T17:25:39Z INFO recoverstalevolume_controller Time to complete {"fetchVolumeStatsFromNodeServerPodLogs": 0.066584637} 2024-07-10T17:25:39Z INFO recoverstalevolume_controller Volume Stats from NodeServer Pod Logs {"Request.Namespace": "default", "Request.Name": "recoverstalevolume-sample", "volume-stas": {"pvc-9d12a2f5-09a9-4eb4-b1f5-2a727249ed2b":"transport endpoint is not connected "}} 2024-07-10T17:25:39Z INFO recoverstalevolume_controller Stale Volume Found {"Request.Namespace": "default", "Request.Name": "recoverstalevolume-sample", "volume": "pvc-9d12a2f5-09a9-4eb4-b1f5-2a727249ed2b"} 2024-07-10T17:25:39Z INFO recoverstalevolume_controller Pod using stale volume {"Request.Namespace": "default", "Request.Name": "recoverstalevolume-sample", "volume-name": "pvc-9d12a2f5-09a9-4eb4-b1f5-2a727249ed2b", "pod-name": "cos-csi-test-app-6b99bd8bf4-5lt7p"} 2024-07-10T17:25:39Z INFO recoverstalevolume_controller Pod deleted. {"Request.Namespace": "default", "Request.Name": "recoverstalevolume-sample"}
禁用IBM Cloud Object Storage附加组件
- 运行以下命令禁用该插件。
示例输出ibmcloud oc cluster addon disable ibm-object-csi-driver --cluster CLUSTER
Data and resources that you created for the add-on might be deleted when the add-on is disabled. Continue? [y/N]> y Disabling add-on ibm-object-csi-driver for cluster XXX... OK
- 验证是否已删除附加组件。
ibmcloud oc cluster addon ls --cluster CLUSTER
从Helm插件迁移到群集插件
-
获取 PVC 的详细信息并选择一个进行迁移。
oc get pvc --all-namespaces -o custom-columns='NAMESPACE:.metadata.namespace,NAME:.metadata.name' | tail -n +2 | while read namespace pvc; do kubectl describe pvc "$pvc" -n "$namespace" | grep 'volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs' > /dev/null ; if [ $? -eq 0 ]; then echo "PVC: $pvc in Namespace: $namespace uses ibm.io/ibmc-s3fs storage provisioner"; fi; done
示例输出
PVC: pvc-test in Namespace: default uses ibm.io/ibmc-s3fs storage provisioner
-
描述 PVC 并获取水桶名称。
oc describe pvc <pvc_name> | grep ibm.io/bucket:
示例输出
ibm.io/bucket: test-s3
-
创建一个与 PVC 名称相同的秘密。
apiVersion: v1 kind: Secret type: cos-s3-csi-driver metadata: name: test-s3 # Name your secret the same name your PVC namespace: default # Specify the namespace where you want to create the secret. In this example, the previous PVC and secret were in the default namespace. data: bucketName: <base64-encoded-bucket-name> apiKey: <base64-encoded-COS-Service-Instance-API-key> accessKey: <base64-encoded-HMAC-access-key> secretKey: <base64-encoded-HMAC-secret-key> stringData: # uid: "3000" # Optional: Provide a uid to run as non root user. This must match runAsUser in SecurityContext of pod spec. mountOptions: |
-
查找 PVC 中使用的存储类。
oc describe pvc <pvc_name> | grep StorageClass:
名为
test-s3
的 PVC 的命令示例。oc describe pvc test-s3 | grep StorageClass:
示例输出
StorageClass: ibmc-s3fs-smart-perf-regional
-
查看新的存储类别 that are available with the add-on and select a replacement class.
- 如果您使用的是
flex
类,请选择一个新的smart
类。 - 如果您使用的是
standard
类,请选择一个新的standard
类。 - 附加组件不再提供
cold
和vault
类,请选择smart
或standard
类。
- 如果您使用的是
-
查看 PVC 的详细信息。
oc describe pvc test-s3
示例输出
Name: pvc-test Namespace: default StorageClass: ibmc-s3fs-smart-perf-regional Status: Bound Volume: pvc-c625474d-31f0-4929-bc3e-feace1fb42fb Labels: <none> Annotations: ibm.io/auto-create-bucket: true ibm.io/auto-delete-bucket: true ibm.io/bucket: bha-test-s23 ibm.io/secret-name: satstoragesecret pv.kubernetes.io/bind-completed: yes pv.kubernetes.io/bound-by-controller: yes volume.beta.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs volume.kubernetes.io/storage-provisioner: ibm.io/ibmc-s3fs Finalizers: [kubernetes.io/pvc-protection] Capacity: 3Gi Access Modes: RWO VolumeMode: Filesystem Used By: test-pod Events: <none>
-
创建一个替代 PVC,使用新的存储类并引用你之前创建的秘密。
apiVersion: v1 kind: PersistentVolumeClaim metadata: name: test-s3 # Enter the same name as the secret you created earlier. spec: accessModes: - ReadWriteOnce resources: requests: storage: 3Gi storageClassName: ibm-object-storage-smart-s3fs
-
确认 PVC 为
Bound
。oc get pvc
-
获取应用程序的详细信息。
oc get pods
-
将应用程序的规模缩小到零。
kubectl scale deployment --replicas=0 my-app
-
创建一个替换部署,引用上一步中创建的 PVC。
-
新部署运行后,可以删除旧部署。
-
对每个要迁移的PVC重复这些步骤。
IBM Cloud Object Storage集群附加存储类
名称 | 回收策略 | 绑定方式 |
---|---|---|
ibm-object-storage-smart-rclone | 删除(T) | 立即 |
ibm-object-storage-smart-rclone-retain | Retain | 立即 |
ibm-object-storage-smart-s3fs | 删除(T) | 立即 |
ibm-object-storage-smart-s3fs-retain | Retain | 立即 |
ibm-object-storage-standard-rclone | 删除(T) | 立即 |
ibm-object-storage-standard-rclone-retain | Retain | 立即 |
ibm-object-storage-standard-s3fs | 删除(T) | 立即 |
ibm-object-storage-standard-s3fs-retain | Retain | 立即 |