K8S你学废了么4——应用配置存储卷-摩杜云开发者社区

一、背景介绍

我们知道程序本身是被装载进内存的一段代码，类似于系统上的cat或者ls程序，就是被装载进内存以实现特定的功能，而简单的程序往往没有什么复杂的定制，越是复杂的应用（如nginx等）根据环境与应用场景的不同，需要进行个性化的定制，大多数应用都可以通过命令行，选项，参数，环境变量和配置文件传递，通常的传递方式有以下几种：

创建 Pod 时设置命令及参数
使用环境变量来设置参数
引用configMap设置参数

二、应用配置存储

1.创建 Pod 时设置命令及参数

创建 Pod 时，可以为其下的容器设置启动时要执行的命令及其参数。如果要设置命令，就填写在配置文件的 command 字段下，如果要设置命令的参数，就填写在配置文件的 args 字段下。一旦 Pod 创建完成，该命令及其参数就无法再进行更改了。

如果在配置文件中设置了容器启动时要执行的命令及其参数，那么容器镜像中自带的命令与参数将会被覆盖而不再执行。如果配置文件中只是设置了参数，却没有设置其对应的命令，那么容器镜像中自带的命令会使用该新参数作为其执行时的参数。

apiVersion: v1
kind: Pod
metadata:
  name: command-demo
spec:
  containers:
  - name: command-demo-container
    image: debian
    command: ["printenv"]
    args: ["HOSTNAME", "KUBERNETES_PORT"]
  restartPolicy: OnFailure

root@master1:~/yaml/chapter4# kubectl apply -f 1.yaml
pod/command-demo created

root@master1:~/yaml/chapter4# kubectl get pod
NAME           READY   STATUS      RESTARTS   AGE
command-demo   0/1     Completed   0          39s

root@master1:~/yaml/chapter4# kubectl logs command-demo
command-demo
tcp://10.96.0.1:443

2.使用环境变量来设置参数

在上面的示例中，我们直接将一串字符作为命令的参数。除此之外，我们还可以将环境变量作为命令的参数。

apiVersion: v1
kind: Pod
metadata:
  name: command-demo
spec:
  containers:
  - name: command-demo-container
    image: debian
    env:
    - name: MESSAGE
      value: "hello world"
    command: ["/bin/echo"]
    args: ["$(MESSAGE)"]

root@master1:~/yaml/chapter4# kubectl apply -f 2.yaml
pod/command-demo created

root@master1:~/yaml/chapter4# kubectl get pod
NAME           READY   STATUS      RESTARTS   AGE
command-demo   0/1     Completed   0          20s

root@master1:~/yaml/chapter4# kubectl logs command-demo
hello world

3.引用configMap设置参数

ConfigMap 是一种 API 对象，用来将非机密性的数据保存到键值对中。使用时，Pod 可以将其用作环境变量、命令行参数或者存储卷中的配置文件。ConfigMap 将你的环境配置信息和容器镜像解耦，便于应用配置的修改。生产环境中，也建议通过ConfigMap注意配置信息。从 v1.19 开始，可以通过将 immutable 字段设置为 true 创建不可变更的 ConfigMap名称。在ConfigMap中，文本数据挂载成文件时采用 UTF-8 字符编码。其他字符编码形式，使用 binaryData 字段

3.1创建时设置configMap键值

通过字面量形式创建键值。如果只是定义字面量，使用命令比yaml文件要来得方便

root@master1:~/yaml/chapter4# kubectl create configmap from-literal  --from-literal=HOSTNAME=ark  --from-literal=SHLVL=7
configmap/from-literal created

root@master1:~/yaml/chapter4# kubectl get cm
NAME                DATA   AGE
from-literal         2      6s

root@master1:~/yaml/chapter4# kubectl describe cm from-literal
Name:         from-literal
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
HOSTNAME:
----
ark
SHLVL:
----
7

BinaryData
====

Events:  <none>

通过文件创建ConfigMap。在键值较多时使用该方法，key是创建时指定的名称，value是文件的内容

root@master1:~/yaml/chapter4# kubectl create configmap from-file --from-file=ngix.conf=n.conf --from-file=fastcgi.conf=f.conf
configmap/from-file created

root@master1:~/yaml/chapter4# kubectl get cm
NAME               DATA   AGE
from-file           2      8s

root@master1:~/yaml/chapter4# kubectl describe cm from-file
Name:         from-file
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
fastcgi.conf:
----
fastcgi_param  REDIRECT_STATUS    200;
......

ngix.conf:
----
user  nginx;
worker_processes  auto;
error_log  /var/log/nginx/error.log notice;
pid        /var/run/nginx.pid;
......

BinaryData
====

Events:  <none>

通过目录创建ConfigMap。如果配置文件有多个可以。此时目录下的文件名成为ConfigMap中的key值，文件内容成为value值

root@master1:~/yaml/chapter4# ls conf.d/
fastcgi.conf  nginx.conf

root@master1:~/yaml/chapter4# kubectl create configmap from-dir --from-file=/root/yaml/chapter4/conf.d/
configmap/from-dir created

root@master1:~/yaml/chapter4# kubectl get cm
NAME               DATA   AGE
from-dir           2      28s

root@master1:~/yaml/chapter4# kubectl get cm from-dir -o yaml
apiVersion: v1
data:
  fastcgi.conf: |2+
    fastcgi_param  SCRIPT_FILENAME    $document_root$fastcgi_script_name;
    fastcgi_param  QUERY_STRING       $query_string;
    fastcgi_param  REQUEST_METHOD     $request_method;
    fastcgi_param  CONTENT_TYPE       $content_type;
......
   nginx.conf: |+
    user  nginx;
    worker_processes  auto;

    error_log  /var/log/nginx/error.log notice;
    pid        /var/run/nginx.pid;
......

通过文件/目录和字面量创建ConfigMap。上述方式中，也可以相互混合使用

root@master1:~/yaml/chapter4# kubectl create configmap from-mix  --from-literal=HOSTNAME=ark  --from-literal=SHLVL=7 --from-file=/root/yaml/chapter4/conf.d/
configmap/from-mix created

root@master1:~/yaml/chapter4# kubectl get cm from-mix
NAME       DATA   AGE
from-mix   4      31s

通过yaml配置清单创建ConfigMap。这也是生产环境中最推荐的方法

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  SPECIAL_LEVEL: very
  SPECIAL_TYPE: charm

root@master1:~/yaml/chapter4# kubectl apply -f 4.yaml
configmap/special-config created

root@master1:~/yaml/chapter4# kubectl get cm special-config
NAME             DATA   AGE
special-config   2      15s

root@master1:~/yaml/chapter4# kubectl describe cm special-config
Name:         special-config
Namespace:    default
Labels:       <none>
Annotations:  <none>

Data
====
SPECIAL_LEVEL:
----
very
SPECIAL_TYPE:
----
charm

BinaryData
====

Events:  <none>

3.2通过环境变量引用configMap键值

使用环境变量引用的方式可以在创建pod时，通过调用configMap的值来注入参数

apiVersion: v1
kind: Pod
metadata:
  name: from-literal
spec:
  containers:
    - name: from-literal
      image: busybox:latest
      imagePullPolicy: IfNotPresent
      command: ["printenv"]
      args: ["WHO", "NUMBER"]
      env:
        - name: WHO
          valueFrom:
            configMapKeyRef:
              name: from-literal
              key: HOSTNAME
        - name: NUMBER
          valueFrom:
            configMapKeyRef:
              name: from-literal
              key: SHLVL
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: from-literal
  namespace: default
data:
  HOSTNAME: ark
  SHLVL: "7"

root@master1:~/yaml/chapter4# kubectl apply -f 5.yaml
pod/from-literal created
configmap/from-literal created

root@master1:~/yaml/chapter4# kubectl get pod
NAME           READY   STATUS      RESTARTS   AGE
from-literal   0/1     Completed   0          4s

root@master1:~/yaml/chapter4# kubectl logs from-literal
ark
7

这种方法有2个约束条件必须先得到满足，否则pod会创建失败：

引用的configMap资源必须实现存在
引用的configMap资源中的键值必须存在

假设将上述示例configMap资源中data字段的“HOSTNAME: ark”去掉，再次执行得到以下结果

root@master1:~/yaml/chapter4# kubectl apply -f 5.yaml
pod/from-literal created
configmap/from-literal created

root@master1:~/yaml/chapter4# kubectl get cm
NAME               DATA   AGE
from-literal       1      7s

root@master1:~/yaml/chapter4# kubectl get pod
NAME           READY   STATUS                       RESTARTS   AGE
from-literal   0/1     CreateContainerConfigError   0          15s

pod创建失败，提示在ConfigMap中找不到键名HOSTNAME的数据

root@master1:~/yaml/chapter4# kubectl logs from-literal
Error from server (BadRequest): container "from-literal" in pod "from-literal" is waiting to start: CreateContainerConfigError

root@master1:~/yaml/chapter4# kubectl describe pod from-literal
......
Events:
  Type     Reason     Age                   From               Message
  ----     ------     ----                  ----               -------
  Normal   Scheduled  2m35s                 default-scheduler  Successfully assigned default/from-literal to node2
  Warning  Failed     20s (x12 over 2m34s)  kubelet            Error: couldn't find key HOSTNAME in ConfigMap default/from-literal

3.3使用configMap存储卷注入键值

通过上面的2种方式都可以让ConfigMap的值注入到pod中，但上述两种方式存在2个弊端：

变量或参数是被加载进pod内存中，如果数据量太大，会占用过多内存
ConfigMap中的值在pod启动时被注入，后期ConfigMap没法自动更新

为了解决该问题，引入了将ConfigMap以存储卷的形式挂载使用，此方式也是生产环境中推荐的方式。

将ConfigMap挂载为存储卷，挂载目录（config）可以事先不存在，ConfigMap中的key值成为文件名，value值成为文件内容

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  SPECIAL_LEVEL: very
  SPECIAL_TYPE: charm
---
apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: busybox:latest
      imagePullPolicy: IfNotPresent
      command: [ "/bin/sh", "-c", "ls /etc/config/" ]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
  restartPolicy: Never

root@master1:~/yaml/chapter4# kubectl apply -f 6.yaml
configmap/special-config created
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS      RESTARTS   AGE
dapi-test-pod   0/1     Completed   0          7s

root@master1:~/yaml/chapter4# kubectl logs dapi-test-pod
SPECIAL_LEVEL
SPECIAL_TYPE

将ConfigMap挂载到指定路径，挂载目录（config）可以事先不存在，示例中仅将ConfigMap中的SPECIAL_LEVEL挂载到/etc/config目录下，文件名称为keys，内容为very

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  SPECIAL_LEVEL: very
  SPECIAL_TYPE: charm
---
apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: busybox:latest
      imagePullPolicy: IfNotPresent
      command: ["/bin/sh"]
      args: ["-c", "ls /etc/config/;cat /etc/config/keys"]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
        items:
        - key: SPECIAL_LEVEL			#ConfigMap中键名
          path: keys							#挂在后的文件名称
  restartPolicy: Never

root@master1:~/yaml/chapter4# kubectl apply -f 7.yaml
configmap/special-config created
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS      RESTARTS   AGE
dapi-test-pod   0/1     Completed   0          6s

root@master1:~/yaml/chapter4# kubectl logs dapi-test-pod
keys
very

权限和可选项，权限可以设定挂载ConfigMap的文件系统权限

......
      configMap:
        name: special-config
        items:
        - key: SPECIAL_LEVEL
          path: key1
          mode: 0644
        - key: SPECIAL_TYPE
          path: key2
          mode: 0600
......

root@master1:~/yaml/chapter4# kubectl apply -f 8.yaml
configmap/special-config created
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS      RESTARTS   AGE
dapi-test-pod   0/1     Completed   0          4s

root@master1:~/yaml/chapter4# kubectl logs dapi-test-pod
total 8
-rw-r--r--    1 root     root             4 Sep 14 05:26 key1
-rw-------    1 root     root             5 Sep 14 05:26 key2

而可选项是指，当optional的值为true时，表示该ConfigMap是可选的，没有也不会影响pod创建

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: busybox:latest
      imagePullPolicy: IfNotPresent
      command: ["/bin/sh"]
      args: ["-c", "ls -lL /etc/config/"]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: special-config
        optional: true
        items:
        - key: SPECIAL_LEVEL
          path: key1
          mode: 0644
        - key: SPECIAL_TYPE
          path: key2
          mode: 0600
  restartPolicy: Never

root@master1:~/yaml/chapter4# kubectl apply -f 8.yaml
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS      RESTARTS   AGE
dapi-test-pod   0/1     Completed   0          4s

但当optional的值为false时，表示该ConfigMap是必选的，没有将无法正常创建pod

root@master1:~/yaml/chapter4# kubectl apply -f 8.yaml
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS              RESTARTS   AGE
dapi-test-pod   0/1     ContainerCreating   0          3s

......
Events:
  Type     Reason       Age                 From               Message
  ----     ------       ----                ----               -------
  Normal   Scheduled    2m14s               default-scheduler  Successfully assigned default/dapi-test-pod to node2
  Warning  FailedMount  12s                 kubelet            Unable to attach or mount volumes: unmounted volumes=[config-volume], unattached volumes=[kube-api-access-mkr9s config-volume]: timed out waiting for the condition
  Warning  FailedMount  7s (x9 over 2m15s)  kubelet            MountVolume.SetUp failed for volume "config-volume" : configmap "special-config" not found

默认挂载ConfigMap存储卷时，ConfigMap会覆盖pod中原有挂载点下的内容（default.conf），通过subpath路径的方式可以保留原有内容，但subpath也有约束条件：使用 ConfigMap 作为 subPath 卷的容器将不会收到 ConfigMap 更新

root@master1:~/yaml/chapter4# kubectl create cm conf.d --from-file conf.d/
configmap/conf.d created

root@master1:~/yaml/chapter4# kubectl get cm
NAME               DATA   AGE
conf.d             2      7s

root@master1:~/yaml/chapter4# ls conf.d/
fastcgi.conf  nginx.conf

apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: nginx:alpine
      imagePullPolicy: IfNotPresent
      command: ["/bin/sh"]
      args: ["-c", "ls /etc/nginx/conf.d/"]
      volumeMounts:
      - name: config-volume
        mountPath: /etc/nginx/conf.d/fastcgi.conf
        subPath: fastcgi.conf
      - name: config-volume
        mountPath: /etc/nginx/conf.d/nginx.conf
        subPath: nginx.conf
  volumes:
    - name: config-volume
      configMap:
        name: conf.d

root@master1:~/yaml/chapter4# kubectl apply -f 9.yaml
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl logs dapi-test-pod
default.conf
fastcgi.conf
nginx.conf

ConfigMap自动更新，这也是ConfigMap最核心的功能了，之所以前面说到ConfigMap挂载为存储卷的目的就是为了他的动态更新。示例中将ConfigMap存储卷挂载到/etc/nginx/conf.d/目录下后，可以看到SPECIAL_LEVEL和SPECIAL_TYPE是链接文件，指向另一个data的链接文件，而data指向实际的目录2023_09_15_05_15_20.3762322962，其下才存放着SPECIAL_LEVEL和SPECIAL_TYPE文件

apiVersion: v1
kind: ConfigMap
metadata:
  name: special-config
  namespace: default
data:
  SPECIAL_LEVEL: very
  SPECIAL_TYPE: charm
---
apiVersion: v1
kind: Pod
metadata:
  name: dapi-test-pod
spec:
  containers:
    - name: test-container
      image: nginx:alpine
      imagePullPolicy: IfNotPresent
      volumeMounts:
      - name: config-volume
        mountPath: /etc/nginx/conf.d
  volumes:
    - name: config-volume
      configMap:
        name: special-config
        optional: false
  restartPolicy: Never

root@master1:~/yaml/chapter4# kubectl apply -f 10.yaml
configmap/special-config created
pod/dapi-test-pod created

root@master1:~/yaml/chapter4# kubectl get pod
NAME            READY   STATUS    RESTARTS   AGE
dapi-test-pod   1/1     Running   0          5s

root@master1:~/yaml/chapter4# kubectl exec -it  dapi-test-pod -- /bin/sh
/ # cd /etc/nginx/conf.d/
/etc/nginx/conf.d # ls -lA
total 0
drwxr-xr-x    2 root     root            47 Sep 15 05:15 ..2023_09_15_05_15_20.3762322962
lrwxrwxrwx    1 root     root            32 Sep 15 05:15 ..data -> ..2023_09_15_05_15_20.3762322962
lrwxrwxrwx    1 root     root            20 Sep 15 05:15 SPECIAL_LEVEL -> ..data/SPECIAL_LEVEL
lrwxrwxrwx    1 root     root            19 Sep 15 05:15 SPECIAL_TYPE -> ..data/SPECIAL_TYPE

/etc/nginx/conf.d # ls -la ..2023_09_15_05_15_20.3762322962/
total 8
drwxr-xr-x    2 root     root            47 Sep 15 05:26 .
drwxrwxrwx    3 root     root           101 Sep 15 05:26 ..
-rw-r--r--    1 root     root             8 Sep 15 05:26 SPECIAL_LEVEL
-rw-r--r--    1 root     root             9 Sep 15 05:26 SPECIAL_TYPE

采用2次连接的好处是，当修改了ConfigMap的内容后，data会自动更新链接指向，这样指向data的SPECIAL_LEVEL和SPECIAL_TYPE文件内容也会跟着进行更新。

需要说明的是，配置文件动态更新了，但服务会不会自动加载配置文件，还得看具体的应用，一般云原生的应用在配置文件发更变化后都会自动触发更新，但示例中nginx还需要手动执行nginx -s reload手动进行更新

root@master1:~/yaml/chapter4# kubectl get cm
NAME               DATA   AGE
special-config     2      9m15s

root@master1:~/yaml/chapter4# kubectl edit cm special-config
configmap/special-config edited

K8S你学废了么4——应用配置存储卷_Pod