k8s-proxy原理-摩杜云开发者社区

kube-proxy支持三种代理模式: 用户空间，iptables和IPVS

在kubectl edit configmap kube-proxy -n kube-system修改代理模式，如果是ipvs的话，可以在scheduler中修改调度算法

[root@k8s-master1 ~]# cat kube-proxy-configmap.yaml
apiVersion: v1
data:
  config.conf: |-
    apiVersion: kubeproxy.config.k8s.io/v1alpha1
    bindAddress: 0.0.0.0
    clientConnection:
      acceptContentTypes: ""
      burst: 0
      contentType: ""
      kubeconfig: /var/lib/kube-proxy/kubeconfig.conf
      qps: 0
    clusterCIDR: 10.244.0.0/16
    configSyncPeriod: 0s
    conntrack:
      maxPerCore: null
      min: null
      tcpCloseWaitTimeout: null
      tcpEstablishedTimeout: null
    enableProfiling: false
    healthzBindAddress: ""
    hostnameOverride: ""
    iptables:
      masqueradeAll: false
      masqueradeBit: null
      minSyncPeriod: 0s
      syncPeriod: 0s
    ipvs:
      excludeCIDRs: null
      minSyncPeriod: 0s
      scheduler: ""
      strictARP: false
      syncPeriod: 0s
    kind: KubeProxyConfiguration
    metricsBindAddress: ""
    mode: "ipvs"   # 此处修改为ipvs
    nodePortAddresses: null
    oomScoreAdj: null
    portRange: ""
    udpIdleTimeout: 0s
    winkernel:
      enableDSR: false
      networkName: ""
      sourceVip: ""
  kubeconfig.conf: |-
    apiVersion: v1
    kind: Config
    clusters:
    - cluster:
        certificate-authority: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
        server: https://192.168.255.146:6443
      name: default
    contexts:
    - context:
        cluster: default
        namespace: default
        user: default
      name: default
    current-context: default
    users:
    - name: default
      user:
        tokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
kind: ConfigMap
metadata:
  creationTimestamp: "2021-09-04T17:24:48Z"
  labels:
    app: kube-proxy
  name: kube-proxy
  namespace: kube-system
  resourceVersion: "12116"
  selfLink: /api/v1/namespaces/kube-system/configmaps/kube-proxy
  uid: 46a52a10-4f8b-46e9-b7c4-059e7f4efc0b


修改完ipvs模式后，还需要重建kube-proxy的pod
[root@k8s-master1 ~]# kubectl get pods -n kube-system
NAME                                  READY   STATUS    RESTARTS   AGE
coredns-9d85f5447-sxvk6               1/1     Running   0          119d
coredns-9d85f5447-wk7z2               1/1     Running   0          119d
etcd-k8s-master1                      1/1     Running   0          119d
kube-apiserver-k8s-master1            1/1     Running   0          119d
kube-controller-manager-k8s-master1   1/1     Running   0          119d
kube-flannel-ds-amd64-52bxh           1/1     Running   0          119d
kube-flannel-ds-amd64-v4zrw           1/1     Running   0          171m
kube-flannel-ds-amd64-zd4m5           1/1     Running   0          119d
kube-proxy-ph9hr                      1/1     Running   0          106m
kube-proxy-vnt8n                      1/1     Running   0          106m
kube-proxy-w5jsf                      1/1     Running   0          106m
kube-scheduler-k8s-master1            1/1     Running   0          119d

[root@k8s-master1 ~]# kubectl delete pod/kube-proxy-ph9hr  pod/kube-proxy-vnt8n pod/kube-proxy-w5jsf -n kube-system

在iptables模式下，service ip仅存在于iptables规则中，没有响应设备，因此是ping不通的。但是ipvs模式下，service ip被绑定在虚拟设备kube-ipvs0中，因此是有响应设备，可以ping通

修改完工作模式为ipvs后，会在每个节点上生成kube-ipvs0虚拟网卡，并将svc的ip地址绑定到该网卡上，这样，当有主机访问到svc地址后，会通过ipvs规则，转入到后端对应的pods中去，关于nodePort会在每个节点上新增ipvs规则，指向对应的pods_ip，所以访问任意节点的ip:NodePort都会转入对应pod

[root@k8s-master1 ~]# ip a | grep ens33  # 本机网卡
2: ens33: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    inet 192.168.255.146/24 brd 192.168.255.255 scope global noprefixroute dynamic ens33
[root@k8s-master1 ~]# kubectl get svc --all-namespaces
NAMESPACE     NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                  AGE
default       kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP                  119d
default       myweb        NodePort    10.96.231.201   <none>        80:30857/TCP             114m
kube-system   kube-dns     ClusterIP   10.96.0.10      <none>        53/UDP,53/TCP,9153/TCP   119d
[root@k8s-master1 ~]# ip a | grep kube-ipvs
11: kube-ipvs0: <BROADCAST,NOARP> mtu 1500 qdisc noop state DOWN group default
    inet 10.96.231.201/32 brd 10.96.231.201 scope global kube-ipvs0
    inet 10.96.0.1/32 brd 10.96.0.1 scope global kube-ipvs0
    inet 10.96.0.10/32 brd 10.96.0.10 scope global kube-ipvs0
[root@k8s-master1 ~]# ipvsadm -ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.255.146:30857 rr
  -> 10.244.2.6:80                Masq    1      0          0         
TCP  10.96.0.1:443 rr
  -> 192.168.255.146:6443         Masq    1      0          0         
TCP  10.96.0.10:53 rr
  -> 10.244.0.2:53                Masq    1      0          0         
  -> 10.244.0.3:53                Masq    1      0          0         
TCP  10.96.0.10:9153 rr
  -> 10.244.0.2:9153              Masq    1      0          0         
  -> 10.244.0.3:9153              Masq    1      0          0         
TCP  10.96.231.201:80 rr
  -> 10.244.2.6:80                Masq    1      0          0         
TCP  10.244.0.0:30857 rr
  -> 10.244.2.6:80                Masq    1      0          0         
TCP  10.244.0.1:30857 rr
  -> 10.244.2.6:80                Masq    1      0          0         
TCP  127.0.0.1:30857 rr
  -> 10.244.2.6:80                Masq    1      0          0         
TCP  172.17.0.1:30857 rr
  -> 10.244.2.6:80                Masq    1      0          0         
UDP  10.96.0.10:53 rr
  -> 10.244.0.2:53                Masq    1      0          0         
  -> 10.244.0.3:53                Masq    1      0          0

iptables: 灵活，功能强大，但是匹配规则呈线性延时，性能较低，只支持轮询，在iptables规则使用权重机制实现 ,可以使用iptables-save | grep service_name名称

Ipvs: 工作在内核态，性能优越，调度算法：rr,wrr,lc,wlc,ip_hash。可以使用ipvsadm -ln查看是否生成规则，一般是service的cluster_ip