StatefulSet 通过 Service + Endpoints 暴露端口供外部访问

什么是StatefulSet

RC、Deployment、DaemonSet都是面向无状态的服务，它们所管理的Pod的IP、名字，启停顺序等都是随机的，而StatefulSet是什么？顾名思义，有状态的集合，管理所有有状态的服务，比如MySQL、MongoDB集群等。
StatefulSet本质上是Deployment的一种变体，在v1.9版本中已成为GA版本，它为了解决有状态服务的问题，它所管理的Pod拥有固定的Pod名称，启停顺序，在StatefulSet中，Pod名字称为网络标识(hostname)，还必须要用到共享存储。
在Deployment中，与之对应的服务是service，而在StatefulSet中与之对应的headless service，headless service，即无头服务，与service的区别就是它没有Cluster IP，解析它的名称时将返回该Headless Service对应的全部Pod的Endpoint列表。
除此之外，StatefulSet在Headless Service的基础上又为StatefulSet控制的每个Pod副本创建了一个DNS域名，这个域名的格式为：
$(podname).(headless server name)
FQDN：$(podname).(headless server name).namespace.svc.cluster.local
例如访问mysql集群Master库：jdbc:mysql://mysql-0.mysql.mysql.svc.cluster.local:3306/test

创建 StatefulSet

K8s 1.23版本，创建 StatefulSet，以 mysql集群 为例,演示怎么通过外部端口访问k8s mysql集群内的数据库：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mysql
  namespace: mysql
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mysql
  serviceName: mysql
  template:
    metadata:
      labels:
        app: mysql
    spec:
      initContainers:
      - name: init-mysql
        image: 10.100.57.178:5000/mysql:8.0.19
        imagePullPolicy: IfNotPresent
        command:
        - bash
        - "-c"
        - |
          set -ex
          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1
          myindex=${BASH_REMATCH[1]}
          echo [mysqld] > /mnt/conf.d/server-id.cnf
          echo server-id=$((100 + ${myindex})) >> /mnt/conf.d/server-id.cnf
          if [[ ${myindex} -eq 0 ]]; then
            cp /mnt/config-map/master.cnf /mnt/conf.d/
          else
            cp /mnt/config-map/slave.cnf /mnt/conf.d/
          fi
        volumeMounts:
        - name: conf
          mountPath: /mnt/conf.d
        - name: config-map
          mountPath: /mnt/config-map
      - name: clone-mysql
        image: 10.100.57.178:5000/mzmuer/xtrabackup:1.0
        imagePullPolicy: IfNotPresent
        command:
        - bash
        - "-c"
        - |
          set -ex
          [[ -d /var/lib/mysql/mysql ]] && exit 0
          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1
          myindex=${BASH_REMATCH[1]}
          [[ ${myindex} -eq 0 ]] && exit 0
          ncat --recv-only mysql-$((${myindex}-1)).mysql 3307 | xbstream -x -C /var/lib/mysql
          xtrabackup --prepare --target-dir=/var/lib/mysql
          # --redo-only 加了会导致为提交的事务不回滚
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: conf
          mountPath: /etc/mysql/conf.d
      containers:
      - name: mysql
        image: 10.100.57.178:5000/mysql:8.0.19
        imagePullPolicy: IfNotPresent
        args: ["--default-authentication-plugin=mysql_native_password"]
        env:
        - name: MYSQL_ALLOW_EMPTY_PASSWORD
          value: "1"
        ports:
        - name: mysql
          containerPort: 3306
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: conf
          mountPath: /etc/mysql/conf.d
        resources:
          requests:
            cpu: 250m
            memory: 256Mi
          limits:
            cpu: 500m
            memory: 512Mi
        livenessProbe:
          exec:
            command: ["mysqladmin", "ping"]
          initialDelaySeconds: 30
          periodSeconds: 10
          timeoutSeconds: 5
        readinessProbe:
          exec:
            command: ["mysql", "-h", "127.0.0.1", "-e", "SELECT 1"]
          initialDelaySeconds: 5
          periodSeconds: 2
          timeoutSeconds: 1
      - name: xtrabackup
        image: 10.100.57.178:5000/mzmuer/xtrabackup:1.0
        imagePullPolicy: IfNotPresent
        ports:
        - name: xtrabackup
          containerPort: 3307
        command:
        - bash
        - "-c"
        - |
          set -ex
          mkdir /test
          cd /var/lib/mysql
          if [[ -s xtrabackup_slave_info ]]; then
            mv xtrabackup_slave_info change_master_to.sql.in
            rm -f xtrabackup_binlog_info
          elif [[ -f xtrabackup_binlog_info ]]; then
            [[ `cat xtrabackup_binlog_info` =~ ^(.*?)[[:space:]]+(.*?)$ ]] || exit 1
            echo -e "CHANGE MASTER TO\nMASTER_LOG_FILE='${BASH_REMATCH[1]}',\nMASTER_LOG_POS=${BASH_REMATCH[2]}" > change_master_to.sql.in
            rm -f xtrabackup_binlog_info
          fi
          if [[ -f change_master_to.sql.in ]]; then
            echo "Waiting for mysqld to be ready (accepting connections)"
            until mysql -h 127.0.0.1 -e "SELECT 1"; do sleep 1; done
            echo "Initializing replication from clone position"
            mv change_master_to.sql.in change_master_to.sql.orig
            mysql -h 127.0.0.1 <<EOF
            $(<change_master_to.sql.orig),
            MASTER_HOST='mysql-0.mysql',
            MASTER_USER='repl',
            MASTER_PASSWORD='repl123.',
            MASTER_CONNECT_RETRY=10;
            START SLAVE;
          EOF
          echo "master slave config ok"
          fi
          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1
          myindex=${BASH_REMATCH[1]}
          if [[ ${myindex} -eq 0 ]]; then
            mysql -h 127.0.0.1 <<EOF
            use mysql;
            delete from user where user='repl' and host='%';flush privileges;
            CREATE USER 'repl'@'%' IDENTIFIED WITH mysql_native_password BY 'repl123.';
            GRANT REPLICATION SLAVE ON *.* TO 'repl'@'%';flush privileges;
            SELECT Host, User, plugin from mysql.user;
          EOF
          else
            Slave_healthy=`mysql -h 127.0.0.1 -e 'show slave status\G' 2> /dev/null |grep -E "Slave_IO_Running|Slave_SQL_Running"|awk '{print $2}'|grep -c Yes`
            if [[ ${Slave_healthy} -eq 2 ]]; then
              echo "master slave healthy ok"
            elif [[ ${Slave_healthy} -eq 1 ]]; then
              echo "master slave healthy no"
              mysql -h 127.0.0.1 <<EOF
              stop slave;
              reset slave;
              start slave;
          EOF
              echo "master slave healthy revert"
            elif [[ ${Slave_healthy} -eq 0 ]]; then
              echo "master slave healthy no"
              mysql -h 127.0.0.1 <<EOF
              reset slave;
              start slave;
          EOF
              echo "master slave healthy revert"
            fi
          fi
          exec ncat --listen --keep-open --send-only --max-conns=1 3307 -c \
            "xtrabackup --backup --slave-info --stream=xbstream --host=127.0.0.1 --user=root"
        volumeMounts:
        - name: data
          mountPath: /var/lib/mysql
          subPath: mysql
        - name: conf
          mountPath: /etc/mysql/conf.d
        resources:
          requests:
            cpu: 100m
            memory: 100Mi
          limits:
            cpu: 200m
            memory: 200Mi
      volumes:
      - name: conf
        emptyDir: {}
      - name: config-map
        configMap:
          name: mysql
  volumeClaimTemplates:
  - metadata:
      name: data
    spec:
      storageClassName: "managed-nfs-storage"
      accessModes: ["ReadWriteMany"]
      resources:
        requests:
          storage: 10Gi

创建第一个外部端口，以访问Master库

apiVersion: v1
kind: Service
metadata:
  name: mysql-0-external
  namespace: mysql
spec:
  type: NodePort
  ports:    
    - port: 3306
      protocol: TCP
      targetPort: 3306
      nodePort: 31306
---
apiVersion: v1
kind: Endpoints
metadata:
  name: mysql-0-external
  namespace: mysql
subsets:  
  - addresses:      
      - ip: "10.244.2.239"   # 根据 pod mysql-0 的 ip 填写
    ports:      
      - port: 3306

创建完成后，用户可以通过 jdbc:mysql://宿主机IP:31306 访问Master库

创建第2个外部端口，以访问Slave库

apiVersion: v1
kind: Service
metadata:
  name: mysql-1-external
  namespace: mysql
spec:
  type: NodePort
  ports:    
    - port: 3306
      protocol: TCP
      targetPort: 3306
      nodePort: 31307
---
apiVersion: v1
kind: Endpoints
metadata:
  name: mysql-1-external
  namespace: mysql
subsets:  
  - addresses:      
      - ip: "10.244.2.240"   # 根据 pod mysql-1 的 ip 填写
    ports:      
      - port: 3306

创建完成后，用户可以通过 jdbc:mysql://宿主机IP:31307 访问Slave库