20210903 (금) kubernetes application customizing

9:38 

 

  Application Customizing

    Container image customizing

    Environment variables 

    Configmap

    Secret

 

### 환경변수 이용한 customizing ###
vagrant@kube-control1:~/work/20210903$ cat myapp-pod-env.yaml
apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod-env
spec:
  containers:
  - name: myapp
    image: devops2341/go-myweb:latest
    env:
    - name: MESSAGE
      value: "Customized Message"
    ports:
    - containerPort: 8080
      protocol: TCP
vagrant@kube-control1:~/work/20210903$ curl http://192.168.9.77:8080
Customized Message
myapp-pod-env

    Configmap

      간단한 텍스트, 설정값, 변수, 설정 파일 등을 Key Value 쌍으로 저장하고 사용할 수 있는 kubernetes object

### configmap 을 통한 Customizing ###       ConfigMap Object 생성(commandline)       kubectl create configmap CONFIGMAP_NAME [option]       --from-file=FILE         - Key/Value가 있는 파일 지정         - Key를 별도로 지정하지 않는 경우 파일이름이 Key가 됨       --from-literal=KEY=VALUE           - Key=Value 를 문자열로 지정 vagrant@kube-control1:~$ kubectl create configmap my-config1 --from-literal=key1=value1 configmap/my-config1 created   vagrant@kube-control1:~$ echo value2 > key2 vagrant@kube-control1:~$ kubectl create configmap my-config2 --from-file key2 configmap/my-config2 created  → 파일만 지정하였으므로 파일명(key2)이 configmap 이름으로 지정됨. vagrant@kube-control1:~$ echo value3 > key3.txt vagrant@kube-control1:~$ kubectl create configmap my-config3 --from-file=key3=key3.txt configmap/my-config3 created → key3라는 configmap 이름을 지정하고, 파일을 지정 vagrant@kube-control1:~$ kubectl describe configmap my-config3 Name:         my-config3 Namespace:    default Labels:       <none> Annotations:  <none> Data ==== key3: ---- value3 Events:  <none> ### manifest 파일을 통한 configmap 생성 ### vagrant@kube-control1:~$ cat my-config4-configmap.yaml apiVersion: v1 kind: ConfigMap metadata:   name: my-config4 data:   key4: value4   key5: value5 vagrant@kube-control1:~$ kubectl create -f my-config4-configmap.yaml vagrant@kube-control1:~$ kubectl describe configmap my-config4 Name:         my-config4 Namespace:    default Labels:       <none> Annotations:  <none> Data ==== key4: ---- value4 key5: ---- value5 Events:  <none> ### configmap 파일을 통한 customizing ### # image를 편집하지 않고, configmap을 통해 변경된 메세지를 출력 vagrant@kube-control1:~$ cat configmap/message  → 디렉터리 안에 message를 포함한 파일 생성 Hello world by ConfigMap vagrant@kube-control1:~$ cat myapp-pod-cm-ref.yaml → manifest 작성 apiVersion: v1 kind: Pod metadata:   name: myapp-pod-cm-ref spec:   containers:   - name: myapp     image: devops2341/go-myweb:latest     env:     - name: MESSAGE       valueFrom:         configMapKeyRef:           name: myapp-message           key: message     args:     - $(MESSAGE)     ports:     - containerPort: 8080       protocol: TCP vagrant@kube-control1:~$ kubectl create -f myapp-pod-cm-ref.yaml  → pod 생성 vagrant@kube-control1:~$ curl http://192.168.233.210:8080 Hello world by ConfigMap  → configmap/message 파일의 message로 출력됨 myapp-pod-cm-ref ### nginx gzip on 을 통한 트래픽 효율화 configmap ### vagrant@kube-control1:~/work/20210903$ cat conf/nginx-gzip.conf server {         listen          80;         server_name     myapp.example.com;         gzip on;         gzip_types      text/plain application/xml;         location / {                 root    /usr/share/nginx/html;                 index   index.html;         } } vagrant@kube-control1:~/work/20210903$ cat myapp-pod-compress.yaml apiVersion: v1 kind: Pod metadata:   name: nginx-pod-compress spec:   containers:   - name: nginx     image: nginx:latest     volumeMounts:     - name: nginx-compress-config       mountPath: /etc/nginx/conf.d     ports:     - containerPort: 80       protocol: TCP   volumes:   - name: nginx-compress-config     configMap:       name: nginx-gzip-config vagrant@kube-control1:~/work/20210903$ curl -H "Accept-Encoding:gzip" -I http://192.168.233.212:80 HTTP/1.1 200 OK Server: nginx/1.21.1 Date: Fri, 03 Sep 2021 02:55:00 GMT Content-Type: text/html Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT Connection: keep-alive ETag: W/"60e46fc5-264" Content-Encoding: gzip vagrant@kube-control1:~/work/20210903$ curl -H "Accept-Encoding:gzip" --output result http://192.168.233.212:80   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current                                  Dload  Upload   Total   Spent    Left  Speed 100   384    0   384    0     0   187k      0 --:--:-- --:--:-- --:--:--  187k vagrant@kube-control1:~/work/20210903$ cat result uR=o�0          ��+�̗S���U5P��                         �y6m                               �D��r5�����交Z���=R����f����Cc��EةW|     ����r$(                                                                                i��ׅ�k¸\f}`@ρ�~���=���;�����s0�(�>����=�r�c�a��G����$&5��ǰ��)_e߯������W�                              �E��{�5j��|�a� �T��3L8�FMZ������l�mK9�5�|�C�0u����Ӱ��*Z.�r��P��                                                       =W/�m���4�g N�'���)���j�ib)0�LZ�+ha�Fu�\��L;kO[ز                                                                                                            ��9��b�u����IZ��R��  ���@��?���3��+�G�ُ��N��UH�*��*��I���[��g�                                           �*d

 

  Secret

    generic (=Opaque) : 일반 secret Key Value 상의 임의의 데이터 (일반)

    → 내용 저장시 base64 로 인코딩하여 저장하게 됨

    docker-registry : Docker registry 인증 정보

    tls : TLS key와 인증서

  

  시크릿 생성(commandline)

  kubectl create secret TYPE NAME [option]

   일반 secret

   kubectl create secret generic NAME [option]

 

   Docker registry 인증 시크릿

  kubectl create secret docker-registry NAME \

   --docker-username=USER --docker-password=PASS \

  --docker-email=EMAIL [--docker-server=SERVER]

 

   TLS 키/인증서 시크릿 생성

    kubectl create secret tls --cert=CERT_FILE --key=KEY_FILE

 

 

manifests 로 secret 생성 : base64인코딩 된 값으로 입력해주어야함.

 

 

### manifests로 secret 생성 : base64 인코딩 된 값으로 입력해주어야함 ### # CLI로 생성시에는 일반값을 넣으면 자동으로 base64 인코딩 됨 # vagrant@kube-control1:~$ cat user-pass-secret.yaml apiVersion: v1 kind: Secret metadata:   name: user-pass-secret type: Opaque data:   username: dXNlcjAxCg==   password: dXNlcjAxcGFzcwo= vagrant@kube-control1:~$ kubectl create -f user-pass-secret.yaml vagrant@kube-control1:~$ kubectl describe secret user-pass-secret Name:         user-pass-secret Namespace:    default Labels:       <none> Annotations:  <none> Type:  Opaque Data ==== password:  11 bytes username:  7 bytes vagrant@kube-control1:~$ kubectl get secrets user-pass-secret -o yaml apiVersion: v1 data:   password: dXNlcjAxcGFzcwo=   username: dXNlcjAxCg== kind: Secret metadata:   creationTimestamp: "2021-09-03T03:32:56Z"   managedFields:   - apiVersion: v1     fieldsType: FieldsV1     fieldsV1:       f:data:         .: {}         f:password: {}         f:username: {}       f:type: {}     manager: kubectl-create     operation: Update     time: "2021-09-03T03:32:56Z"   name: user-pass-secret   namespace: default   resourceVersion: "813060"   selfLink: /api/v1/namespaces/default/secrets/user-pass-secret   uid: e830b4c8-44e3-420a-b0e7-d78d800858f2 type: Opaque ### tls 인증서 시크릿 생성### vagrant@kube-control1:~/work/20210903$ mkdir nginx-tls vagrant@kube-control1:~/work/20210903$ openssl genrsa -out nginx-tls/nginx-tls.key 2048 Generating RSA private key, 2048 bit long modulus (2 primes) ........+++++ .........................................................................................................................................+++++ e is 65537 (0x010001) vagrant@kube-control1:~/work/20210903$ openssl req -new -x509 -key nginx-tls/nginx-tls.key \ -out nginx-tls/nginx-tls.crt \ -days 365 -subj /CN=myapp.example.com req: Use -help for summary. vagrant@kube-control1:~/work/20210903$ openssl req -new -x509 -key nginx-tls/nginx-tls.key \ > -out nginx-tls/nginx-tls.crt \ > -days 365 -subj /CN=myapp.example.com vagrant@kube-control1:~/work/20210903$ ls nginx- ls: cannot access 'nginx-': No such file or directory vagrant@kube-control1:~/work/20210903$ ls nginx-tls/ nginx-tls.crt  nginx-tls.key vagrant@kube-control1:~/work/20210903$ kubectl create secret tls nginx-tls-secret \ > --cert=nginx-tls/nginx-tls.crt \ > --key=nginx-tls/nginx-tls.key secret/nginx-tls-secret created vagrant@kube-control1:~/work/20210903$ kubectl describe secret nginx-tls-secret Name:         nginx-tls-secret Namespace:    default Labels:       <none> Annotations:  <none> Type:  kubernetes.io/tls Data ==== tls.crt:  1135 bytes tls.key:  1675 bytes vagrant@kube-control1:~$ cat conf/nignx-tls.conf server {         listen                  80;         listen                  443;         server_name             myapp.example.com;         ssl_certificate         /etc/nginx/ssl/tls.crt;         ssl_certificate_key     /etc/nginx/ssl/tls.key;         ssl_protocols           TLSv1 TLSv1.1 TLSv1.2;         ssl_ciphers             HIGH:!aNULL:!MD5;         location / {                 root    /usr/share/nginx/html;                 index   index.html;         } } vagrant@kube-control1:~$ kubectl create configmap nginx-tls-config --from-file=conf/nginx-tls.conf configmap/nginx-tls-config created vagrant@kube-control1:~$ kubectl describe configmap nginx-tls-config Name:         nginx-tls-config Namespace:    default Labels:       <none> Annotations:  <none> Data ==== nginx-tls.conf: ---- server {   listen                   80;   listen                   443 ssl;   server_name            myapp.example.com;   ssl_certificate        /etc/nginx/ssl/tls.crt;   ssl_certificate_key  /etc/nginx/ssl/tls.key;   ssl_protocols          TLSv1 TLSv1.1 TLSv1.2;   ssl_ciphers            HIGH:!aNULL:!MD5;   location / {     root   /usr/share/nginx/html;     index  index.html;   } } Events:  <none> vagrant@kube-control1:~/work/20210903$ cat nginx-pod-https.yaml apiVersion: v1 kind: Pod metadata:   name: nginx-pod-https spec:   containers:   - name: nginx-tls     image: nginx:latest     volumeMounts:     - name: nginx-tls-config       mountPath: /etc/nginx/conf.d     - name: https-cert       mountPath: /etc/nginx/ssl       readOnly: true     ports:     - containerPort: 80       protocol: TCP     - containerPort: 443       protocol: TCP   volumes:   - name: nginx-tls-config     configMap:       name: nginx-tls-config   - name: https-cert     secret:       secretName: nginx-tls-secret vagrant@kube-control1:~/work/20210903$ kubectl create -f nginx-pod-https.yaml pod/nginx-pod-https created vagrant@kube-control1:~/work/20210903$ kubectl describe configmaps nginx-tls-config Name:         nginx-tls-config Namespace:    default Labels:       <none> Annotations:  <none> Data ==== nginx-tls.conf: ---- server {   listen                   80;   listen                   443 ssl;   server_name            myapp.example.com;   ssl_certificate        /etc/nginx/ssl/tls.crt;   ssl_certificate_key  /etc/nginx/ssl/tls.key;   ssl_protocols          TLSv1 TLSv1.1 TLSv1.2;   ssl_ciphers            HIGH:!aNULL:!MD5;   location / {     root   /usr/share/nginx/html;     index  index.html;   } } Events:  <none> vagrant@kube-control1:~/work/20210903$ kubectl get pods -o wide NAME                 READY   STATUS    RESTARTS   AGE     IP                NODE         NOMINATED NODE   READINESS GATES nginx-pod-https      1/1     Running   0          8s      192.168.233.213   kube-node2   <none>           <none> vagrant@kube-control1:~/work/20210903$ curl https://192.168.233.213 -k -v *   Trying 192.168.233.213:443... * TCP_NODELAY set * Connected to 192.168.233.213 (192.168.233.213) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: *   CAfile: /etc/ssl/certs/ca-certificates.crt   CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 * ALPN, server accepted to use http/1.1 * Server certificate: *  subject: CN=myapp.example.com *  start date: Sep  3 03:54:30 2021 GMT *  expire date: Sep  3 03:54:30 2022 GMT *  issuer: CN=myapp.example.com *  SSL certificate verify result: self signed certificate (18), continuing anyway. > GET / HTTP/1.1 > Host: 192.168.233.213 > User-Agent: curl/7.68.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Server: nginx/1.21.1 < Date: Fri, 03 Sep 2021 05:35:05 GMT < Content-Type: text/html < Content-Length: 612 < Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT < Connection: keep-alive < ETag: "60e46fc5-264" < Accept-Ranges: bytes < <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style>     body {         width: 35em;         margin: 0 auto;         font-family: Tahoma, Verdana, Arial, sans-serif;     } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> * Connection #0 to host 192.168.233.213 left intact vagrant@kube-control1:~/work/20210903$ kubectl get pods NAME                 READY   STATUS    RESTARTS   AGE myapp-pod-arg        1/1     Running   0          4h46m myapp-pod-cm-ref     1/1     Running   0          3h32m myapp-pod-env        1/1     Running   0          4h36m nginx-pod            1/1     Running   0          3h22m nginx-pod-compress   1/1     Running   0          163m nginx-pod-https      1/1     Running   0          3m36s vagrant@kube-control1:~/work/20210903$ kubectl describe nginx-pod-https error: the server doesn't have a resource type "nginx-pod-https" vagrant@kube-control1:~/work/20210903$ kubectl describe pod nginx-pod-https Name:         nginx-pod-https Namespace:    default Priority:     0 Node:         kube-node2/192.168.200.22 Start Time:   Fri, 03 Sep 2021 05:34:40 +0000 Labels:       <none> Annotations:  cni.projectcalico.org/containerID: 5579ce650c5a32bbc56a210992bb502e7313ab2265baa38fd11a26ccd54582b9               cni.projectcalico.org/podIP: 192.168.233.213/32               cni.projectcalico.org/podIPs: 192.168.233.213/32 Status:       Running IP:           192.168.233.213 IPs:   IP:  192.168.233.213 Containers:   nginx-tls:     Container ID:   docker://cb6715e538da098807f12fc2183d4a7c2d377d6bb0f6a6968739abb1ec11adf6     Image:          nginx:latest     Image ID:       docker-pullable://nginx@sha256:4d4d96ac750af48c6a551d757c1cbfc071692309b491b70b2b8976e102dd3fef     Ports:          80/TCP, 443/TCP     Host Ports:     0/TCP, 0/TCP     State:          Running       Started:      Fri, 03 Sep 2021 05:34:45 +0000     Ready:          True     Restart Count:  0     Environment:    <none>     Mounts:       /etc/nginx/conf.d from nginx-tls-config (rw)       /etc/nginx/ssl from https-cert (ro)       /var/run/secrets/kubernetes.io/serviceaccount from default-token-w26v7 (ro) Conditions:   Type              Status   Initialized       True   Ready             True   ContainersReady   True   PodScheduled      True Volumes:   nginx-tls-config:     Type:      ConfigMap (a volume populated by a ConfigMap)     Name:      nginx-tls-config     Optional:  false   https-cert:     Type:        Secret (a volume populated by a Secret)     SecretName:  nginx-tls-secret     Optional:    false   default-token-w26v7:     Type:        Secret (a volume populated by a Secret)     SecretName:  default-token-w26v7     Optional:    false QoS Class:       BestEffort Node-Selectors:  <none> Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                  node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events:   Type    Reason     Age    From               Message   ----    ------     ----   ----               -------   Normal  Scheduled  3m50s  default-scheduler  Successfully assigned default/nginx-pod-https to kube-node2   Normal  Pulling    3m49s  kubelet            Pulling image "nginx:latest"   Normal  Pulled     3m45s  kubelet            Successfully pulled image "nginx:latest" in 3.339594365s   Normal  Created    3m45s  kubelet            Created container nginx-tls   Normal  Started    3m45s  kubelet            Started container nginx-tls vagrant@kube-control1:~/work/20210903$ curl http://192.168.9.94 <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style>     body {         width: 35em;         margin: 0 auto;         font-family: Tahoma, Verdana, Arial, sans-serif;     } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> vagrant@kube-control1:~/work/20210903$ curl https://192.168.233.213 -k -v *   Trying 192.168.233.213:443... * TCP_NODELAY set * Connected to 192.168.233.213 (192.168.233.213) port 443 (#0) * ALPN, offering h2 * ALPN, offering http/1.1 * successfully set certificate verify locations: *   CAfile: /etc/ssl/certs/ca-certificates.crt   CApath: /etc/ssl/certs * TLSv1.3 (OUT), TLS handshake, Client hello (1): * TLSv1.3 (IN), TLS handshake, Server hello (2): * TLSv1.2 (IN), TLS handshake, Certificate (11): * TLSv1.2 (IN), TLS handshake, Server key exchange (12): * TLSv1.2 (IN), TLS handshake, Server finished (14): * TLSv1.2 (OUT), TLS handshake, Client key exchange (16): * TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1): * TLSv1.2 (OUT), TLS handshake, Finished (20): * TLSv1.2 (IN), TLS handshake, Finished (20): * SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384 * ALPN, server accepted to use http/1.1 * Server certificate: *  subject: CN=myapp.example.com *  start date: Sep  3 03:54:30 2021 GMT *  expire date: Sep  3 03:54:30 2022 GMT *  issuer: CN=myapp.example.com *  SSL certificate verify result: self signed certificate (18), continuing anyway. > GET / HTTP/1.1 > Host: 192.168.233.213 > User-Agent: curl/7.68.0 > Accept: */* > * Mark bundle as not supporting multiuse < HTTP/1.1 200 OK < Server: nginx/1.21.1 < Date: Fri, 03 Sep 2021 05:35:05 GMT < Content-Type: text/html < Content-Length: 612 < Last-Modified: Tue, 06 Jul 2021 14:59:17 GMT < Connection: keep-alive < ETag: "60e46fc5-264" < Accept-Ranges: bytes < <!DOCTYPE html> <html> <head> <title>Welcome to nginx!</title> <style>     body {         width: 35em;         margin: 0 auto;         font-family: Tahoma, Verdana, Arial, sans-serif;     } </style> </head> <body> <h1>Welcome to nginx!</h1> <p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p> <p>For online documentation and support please refer to <a href="http://nginx.org/">nginx.org</a>.<br/> Commercial support is available at <a href="http://nginx.com/">nginx.com</a>.</p> <p><em>Thank you for using nginx.</em></p> </body> </html> * Connection #0 to host 192.168.233.213 left intact → https로 웹서비스