Online Material - Part 2

Welcome to the online material for Part 2 of the Introduction to Research Computing on Kubernetes workshop.

In this part we will cover:

Deployments
Services
Ingresses
Building Image in RCD Gitlab
Deploying a science gateway

Deployments

Kubernetes deployments are similar to Job in that they provide a way to automatically manage creation and scheduling of pods. Deployments, in particular, are geared towards keeping some number of replica pods running at all times. Deployments are useful for things like web servers and databases.

Let's create a deployment to spin up a bunch of nginx pods. Put the following contents into a deployment1.yml file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment-USERNAME
  labels:
    app: nginx-USERNAME
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx-USERNAME
  template:
    metadata:
      labels:
        app: nginx-USERNAME
    spec:
      containers:
        - name: nginx
          image: nginx:1.14.2
          resources:
            limits:
              memory: 200Mi
              cpu: 1
            requests:
              memory: 50Mi
              cpu: 50m

There are a few interesting things to point out about this Deployment specification:

We can specify the replicas (right now set to 2).
We specify labels on the pod template and a matchLabels selector on the Deployment. This is required by the Deployment and will also be used later when we create a Service.

We could now create this deployment using kubectl create -f deployment1.yml but this time let us use kubectl apply:

./kubectl apply -f deployment1.yml

The apply command within kubectl will create the Kubernetes objects if they do not exists. If they do exists, kubectl will attempt to modify the object to have it match any changes on disk. We will be able to make changes to our deployment and re-run kubectl apply without having to delete the object first.

We should be able to see our deployment with:

./kubectl get deployments

And see the pods that this deployment creates using:

./kubectl get pods

We should see 2 pods created.

Scaling

Let's try scaling the deployment. Edit the file to have replicas: 3 and run:

./kubectl apply -f deployment1.yml

Now if you run ./kubectl get pods you should see 3 pods.

Rollouts

Deployments also allow graceful rollouts. Edit the deployment1.yml file again and change the image from nginx:1.14.2 to nginx:1.25.3. Then run:

./kubectl apply -f deployment1.yml && ./kubectl rollout status -f deployment1.yml

This will apply the changes to the deployment, then run the rollout status command which will watch the changes to the deployment.

Self Healing

Another useful feature of deployments are their self-healing nature. Let's delete a pod and see how it self-heals. First, list the pods again:

./kubectl get pods

Pick a random pod from your deployment and delete it:

./kubectl delete pod nginx-deployment-<suffix>

If we then list the pods you should see that a third pod will be automatically created again.

Port Forwarding

Each of the three nginx pods is serving a basic web service on port 80. We can forward this port back to our original machine using kubectl port-forward. You will need the name of one of the pods in the deployment (reminder: you can run kubectl get pods to see the list of pods). Run:

./kubectl port-forward pod/<pod-name> 8080:80

This should forward port 8080 on your local computer to port 80 (the standard web port) on the pod. Once this command is running, try opening http://localhost:8080 in your browser.

You should see the following screen:

Welcome to nginx

You can press Ctrl-C to stop the port-forward process.

Let us update the

Service

Manually picking a pod to connect to is extra work and brittle. It defeats some of the advantages of having an automatically healing deployment. Services provide a load balancer. It provides a single endpoint and if we connect to it, the service will automatically route the traffic to one of the healthy pods that match a selector.

Create a file service1.yml with the following contents:

apiVersion: v1
kind: Service
metadata:
  name: nginx-svc-USERNAME
spec:
  ports:
    - port: 80
      protocol: TCP
      targetPort: 80
  selector:
    app: nginx-USERNAME
  type: ClusterIP

You can now create it:

./kubectl create -f service1.yml

You can list your service with:

./kubectl get services

This is a ClusterIP service which means it is allocation a cluster internal IP and DNS. Within the cluster, you can now access this at http://nginx-svc-USERNAME. To test this, you can exec into one of the pods and try to curl http://nginx-svc-USERNAME:

./kubectl exec -it <pod-name> -- /bin/bash

curl http://nginx-svc-USERNAME/

If we want to confirm that these requests are being split across different pods, you can open a new terminal and have all the logs from all the pods:

kubectl logs --prefix -f -l 'app=nginx-USERNAME'

If you continue to make curl requests while the logs are running you will see that they end up being serviced by different pods.

If you want to access this service outside the cluster, you can use the port forwarding command:

./kubectl port-forward svc/nginx-svc-USERNAME 8080:80

Ingress

If we want to expose a service outside of the Kubernetes cluster, we use an Ingress object. Ingresses typically can provide load balancing, SSL termination and name-based routing (you can have different domains point at different services). Different Kubernetes clusters have different implementations of Ingress controllers (and some clusters don't support Ingresses at all). In Nautilus, the Ingress uses HAProxy Ingress controller.

Create a file called ingress.yml with the following contents:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: haproxy
  name: nginx-ingress-USERNAME
spec:
  rules:
    - host: nginx-USERNAME.nrp-nautilus.io
      http:
        paths:
          - backend:
              service:
                name: nginx-svc-USERNAME
                port:
                  number: 80
            path: /
            pathType: Prefix
  tls:
    - hosts:
        - nginx-USERNAME.nrp-nautilus.io

You can create the ingress with:

./kubectl create -f ingress.yml

You service and deployment pods should now be exposed outside of the cluster as https://nginx-USERNAME.nrp-nautilus.io. The ingress will even handle SSL termination for you.

Cleanup

Make sure you clean up the resources we've created:

./kubectl delete deployment nginx-deployment-USERNAME
./kubectl delete service nginx-svc-USERNAME
./kubectl delete ingress nginx-ingress-USERNAME

Deploying Image from RCD Gitlab

First login to git.rcd.clemson.edu.
Fork the palmetto/intro-k8s repo.
Enable CI/CD and Container Registry at Project -> Settings -> General -> Visibility, project features, permissions
Create an access token that can be used to pull registry images:
1. Settings -> Access Tokens -> Add new token
2. Token name: intro-k8s-USERNAME
3. Role: Developer
4. Scopes: only "read_registry"
Create a registry credential Kubernetes secret file:

./kubectl create secret docker-registry reg-cred-USERNAME \
			--docker-server="registry.rcd.clemson.edu" \
			--docker-username="k8s-intro-USERNAME" \
			--docker-password="<password>" \
			--dry-run="client" \
			-o yaml > reg-cred.yml

Apply it:

./kubectl apply -f reg-cred.yml

You should now be set to deploy using your images. First let's build the images. In Gitlab, select Build -> Pipelines -> Run Pipelines.

While that is building, download the k8s.yml file. Edit it so that all instances of USERNAME are replaced with your user name.

Then try to apply it:

kubectl apply -f k8s.yml

Once it is running, test by going to https://gateway-USERNAME.nrp-nautilus.io.

Online Material - Part 2

Deployments​

Scaling​

Rollouts​

Self Healing​

Port Forwarding​

Service​

Ingress​

Cleanup​

Deploying Image from RCD Gitlab​