TLS with Kubernetes Gateway API

In the previous blog, we looked into Kubernetes Gateway API implementation. We saw a demo on gradually shifting traffic from Ingress to Gateway API. Here, I will share a tutorial on securing the traffic in Gateway API using TLS. 

Steps to implement TLS with Kubernetes Gateway API

I will follow the given steps for the demo:

• Step #0: Demo overview and prerequisites
• Step #1: Deploy the application, cert-manager, and ClusterIssuer
• Step #2: Deploy the Gateway with HTTP and HTTPS listeners
• Step #3: Attach HTTPRoutes to the Gateway listeners
• Step #4: Access the Gateway

Step #0: Demo overview and prerequisites

To show TLS with K8s Gateway API, I will deploy a Gateway with 2 listeners: HTTP and HTTPS. Both listeners will have an HTTPRoute attached to it, which routes the traffic to the service as shown in the image below:

We will check if we can access the application through the secure HTTPS route.

The prerequisites for the demo are the following:

• A controller to implement Kubernetes Gateway API. I have used Istio Ingress for the demo.
• At the time of writing this piece, the cert-manager needs to have the experimental gateway feature manually enabled for it to work with Gateway API. For that, add –feature-gates=ExperimentalGatewayAPISupport=true to the cert-manager-controller container args:

```
containers:
  - name: cert-manager-controller
    image: "quay.io/jetstack/cert-manager-controller:v1.14.1"
    imagePullPolicy: IfNotPresent
    args:
      - --v=2
      - --cluster-resource-namespace=$(POD_NAMESPACE)
      - --leader-election-namespace=kube-system
      - --acme-http01-solver-image=quay.io/jetstack/cert-manager-acmesolver:v1.14.1
      - --max-concurrent-challenges=60
      - --feature-gates=ExperimentalGatewayAPISupport=true ######### REQUIRED FOR CERTIFICATE TO WORK WITH K8S GATEWAY API

```

I have already configured and uploaded the resources used for the demo to the IMESH GitHub repository.

If you are interested in watching the demo in action, here is the video:

Step #1: Deploy the application, cert-manager, and ClusterIssuer

I’m deploying a simple echoserver service (application.yaml) to tls-gw-api namespace (namespace.yaml):

kubectl apply -f app/application.yaml

Now, let’s deploy the cert-manager (cert-manager.yaml) which has the Gateway feature enabled:

kubectl apply -f certificate/cert-manager.yaml

The cert-manager is up and running:

Now, I’m using the following ClusterIssuer (ssl-prod-cluster-issuer.yaml) for the prod cluster as the Certificate Authority (CA) to manage the certificates for any namespace in the cluster:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: prod-cluster-issuer
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: prod-cluster-issuer
    solvers:
  	- http01:
          gatewayHTTPRoute:
        	parentRefs:
          	  - kind: Gateway
                name: k8s-gateway
                namespace: tls-gw-api

I’m using the HTTP-01 challenge for the certificates to auto-renew in the cluster. The gatewayHTTPRoute field specifies the gateway where the HTTP challenge needs to be solved and certificates to be auto-renewed.

Applying the issuer:

kubectl apply -f certificate/ssl-prod-cluster-issuer.yaml

You can check its deployment status by running the command:

kubectl get clusterissuer

That sums up the configurations for the cert-manager. Now, let’s look at the Kubernetes Gateway API CRDs.

Step #2: Deploy the Gateway CRD

Following is the configuration for the Gateway resource (k8s-gateway.yaml):

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: k8s-gateway
  namespace: tls-gw-api
  annotations:
    cert-manager.io/cluster-issuer: prod-cluster-issuer
    # cert-manager.io/cluster-issuer: staging-cluster-issuer
    service.beta.kubernetes.io/port_80_no_probe_rule: "true" # FOR AZURE
    service.beta.kubernetes.io/port_443_no_probe_rule: "true" # FOR AZURE
spec:
  gatewayClassName: istio
  listeners:
    - name: http-listener
  hostname: "*.imesh.ai"
  port: 80
  protocol: HTTP
  allowedRoutes:
      namespaces:
          from: All
    - name: https-listener
  hostname: test.imesh.ai
  port: 443
  protocol: HTTPS
  tls:
      mode: Terminate
      certificateRefs:
          - name: "test-imesh-cert"
  allowedRoutes:
      namespaces:
          from: All

• In the above file, metadata specifies the gateway (k8s-gateway) and the respective namespace (tls-gw-api).
• The annotation cert-manager.io/cluster-issuer: prod-cluster-issuer specifies the ClusterIssuer configured in step #1. Make sure to change the issuer and annotation if you are using a namespace issuer (i.e., Issuer resource).
• gatewayClassName shows the controller (Istio) used for implementing the Gateway API resources. Istio is already installed in the cluster.
• listeners field lists http-listener and https-listener which use HTTP and HTTPS protocols, respectively. http-listener has the hostname *.imesh.ai while https-listener has test.imesh.ai.
  • Note that an HTTP listener is mandatory even if you need only the HTTPS route or TLS. Without an HTTP listener, the certificate will not be able to solve the HTTP challenge since it happens in plain-text/HTTP format. For better security, it is advisable to add all your application-level routes to the HTTPS listener and just have the HTTP listener for certificate renewal purposes.
• tls field refers to the tls termination behavior (termination) and the certificates used by the Gateway. The certificate’s secret information is mentioned under certificateRefs.

Applying the Gateway:

kubectl apply -f gw-api/k8s-gateway.yaml

The Gateway is running:

Now, let’s create routes for the Gateway listeners to take the requests to the echoserver service.

Step #3: Attach HTTPRoutes to HTTP and HTTPS listeners

I have created 2 HTTPRoute resources: insecure-http-route and secure-http-route. In both resources, the sectionName field refers to the respective listener in the Gateway to which the route gets attached.

In the insecure-http-route resource, the sectionName is http-listener which means that the HTTPRoute gets attached to the HTTP listener in the Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: insecure-http-route
  namespace: tls-gw-api
spec:
  parentRefs:
    - kind: Gateway
  name: k8s-gateway
  sectionName: http-listener
``` 

Similarly, secure-http-route attaches to the https-listener in the Gateway.

I have also added filter chains to inject request and response headers to identify whether the request comes from secure or insecure routes. The below configuration is from secure-http-route, and it injects HTTPS-Secure header:

``` filters:
  - type: ResponseHeaderModifier
    responseHeaderModifier:
      add:
        - name: PROTOCOL
          value: HTTPS-Secure
  - type: RequestHeaderModifier
    requestHeaderModifier:
      add:
        - name: PROTOCOL
          value: HTTPS-Secure
```

Similarly, insecure-http-route injects HTTP-Insecure value to the header.

Both the HTTPRoute resources route traffic from the Gateway to echoserver service in port 80.

Let us apply the routes:

kubectl apply -f gw-api/insecure-http-route.yaml
kubectl apply -f gw-api/secure-http-route.yaml

You can verify the routes attached to the Gateway listeners by describing the Gateway:

kubectl describe gateway k8s-gateway -n tls-gw-api 

We can also check if the certificate is generated:

kubectl get certificate -n tls-gw-api

Now, let us verify if we can access the application from the browser through both HTTP and HTTPS routes.

Step #4: Access the Gateway

Let us try the insecure, HTTP route by typing in http://test.imesh.ai in the browser:

You can see the protocol=HTTP-Insecure in the request headers, which means that it is coming from the insecure HTTP route.

If I try https://test.imesh.ai, you can see that the protocol value is HTTPS-Secure, meaning the request comes from the secure HTTP route.
 

You can also view the certificate issued for the connection:

IMESH Enterprise Gateway API support

The TLS setup with Gateway API that we saw above is a simple one. In real-life production clusters, DevOps and architects deal with multiple certificates used for different domains. Managing certificates and their auto-renewal can quickly become complicated in those scenarios. IMESH provides support for Gateway API challenges, and you can contact us anytime if you need help.