Anas T 
Migrating the Ingress resources all at once is not recommended. Instead, DevOps and architects should implement a gradual rollout strategy to ensure a smooth switch from Ingress to Kubernetes Gateway API.
Here, let us briefly go through the reasons why you should migrate. Then, we will see the 3 R’s strategy to migrate from Ingress to Gateway API, without causing downtime.
Why should you migrate from Ingress to Gateway API
Kubernetes Ingress is officially frozen; new developments with respect to Ingress configuration will happen in Gateway API CRDs. Since there is no kind Ingress in the Gateway API, DevOps managers and architects will need to make the switch from Ingress, eventually.
Migrating from Ingress is justified even if we compare the capabilities of both Ingress and Gateway API CRDs. Gateway API far outweighs Ingress in the following aspects:
- Better RBAC and Multitenancy: Unlike Ingress — which is a single resource without proper role separation — Gateway API CRDs have a role-oriented design which makes them a good fit for multitenant, shared Kubernetes clusters.
- Standard, controller-independent specifications: Ingress needs to be configured with annotations except for basic routing functionalities. The annotations are controller-dependent and are not portable across implementations. Gateway API standardizes the specifications for basic and advanced network management without annotations while providing seamless portability across a variety of Ingress controller and service mesh implementations.
- Advanced traffic management: Gateway API CRDs provide out-of-the-box support for advanced routing configurations, like header modification, canary and blue-green rollout, etc. Unlike Ingress, DevOps managers and architects can have multiple backends per route in Gateway API and configure network rules without writing and testing tedious custom annotations.
- Extensibility: Ingress is extensible with annotations, but the scope is limited since annotations are just strings of key-value pairs. Gateway API allows extensibility with custom filters, policies, and CRDs.
Check out the blog Kubernetes Gateway API vs Ingress to learn more about each of the above points with examples.
The 3 R’s migration strategy: research, rollout, repeat
The migration strategy we are sharing here is not specific to any implementation. It focuses on a phased rollout approach for a safer switch from Ingress to Gateway API. Needless to say, DevOps and architects need a solid understanding of Ingress and the Gateway API CRDs to carry out a successful migration.
Step #1: Research
The research phase entails questions DevOps and architects need to ask regarding Gateway API feasibility and application scope, while planning the migration.
Gateway API feasibility
There can be 2 scenarios with respect to migration — either the DevOps team wants to adopt Gateway API CRDs with the current Ingress controller, or they want to implement the CRDs using a new controller.
In the latter case, it is important to research Ingress controller and service mesh implementations for Gateway API and finalize one. The team should also be clear about configuring Ingress rules with Gateway API CRDs.
So the questions to ask are,
- Which vendor to consider in case of migrating to a new Ingress controller?
- How to map current Ingress configurations to equivalent concepts in Gateway API?
Application scope
DevOps and architects should be wary of changes that can happen to the application or infrastructure with Gateway API migration. For example, there can be controller-dependent features that need to be implemented with Gateway API.
At the time of writing this, RequestHeaderModifier and RequestRedirect HTTP filter types are in core supported features, and RequestMirror, ResponseHeaderModifier, and UrlRewrite are in extended support.
So, following are some questions to ask in this stage:
- How to migrate implementation-specific features?
- Should deployment strategies be changed?
- Is the migration going to change the current infra and request flow? If yes, how?
- How to ensure appropriate RBAC policies with Gateway API resources at different levels?
Asking these questions and gathering insights will save a lot of trouble during the actual migration. Once it is done, proceed to rollout.
Step #2: Rollout
This is where the actual migration happens. DevOps and architects can start the migration by making a POC.
Create a proof of concept (POC) with non-critical workloads
Take a few non-critical workloads in the dev environment and rewrite their Ingress configuration with Gateway API resources. You can deploy a few services in a separate test environment for the POC so they don’t interfere with your current workflows.
If you plan to migrate to a new controller, use it in this phase to implement Gateway API. Make sure that controller-dependent features can be implemented using the new controller. Check if the Ingress configurations are working well for the services and move ahead to the next step.
Start progressive delivery in the dev cluster using the canary rollout strategy
Migrate more workloads in the dev cluster to Gateway API. Shift traffic from Ingress to Gateway API using the canary deployment strategy. That is, let the Ingress and Gateway API resources coexist and use an external load balancer to route the traffic between the two (see the image below).
Route most of the traffic to the Ingress and a small portion to the Gateway API. You may start by routing 10% of the total traffic routed to your service, through Gateway API. Test any additional filters, matching rules, and extensions (if any) in the Gateway API routes.
Test and validate
Throughout the rollout process, DevOps and architects need to ensure that the controller-specific configurations and features achieved with Ingress can also be achieved with Gateway API. You can also test RBAC policies with Gateway API for further fine-grained security.
Continue testing the APIs after migration and if metrics and logs are fine, gradually increase the traffic percentage to Gateway API. Perform load testing and other benchmarking measurements to leave no room for errors. Based on the confidence in the performance and behavior of Gateway API, release it completely.
Step #3: Repeat
The stage involves repeating the steps taken in the rollout stage but in the staging and production environment. The configurations will mostly be the same, except for changes in the naming. DevOps and architects can first start with the staging environment and then move on to migrate production workloads.
You need to perform rigorous testing and performance benchmarking to prevent any downtime. The rollout and repeat stages follow a clean-up process, where the Ingress resources are decommissioned after Gateway API handles 100% of the traffic.
Kubernetes Gateway API migration in action
Watch the following video to see a demo on how to shift traffic from Ingress to K8s Gateway API and implement it:
The demo shows:
- A sample banking application deployed and exposed through Ingress using the Nginx Ingress controller.
- Later, Gateway API CRDs are used to expose the same application, and the traffic is gradually shifted from Ingress to Gateway API using a load balancer.
- For the demo, Istio Ingress is used for Gateway implementation.
You can also go through it in the blog, Kubernetes Gateway API Implementation in 3 Steps. Feel free to get in touch if you need help with the implementation.
