Anas T 
Some architects, cloud engineers, and DevOps folks often say, “Microservices are small monoliths.” This stems from the complexity of dealing with a number of services, especially managing and configuring their network rules and security aspects.
When clients make requests to microservices spread over multiple clusters and clouds in a distributed system, tracking each request to ensure security and proper routing rules becomes tedious. Ideally, backend services should not be doing that, as they should be left alone with delivering the business logic. This is where an API gateway, a single entry point for all requests, comes in. Let us see what an API gateway is, and what features and benefits it offers.
What is an API Gateway?
An API gateway is a server (or L7 proxy) between a client and microservices that acts as a centralized entry point for all clients into the system. It is a reverse proxy that accepts client API calls and forwards them to the appropriate microservice (refer to Fig. A below).
By providing an API for each client, API gateways encapsulate the complexity of the underlying system and let clients talk to it instead of calling specific services. They also perform security checks (authentication and authorization) before the traffic reaches the service, thus leaving services to focus on their core functionalities.
Fig. A – API gateway implementation conceptual diagram for microservices architecture
The need for API gateways for microservices
The challenges posed by the direct client-to-microservices pattern of communication led to the popularity of API gateways. Let us go through some of them.
The problem of service discovery and traffic routing
For direct client-to-microservice connection, the clients have to know the specific endpoints of service instances. But keeping track of endpoints adds complexity for clients because of the dynamic (de)scaling of services. Also, if clients are coupled to the services, scaling becomes an issue as it will require configuration changes on the client side. Besides, routing traffic based on certain attributes, such as geography (geo-routing), is hard to configure when clients invoke services directly.
Security concerns
Publicly exposing service endpoints for direct client-to-service communication causes security concerns. It increases the attack surface for intruders and leaves backend services prone to threats, such as packet sniffing, man-in-the-middle attack, etc. Besides, direct client-to-microservices put the burden of authenticating and authorizing API calls on services, instead of letting them focus on delivering the business logic.
Diverse protocols affecting interoperability
The flexibility provided by microservices architecture lets developers build services using the language (Python, Java, Go) they choose. Similarly, they can implement those services in different API types, such as REST, gRPC, and others. In a direct client-to-microservice communication pattern, it then becomes a requirement for the client to understand and communicate using different protocols. This adds additional complexity as the client application will require more code and logic.
Latency caused by round trips
Consider a product page from the Amazon website. Some attributes like product pricing, quantity, and reviews will be deployed as different services in the backend. If the client is invoking services directly, it will have to make separate requests for each service (product price, reviews, quantity, etc.) to retrieve the required information, since there is no mechanism to cache a response from the upstream services. These calls add to the overhead of establishing multiple connections, and the round trips caused by these network requests add to latency and suboptimal user experience.
API gateways’ architecture is in a way that it mitigates the challenges caused by direct client-to-microservice connections and provides a variety of features.
API gateway traffic flow
An API gateway is an L7 proxy that abstracts the traffic management out of front-end microservices, which are usually requested by clients. API gateways can read and understand the HTTP messages (refer to the picture below), so they can apply filters or take actions on the traffic.
HTTP message structure (source)
A request flows through multiple steps at the API gateway. The below image (Fig. B) represents an API gateway sitting at the edge of a Kubernetes cluster and the stages through which a request flows.
Fig. B – Traffic flow of an incoming gRPC request through the API gateway
- Request validation: First, API gateways validate an incoming request by checking the request method, headers, and body, to ensure that it complies with predefined rules. Also, requests are whitelisted or blacklisted based on the gateway’s allow-list and deny-list, to exercise strict access control.
- Authentication and authorization (AuthN/Z): The gateway authenticates and authorizes the validated request by verifying credentials, such as API keys (authentication), and then implements authorization policies, such as RBAC (role-based access control). API gateway may also use the help of an IdP (identity provider) for exhaustive authN/Z checks, like multi-factor authentication (MFA).
- Service discovery: API gateway uses service discovery component to locate the appropriate backend service for the authorized request. It does that by querying the service registry or by using dynamic DNS.
- Protocol translation: If necessary, API gateways can translate protocols between clients and microservices. In the above image, the client sends a gRPC request that is translated into REST for the “Cart” service to understand. Also, the gateway transforms the response into the public-facing protocol (REST to gRPC format here) before returning the response to the client.
- Dynamic routing: Once the backend service is located and the request protocol is translated if needed, the API gateway routes the request to the appropriate service instance or load balancer. Gateways typically have routing rules defined in their configurations.
Once the API gateway completes the above steps, it will return the response from the service back to the client. However, note that the outlined steps may vary depending on the way the gateway is configured and the implementation of additional features.
API gateway features
API gateways provide many features apart from the critical ones mentioned above.
- Advanced traffic routing: API gateways can manage and control the way API traffic is distributed among the services. They can perform load balancing and traffic splitting, and act as a reverse proxy and forward proxy. API gateway also implements progressive delivery, such as blue-green deployments.
- Rate limiting: API gateways can limit requests based on the IP address or HTTP headers. It helps services from overloading with requests and preventing malicious attacks, such as denial-of-service (DoS). Rate limiting also helps API providers implement different monetization strategies (tiered pricing model, for example).
- Error handling: API gateways can handle and standardize error responses if an API request fails due to issues, such as internal server errors, authentication failures, and invalid input. The HTTP status code of the error and the error message can be customized for the client using the gateway, helping clients understand and handle/debug it correctly.
- Circuit break: API gateways can implement circuit breaker patterns that help backend services from overloading with requests that are failing. The circuit-breaking pattern will stop forwarding requests to the failing service and return the appropriate error code. It helps in preventing the error from cascading to healthy upstream services and makes the API infrastructure resilient.
- Observability: API gateways provide logging, monitoring, and analytics for operational observability. It helps in understanding the performance (API usage) and behavior of the API infrastructure, resulting in improved visibility, security, and reduced downtime.
- Caching: API gateway can cache and serve responses made from previous requests for the clients. If API caching is enabled, the gateway will check a request against cached responses and forward one if it exists, thus eliminating the need to make a new, same request to the backend service every time. This will significantly reduce the load on services and response latency, and improve API performance and user experience.
- SSL termination: API gateway can perform SSL termination, which involves decrypting an incoming SSL connection from the client on behalf of the backend services. Offloading SSL termination along with other security features like authN/Z will leave services focused on their primary responsibilities.
All these features provided by API gateways provide enormous benefits in managing a distributed system of services.
Benefits of API gateway
API gateway implementation helps organizations reap the following benefits, among others.
Improved application security
As a centralized point for API management, gateways hide the services and the underlying infrastructure from being exposed publicly. This makes it difficult for attackers trying to bring down the application, particularly by overwhelming the services with requests (DoS attack). Since gateways process every request before reaching the backend, they can apply rate limiting against such an attack. Other security features, such as request validation, authN/Z, circuit breaking, and policy enforcement, coupled with logging and monitoring, makes API gateway contribute to the overall security of the applications.
Enhanced flexibility in handling and scaling microservices
API gateways decouple external clients from internal microservices. This gives high flexibility for DevOps and Infrastructure Engineers in making changes to the backend services without requiring updates to configurations in the client applications. Clients can still make requests through the gateway and get responses without knowing about the changes the backend has undergone. Many important functionalities, such as authN/Z and load balancing, will be taken care of by the gateway. Offloading these responsibilities to the gateway helps developers write less amount of code for applications, which fosters innovation and enables rapid releases.
Better monetization for API providers
API monetization is all about productizing the API for third-party consumers. API gateway provides a better way for companies to monetize their APIs for generating revenue or to cover the operational cost of maintaining the APIs. The gateway helps in connecting the client requests to a billing system, thus providing a centralized billing and metering mechanism for API providers. This helps companies track API usage and collect payments for the service by implementing different pricing models, such as pay-as-you-go, tiered, and unit-based, for API consumers.
Improved user experience (UX)
API gateway eases clients from making too many requests by making requests to the underlying services and aggregating them. That is, a single request to the gateway will be sufficient for client applications, which will significantly reduce the latency. And in the case of frequently repeated requests, the gateway can promptly serve cached responses without forwarding the request to the backend. Also, with monitoring and logging features, API gateways make it easier to track and troubleshoot any performance issues, which helps in minimizing application downtime. All of these help in improving the application’s performance, reliability, and user experience.
Top 3 open-source API gateway tools
While evaluating an API gateway tool, organizations can look for open-source tools, cloud service providers, or enterprise editions. If open-source is your priority, we have outlined the top three open-source API gateway tools, based on the factors such as ease of use, flexibility, and scalability.
Tyk API Gateway
Tyk provides a fully open-source gateway that supports multiple protocols like REST, GraphQL, and gRPC. It has no third-party dependency apart from Redis, and it is one of the fastest gateways available today.
Below are some features of Tyk API gateway:
- Use any protocol: REST, SOAP, GraphQL, gRPC, and TCP
- AuthN/Z using OIDC, JWT, Client Certificates, and more
- Create extensible plugins using any language (Python, JS, Go)
- Tyk Operator for Kubernetes-native declarative API
- Allow browser-based requests by enabling CORS
- API versioning and lifecycle management
- Detailed API usage data with analytics logging
Kong API Gateway
Kong API Gateway is a cloud-native gateway, suited for both multicloud and hybrid cloud deployments. The gateway is also Kubernetes-native with the help of its own Kubernetes ingress controller. Kong is known for its flexibility and extensibility through modules and plugins.
Some open-source features of Kong API Gateway include the following:
- End-to-end automation to drive a GitOps flow of API design and execution
- Kubernetes Ingress Controller to deploy APIs to K8s
- Basic traffic control plugins, such as basic rate limiting and lightweight caching
- Simple data transformations
- gRPC transformations to backend gRPC services
- AuthN/Z (HMAC, JWT key auth, limited 0Auth 2.0 and LDAP, bot detection, ACLs)
- Simple logging (file logging, HTTP logging, basic StatsD, TCP/UDP logging)
KrakenD API Gateway
A high-performing API gateway, KrakenD is built with a serverless architecture that provides true linear scalability. It helps in scaling out without a single point of failure. KrakenD runs on on-prem, hybrid, or cloud, and is extensible with plugins and embedded scripts.
The open-source version of KrakenD offers the following features:
- A visual tool called KrakenD Designer to generate KrakenD configurations
- Multi-format configuration (JSON, YAML, TOML, HCL, and more)
- Build custom Go plugins and embed them with a ready-to-use image
- Data transformation, HTTP cache headers for CDN, and automatic output encoding
- Zero-trust parameter forwarding, preventing clickjacking, sniffing, and cross-site scripting
- JWT claim-based routing and traffic shadowing/mirroring
- Logging, tracing, and metrics (Jaeger, Zipkin, ELK stack dashboard, Prometheus, and more)
Is API gateway a silver bullet?
No, it is not. Like any other tool, API gateways come with a set of challenges. Here are a few of them:
- Traffic handling limited to edge
- Inability to handle north-south traffic
- Lack of visibility into the internal communication
- No control over the network inside the cluster
- Unsecured east-west traffic
- Cannot implement progressive delivery (such as canary)
Explore these challenges of API gateways in detail and understand why considering a service mesh platform, like Istio, would be ideal: API gateway vs Istio service mesh.
About IMESH
IMESH helps organizations simplify and secure their network of cloud-native applications by onboarding and adopting Istio and Envoy. The Istio enterprise support we provide helps to deploy Istio into production with no operational hassle. Book a free 30-mins consultation with one of our Istio experts, or contact us if you need enterprise Istio support.
