Here, we will see the other type of Istio rate limiting, i.e., Istio global rate limiting. I will explain what it is and how it works, and will show you how to configure Istio global rate limiting per client IP basis.<\/p>\n\n\n\n
Istio global rate limiting<\/h2>\n\n\n\n
Rate limiting applied on an application or service in the Istio mesh, where the application\/Envoy proxy makes gRPC calls to a global rate limit service for request quota, is called Istio global rate limiting.<\/p>\n\n\n
![\"Istio](\"https:\/\/imesh.ai\/blog\/wp-content\/uploads\/2023\/11\/Istio-global-rate-limiting-diagram.png\")
<\/figure>\n<\/div>\n\n\nIstio global rate limiting workflow diagram<\/em><\/p>\n\n\n\n Here is how Istio global rate limiting works:<\/p>\n\n\n\n Istio global rate limiting helps DevOps and architects set rate limiting per IP or dynamic header values. They can apply a rate limiter at the service\/application level \u2014 instead of a service instance\/proxy level \u2014 using Istio global rate limiting.<\/strong><\/p>\n\n\n\n Note that you can also set a rate limiter at the service level by configuring the appropriate workloadSelector<\/em> in Istio local rate limiting. However, the token bucket quota will not be shared among the replica pods\/Envoy proxies, and each proxy will have its own token bucket.<\/p>\n\n\n\n That is, if there is a service with 3 replicas and you set the rate limiter to 5 for the service, each pod will get a token bucket with a maximum of 5 tokens. It will accumulate to a maximum of 15 requests after which the rate limiter will be triggered.<\/p>\n\n\n\n In global rate limiting the token bucket quota is shared, i.e. the service will only accept 5 requests in total, regardless of the number of replicas it has.<\/p>\n\n\n\n Let us see the steps in action.<\/p>\n\n\n\n In the local rate limiting, we configured the token bucket in the Envoy proxy\/pod itself. The proxy then rate-limited requests based on the configuration.<\/p>\n\n\n\n Global rate limiting requires some extra components, including a ConfigMap <\/em>for the global rate limit service.<\/p>\n\n\n\n DevOps and architects can define the request quota for the service that needs rate limiting in the ConfigMap <\/em>resource <\/em>(sample ConfigMap <\/em>YAML<\/a> below).<\/p>\n\n\n\n (You can see that the domain <\/em>value here is set to productpage-ratelimit<\/em>. The value should be the same in the EnvoyFilter <\/em>CRD, which you will see in step #3, for the rate limiter to work.)<\/p>\n\n\n\n The way the reference implementation of the ConfigMap<\/em> (i.e., the global rate limiting service) works is that each list of descriptors<\/a> on the same level is treated as an OR operation, while the nested descriptors are ANDed together.<\/p>\n\n\n\n The above configuration has nested descriptors. A request will be checked for any header value (since I haven\u2019t given a specific header value) and for the path \/get<\/em>. Only the requests that satisfy both conditions will be rate-limited by 5 requests per minute.<\/p>\n\n\n\n A key thing to note here is that the order and the number of the descriptors should be the same as defined for rate limit actions<\/a> in the EnvoyFilter <\/em>resource.<\/p>\n\n\n\n Once you apply the ConfigMap <\/em>YAML, you can proceed to step #2.<\/p>\n\n\n\n Istio provides a Redis-based, sample rate limit service to set up global rate limiting. The Redis database stores the request quota and caches the request count from the rate limit service.<\/p>\n\n\n\n So there are 2 deployments to apply: rate limit service and Redis.<\/strong> See their configurations here<\/a>.<\/p>\n\n\n\n Note:<\/p>\n\n\n\n I have already explained most of the fields on an EnvoyFilter <\/em>resource in the Istio local rate limiting blog<\/a>. Please have a look at it if you haven\u2019t already.<\/p>\n\n\n\n There are 2 EnvoyFilters <\/em>in the sample EnvoyFilter CRD here<\/em><\/a>:<\/p>\n\n\n\n The Istio global rate limiter should work after you apply the EnvoyFilter<\/em> CRD. You can test the configuration using curl<\/em>:<\/p>\n\n\n\n One of the crucial use cases of Istio global rate limiting is that it helps DevOps and architects set rate limiting per client IP. The global rate limiter is more suitable for IP address-based rate limiting since it supports dynamic rate limits, unlike the local rate limiting filter.<\/strong><\/p>\n\n\n\n Dynamic rate limits mean that the global rate limiter will first check for specific values in the descriptor and give a token bucket accordingly \u2014 but if no values are defined then every value for that descriptor is given a separate token bucket.<\/p>\n\n\n\n So if you add a descriptor entry for IP address without mentioning a specific remote address, requests from each IP will receive a token bucket. The requests from the IPs will be rate-limited once they exhaust their individual request quota (see the example below).<\/p>\n\n\n\n With the local rate limiter, you cannot configure dynamic rate limits. You have to specify a value for every descriptor, which means you will end up having one entry per IP address.<\/p>\n\n\n\n For example, you will have to configure a descriptor entry for the remote address 10.0.0.1, another one for 10.0.0.2, and so on. Requests from IPs not specified in the descriptor will not be rate-limited.<\/p>\n\n\n\n Let us see a sample Istio global rate limiting at ingress based on the client\u2019s remote address.<\/p>\n\n\n\n The configuration is similar to how we set it up for the local rate limiter for the Ingress gateway in the previous blog<\/a> (scroll down towards the end to see it). We changed the workloadSelector <\/em>and context <\/em>to istio: ingressgateway <\/em>and GATEWAY<\/em>, respectively.<\/p>\n\n\n\n The change when configuring global rate limiting per IP is that you have to define an additional remote_address <\/em>action in the EnvoyFilter <\/em>CRD, where rate limiting actions for the service can be defined:<\/p>\n\n\n\n After that, you can configure the global rate limiter service to use a dedicated token bucket for each value of the remote address by not specifying<\/em> any address at all.<\/p>\n\n\n\n Since the configuration for the rate limiter service is configured in the ConfigMap<\/em>, you can make that change under the descriptors <\/em>field in the ConfigMap<\/em> resource:<\/p>\n\n\n\n Once you successfully configure and apply this, you can see that requests through the ingress gateway show up in the rate limit service log in the following format:<\/p>\n\n\n\n The key thing to note here is that we got the remote address, but it belongs to a node in the cluster, not the client.<\/p>\n\n\n\n To get around this, we need to configure the load balancer or gateway service to preserve the client IP as mentioned in the Istio docs for the Istio ingress gateway<\/a>.<\/p>\n\n\n\n You can use the following command and update the ingress gateway to set externalTrafficPolicy: Local<\/em> to preserve the original client source IP on the ingress gateway:<\/p>\n\n\n\n A similar configuration can be done to rate limit per client IP if you are using Nginx ingress.<\/p>\n\n\n\n Setting dynamic values using Istio global rate limiting allows DevOps and architects to set up advanced rate-limiting configurations.<\/p>\n\n\n\n For example, you can use a header-to-metadata filter<\/a> in the EnvoyFilter <\/em>resource and use regular expressions to get only the URL path that you need, without any query parameters:<\/p>\n\n\n\n The above regular expression substitution gets only the URL path without any query parameters and creates metadata in the request under the key url<\/em>, in the ratelimiter<\/em> namespace.<\/p>\n\n\n\n You can then match the inbound request on the desired port (I have used port 8000) by using the url <\/em>metadata key in the VIRTUAL_HOST <\/em>configuration:<\/p>\n\n\n\n The above configuration practically cleans up the URL by removing unnecessary query parameters from the request path, taking just the url <\/em>value.<\/p>\n\n\n\n Alternatively, you can use the header-to-metadata filter to pull any specific query parameter or cookies (user ID, for example) from the request path using regex and set a rate limiter based on that. <\/p>\n\n\n\n DevOps and architects may consider the following pointers while trying to decide which type of Istio rate limiting they want to implement:<\/p>\n\n\n\n If you are a DevOps engineer or architect using Istio, it is a given to protect services \u2014 such as login and payment \u2014 against DoS and brute force attacks using Istio rate limiting. It will also help you prevent service overload and maintain an available infrastructure all the time.<\/p>\n\n\n\n\n
<\/li>\n\n\n\n
<\/li>\n\n\n\n
<\/li>\n\n\n\n
<\/li>\n\n\n\nThe difference in token bucket allocation between Istio global rate limiting and local rate limiting at the service level<\/h3>\n\n\n\n
Steps to configure Istio global rate limiter<\/h2>\n\n\n\n
\n
Step #1: Apply ConfigMap <\/em>for the rate limit service<\/h3>\n\n\n\n
apiVersion: v1\nkind: ConfigMap\nmetadata:\n name: ratelimit-config\n namespace: istio-system\ndata:\n config.yaml: |\n domain: productpage-ratelimit\n descriptors:\n - key: ratelimitheader\n descriptors:\n - key: PATH\n value: \"\/get\"\n rate_limit:\n unit: minute\n requests_per_unit: 5<\/code><\/pre>\n\n\n\nStep #2: Deploy rate limit service and Redis<\/h3>\n\n\n\n
\n
<\/li>\n\n\n\nkubectl rollout restart deploy [rate_limit_service_name] -n [namespace] <\/code><\/pre>\n\n\n\nStep #3: Apply EnvoyFilter CRD<\/h3>\n\n\n\n
\n
...\ndomain: productpage-ratelimit\nfailure_mode_deny: true\ntimeout: 10s\nrate_limit_service:\n grpc_service:\n envoy_grpc:\n cluster_name: outbound|8081||ratelimit.istio-system.svc.cluster.local\n authority: ratelimit.istio-system.svc.cluster.local\n transport_api_version: V3\n...<\/code><\/pre>\n\n\n\n\n
...\npatch:\n operation: MERGE\n # Applies the rate limit rules.\n value:\n rate_limits:\n - actions: # any actions in here\n - request_headers:\n header_name: \"x-rate-limit-please\"\n descriptor_key: \"ratelimitheader\"\n skip_if_absent: true\n - request_headers:\n header_name: \":path\"\n descriptor_key: \"PATH\"\n...<\/code><\/pre>\n\n\n\ncurl -v <url>\/get -H \"x-rate-limit-please: true\"<\/code><\/pre>\n\n\n![\"Mindtickle](\"https:\/\/imesh.ai\/blog\/wp-content\/uploads\/2024\/01\/Mindtickle-case-study-1.png\")
<\/a><\/div><\/div><\/div>\n\n\n\nIstio rate limiting per IP<\/h2>\n\n\n\n
Example for IP-based Istio global rate limiting at ingress<\/h3>\n\n\n\n
...\npatch:\n operation: MERGE\n # Applies the rate limit rules.\n value:\n rate_limits:\n - actions: # any actions in here\n - remote_address: {}\n - request_headers:\n header_name: \":path\"\n descriptor_key: \"PATH\"\n...<\/code><\/pre>\n\n\n\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: ratelimit-config\n namespace: istio-system\ndata:\n config.yaml: |\n domain: productpage-ratelimit\n descriptors:\n - key: remote_address\n descriptors:\n - key: PATH\n value: \"\/get\"\n rate_limit:\n unit: minute\n requests_per_unit: 5<\/code><\/pre>\n\n\n\n\"got descriptor: (remote_address=10.224.0.10),(PATH=\/get)<\/code><\/pre>\n\n\n\nkubectl patch svc istio-ingressgateway -n istio-system -p '{\"spec\":{\"externalTrafficPolicy\":\"Local\"}}'<\/code><\/pre>\n\n\n\nAdvanced configurations with Istio global rate limiting<\/h2>\n\n\n\n
...\npatch:\n operation: INSERT_BEFORE\n value:\n name: envoy.filters.http.header_to_metadata\n typed_config:\n \"@type\": type.googleapis.com\/envoy.extensions.filters.http.header_to_metadata.v3.Config\n request_rules:\n - header: \":path\"\n on_header_present:\n metadata_namespace: ratelimiter\n key: url\n regex_value_rewrite:\n pattern:\n google_re2: {}\n regex: '^(\\\/[\\\/\\d\\w-]+)(\\?)?.*'\n substitution: '\\1'\n...<\/code><\/pre>\n\n\n\n...\n- applyTo: VIRTUAL_HOST\n match:\n context: SIDECAR_INBOUND\n routeConfiguration:\n vhost:\n name: \"inbound|http|8000\"\n patch:\n operation: MERGE\n value:\n rate_limits:\n - actions:\n - metadata:\n descriptor_key: url\n metadata_key:\n key: ratelimiter\n path:\n - key: url\n...<\/code><\/pre>\n\n\n\nIstio global or local rate limiting: which one to use?<\/h2>\n\n\n\n
\n
<\/li>\n\n\n\n
<\/li>\n\n\n\n
<\/li>\n\n\n\n
<\/li>\n\n\n\nLeverage Istio rate limiting to protect crucial services<\/h2>\n\n\n\n