Implementing stronger RBAC and Multitenancy in Kubernetes using Istio

Background of multi tenancy 

DevOps and solution architects often implement RBAC and multitenancy in their Kubernetes infrastructure to achieve isolation of workspace and allow authorized persons to access resources with least privilege resources. 

The implementation of RBAC and multitenancy can be very simple or complicated, and this depends of the following parameters:

1. Application- microservice based architecture
2. Deployment speed & frequency- High or low
3. Architecture- Multi-cloud cloud/Hybrid cloud/ On-prem 
4. Environment- Dev/stage/prod
5. Teams/Groups- Developers, DevOps, SREs, Marketing/Pre-sales
6. Users- application developer, full stack developer, web-API engineer
7. Budget in hand
8. Goal- Simple administration vs granular security 
9. Geography team- Multi-geo

In this article we will discuss how to implement RBAC and multitenancy (simple or complicated) in Kubernetes using open source Istio service mesh. 

Enabling RBAC for K8s service accounts and users using K8s resources

We have discussed at length about how to enable Kubernetes RBAC using various resources such as- Role, ClusterRole, RoleBinding, ClusterRoleBinding. The idea of using all the Kubernetes RBAC resources is to create various actions that can be taken on namespaces or resources and then allocate (or bind) the actions to users and service accounts. To restrict IPs i.e. ingress and egress traffic, network policy resources (refer the image below) in Kubernetes can be used. 

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: banking-network-policy
  namespace: dev-namespace
spec:
  podSelector:
    matchLabels:
      role: db
  policyTypes:
    - Ingress
    - Egress
  ingress:
    - from:
        - ipBlock:
            cidr: 172.17.0.0/16
            except:
              - 172.17.1.0/24
  egress:
    - to:
        - ipBlock:
            cidr: 10.0.0.0/24
      ports:
        - protocol: TCP
          port: 5978

However, there can be certain limitations of using K8s RBAC.

Limitations of using Kubernetes RBAC for workloads and services

You see, the Kubernetes RBAC is good for implementing users’ access to certain resources by applying checks on API servers. It is very good when you want to allow developers to perform experiments on a cluster or a namespace or testers to deploy services and test in testing workspaces. 

But what about authorization of workloads or services? How to implement granular controls to workload authorization in production environments. Some of the limitations are: 

1. K8s RBAC is challenging to implement for granular usage. Imagine you want to restrict accounts to use few resources in an endpoint (a service in a namespace) or want to control REST operations on the pods, K8s RBAC will not be sufficient in this case. 
2. Difficult to create RBAC policies in Roles and RoleBinding. For e.g. in a large organization there can be multiple clusters and namespaces. There can be 100+ services running in all these namespaces and is handled by various application teams. And many of these services need to talk to each other. DevOps may end up creating too many Roles, RoleBinding, ClusterRoles, and ClusterRoleBinding objects. 
3. Similarly in large set up updating and deleting an RBAC policy in the Kubernetes resource can also be daunting. 
4. For ingress and egress traffic (or east-west and north-south traffic) one has to create additional Network policies along with Kubernetes RBAC. 

To make the RBAC implementation simple, Istio can be used. 

Note: It is not Kubernetes RBAC vs Istio, but Kubernetes RBAC and Istio for better manageability and implementation of RBAC and multitenancy for users and production workloads. 

Achieving RBAC multitenancy with Istio

In case you are using Istio for simplifying your network, security or observability you can use Istio service mesh for implementing RBAC and multitenancy in your Kuberntes workloads and for your teams. 

Note: The approach of Kubernetes RBAC is user-first i.e. defining what user can do what operations. But Istio uses the resource-first approach i.e. it dictates who can do what on a particular service (and its resources). The advantage of Istio’s approach is manageability of the RBAC and multitenancy rules (we will see later in this article) with its Authorization policy.

There can be multiple easy-to-complicated scenarios where Istio can be used to implement RBAC:

1. Single cluster-multiple namespace
2. Multicluster-multiple namespaces
3. Multicloud- multiple cluster (multiple namespace)

We will look into each scenario and understand how Istio can be used for RBAC and multi-tenancy. 

Scenario1: RBAC for single cluster-multiple namespace using Istio

Let us take an example where there are multiple namespaces for various non-prod environments which have to be shared among developers and Devops. In the image below, we have dev-namspace and staging-namepsace where a banking application with multiple services are deployed. And each namespace will be allowed the editing rights to certain team members. The banking application contains 3 services- profile, transactions and account-summary- they talk to each other and share data. 

One can achieve isolation of workspace and allow least privilege access to team members using Kubernetes RBAC- i.e. through defining Role and RoleBinding, and workloads authorization can be implemented using Istio Authorization policy ( sample below):

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: restrict-access-policy  namespace: stage-ns
spec:
  selector:
    matchLabels:
      app: account-summary
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/stage-ns/sa/profile","cluster.local/ns/stage-ns/sa/transaction" ]
    - source:
        namespaces: ["dev-namespace"]
    to:
    - operation:
        methods: ["GET","PUT","POST"]

The best part is that as the network and security is abstracted from the application using Istio service mesh, it is easy to implement and modify authorization policies or rules. The above Authorization policy in Istio is implemented to side-car proxies which will validate any REST API request to any workloads. 

In case you have an end-point for a microservices and there are many resources to be used; for e.g. account-summary microservice has 2 resources i.e. current-year and previous-year for getting the account summary for current year or  previous year. In that case, you can use path (or URI) in the Istio Authorization policy. Check the sample Istio Authorization policy below. 

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: account-summary
  namespace: staging-ns
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/stage-ns/sa/profile"]
    - source:
        namespaces: ["staging-ns"]
    to:
    - operation:
        methods: ["GET"]
        paths: ["/previous-year", "/current-year"]

Note, this is just a few use-cases, you can apply any RBAC use-cases to services using Istio. The Istio Authorization policy provides various actions such as ALLOW,  DENY, CUSTOM that can be applied to any workloads for the REST calls. Read more about Istio Authorization policy.

You can also watch the video on how to set up RBAC and multitenancy using Istio service mesh. 

Scenario2: RBAC for multiple cluster with multiple namespace using Istio

Let’s say there is another scenario where the production clusters are different from the development or staging clusters. In those cases too Istio can be used for practicing RBAC. However, it depends on the implementation of the Istio. For e.g. if you have resource constraints you can configure one Istio control plane in one cluster and then manage all the workloads in all the cluster using the Istio control plane. You can use an east-west gateway to implement multiple cloud Istio.

For user management you can use Kubernetes RBAC, and for workload authorization you can use Istio Authorization policy. 

Scenario3- RBAC for Multicloud and multi cluster application using Istio

Similarly there can be an application whose services are stored in multiple clusters for high-availability. In the image below, a passive version of the transaction service is deployed into multiple clouds (another Kubernetes cluster) to ensure HA.  Again RBAC can be implemented using Istio authorization policy. But it depends on the budget and level of logical partitioning and control one wants to achieve. Either one or multiple Istio can be implemented to control all the Kubernetes cluster across multiple clouds and RBAC and multi tenancy can be implemented accordingly. 

Note: For large teams with different projects or products, separate Istio can be implemented and let there be separate owners for each Istio infra. In this way there will be siloes authorization policies based on the product requirements.