In this blog post, we will continue to explain how network policies work and show how they can control traffic in a Kubernetes cluster. If you are not familiar with network policies or missed part 1 of this blog post series, please check it here.
For the demo, we will use kind and spin up one master and one worker Kubernetes cluster without a CNI. As you remember from the previous post, we need a supporting CNI installed in the cluster; otherwise, they will not be effective. In our demo, we will use Calico, one of the popular CNI that supports network policies.
Here is my kind configuration file (cluster.yaml).
kind: Cluster apiVersion: kind.x-k8s.io/v1alpha4 nodes: - role: control-plane image: kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600 - role: worker image: kindest/node:v1.19.1@sha256:98cf5288864662e37115e362b23e4369c8c4a408f99cbc06e58ac30ddc721600 networking: disableDefaultCNI: true podSubnet: 192.168.0.0/16
First, I will start the cluster with the command below.
kind create cluster --config cluster.yaml
So our cluster is ready, and the next step is deploying the Calico. Here I deploy it using the command:
kubectl create -f https://docs.projectcalico.org/v3.8/manifests/calico.yaml
In a couple of minutes, calico pods will be ready.
Now we can start to deploy some applications and see how we’ll limit traffic with network policies.
First, let’s remember our architecture. We have four namespaces in our cluster:
Here, you may think that we have a 3-tier web application stack, and we need to limit traffic between these tiers. Also, we need to deny traffic from the development namespace to our application stack.
So let’s carry on by creating our namespaces and apps.
Here is the manifest file for the namespaces.
apiVersion: v1 kind: Namespace metadata: name: development labels: env: development --- apiVersion: v1 kind: Namespace metadata: name: frontend labels: env: frontend --- apiVersion: v1 kind: Namespace metadata: name: backend labels: env: backend --- apiVersion: v1 kind: Namespace metadata: name: database labels: env: database
And I create them using the command
kubectl create -f namespace.yaml
Next, we will deploy our pods and services to test the connection between them and here is the pod.yaml.
apiVersion: v1 kind: Pod metadata: name: development-app namespace: development labels: env: development spec: containers: - name: dev-nginx image: nginx --- apiVersion: v1 kind: Pod metadata: name: frontend-app namespace: frontend labels: env: frontend spec: containers: - name: frontend-nginx image: nginx --- apiVersion: v1 kind: Pod metadata: name: backend-app namespace: backend labels: env: backend spec: containers: - name: backend-nginx image: nginx --- apiVersion: v1 kind: Pod metadata: name: database namespace: database labels: env: database spec: containers: - name: db image: mysql:8.0.23 env: - name: MYSQL_RANDOM_ROOT_PASSWORD value: "true"
And here is the service.yaml.
apiVersion: v1 kind: Service metadata: name: development-service namespace: development spec: selector: env: development ports: - protocol: TCP port: 80 targetPort: 80 --- apiVersion: v1 kind: Service metadata: name: frontend-service namespace: frontend spec: selector: env: frontend ports: - protocol: TCP port: 80 targetPort: 80 --- apiVersion: v1 kind: Service metadata: name: backend-service namespace: backend spec: selector: env: backend ports: - protocol: TCP port: 80 targetPort: 80 --- apiVersion: v1 kind: Service metadata: name: database-service namespace: database spec: selector: env: database ports: - protocol: TCP port: 3306 targetPort: 3306
So as you can see, we deploy three nginx containers and a MySQL database container. When they are up and running, we’ll start the tests.
Again I deploy pods and services using the commands below.
kubectl create -f pod.yaml
kubectl create -f service.yaml
Right now, without any network policies, we can access from any pod to any pod and between all namespaces. This is the default behavior of Kubernetes when you don’t configure network policies. We can directly make some tests and see that we can reach, for example, from the development pod in the development namespace to the MySql database in the database namespace.
Since we are using nginx images in our pods, we can test the connection with the following curl command
kubectl exec -it -n development development-app -- curl telnet://database-service.database:3306 -v
As expected, we can connect to the database without any issue, and it looks scary :). So our next step should be to prevent this and deploy some network policies into our cluster.
Before deploying a network policy, let’s look at the spec of a network policy.
Like the other objects in Kubernetes, there are the same mandatory fields when defining a network policy. The configuration is set at spec, and here are the fields:
(We will see more examples and see how these fields are used)
Network policies in action!
First of all, we need to deny everything in all namespaces by default. The reason for this is that we may deploy other namespaces and other resources in our cluster, and we will be able to block the traffic generated by these resources.
So let’s deploy our first network policy.
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: development spec: podSelector: {} policyTypes: - Ingress - Egress egress: - ports: # Allow DNS Resolution - port: 53 protocol: UDP - port: 53 protocol: TCP --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: frontend spec: podSelector: {} policyTypes: - Ingress - Egress egress: - ports: # Allow DNS Resolution - port: 53 protocol: UDP - port: 53 protocol: TCP --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: backend spec: podSelector: {} policyTypes: - Ingress - Egress egress: - ports: # Allow DNS Resolution - port: 53 protocol: UDP - port: 53 protocol: TCP --- apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: default-deny-all namespace: database spec: podSelector: {} policyTypes: - Ingress - Egress egress: - ports: # Allow DNS Resolution - port: 53 protocol: UDP - port: 53 protocol: TCP
Let’s see one of them in detail, the database one:
We’ll deploy these network policies and then make another test as we did previously.
kubectl create -f np-default-deny-all.yaml
Let's do the same test as before to check if we can still access the database.
That looks better; we can't connect to the database from the development namespace.
We can carry on with deploying out other network policies. So:
1 - In frontend namespace: We need to allow traffic from frontend to backend, type Egress. (allow-egress-to-backend.yaml)
2 - In backend namespace: We need to allow traffic from frontend to backend, type Ingress. Also, allow traffic from backend to database, type Egress. (allow-ingress-from-frontend-and-egress-to-db.yaml)
3 - In database namespace: We need to allow traffic from backend to database, type Ingress. (allow-ingress-from-backend.yaml)
Let's check the manifest files for each of the network policies
allow-egress-to-backend.yaml
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: frontend-network-policy namespace: frontend spec: podSelector: matchLabels: env: frontend policyTypes: - Egress egress: - to: - namespaceSelector: matchLabels: env: backend podSelector: matchLabels: env: backend ports: - protocol: TCP port: 80
allow-ingress-from-frontend-and-egress-to-db.yaml
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: backend-network-policy namespace: backend spec: podSelector: matchLabels: env: backend policyTypes: - Ingress - Egress ingress: - from: - namespaceSelector: matchLabels: env: frontend podSelector: matchLabels: env: frontend ports: - port: 80 protocol: TCP egress: - to: - namespaceSelector: matchLabels: env: database podSelector: matchLabels: env: database ports: - port: 3306 protocol: TCP
allow-ingress-from-backend.yaml
apiVersion: networking.k8s.io/v1 kind: NetworkPolicy metadata: name: database-network-policy namespace: database spec: podSelector: matchLabels: env: database policyTypes: - Ingress ingress: - from: - namespaceSelector: matchLabels: env: backend podSelector: matchLabels: env: backend ports: - port: 3306 protocol: TCP
We can start deploying these and test them immediately.
kubectl create -f allow-egress-to-backend.yaml
kubectl create -f allow-ingress-from-backend.yaml
kubectl create -f allow-ingress-from-frontend-and-egress-to-db.yaml
Finally, we have deployed all the network policies in our cluster.
I know I'm not able to reach from development namespace to any other so I'll only test the important ones.
Test 1: From frontend to backend
kubectl exec -it -n frontend frontend-app -- curl backend-service.backend
Test 2: From frontend to database. (It should fail)
kubectl exec -it -n frontend frontend-app -- curl telnet://database-service.database:3306 -v
Test 3: From backend to database . (It should work)
kubectl exec -it -n backend backend-app -- curl telnet://database-service.database:3306 -v
That's it, we successfully deployed our network policies, and we can now control our traffic in our Kubernetes cluster. I hope you find it helpful, and if you enjoyed reading it, please don't forget to share.
In our next blog post, we will continue to share other Kubernetes and DevSecOps related information.
We also offer a Free Kubernetes Security Audit; if you’d like to hear more about it, please take a look at it here.