Applying principle of least privilege to CNFs

[taylor]

CNFs should always run with the least amount of privileges required to function properly

Reference:

• Chapter 1. Approaching Kubernetes Security
  from Kubernetes Security by Liz Rice, Michael Hausenblas

  Least Privilege: The principle of least privilege tells us to restrict access so that different components can access only the information and resources they need to operate correctly. In the event of a component being compromised, an attacker can reach only the subset of information and resources available to that component. This limits the “blast radius” of the attack.

• https://redhat-connect.gitbook.io/best-practices-guide/principle-of-least-privilege
• https://kubernetes.io/docs/concepts/security/overview/

Related best practice ideas:

• CNF's containers should run unprivileged
• Container should execute process as non-root user

[taylor]

Related issue for adding a best practice https://github.com/cncf/cnf-wg/issues/67

[iawells]

Obvious question: why (which I think we can justify, but 'do this' is not as good as 'do this because...')

[electrocucaracha]

@iawells you mean something like this: Using least privileged containers can reduce the attack surface and preserve the inmutability of compute node

[iawells]

We were discussing this yesterday.

Using least privilege (in its strict form) is an application design technique that you give apps teh minimum privilege available that is required to get their jobs done. In k8s today you might argue that this justifies the use of privileged containers, which is the least privilege available for certain tasks. In fact, least privilege doesn't really say what the platform offers as much as what the app does with it.

I think we have a related set of needs, which is better described as something other than least privilege: the platform should:

• delegate privilege to apps in a way that wouldn't endanger its own job to provide a stable environment that isolates apps
• delegate it in a fine-grained way that benefits least privilege in apps (so if an app is only going to want priv A it shouldn't get B, C, D)

The first of those is saying that there's an upper limit on the privilege that an platform will offer. Today, for instance, a privileged container gets access to all netns'es, which means that an app can break the platform's management networking. If the rule we're looking for is that 'a platform will delegate only rights to applications that preserve the stability of the platform' - which you can reduce to 'if the platform is broken then the platform team are the cause of the problem and the platform team can fix it' - we are ensuring that an application cannot break the platform (and we would want to extend this a little to 'the application cannot break other applications').

The second of these is saying that we should be careful how big a dose of privilege we offer. If I can only give an app access rights to delete any pod, it can start deleting the pod of other apps. That would be bad. We will need access rights scoped to an application - 'a platform can delegate limited rights to an application that prevent it interfering with other applications'.

[taylor]

Potential best practices related to the least privilege principle

• No root in a container
  • To avoid compromised apps doing even more damage (3)
  • To avoid uncompromised apps stomping on
• Manage access that apps have to k8s api so they can only use appropriate APIs to their circumstances
  • Limit access to K8s APIs from applications
  • To prevent change of network attachment points (1, 2)
  • To prevent messing with other people’s containers, networking (1)
  • To prevent change to system-wide properties (e.g. host labels, taints) that affect all apps (1)
• No privilege or capabilities granted to containers
  • To prevent breaking platform (2 and 1 by implication)
• Containers should be runtime isolated from containers in another Pod
  • Containers should be runtime isolated from one another
• Containers should be runtime isolated from the host
• Shared resource access between containers in different Pods should be managed by orchestration
• Use a restricted Security Context where possible
  • https://kubernetes.io/docs/tasks/configure-pod-container/security-context/
  • https://docs.openshift.com/container-platform/4.7/authentication/managing-security-context-constraints.html
• Pods should not mount host directories as volumes
• Pods should not use host name spaces
• Use Kubernetes network policies to control traffic between pods and clusters
• Prevent overly permissive access to secrets
• If possible use a read-only root filesystem
• Use the default (masked) /proc filesystem mount
• Drop unused and unnecessary Linux capabilities
• Give each application its own Kubernetes Service Account
  • Do not mount the service account credentials in a container if it does not need to access the Kubernetes API
• Use SELinux options for more fine-grained process controls
• Use Kubernetes-native security controls to reduce operational risk

[iawells]

Given that we know right now that CNFs will not run well or at all without at least some form of elevated privilege, I propose that we have a best practice that says 'here's a process for least privilege implementations'. Basically:

1. Least privilege involves running as a non-admin k8s user, independent of users used for other applications on the platform and limited to the management of the pods that form a part of the application and not the infrastructure between applications or of other pods
2. Least privilege involves running without host networking rights
3. Least privilege involves running without elevated rights in the container (such as CAP_NET_ADMIN)

This is what you will strive for. However, we appreciate there may be exceptions, and exceptions should be documented and ship with the application. Exceptions should detail privileges required higher than the above, where they are required (e.g. individual pods, an additional elevated account for install and upgrade ops,etc.); and will include a reason for requiring the elevated privilege. This is an input to the operations team's audit process and they should consider its content when deciding whether the application meets their security requirements before adding the application to their setup.

The idea is that this best practice details how to allow for problems that a strict 'least privilege' policy would cause. It also allows us to flesh out our Platonic ideal of least privilege.

[taylor]

@wavell said (in https://github.com/cncf/cnf-wg/issues/67#issuecomment-886899047):

least capabilities within non or 'unprivileged' processes, can be a more fine grained version of this best practice ... especially for networking/bpf cnfs:
https://man7.org/linux/man-pages/man7/capabilities.7.html

[taylor]

@fkautz and @iawells have both mentioned CAP_NET_ADMIN as a capability that can be problematic. A process in a container could cause problems with network traffic in a Pod for instance.

[fkautz]

CAP_NET_ADMIN is close to root access. The capability in question is CAP_NET_RAW which allows direct access to network traffic as raw IO. Pods can craft and receive packets arbitrarily with this capability. All pods by default have this capability to respond to icmp ping. This should be mitigated via seccomp or another police enforcement framework.

On Mon, Jul 26, 2021 at 11:03 AM Taylor Carpenter ***@***.***> wrote: @fkautz <https://github.com/fkautz> and @iawells <https://github.com/iawells> have both mentioned CAP_NET_ADMIN as a capability that can be problematic. A process in a container could cause problems with network traffic in a Pod for instance. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/cncf/cnf-wg/discussions/28#discussioncomment-1052576>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABBEGR64I6KPX76UY24ANLTZWPPHANCNFSM4UUCMYMQ> .

-- -- Frederick F. Kautz IV

[tliron]

Close ... and yet still different. :)

I think it's worth going through these three things separately and detailing exactly what they provide and at what cost for isolation/security:

1. root
2. privileged: true
3. capabilities: [ ... ]

It's also worth going into individual capabilities. There are a few specifically for networking and they are different from each other (sometimes you need to use 2+ together).

[taylor]

Yes for covering them individually @tliron. The list above ☝🏻 https://github.com/cncf/cnf-wg/discussions/28#discussioncomment-1052082 is suppose to be for individual best practices which need to be gone through separately, communicating why they are useful, for what context, and any caveats.

The three you list (root, privileged: true, and capabilities) are part of the large set of things to look at under this discussion: Applying the principle of least privilege to CNFs.

[fkautz]

We should go over CAP_NET_ADMIN and CAP_NET_RAW at the minimum. There is also CAP_NET_BIND_SERVICE and CAP_NET_BROADCAST which may be interesting to review.

[fkautz]

Considering the scope of the ask, we should probably go over other capabilities as well. See https://man7.org/linux/man-pages/man7/capabilities.7.html for a current list.

[taylor]

Best Practices for Securing and Hardening Container Images: https://tanzu.vmware.com/developer/guides/containers/security-best-practices/

• Non-root containers
• Use arbitrary UUIDs

[taylor]

Recommendation: Container should run in an unprivileged mode -> https://github.com/cncf/cnf-wg/discussions/25