Access policy CRDS, privileges and namespaces

[zivnevo]

There is a profound use case for letting cluster admins set "high-priority" access policies that cannot be overridden by non-admin users. This allows enforcing corporate rules, decisions from CISO team, and various regulatory requirements. The PDP currently supports such a two-tier approach, storing privileged and non-privileged policies separately, and evaluating them in the right order.

We should also consider the two intended operation modes for ClusterLink:

• The privileged operation mode, which we currently focus on, and in which a cluster admin installs a single instance of ClusterLink in a dedicated namespace and installs all relevant CRDs
• The non-privileged operation mode, in which every namespace owner may install a separate copy of ClusterLink to his/her namespace.

The privileges here should not be confused with the two tiers of access policies, though the two types of privileges are related.

This discussion comes to decide how the CRD interface looks like. We consider 3 options:

1. Two separate CRDs: a cluster-wide CRD for privileged policies, and a namespaced CRD for non-privileged policies.
2. Two separate CRDs, both namespaced: considering only privileged policies in ClusterLink's namespace and only non-privileged in user namespace
3. A single CRD for access policies. Access policies installed in the ClusterLink namespace are considered privileged. Access policies installed in user namespaces are considered non-privileged.

Option 1 is pretty much consistent with how K8s NetworkPolicies and AdminNetworkPolicies work. However, maintaining 2 CRDs complicates the controller, and requires more code in general, compared to Option 3.

@orozery , @elevran , @kfirtoledo