From dc2cf098b40ba90d213cbc840474770b80cd1abf Mon Sep 17 00:00:00 2001 From: Gerrit91 Date: Thu, 21 Mar 2024 15:44:20 +0100 Subject: [PATCH] Try integrate proposals from #187. --- docs/src/development/proposals/MEP4/README.md | 893 +++--------------- 1 file changed, 145 insertions(+), 748 deletions(-) diff --git a/docs/src/development/proposals/MEP4/README.md b/docs/src/development/proposals/MEP4/README.md index a14fe9b1dd..a41f89bf0c 100644 --- a/docs/src/development/proposals/MEP4/README.md +++ b/docs/src/development/proposals/MEP4/README.md @@ -6,22 +6,16 @@ In the past we decided to treat the metal-api as a "low-level API", i.e. the API does not specifically deal with projects and tenants. A user with editor access can for example assign machines to every project he desires, he can see all the machines available and can control them. -Even though we always tried to reserve the possibility to sell metal-stack as a bare metal machine provider only, the ultimate objective to us has always been to create an API for Kubernetes clusters. Hence, we tried to keep the metal-api code base as small as possible and we added resource scoping to a "higher-level API", the cloud-api, a component that is not open-source. From there, a user would be able to only see his own clusters. +Even though we always tried to reserve the possibility to sell metal-stack as a bare metal machine provider only, the ultimate objective to us has always been to create an API for Kubernetes clusters. Hence, we tried to keep the metal-api code base as small as possible and we added resource scoping to a "higher-level APIs". From there, a user would be able to only see his own clusters and IP addresses. -As time passed by, things changed: The metal-stack has become an open-source project and people are willing to adopt. Adopters don't have the cloud-api and they are interested in putting their own technologies on top of the metal-stack infrastructure. Introducing multi-tenancy to the metal-api is a serious chance of making our product better and more successful as it opens the door for: +As time passed by, things changed: The metal-stack has become an open-source project and people are willing to adopt. Adopters who want to put their own technologies on top of the metal-stack infrastructure don't have those "higher-level APIs", which requires them to implement them on their own. Introducing multi-tenancy to the metal-api is a serious chance of making our product better and more successful as it opens the door for: -- Becoming a "fully-featured" cloud provider with bare metal servers, networks and ip addresses treated as first-class citizens +- Becoming a "fully-featured" API - Letting untrusted / third-parties work with the API -- Narrowing down attack surfaces (through user roles and fine-grained permissions) +- Narrowing down attack surfaces (through roles and fine-grained permissions) - Gaining performance through resource scopes -- Using the Gardener Dashboard (no need for individual metal-stack deployments per tenant) - Discouraging people to implement their own scoping layers in front of the metal-stack -With these traits in place, wild ideas for the future come to mind: - -- Having a public offering of metal-stack -- Allowing users to bring their own partitions (managed control plane) - ## Table of Contents ```@contents @@ -29,7 +23,7 @@ Pages = ["README.md"] Depth = 5 ``` -# Requirements +## Requirements These are some general requirements / higher objectives that MEP-4 has to fulfill. @@ -41,6 +35,82 @@ These are some general requirements / higher objectives that MEP-4 has to fulfil - Project scoping (disallow resource access to resources of other projects, except a user acts "on-behalf") - Access tokens in self-service for technical user access +## Implementation + +We gathered a lot of knowledge while implementing the backend for metalstack.cloud. The goal is now to use the same technology and adopt that to the metal-api, this includes: + +- gRPC in combination with connectrpc +- OPA for making auth decisions +- REST HTTP only for OIDC login flows + +### API Definitions + +The API definitions should be located on a separate Github repository separate from the server implementation. The proposed repository location is: https://github.com/metal-stack/api. + +This repository contains the `proto3` specification of the exposed metal-stack api. This includes the messages, simple validations, services and the access permission to these services. The input parameters for the authorization in the backend are generated from the `proto3` annotations. + +Client implementations for the most relevant languages (go, python) are generated automatically. + +This api is divided into end user and admin access at the top level. The proposed APIs are: + +- `api.v2`: For end-user facing services +- `admin.v2`: For operators and controllers which need access to unscoped entities + +The methods of the API can have different role scope: + +- `tenant`: Tenant-scoped methods, e.g. project creation (tenant needs to be provided in the request payload) +- `project`: Project-scoped methods, e.g. machine creation (tenant needs to be provided in the request payload) +- `admin` Admin-scoped methods, e.g. unscoped tenant list or switch register + +And has methods with different visibility scopes: + +- `self`: Methods that only the logged in user can access, e.g. show permissions with the presented token +- `public`: Methods that do not require any specific authorization +- `private`: Methods that are not exposed + +### API Server + +The API server implements the services defined in the API and validates access to a method using OPA with the JWT tokens passed in the requests. The server is implemented using the connectrpc.com framework. + +The API server implements the login flow through OIDC. After successful authentication, the API server derives user permissions from the OIDC provider and issues a new JWT token which is passed on to the user. The tokens including the permissions are stored in a redis compatible backend. + +With these tokens, users can create Access Tokens for CI/CD or other use cases. + +JWT Tokens can be revoked by admins and the user itself. + +❓ Discuss: Should we create a new repository (https://github.com/metal-stack/api-server) or can we locate the new API in the existing metal-api project beneath v2. + +#### Relationship to the metal-api + +To allow consumers to migrate to the `v2` API gradually, the business logic of the metal-api must be maintained in parallel with the new `v2` api. + +To achieve this with it is required to extract the backend implementation, currently the `cmd/internal` package should be factored out to a consumable repository at `github.com/metal-stack/api-server/pkg/`. We will try to migrate the rethinkdb backend implementation to a generic approach during this effort. + +### Migration of the Consumers + +To allow consumers to migrate to the `v2` API gradually, both apis, the new and the old, are deployed in parallel. In the control-plane both apis are deployed side-by-side behind the ingress. `api.example.com` is forwarded to `metal-api` and `metal.example.com` is forwarded to the new `api-server`. + +There are a lot of consumers of metal-api, which need to be migrated: + +- ansible +- firewall-controller +- firewall-controller-manager +- gardener-extension-auth +- gardener-extension-provider-metal + - Do not point the secret bindings to a the shared provider secret in the seed anymore. Instead, use individual provider-secret containing project-scoped API access tokens in the Gardener project namespaces. +- machine-controller-manager-provider-metal +- metal-ccm +- metal-console +- metal-bmc +- metal-core +- metal-hammer +- metal-image-cache-sync +- metal-images +- metal-metrics-exporter +- metal-networker +- metalctl +- pixie + ## User Scenarios This section gathers a collection of workflows from the perspective of a user that we want to provide with the implementation of this proposal. @@ -52,115 +122,60 @@ A regular user wants to create a project to later maintain multiple resources in Requirements: Tenant was created - The user has to login before he can interact with the API. The login command will open a browser window as implemented already. - ``` - $ metalctl login - ``` + + ```bash + metalctl login + ``` + - A user is always associated with a tenant. - ``` - $ metalctl whoami - UserId: gerrit - Email: gerrit@gerrit.gerrit - Tenant: metal-stack - Issuer: https://dex.test.io/dex - Expires at Thu Jul 22 00:07:09 CEST 2021 - ``` -- The user already has permissions to create a project and can only see projects of his own tenant. - ``` - $ metalctl show-permissions - Project: - Roles: - metal-project-creator - Permissions: - metal.v2.project.list - metal.v2.project.get - metal.v2.project.delete - metal.v2.project.create - metal.v2.project.update - Resources: - * - ``` -- The user can also lookup the role binding, which actually gives him the permissions for project creation. - ``` - $ metalctl rolebinding list - ID NAME TENANT PROJECT - 40beeacc-4eba-4751-9c63-d82d81994d17 metal-project-creator metal-stack - 66d19904-5de2-4c36-9fd2-cfdafe5d8ae2 metal-admin - $ metalctl rolebinding describe 40beeacc-4eba-4751-9c63-d82d81994d17 - { - "id": "40beeacc-4eba-4751-9c63-d82d81994d17", - "name": "metal-project-creator", - "tenantid": "metal-stack", - "projectid": "", - "roles": [ - { - "id": "8c6b404e-02af-4498-a124-86c8eb49e12e", - "name": "metal-project-creator", - "permissions": [ - "metal.v2.project.list", - "metal.v2.project.get", - "metal.v2.project.delete", - "metal.v2.project.create", - "metal.v2.project.update" - ] - } - ], - "userids": ["gerrit"], - "resources": ["*"], - "oidcgroups: [] - } - ``` -- The user can create the project. - ``` - $ metalctl project create --name my-project - ... - ``` + + ```bash + $ metalctl whoami + UserId: gerrit + Email: gerrit@gerrit.gerrit + Tenant: metal-stack + Issuer: https://metal.example.com + ProjectRoles: + TenantRoles: + metal-stack-tenant-a: OWNER + Expires at Thu Jul 22 00:07:09 CEST 2024 + ``` + +- The user can create a project. + + ```bash + metalctl project create --name my-project + ``` + - The user has the option to direct all requests to a certain project (just like context switch) + + ```bash + metalctl ctx update my-ctx --default-project 793bb6cd-8b46-479d-9209-0fedca428fe1 + ``` + +- The user automatically acquired the owner role for the project he created. + + ```bash + $ metalctl whoami + UserId: gerrit + Email: gerrit@gerrit.gerrit + Tenant: metal-stack + Issuer: https://metal.example.com + ProjectRoles: + 793bb6cd-8b46-479d-9209-0fedca428fe1: OWNER + TenantRoles: + metal-stack-tenant-a: OWNER + Expires at Thu Jul 22 00:07:09 CEST 2024 ``` - $ metalctl project set 793bb6cd-8b46-479d-9209-0fedca428fe1 - You are now acting on project 793bb6cd-8b46-479d-9209-0fedca428fe1. - ``` -- The user automatically got the `metal-project-owner` default role for the project (auto default role association can also be turned off) including typical owner permissions. - ``` - $ metalctl show-permissions - Project: 793bb6cd-8b46-479d-9209-0fedca428fe1 - Roles: - metal-project-creator - metal-project-owner - Permissions: - metal.v2.image.list - metal.v2.image.get - metal.v2.image.get-latest - metal.v2.ip.list - metal.v2.ip.get - metal.v2.ip.create - metal.v2.ip.delete - metal.v2.ip.update - metal.v2.machine.list - metal.v2.machine.get - metal.v2.machine.create - metal.v2.machine.delete - metal.v2.machine.update - metal.v2.network.list - metal.v2.network.get - metal.v2.network.create-child - metal.v2.network.delete-child - metal.v2.network.update-child - metal.v2.project.list - metal.v2.project.get - metal.v2.project.delete - metal.v2.project.create - metal.v2.project.update - ...more typical user permissions... - Resources: - * - ``` + ### Machine Creation A regular user wants to create a machine resource. Requirements: Project was created, permissions are present -- By default, the user can see networks that were provided by the admin. +- The user can see networks that were provided by the admin. + ``` $ metalctl network ls ID NAME PROJECT PARTITION NAT SHARED PREFIXES IPS @@ -168,6 +183,7 @@ Requirements: Project was created, permissions are present tenant-super-network-fra-equ01 Project Super Network fra-equ01 false false 10.128.0.0/14  ● underlay-fra-equ01 Underlay Network fra-equ01 false false 10.0.0.0/16  ● ``` + - The user has to set the project scope first or provide `--project` flags for all commands. ``` $ metalctl project set 793bb6cd-8b46-479d-9209-0fedca428fe1 @@ -215,117 +231,32 @@ Admins should be able to see "everything", even resources of tenant's regular us 1. Admins have set up the environment and are likely to have access to all the databases, such they were able to compromise the environment anyway - The user has permissions to create a project. - ``` - $ metalctl show-permissions - Project: - Roles: - metal-admin - Permissions: - metal.v2.* - Resources: - * - ``` -- The admin user can see all machines there are. - ``` - $ metalctl machine ls -A - ID LAST EVENT WHEN AGE HOSTNAME PROJECT SIZE IMAGE PARTITION - 00000000-0000-0000-0000-ac1f6b7befb2 Phoned Home 4s 50d 4h storage-2 a0a4c959-b191-4b4d-8483-951ccc2ff3f1 c1-xlarge-x86 Centos 7 20210415 fra-equ01 - 00000000-0000-0000-0000-0025905f207a Phoned Home 5s 103d 15m shoot--phjjb...-firewall-0f96d 5820c4e7-fbd4-4e4b-a40b-2b83eb34bbe1 s1-large-x86 Firewall 2 Ubuntu 20210304 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7b77c8 Phoned Home 5s 65d 8h shoot--test-...-firewall-cf20f 00000000-0000-0000-0000-000000000001 c1-xlarge-x86 Firewall 2 Ubuntu 20210316 fra-equ01 - 00000000-0000-0000-0000-ac1f6bd390a6 Waiting 59s fra-equ01 - 00000000-0000-0000-0000-0025905f2032 Phoned Home 4s 8d 1h shoot--p5c7f...-firewall-c72be 5410a72d-14c1-424e-9896-71c7ee84d393 s1-large-x86 Firewall 2 Ubuntu 20210606 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7aeb90 Phoned Home 4s 50d 4h storage-0 a0a4c959-b191-4b4d-8483-951ccc2ff3f1 c1-xlarge-x86 Centos 7 20210415 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7aeb76 Phoned Home 4s 1d 16h shoot--pskqm...p-0-576df-6f552 b5f26a3b-9a4d-48db-a6b3-d1dd4ac4abec c1-xlarge-x86 Debian 10 20210719 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7d7efa Phoned Home 56s 105d 2h shoot--phjjb...p-0-77fc5-sr5zn 5820c4e7-fbd4-4e4b-a40b-2b83eb34bbe1 s2-xlarge-x86 Debian 10 20210316 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7b77cc Phoned Home 4s 50d 4h storage-1 a0a4c959-b191-4b4d-8483-951ccc2ff3f1 c1-xlarge-x86 Centos 7 20210415 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7d7e32 Phoned Home 4s 105d 2h shoot--phjjb...p-0-77fc5-qldng 5820c4e7-fbd4-4e4b-a40b-2b83eb34bbe1 s2-xlarge-x86 Debian 10 20210316 fra-equ01 - 11111111-2222-3333-4444-aabbccddeeff Waiting 6s s1-large-broken-x86 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7bed26 Phoned Home 49s c1-xlarge-x86 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7b77e4 Phoned Home 4s 1d 4h shoot--p9sk4...-firewall-5b3b7 7b29e8ea-e4c8-4aaa-9e42-70dd39d20108 s1-large-x86 Firewall 2 Ubuntu 20210606 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7bed30 Phoned Home 4s 105d 2h shoot--test-...ker-5f9b5-ltk9g 00000000-0000-0000-0000-000000000001 c1-xlarge-x86 Ubuntu 20.04 20210316 fra-equ01 - 00000000-0000-0000-0000-ac1f6bd3909c Phoned Home 4s 113d 31m storage-firewall a0a4c959-b191-4b4d-8483-951ccc2ff3f1 c1-large-x86 Firewall 2 Ubuntu 20210316 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7beb80 Phoned Home 5s 1d 16h shoot--p5c7f...p-0-5968d-gtk95 5410a72d-14c1-424e-9896-71c7ee84d393 c1-xlarge-x86 Debian 10 20210719 fra-equ01 - 00000000-0000-0000-0000-ac1f6b7d7dc6 Phoned Home 4s 105d 2h shoot--phjjb...p-0-77fc5-k5wp6 5820c4e7-fbd4-4e4b-a40b-2b83eb34bbe1 s2-xlarge-x86 Debian 10 20210519 fra-equ01 - 256b1c00-be6d-11e9-8000-3cecef22b288 Phoned Home 5s 8d 1h shoot--p5c7f...-firewall-9c9af 5410a72d-14c1-424e-9896-71c7ee84d393 n1-medium-x86 Firewall 2 Ubuntu 20210606 fra-equ01 - 6f440a00-be4d-11e9-8000-3cecef22f91c Phoned Home 4s 6d 42m shoot--pskqm...-firewall-44522 b5f26a3b-9a4d-48db-a6b3-d1dd4ac4abec n1-medium-x86 Firewall 2 Ubuntu 20201214 fra-equ01 - 00000000-0000-0000-0000-ac1f6bd39026 Waiting 56s c1-large-x86 fra-equ01 - 00000000-0000-0000-0000-ac1f6bd390b2 Phoned Home 4s 1d 14h shoot--p5c7f...p-0-b5f87-pjpk2 5410a72d-14c1-424e-9896-71c7ee84d393 c1-large-x86 Debian 10 20210719 fra-equ01 - ``` - -### Create Tenant and Project on Behalf -- The user requires permissions to create a tenant and project, this can be achieved through a role binding (e.g. provided by the admin): + ```bash + $ metalctl whoami + UserId: gerrit + Email: gerrit@gerrit.gerrit + Tenant: metal-stack + Issuer: https://metal.example.com + ProjectRoles: + 793bb6cd-8b46-479d-9209-0fedca428fe1: OWNER + TenantRoles: + metal-stack-tenant-a: OWNER + Expires at Thu Jul 22 00:07:09 CEST 2024 + AdminRoles: + *: OWNER ``` - $ metalctl rolebinding list - ID NAME TENANT PROJECT - 40beeacc-4eba-4751-9c63-d82d81994d17 metal-project-creator metal-stack - 66d19904-5de2-4c36-9fd2-cfdafe5d8ae2 metal-admin - 45a72f32-5acb-47cd-8ac9-49764587ad46 metal-tenant-creator new-tenant - $ metalctl rolebinding describe 45a72f32-5acb-47cd-8ac9-49764587ad46 - { - "id": "45a72f32-5acb-47cd-8ac9-49764587ad46", - "name": "metal-tenant-creator", - "tenantid": "new-tenant", - "projectid": "", - "roles": [ - { - "id": "17fa35be-8c73-46fe-8aaf-284b9531e33d", - "name": "metal-tenant-creator", - "permissions": [ - "metal.v2.tenant.list", - "metal.v2.tenant.get", - "metal.v2.tenant.delete", - "metal.v2.tenant.create", - "metal.v2.tenant.update", - "metal.v2.project.list", - "metal.v2.project.get", - "metal.v2.project.delete", - "metal.v2.project.create", - "metal.v2.project.update" - ] - } - ], - "userids": ["gerrit"], - "resources": ["*"], - "oidcgroups: [] - } - $ metalctl whoami - UserId: gerrit - Email: gerrit@gerrit.gerrit - Tenant: metal-stack - Issuer: https://dex.test.io/dex - Expires at Thu Jul 22 00:07:09 CEST 2021 +- The admin user can see all machines there are. - $ metalctl show-permissions - Project: - Roles: - metal-tenant-creator - Permissions: - metal.v2.project.list - metal.v2.project.get - metal.v2.project.delete - metal.v2.project.create - metal.v2.project.update - metal.v2.tenant.list - metal.v2.tenant.get - metal.v2.tenant.delete - metal.v2.tenant.create - metal.v2.tenant.update - Resources: - * - ``` -- The tenant can now be created for the tenant `new-tenant` even though the logged in tenant is from `metal-stack`. - ``` - $ metalctl tenant create --name new-tenant - ... - $ metalctl project create --name new-project + ```bash + $ metalctl admin machine ls ... ``` -### Custom Role Permissions +### Limited API Access Through Technical Users -A user creates a custom role `ci-builder` and a project token for it: +A user creates a custom role `ci-builder` and a project token for it. A user cannot elevate his permissions, which needs to be prevented by the API. - The user has permissions to create a role. ``` @@ -410,29 +341,6 @@ A user creates a custom role `ci-builder` and a project token for it: ❓ Discuss: Should there be a `masterdatactl` (something like an extension of `metalctl`) to maintain tenants, projects, quotas, users, roles and annotations for accounting? -## Centralized vs. Decentralized User Management - -Highly regulated environments usually maintain a central place where permissions are requested and approved. This use-case has to be considered, too, for the implemenation of this proposal. - -- TBD (Dex, Keycloak, User Management) -- Initial user workflow -- Connect to external user management? - -Advantages of Decentralized: - -- Much easier to adopt -- More developer-friendly -- Better to unit test - -Advantages of Centralized: - -- Compliance (regulatory compliance) -- Defined permission workflow - -❓ Discuss: If we go with the decentralized approach, how can we marry this with an existing centralized environment? - -Idea: Create a group-rolebinding-controller similar to a Kubernetes controller we already have that creates role bindings that associates OIDC groups / users with roles. Allow external user backends like LDAP. Auto-create users (and tenants?) from OIDC on login. - ### Scopes for Resources The admins / operators of the metal-stack should be able to provide _global_ resources that users are able to use along with their own resources. In particular, users can view and use _global_ resources, but they are not allowed to create, modify or delete them. @@ -465,514 +373,3 @@ Where possible, users should be capable of creating their own resource entities. !!! info Example: A user can make use of the file system layouts provided by the admins, but can also create own layouts. Same applies for images. As soon as a user creates own resources, the user takes over the responsibility for the machine provisioning to succeed. - -### Auditing - -With the new API we also have the chance of introducing API auditing throughout the resources. - -The auditing can be configured with different emitters, like: - -- Rethinkdb (can be accessed through `metalctl audit ls`) -- Splunk -- Elasticsearch -- ... - -And contain: - -- User information (name, mail) -- Requested endpoint path -- Permissions -- OPA decision -- Status code - -## Implementation - -This section contains implementation details for fulfilling the requirements section. - -- Introducing API V2 -- Introducing OPA -- Common filter queries for all resources - -### Data Model - -This subsection contains definitions for the entities that have to be created or updated. - -```golang -package v2 - -import "time" - -type Meta struct { - ID string `json:"id" description:"the unique ID of this entity" unique:"true" required:"true"` - Kind string `json:"kind,omitempty"` - ApiVersion string `json:"apiversion,omitempty"` - Version int64 `json:"version,omitempty"` - CreatedTime *time.Time `json:"created_time,omitempty"` - UpdatedTime *time.Time `json:"updated_time,omitempty"` - Annotations map[string]string `json:"annotations,omitempty"` - Labels []string `json:"labels,omitempty"` -} - -type Describable struct { - Name *string `json:"name,omitempty" description:"a readable name for this entity" optional:"true"` - Description *string `json:"description,omitempty" description:"a description for this entity" optional:"true"` -} - -type Common struct { - Meta - Describable -} - -// Role is a struct that groups permissions. -// When it points to a project, this role is project specific otherwise it's considered global. -type Role struct { - Common - Active bool `json:"active" description:"whether this role is currently active or not"` - ProjectID string `json:"projectid" description:"the project this role belongs to, global if empty"` - Permissions []string `json:"permissions" description:"permissions that this role consists of"` -} - -// RoleBinding associates roles with subjects. -type RoleBinding struct { - Common - TenantID string `json:"tenantid" description:"the tenant this role binding belongs to"` - ProjectID string `json:"projectid" description:"the project this role binding belongs to"` - RoleIDs []string `json:"roleids" description:"the roles that this binding associates with the subjects"` - UserIDs []string `json:"userids" description:"the users that this binding associates with the roles"` - TokenIDs []string `json:"tokenids" description:"the project tokens that this binding associates with the roles"` - ResourceIDs []string `json:"resourceids" description:"the resources that this binding associates with the roles, be aware that these are flattened when multiple role bindings apply"` - OIDCGroups []string `json:"oidcgroups" description:"the oidc group claims that this binding associates with the roles"` -} - -// User is a human consumer of the API. -type User struct { - Identifiable - Name *string `json:"name,omitempty" description:"a readable name for this entity" optional:"true"` - EMail *string `json:"mail,omitempty" description:"a public mail address of this user" optional:"true"` - AvatarURL *string `json:"avatarurl,omitempty" description:"a url to an avatar image" optional:"true"` - Bio *string `json:"bio,omitempty" description:"a brief biography of this user" optional:"true"` - LogonType string `json:"logontype" description:"the type how this user logs in [password|oidc]"` -} - -type Project struct { - Common - TenantID string `json:"tenantid,omitempty"` - Quotas *QuotaSet `json:"quotas,omitempty"` -} - -type ProjectToken struct { - Common - ProjectID string `json:"projectid" description:"the project that this token belongs to"` - Creator string `json:"creator" description:"documents the user id of the creator of this token"` -} - -type Tenant struct { - Meta *Meta `json:"meta,omitempty"` - Name string `json:"name,omitempty"` - Description string `json:"description,omitempty"` - Quotas *QuotaSet `json:"quotas,omitempty"` -} -``` - -We will also need some new configuration options: - -```golang -type Configuration struct { - // DefaultRoles are roles that will be automatically applied to the creator for created project through a role binding - DefaultRoles []string - // DefaultProjectQuotas are quotas that will be automatically applied to new projects - DefaultProjectQuotas *QuotaSet - // DefaultTenantQuotas are quotas that will be automatically applied to new tenants - DefaultTenantQuotas *QuotaSet - // AutoCreateOIDCUsers will automatically create a user on OIDC login if the user doesn't exist in the database - AutoCreateOIDCUsers bool -} -``` - -![](data_model.svg) - -### API v2 - -We will need an API v2, which gives all components and users time to slowly change to the new API while maintaining the current functionality of the endpoints in the metal-api. - -- Introduce new paths that allow decisions about access very early in the game (only from endpoint and the user permissions) -- Get rid off layer HMAC auth (replaced by project tokens) -- Move internal endpoints to gRPC (e.g. register machine), remove daisy-chained swagger client from metal-core -- Remove request wrapper structs from metal-go (https://github.com/metal-stack/metal-go/issues/33) - -We can start working on this for every resource individually. The work will potentially turn into a new shared layer with common functionality between v1/v2 (e.g. code for machine creation) with the API-specific packages only becoming responsible for mapping requests. - -### Resource Paths - -Suggested convention for path layout: `/:api-version/:resource/:id[/][?project-id=:projectid][?tenant-id=:tenantid]` - -The project and tenant ids become part of the path layout because extracting those fields from the request payload before handing it to the business logic is hard to achieve. - -The `tenant-id` param is only used for resources that have no project scopes, e.g. `projects`. - -In order to create the input for OPA, we need a call to the backend on the user + project ID. The response contains the permissions for this user. - -### Permission Wildcards - -Permissions should be able to match wildcards. - -Example for admin user: - -``` -$ metalctl project show-permissions -Roles: - metal-project-owner -Permissions: - metal.v2.* -Resources: - * -``` - -#### OPA Example - -Let's imagine a user issuing the following request: - -``` -metalctl image describe ubuntu-20.04.20210606 -``` - -This will call `GET /v2/image/ubuntu-20.04.20210606?project-id=my-project`. - -Here is an example rule set for OPA (you can try it on the [rego playground](https://play.openpolicyagent.org/)): - -``` -package api.v2.metalstack.io.authz - -permissions = { - "metal.v2.image.get", - "metal.v2.image.list", - "metal.v2.image.create", - "metal.v2.image.update", - "metal.v2.image.delete", - "metal.v2.admin.image.create", - "metal.v2.admin.image.update", - "metal.v2.admin.image.delete" -} - -default decision = {"allow": false} - -match(e) { - glob.match(input.backend.permissions, [","], e) -} - -decision = {"allow": true} { - match(e.permission) -} - -decision = {"allow": false, "reason": reason} { - not match(e.permission) - reason := sprintf("missing permission on %s", [e.permission]) -} - -e = {"permission": permissions["metal.v2.image.get"]} { - some id - input.request.method = "GET" - input.request.path = ["v2", "image", id] - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.image.list"]} { - input.request.method = "GET" - input.request.path = ["v2", "image"] -} - -e = {"permission": permissions["metal.v2.image.create"]} { - some id - input.request.method = "PUT" - input.request.path = ["v2", "image", id] - input.request.project != "" - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.image.update"]} { - some id - input.request.method = "POST" - input.request.path = ["v2", "image", id] - input.request.project != "" - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.image.delete"]} { - some id - input.request.method = "DELETE" - input.request.path = ["v2", "image", id] - input.request.project != "" - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.admin.image.create"]} { - some id - input.request.method = "PUT" - input.request.path = ["v2", "image", id] - input.request.project == "" - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.admin.image.update"]} { - some id - input.request.method = "POST" - input.request.path = ["v2", "image", id] - input.request.project == "" - glob.match(input.backend.resources, [","], id) -} - -e = {"permission": permissions["metal.v2.admin.image.delete"]} { - some id - input.request.method = "DELETE" - input.request.path = ["v2", "image", id] - input.request.project == "" - glob.match(input.backend.resources, [","], id) -} -``` - -OPA input: - -```json -{ - "token": { - "user": "gerrit", - "tenant": "metal-stack" - }, - "request": { - "method": "GET", // from request - "path": ["v2", "image", "00000000-0000-0000-0000-ac1f6b7befb2"], // split on request path - "project": "793bb6cd-8b46-479d-9209-0fedca428fe1" // from request query param - }, - "backend": { - // composed from the user's general permissions and the permissions of the project in the request's query param - "permissions": "{metal.v2.image.*}", - "resources": "{*}" - } -} -``` - -OPA output: - -```json -{ - "decision": { - "allow": true - }, - "e": { - "permission": "metal.v2.image.get" - }, - "permissions": [ - "metal.v2.admin.image.create", - "metal.v2.admin.image.delete", - "metal.v2.admin.image.update", - "metal.v2.image.create", - "metal.v2.image.delete", - "metal.v2.image.get", - "metal.v2.image.list", - "metal.v2.image.update" - ] -} -``` - -### Endpoints - -This section defines the new endpoints for the API. - -❓ Should we make "list and search" a single endpoint? - -#### File System Layout - -| Endpoint | Method | Permission | Notes | -| :---------------------------------------------- | :----- | :------------------------------------- | :-------------------------------------------- | -| /v2/filesystemlayout | GET | metal.v2.filesystemlayout.list | | -| /v2/filesystemlayout | POST | metal.v2.filesystemlayout.search | New! | -| /v2/filesystemlayout/:id | GET | metal.v2.filesystemlayout.get | | -| /v2/filesystemlayout/:id | PUT | metal.v2.admin.filesystemlayout.create | | -| /v2/filesystemlayout/:id | POST | metal.v2.admin.filesystemlayout.update | | -| /v2/filesystemlayout/:id | DELETE | metal.v2.admin.filesystemlayout.delete | | -| /v2/filesystemlayout/:id?project-id=:project-id | PUT | metal.v2.filesystemlayout.create | | -| /v2/filesystemlayout/:id?project-id=:project-id | POST | metal.v2.filesystemlayout.update | | -| /v2/filesystemlayout/:id?project-id=:project-id | DELETE | metal.v2.filesystemlayout.delete | | -| /v2/filesystemlayout/:id/matches | POST | metal.v2.filesystemlayout.matches | Is there a better name, can this be combined? | -| /v2/filesystemlayout/try | POST | metal.v2.filesystemlayout.try | Is there a better name, can this be combined? | - -#### Firewall - -| Endpoint | Method | Permission | Notes | -| :------------------------ | :----- | :-------------------------------- | :------------------------ | -| /v2/firewall | GET | metal.v2.firewall.list | | -| /v2/firewall | POST | metal.v2.firewall.search | Was `/find` before | -| /v2/firewall/:id | GET | metal.v2.firewall.get | | -| /v2/firewall/:id/allocate | POST | metal.v2.firewall.create-specific | New, now equivalent to IP | -| /v2/firewall/allocate | POST | metal.v2.firewall.create | | - -❓ Should there be the same endpoints as for machines, too, re-using the same code? - -#### Firmware - -| Endpoint | Method | Permission | Notes | -| :------------------------------------------ | :----- | :----------------------------- | :---- | -| /v2/firmware | GET | metal.v2.firmware.list | | -| /v2/firmware/:kind/:vendor/:board/:revision | PUT | metal.v2.admin.firmware.create | | -| /v2/firmware/:kind/:vendor/:board/:revision | DELETE | metal.v2.admin.firmware.delete | | - -Users should not be able to upload / delete own firmware as it can mess up the environment. - -This resource is a little different because it uses an S3 bucket for persistence and not the regular metal-db. - -#### Health - -| Endpoint | Method | Permission | Notes | -| :--------- | :----- | :------------------ | :---- | -| /v2/health | GET | metal.v2.health.get | | - -❓ This endpoint was public before (such that health checks can easily be performed through K8s deployments). To reduce attack surface this should require a permission. - -#### IP - -| Endpoint | Method | Permission | Notes | -| :------------------ | :----- | :-------------------------- | :-------------------- | -| /v2/ip | GET | metal.v2.ip.list | | -| /v2/ip | POST | metal.v2.ip.search | Was `/find` before | -| /v2/ip/:id | GET | metal.v2.ip.get | | -| /v2/ip/:id | POST | metal.v2.ip.update | now has `:id` in path | -| /v2/ip/:id | DELETE | metal.v2.ip.delete | Was `/free` before | -| /v2/ip/:id/allocate | PUT | metal.v2.ip.create-specific | | -| /v2/ip/allocate | POST | metal.v2.ip.create | | - -#### Image - -| Endpoint | Method | Permission | Notes | -| :----------------------------------- | :----- | :-------------------------- | :---- | -| /v2/image | GET | metal.v2.image.list | | -| /v2/image | POST | metal.v2.image.search | New! | -| /v2/image/:id | GET | metal.v2.image.get | | -| /v2/image/:id/latest | GET | metal.v2.image.get-latest | | -| /v2/image/:id | PUT | metal.v2.admin.image.create | | -| /v2/image/:id | POST | metal.v2.admin.image.update | | -| /v2/image/:id | DELETE | metal.v2.admin.image.delete | | -| /v2/image/:id?project-id=:project-id | PUT | metal.v2.image.create | | -| /v2/image/:id?project-id=:project-id | POST | metal.v2.image.update | | -| /v2/image/:id?project-id=:project-id | DELETE | metal.v2.image.delete | | - -#### Machine - -| Endpoint | Method | Permission | Notes | -| :----------------------------------------------- | :----- | :----------------------------------- | :--------------------------------------------------- | -| /v2/machine | GET | metal.v2.machine.list | | -| /v2/machine | POST | metal.v2.machine.search | Was `/find` before | -| /v2/machine/:id | GET | metal.v2.machine.get | | -| /v2/machine/:id | POST | metal.v2.machine.update | now has `:id` in path | -| /v2/machine/:id | DELETE | metal.v2.machine.delete | Was `/free` before | -| /v2/machine/:id/event | Get | metal.v2.machine.list-events | | -| /v2/machine/:id/state | Get | metal.v2.machine.get-state | Was `/free` before | -| /v2/machine/:id/allocate | PUT | metal.v2.machine.create-specific | | -| /v2/machine/:id/console-password | POST | metal.v2.machine.console-password | Was `/consolepassword` before, now has `:id` in path | -| /v2/machine/:id/update-firmware | POST | metal.v2.machine.update-firmware | `:id` position has moved | -| /v2/machine/:id/remove | DELETE | metal.v2.machine.admin.delete | | -| /v2/machine/:id/reinstall | POST | metal.v2.machine.reinstall | | -| /v2/machine/:id/power/bios | POST | metal.v2.machine.power.boot-bios | | -| /v2/machine/:id/power/disk | POST | metal.v2.machine.power.boot-disk | | -| /v2/machine/:id/power/pxe | POST | metal.v2.machine.power.boot-pxe | | -| /v2/machine/:id/power/off | POST | metal.v2.machine.power.off | | -| /v2/machine/:id/power/on | POST | metal.v2.machine.power.on | | -| /v2/machine/:id/power/reset | POST | metal.v2.machine.power.reset | | -| /v2/machine/:id/power/chassis-identify-led-state | POST | metal.v2.machine.power.led-state | Now beneath `/power` | -| /v2/machine/:id/power/chassis-identify-led-on | POST | metal.v2.machine.power.led-state-on | | -| /v2/machine/:id/power/chassis-identify-led-off | POST | metal.v2.machine.power.led-state-off | | -| /v2/machine/allocate | POST | metal.v2.machine.create | | -| /v2/machine/ipmi | GET | metal.v2.machine.ipmi.list | | -| /v2/machine/ipmi | POST | metal.v2.machine.ipmi.search | Was `/find` before | -| /v2/machine/ipmi/:id | GET | metal.v2.machine.ipmi.get | | - -❗ Removed: - -- `/ipmi` bmc-catcher reporting will be moved to gRPC API -- `/register` will be moved to gRPC API -- `/event` additions will be moved to gRPC API -- `/finalize-allocation` will be moved to gRPC API -- `/abort-reinstall` will be moved to gRPC API - -#### Network - -| Endpoint | Method | Permission | Notes | -| :------------------------------------- | :----- | :---------------------------- | :---- | -| /v2/network | GET | metal.v2.network.list | | -| /v2/network | POST | metal.v2.network.search | New! | -| /v2/network/:id | GET | metal.v2.network.get | | -| /v2/network/:id | PUT | metal.v2.admin.network.create | | -| /v2/network/:id | POST | metal.v2.admin.network.update | | -| /v2/network/:id | DELETE | metal.v2.admin.network.delete | | -| /v2/network/:id?project-id=:project-id | PUT | metal.v2.network.create-child | | -| /v2/network/:id?project-id=:project-id | POST | metal.v2.network.update-child | | -| /v2/network/:id?project-id=:project-id | DELETE | metal.v2.network.delete-child | | - -#### Partition - -| Endpoint | Method | Permission | Notes | -| :--------------------- | :----- | :------------------------------ | :---- | -| /v2/partition | GET | metal.v2.partition.list | | -| /v2/partition | POST | metal.v2.partition.search | New! | -| /v2/partition/:id | GET | metal.v2.partition.get | | -| /v2/partition/:id | PUT | metal.v2.admin.partition.create | | -| /v2/partition/:id | POST | metal.v2.admin.partition.update | | -| /v2/partition/:id | DELETE | metal.v2.admin.partition.delete | | -| /v2/partition/capacity | GET | metal.v2.partition.capacity | | -#### Project - -TBD - -#### Size - -| Endpoint | Method | Permission | Notes | -| :--------------------- | :----- | :------------------------- | :------------------------------------- | -| /v2/size | GET | metal.v2.size.list | | -| /v2/size | POST | metal.v2.size.search | New! | -| /v2/size/:id | GET | metal.v2.size.get | | -| /v2/size/:id | PUT | metal.v2.admin.size.create | | -| /v2/size/:id | POST | metal.v2.admin.size.update | | -| /v2/size/:id | DELETE | metal.v2.admin.size.delete | | -| /v2/size/from-hardware | GET | | Is there a better name? Who uses this? | - -#### Switch - -This really is a pure admin-resource. - -| Endpoint | Method | Permission | Notes | -| :------------- | :----- | :------------------------- | :---- | -| /v2/switch | GET | metal.v2.admin.size.list | | -| /v2/switch | POST | metal.v2.admin.size.search | New! | -| /v2/switch/:id | GET | metal.v2.admin.size.get | | - -❗ Removed: - -- All non-read endpoints, can be moved to gRPC API - -#### Tenant - -TBD - -#### Version - -| Endpoint | Method | Permission | Notes | -| :---------- | :----- | :------------------- | :---- | -| /v2/version | GET | metal.v2.version.get | | - -❓ This endpoint was public before (such that it can easily be shown in badges). To reduce attack surface this should require a permission. - -### Scoping - -We want a common scoping logic on the database for all resources. Therefore, all resources must use the same database fields that can be used for filtering. - -| Scope | Field Name | -| :------- | :--------- | -| Tenant | tenant_id | -| Project | project_id | -| Resource | id | - -## Migration Path - -### Migrate Existing Gardener Projects to New API - -- Do not point the secret bindings to a the shared provider secret in a partition. Create an individual provider-secret for the logged in tenant. The Gardener needs to use this tenant-specific provider secret to talk to the metal-api, do not give the Gardener HMAC access anymore. -- The provider secret partition mapping can be removed from the cloud-api config and from the deployment