spread/google: add support for service accounts #187

bboozzoo · 2024-06-14T09:30:30Z

Add support for attaching service accounts to the instances created in GCP.

And we can now automatically get access to Ubuntu Pro features (if your account permits it):

google:ubuntu-pro-22.04-64 .../mini/hello# pro status
SERVICE          ENTITLED  STATUS       DESCRIPTION
anbox-cloud      yes       disabled     Scalable Android in the cloud
esm-apps         yes       enabled      Expanded Security Maintenance for Applications
esm-infra        yes       enabled      Expanded Security Maintenance for Infrastructure
fips-preview     yes       disabled     Preview of FIPS crypto packages undergoing certification with NIST
fips-updates     yes       disabled     FIPS compliant crypto packages with stable security updates
livepatch        yes       enabled      Canonical Livepatch service
usg              yes       disabled     Security compliance and audit tools

For a list of all Ubuntu Pro services, run 'pro status --all'
Enable services with: pro enable <service>

                Account: snapd-spread
           Subscription: snapd-spread
            Valid until: Fri Dec 31 00:00:00 9999 UTC
Technical support level: essential

bboozzoo · 2024-06-14T11:27:15Z

README.md

+backends:
+    google:
+        key: $(HOST:echo $GOOGLE_JSON_FILENAME)
+        google-service-account: $(HOST:echo $GOOGLE_SERVICE_ACCOUNT)


perhaps this could be a boolean switch? The problem is that even through the key type is service_account, the actual role associated with the identity of the request may not have the permissions to attach service account to instances. As we learned the hard way, what you need is compute.instances.setServiceAccount permission, which wasn't set for spread user role in our GCP project.

Alternative I discussed with @sergiocazzolato is to maybe have this as a property of the actualy system eg. attach-service-account: true, which then defaults to using the service account client_email present in the client authentication file.

thp-canonical · 2024-06-14T13:12:21Z

spread/project.go

@@ -82,6 +82,9 @@ type Backend struct {

 	Priority OptionalInt
 	Manual   bool
+
+	// Only relevant for the Google backend
+	GoogleServiceAccount string `yaml:"google-service-account"`


Looking at existing keys above, this would be the first that namespaces the name (prefix Google / google-) based on the backend, existing backend-specific fields (Allocate, Discard, Memory, Plan, Location, Storage) all don't have prefixes that mention their backend.

..which would suggest adding it as ServiceAccount string (with service-account in YAML) next to the existing google-specific members (Plan, Location, Storage)?

The field has now been moved to the system level

thp-canonical · 2024-06-14T13:15:47Z

spread/google.go

+		// account to the instance
+		params["serviceAccounts"] = []googleParams{{
+			"email":  serviceAccount,
+			"scopes": []string{"https://www.googleapis.com/auth/cloud-platform"},


Do the scopes also need to be parameterized (fewer/more/different scopes), or is that the only sensible value to be used when using Spread?

AFAIU it should match the same default scope we use when setting up the client https://github.com/snapcore/spread/blob/ded9133cdbceaf01f8a1c9decf6ff9ea56e194d6/spread/google.go#L776

thp-canonical · 2024-06-14T13:16:18Z

README.md

@@ -894,6 +894,22 @@ Service accounts are best as they can be further constrained and not be
 associated with your overall authenticated access. Do not ship your own
 credentials to remote systems.

+In order to attach a service account to the instance created by spread use the


Maybe make "service account" a link to documentation from Google Cloud what a service account is and when one would use it?

This is mentioned in the paragraph just above.

Add support for attaching service accounts to the instances created in GCP. Signed-off-by: Maciej Borzecki <[email protected]>

bboozzoo · 2024-06-24T14:39:41Z

@thp-canonical @sergiocazzolato @ZeyadYasser I have reworked this a bit, please have a look. The backend level field was confusing and I've changed this completely. Since there is no reasonable scenario in which the service account would use a different 'email' address than one in the key, the actual value gets derived from the authentication key. The process is identical to what golang.org/x/oauth2/google package does. Now, a system level property governs whether you want to attach a service account to an instance or not. This distinctions is still needed, as the IAM role assined to the account may not permit to attach service accounts, and so just attempting to attach it by default one would not be able to create any instances if the permissions scope was intentionally limited.

zyga

Looks good, two comments.

zyga · 2024-06-24T18:32:03Z

README.md

+        key: $(HOST:echo $GOOGLE_JSON_FILENAME)
+        ...
+        systems:
+            - system-with-service-account:


I'd make this more realistic or more clarly a placeholder text.

zyga · 2024-06-24T18:34:01Z

spread/google.go

+				// already a raw JSON blob
+				raw = []byte(p.backend.Key)
+			} else {
+				raw, err = ioutil.ReadFile(p.backend.Key)


I think this is just os.ReadFile now

spread is go 1.13 😞

ZeyadYasser

I am bit worried of giving too much privileges (see, edit, configure, and delete Google Cloud data) [1] to test instances. I feel that explicitly passing a dedicated service account/scopes could be safer.

I might be missing context here that would make this a non-issue, please correct me if I am wrong.

[1] https://developers.google.com/identity/protocols/oauth2/scopes#ml

ZeyadYasser · 2024-06-24T21:00:54Z

spread/google.go

+		// account to the instance
+		params["serviceAccounts"] = []googleParams{{
+			"email":  p.serviceAccount,
+			"scopes": []string{"https://www.googleapis.com/auth/cloud-platform"},


This scope feels too powerful to give to an instance running, Do we know the minimum scopes we need for ubuntu pro?
if the scope should match the default scope for the key, would it make sense to avoid attaching the service account from the auth key to the machine and instead explicitly pass a less privileged (setup by IS) service account with minimum set of scopes needed?

This is the exact same scope we use when creating GCP client in https://github.com/snapcore/spread/blob/master/spread/google.go#L776 it's also the scope of the token, but the actual permissions. What you can do is not driven by the token scope but by the IAM role assigned to your account. In fact, your application default account, or even the github account we use in default in snapd spred are unable to attach the instances because they lack the necessary permissions in their assigned roles roles.

This is the exact same scope we use when creating GCP client in https://github.com/snapcore/spread/blob/master/spread/google.go#L776

But the GCP client is spread, right? and the key owner is the one invoking spread (consciously) so it is expected and not a security issue here.

it's also the scope of the token, but the actual permissions.

Yes, but since the user can run spread that means their attached svc account can do CRUD operations on GCP instances.

My worry, is giving this power to an Ubuntu instance with lots of installed software (large attack surface), all of which would be able to do CRUD operations on the company's internal GCP account. The attack surface grows from just spread's code base, to Ubuntu + a bunch of installed software.

Again, I might be missing a puzzle piece here or some context that could make this concern a non issue.

You cannot attach a service account unless the role assigned to your account is allowed to do so. The application default account all of use does not have that. The service account used by spread runners does not have that either. In fact you even won't be able to start an instance with the service account attached, all you get back is:

cannot allocate new Google server for ubuntu-pro-22.04-64: required 'compute.instances.setServiceAccount' permission for 'projects/snapd-spread/zones/us-east1-b/instances/jun250926-154636'

You are not giving power to anyone, unless you generate a key for them in GCP admin console and give them the key they can use to pass to spread.

I was getting my concerns from https://cloud.google.com/iam/docs/attach-service-accounts:

When the resource needs to access other Google Cloud services and resources, it uses its attached service account as its identity. For example, if you attach a service account to a Compute Engine instance, and the applications on the instance use a client library to call Google Cloud APIs, those applications automatically authenticate as the attached service account.

but since you tested it on launched machine, then I am confusing this with something else.

Yes, that's the whole point of attaching a service account to the machine. This way we as long as the account we use entitled to use Ubuntu Pro, the ubuntu-advantage-client will be able to automatically enable relevant support at boot time. There is no need to share or carry yet another token.

bboozzoo force-pushed the bbboozzoo/gcp-service-account-upstream branch from a1696e0 to 98757d7 Compare June 14, 2024 09:41

bboozzoo commented Jun 14, 2024

View reviewed changes

thp-canonical reviewed Jun 14, 2024

View reviewed changes

spread/google: add support for service accounts

07c7be1

Add support for attaching service accounts to the instances created in GCP. Signed-off-by: Maciej Borzecki <[email protected]>

bboozzoo force-pushed the bbboozzoo/gcp-service-account-upstream branch from 98757d7 to 07c7be1 Compare June 24, 2024 14:33

bboozzoo requested a review from thp-canonical June 24, 2024 17:10

zyga approved these changes Jun 24, 2024

View reviewed changes

ZeyadYasser reviewed Jun 24, 2024

View reviewed changes

ZeyadYasser approved these changes Jun 25, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

spread/google: add support for service accounts #187

spread/google: add support for service accounts #187

bboozzoo commented Jun 14, 2024 •

edited

Loading

bboozzoo Jun 14, 2024

bboozzoo Jun 14, 2024

thp-canonical Jun 14, 2024

thp-canonical Jun 14, 2024

bboozzoo Jun 24, 2024

thp-canonical Jun 14, 2024

bboozzoo Jun 14, 2024

thp-canonical Jun 14, 2024

bboozzoo Jun 24, 2024

bboozzoo commented Jun 24, 2024

zyga left a comment

zyga Jun 24, 2024

zyga Jun 24, 2024

bboozzoo Jun 25, 2024

ZeyadYasser left a comment

ZeyadYasser Jun 24, 2024

bboozzoo Jun 25, 2024

ZeyadYasser Jun 25, 2024

bboozzoo Jun 25, 2024

ZeyadYasser Jun 25, 2024

bboozzoo Jun 25, 2024

spread/google: add support for service accounts #187

Are you sure you want to change the base?

spread/google: add support for service accounts #187

Conversation

bboozzoo commented Jun 14, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bboozzoo commented Jun 24, 2024

zyga left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ZeyadYasser left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bboozzoo commented Jun 14, 2024 •

edited

Loading