Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: exposing cataloging IAM Role as a prop parameter & limiting default permissions #357

Merged
merged 4 commits into from
Oct 9, 2024

Conversation

rpawlaszek
Copy link
Contributor

Fixes #336

This PR includes:

  • allowing passing an IAM to be used by the flow for the purposes of cataloging
  • limiting the default cataloging IAM Role's permissions to only the database and tables it works with
  • adding code comments to the S3 destination types
  • adding S3 cataloging unit tests

@rpawlaszek rpawlaszek requested a review from kozub October 8, 2024 10:07
@rpawlaszek rpawlaszek self-assigned this Oct 8, 2024
Copy link
Contributor

@blimmer blimmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will work. Only main question is RE: glue:UntagResource.

Also, as a side note, this PR was slightly difficult to review without "hide whitespace" changes toggled on. If the code needs to be formatted, it might be nice to do that as a separate PR to make future reviews simpler 👀

});
return { role, policy };
role.addToPolicy(
new PolicyStatement({
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
new PolicyStatement({
// https://docs.aws.amazon.com/appflow/latest/userguide/security_iam_id-based-policy-examples.html#security_iam_id-based-policy-examples-access-gdc
new PolicyStatement({

template.resourceCountIs('AWS::S3::Bucket', 0);
template.resourceCountIs('AWS::AppFlow::Flow', 1);
template.resourceCountIs('AWS::IAM::Role', 1);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to add a test on the role properties to ensure it's scoped to the database, schema, and tables.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look at the updated test. Now there is a strict assertion on the actions and resources for the IAM Policy.

'glue:BatchDeleteTable',
'glue:DeletePartition',
],
resources: ['*'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The only action I'm unsure about is glue:UntagResource, which operates on a bunch of different resources: https://docs.aws.amazon.com/service-authorization/latest/reference/list_awsglue.html#awsglue-connection

Screenshot 2024-10-08 at 07 30 48

My guess is that it only deals with database and schemas, but I'm not positive.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please take a look here. One can tag/untag databases so you're right. I don't think we need to break down this policy into smaller as the combination of actions and resources gives us least privilege

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern here is if we need to allow glue:UntagResource on * to retain good behavior. But, like you said, it might just need to work with catalog/database/table, so we could start there.

Just noting that if something breaks we might need a separate statement that allows Tag/Untag resource on a wider swath of resources.

@rpawlaszek rpawlaszek requested a review from blimmer October 8, 2024 18:46
Copy link
Contributor

@blimmer blimmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

'glue:BatchDeleteTable',
'glue:DeletePartition',
],
resources: ['*'],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My concern here is if we need to allow glue:UntagResource on * to retain good behavior. But, like you said, it might just need to work with catalog/database/table, so we could start there.

Just noting that if something breaks we might need a separate statement that allows Tag/Untag resource on a wider swath of resources.

@rpawlaszek rpawlaszek requested a review from psrebniak October 9, 2024 07:29
Copy link
Contributor

@psrebniak psrebniak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good, approved

@rpawlaszek rpawlaszek merged commit a5448e8 into main Oct 9, 2024
10 of 12 checks passed
@mergify mergify bot deleted the rpawlasz/s3-glue-catalog-upgrade branch October 9, 2024 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Glue Database Role is too Permissive
3 participants