Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add get_aws_accounts typed query #3470

Merged
merged 3 commits into from
Apr 27, 2023

Conversation

hemslo
Copy link
Contributor

@hemslo hemslo commented Apr 24, 2023

Add get_aws_accounts typed query using GraphQL query variables. This will be used in terraform-tgw-attachments integration.

Rely on app-sre/qontract-server#200 for server side support.

🤖 Generated by Copilot at 8c88cf3

This pull request adds typed queries and data classes for AWS accounts and their Terraform state. It uses qenerate to generate Python data classes from GraphQL definitions, and pydantic to validate and serialize the data. It also adds unit tests for the typed query functions. The main files affected are reconcile/gql_definitions/terraform_tgw_attachments/aws_accounts.py and reconcile/typed_queries/terraform_tgw_attachments/aws_accounts.py.

🤖 Generated by Copilot at 8c88cf3

  • Add GraphQL fragment definitions and generated Python data classes for common fields and Terraform state of AWS accounts (link, link, link, link)
  • Add GraphQL query definition and generated Python data classes for querying AWS accounts by name with the common fields and Terraform state fragments (link, link)
  • Add convenience function query in reconcile/gql_definitions/terraform_tgw_attachments/aws_accounts.py that queries and parses the data into the data classes (link)
  • Add typed query function get_aws_accounts in reconcile/typed_queries/terraform_tgw_attachments/aws_accounts.py that takes a GqlApi instance and an optional name argument, and returns a list of AWSAccountV1 instances using the query function (link)
  • Add unit tests for get_aws_accounts in reconcile/test/test_typed_queries/terraform_tgw_attachments/test_get_aws_accounts.py that use pytest fixtures and mock data classes and GqlApi instances, and assert the expected results and variables (link)

Copy link
Member

@chassing chassing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TBH, I don't like such general-purpose queries. With our new GQL classes, it's much easier to fine-grain define all required attributes in a query instead of reusing existing ones. It would be better to define some important and commonly needed attributes as a fragment and reuse this fragment in gql queries defined for special use cases or integration needs.

@hemslo
Copy link
Contributor Author

hemslo commented Apr 24, 2023

This is equivalent to queries.get_aws_accounts, but with typed query, there are many combinations of different usages, I'm afraid it may be too many similar pieces if we split them.

@chassing
Copy link
Member

That is what I mean, do not follow the same pattern/path as with queries.get_aws_account. Having similar gql queries and similar GQL classes, but dedicated per integration, is completely fine, e.g. https://github.com/app-sre/qontract-reconcile/blob/master/reconcile/gql_definitions/terraform_cloudflare_dns/app_interface_cloudflare_dns_settings.gql#L4 and https://github.com/app-sre/qontract-reconcile/blob/master/reconcile/gql_definitions/terraform_cloudflare_users/app_interface_setting_cloudflare_and_vault.gql#L4

See the gql_definitions directory; each "newer" integration defines its own queries but reuses some fragment if it's worth.

@hemslo
Copy link
Contributor Author

hemslo commented Apr 24, 2023

Cool, then I'll update this to only keep parts needed for terraform-tgw-attachments, it's hard to give it a proper name as the same structure is required by both terraform-tgw-attachments and terraform-vpc-peerings

@chassing
Copy link
Member

Having 2 directories in gql_definitions (terraform-tgw-attachments and terraform-vpc-peerings) would be great. As I said, maybe it's worth having a proper fragment (in fragments) as well.

@hemslo hemslo force-pushed the aws-accounts-typed-query branch from 3c6c3a2 to 8c88cf3 Compare April 26, 2023 05:24
@hemslo
Copy link
Contributor Author

hemslo commented Apr 26, 2023

@chassing I updated gql definitions and queries to fit terraform-tgw-attachments, is this the expected structure?
If yes, then changes in my previous pr should also move to terraform-tgw-attachments dirs, right?

Copy link
Member

@chassing chassing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks for all your effort! ❤️

@hemslo hemslo force-pushed the aws-accounts-typed-query branch from 8c88cf3 to ce949ac Compare April 27, 2023 01:28
@hemslo hemslo merged commit 8257677 into app-sre:master Apr 27, 2023
@hemslo hemslo deleted the aws-accounts-typed-query branch April 27, 2023 02:30
jonmosco pushed a commit to jonmosco/qontract-reconcile that referenced this pull request May 11, 2023
* Add AWSAccounts gql definition
* Add get_aws_accounts typed query
* Update aws_accounts gql and typed_query to fit terraform_tgw_attachments

Signed-off-by: Di Wang <[email protected]>
bkez322 pushed a commit to bkez322/qontract-reconcile that referenced this pull request Jul 13, 2023
* Add AWSAccounts gql definition
* Add get_aws_accounts typed query
* Update aws_accounts gql and typed_query to fit terraform_tgw_attachments

Signed-off-by: Di Wang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants