Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add django-tasks-scheduler project #31

Merged
merged 21 commits into from
Aug 5, 2024
91 changes: 61 additions & 30 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,39 +112,68 @@ Django Commons packages.

### Terraform changes to add a new project

#### Using the GitHub UI

1. Transfer the existing repository to the Django Commons organization using the GitHub UI, so old information is
preserved.
2. Make sure the there are no teams `repo-name`, `repo-name-admins` and `repo-name-committers` in the Django Commons
organization. Teams can be viewed [here](https://github.com/orgs/django-commons/teams). The teams will be created by
the terraform apply process.

#### Locally

Assuming repository name is `repo-name`:

1. In [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars),
add the new repository to the `repositories` section:
1.
In [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars),
add the new repository to the `repositories` section:

```terraform
repositories = {
# ...
"repo-name" = {
description = "repo description"
allow_auto_merge = false # optional, default is false
allow_merge_commit = false # optional, default is false
allow_rebase_merge = false # optional, default is false
allow_squash_merge = false # optional, default is false
allow_update_branch = false # optional, default is false
enable_branch_protection = true # optional, default is true
has_discussions = true # optional, default is true
has_downloads = true # optional, default is true
has_wiki = false # optional, default is false
is_template = false # optional, default is false
push_allowances = []
required_status_checks_contexts = [] # optional, default is []
template = "" # optional, default is ""
topics = []
visibility = "public" # optional, default is "public"
skip_team_creation = false # Optional, default is false => create 3 teams for the repository
admins = [] # Members of the repository's admin and repository teams. Have admin permissions
committers = [] # Members of the repository's committers and repository teams. Have write permissions
members = [] # Members of the repository team. Have triage permissions
}
# ...
"repo-name" = {
description = "repo description"
homepage_url = "" # optional, default is ""
allow_auto_merge = false # optional, default is false
allow_merge_commit = false # optional, default is false
allow_rebase_merge = false # optional, default is false
allow_squash_merge = true # optional, default is true
allow_update_branch = true # optional, default is true
delete_branch_on_merge = true # optional, default is true
has_discussions = true # optional, default is true
has_downloads = true # optional, default is true
has_wiki = false # optional, default is false
is_template = false # optional, default is false
push_allowances = []
template = "" # optional, default is ""
topics = []
visibility = "public" # optional, default is "public"
skip_team_creation = optional(bool, false) # Do not create teams for repository
enable_branch_protection = true # optional, default is true
required_status_checks_contexts = [] # optional, default is []
admins = [] # Members of the repository's admin and repository teams. Have admin permissions
committers = [] # Members of the repository's committers and repository teams. Have write permissions
members = [] # Members of the repository team. Have triage permissions
}
}
```

2. import the repository into the terraform state by
cunla marked this conversation as resolved.
Show resolved Hide resolved
running `terraform import -var-file=production/org.tfvars -var-file=production/repositories.tfvars -var github_token=... 'github_repository.this["repo-name"]' repo-name`
3. Run `terraform plan -var-file=production/org.tfvars -var-file=production/repositories.tfvars -var github_token=...`
tim-schilling marked this conversation as resolved.
Show resolved Hide resolved
to see the changes that will be made.

4. Create a pull-request to `main` branch. This will trigger terraform to plan the changes in the organization to be
executed.
Review the changes and make sure they align with the request.
5. Merge the pull request. This will trigger terraform to apply the changes in the organization.

The expected changes:

- New teams `repo-name`, `repo-name-admins`, `repo-name-committers` with the relevant members based on the
repository's description.
- The repository changes are accepted by the project maintainers.

## Remove Project Playbook

1. Confirm there's agreement amongst current project maintainers to move project out of Django Commons
Expand All @@ -158,10 +187,12 @@ repositories = {

1. Remove the repository from the `repositories` section
in [`terraform/production/respositories.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/respositories.tfvars)
2. Remove the parent team and child teams for the repository from the `teams_repositories`
and `teams_repositories_privileged` sections in
[`terraform/production/teams.tfvars`](https://github.com/django-commons/controls/blob/main/terraform/production/teams.tfvars)
3. Create a pull-request to `main` branch. This will trigger terraform to plan the changes in the organization to be
2. Create a pull-request to `main` branch. This will trigger terraform to plan the changes in the organization to be
executed.
Review the changes and make sure they align with the request.
4. Merge the pull request. This will trigger terraform to apply the changes in the organization.
3. Merge the pull request. This will trigger terraform to apply the changes in the organization.

The expected changes:

- The repository will be removed from the organization.
- The repository's teams will be removed from the organization.
21 changes: 13 additions & 8 deletions terraform/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ GitHub Organization as Terraform
- `production/*.tfvars` - instances, should strictly follow the types in `variables.tf`.
- `main.tf` - build configuration based on instances values from `production.tfvars` (or, if not defined explicitly,
then default value from `variables.tf`)
- `resources-*.tf` - define resources, like `github_repository`, `github_team`, etc.
- `tfstate.json` - Current state file, pulled using `terraform import ..`

# Why Terraform?
Expand All @@ -34,24 +35,27 @@ All changes should be made in `production/*.tfvars`:
repositories = {
"repo-name" = {
description = "repo description"
homepage_url = "" # optional, default is ""
allow_auto_merge = false # optional, default is false
allow_merge_commit = false # optional, default is false
allow_rebase_merge = false # optional, default is false
allow_squash_merge = false # optional, default is false
allow_update_branch = false # optional, default is false
enable_branch_protection = true # optional, default is true
allow_squash_merge = true # optional, default is true
allow_update_branch = true # optional, default is true
delete_branch_on_merge = true # optional, default is true
has_discussions = true # optional, default is true
has_downloads = true # optional, default is true
has_wiki = false # optional, default is false
is_template = false # optional, default is false
push_allowances = []
required_status_checks_contexts = [] # optional, default is []
template = "" # optional, default is ""
topics = []
visibility = "public" # optional, default is "public"
visibility = "public" # optional, default is "public"
skip_team_creation = optional(bool, false) # Do not create teams for repository
admins = optional(set(string), []) # Members of the repository admin team
committers = optional(set(string), []) # Members of the repository committers team
enable_branch_protection = true # optional, default is true
required_status_checks_contexts = [] # optional, default is []
admins = [] # Members of the repository's admin and repository teams. Have admin permissions
committers = [] # Members of the repository's committers and repository teams. Have write permissions
members = [] # Members of the repository team. Have triage permissions
}
# ...
}
Expand All @@ -72,6 +76,7 @@ To do so, you can use the following steps:
4. Make changes to `production/*.tfvars` to reflect the desired state (add/update users, repositories, teams, etc.)
5. To see what changes between the current state of the GitHub organization and the plan
run: `terraform plan -var-file=production/org.tfvars -var-file=production/repositories.tfvars -var github_token=...`
6. To apply the changes, run: `terraform apply -var-file=production/org.tfvars -var-file=production/repositories.tfvars -var github_token=...`
6. To apply the changes,
run: `terraform apply -var-file=production/org.tfvars -var-file=production/repositories.tfvars -var github_token=...`

[1]: https://developer.hashicorp.com/terraform/tutorials/it-saas/github-user-teams#configure-your-credentials
19 changes: 15 additions & 4 deletions terraform/production/repositories.tfvars
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,12 @@ repositories = {
"controls" = {
description = "The controls for managing Django Commons projects"
enable_branch_protection = false
allow_merge_commit = true
allow_rebase_merge = true
allow_squash_merge = true
allow_merge_commit = true
allow_rebase_merge = true
allow_squash_merge = true
topics = []
push_allowances = []
skip_team_creation = true
skip_team_creation = true
cunla marked this conversation as resolved.
Show resolved Hide resolved
}

"membership" = {
Expand Down Expand Up @@ -52,4 +52,15 @@ repositories = {
members = [
]
}

"django-tasks-scheduler" = {
description = "Schedule async tasks using redis pub/sub."
homepage_url = "https://django-tasks-scheduler.readthedocs.io/"

admins = [
"cunla",
]
committers = []
members = []
}
}
13 changes: 10 additions & 3 deletions terraform/resources-repo-committer-teams.tf
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,10 @@ resource "github_team" "repo_committer_team" {

# Add the people to the team
resource "github_team_members" "repo_committer_team_members" {
for_each = {for k, v in var.repositories : k => v if v.skip_team_creation == false}
for_each = {
for k, v in var.repositories : k => v
if v.skip_team_creation == false && length(v.committers) > 0
}

team_id = github_team.repo_committer_team[each.key].id

Expand All @@ -19,13 +22,17 @@ resource "github_team_members" "repo_committer_team_members" {

content {
username = members.value
role = contains(var.admins, members.value) ? "maintainer" : "member"
role = contains(each.value.admins, members.value) ? "maintainer" : "member"
}
}
}

# Define the team's permissions for the repositories
resource "github_team_repository" "repo_committer_team_access" {
for_each = {for k, v in var.repositories : k => v if v.skip_team_creation == false}
for_each = {
for k, v in var.repositories : k => v
if v.skip_team_creation == false
}
repository = each.key
team_id = github_team.repo_committer_team[each.key].id
permission = "maintain"
Expand Down
1 change: 1 addition & 0 deletions terraform/resources-repos.tf
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
resource "github_repository" "this" {
for_each = var.repositories

homepage_url = each.value.homepage_url
allow_auto_merge = each.value.allow_auto_merge
allow_merge_commit = each.value.allow_merge_commit
merge_commit_title = each.value.merge_commit_title
Expand Down
78 changes: 78 additions & 0 deletions terraform/tfstate.json
Original file line number Diff line number Diff line change
Expand Up @@ -448,6 +448,84 @@
"sensitive_attributes": [],
"private": "eyJzY2hlbWFfdmVyc2lvbiI6IjEifQ=="
},
{
"index_key": "django-tasks-scheduler",
"schema_version": 1,
"attributes": {
"allow_auto_merge": false,
"allow_merge_commit": true,
"allow_rebase_merge": true,
"allow_squash_merge": true,
"allow_update_branch": false,
"archive_on_destroy": null,
"archived": false,
"auto_init": false,
"default_branch": "master",
"delete_branch_on_merge": false,
"description": "Schedule async tasks using redis pub/sub.",
"etag": "W/\"eafd212fb05626bb9b6c36a2e1c22f85eca859f6249f96d8ad860a3f1b832f48\"",
"full_name": "django-commons/django-tasks-scheduler",
"git_clone_url": "git://github.com/django-commons/django-tasks-scheduler.git",
"gitignore_template": null,
"has_discussions": true,
"has_downloads": true,
"has_issues": true,
"has_projects": true,
"has_wiki": true,
"homepage_url": "https://django-tasks-scheduler.readthedocs.io/",
"html_url": "https://github.com/django-commons/django-tasks-scheduler",
"http_clone_url": "https://github.com/django-commons/django-tasks-scheduler.git",
"id": "django-tasks-scheduler",
"ignore_vulnerability_alerts_during_read": null,
"is_template": false,
"license_template": null,
"merge_commit_message": "PR_TITLE",
"merge_commit_title": "MERGE_MESSAGE",
"name": "django-tasks-scheduler",
"node_id": "R_kgDOJzK5Kg",
"pages": [],
"primary_language": "Python",
"private": false,
"repo_id": 657635626,
"security_and_analysis": [
{
"advanced_security": [],
"secret_scanning": [
{
"status": "disabled"
}
],
"secret_scanning_push_protection": [
{
"status": "disabled"
}
]
}
],
"squash_merge_commit_message": "COMMIT_MESSAGES",
"squash_merge_commit_title": "COMMIT_OR_PR_TITLE",
"ssh_clone_url": "[email protected]:django-commons/django-tasks-scheduler.git",
"svn_url": "https://github.com/django-commons/django-tasks-scheduler",
"template": [],
"topics": [
"background-jobs",
"django",
"django-application",
"job-queue",
"python",
"redis",
"redis-queue",
"scheduled-jobs",
"scheduled-tasks",
"task-queue"
],
"visibility": "public",
"vulnerability_alerts": true,
"web_commit_signoff_required": false
},
"sensitive_attributes": [],
"private": "eyJzY2hlbWFfdmVyc2lvbiI6IjEifQ=="
},
{
"index_key": "membership",
"schema_version": 1,
Expand Down
1 change: 1 addition & 0 deletions terraform/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ variable "repositories" {
delete_branch_on_merge = optional(bool, true)
has_discussions = optional(bool, true)
has_downloads = optional(bool, true)
homepage_url = optional(string, "")
has_wiki = optional(bool, false)
is_template = optional(bool, false)
push_allowances = optional(list(string), [])
Expand Down