refactor: Use vars/RedHat_N.yml symlink for CentOS, Rocky, Alma where…

…ver possible We have a lot of requests to support Rocky and Alma in various system roles. The first part of adding support is adding `vars/` files for these platforms. In almost every case, for a given major version N, the vars file RedHat_N.yml can be used for CentOS, Rocky, and Alma. Rather than making a copy of the RedHat_N.yml file, just use a symlink to reduce size and maintenance burden, and standardize this across all system roles for consistency. NOTE: OracleLinux is not a strict clone, so we are not going to do this for OracleLinux at this time. Support for OracleLinux will need to be done in separate PRs. For more information, see linux-system-roles/cockpit#130 Note that there may be more work to be done to the role to fully support Rocky and Alma. Many roles have conditionals like this: ```yaml some_var: "{{ 'some value' if ansible_distribution in ['CentOS', 'RedHat'] else 'other value' }}" another_var: "{{ 'some value' if ansible_distribution in ['CentOS', 'Fedora', 'RedHat'] else 'other value' }}" ... - name: Do something when: ansible_distribution in ['CentOS', 'RedHat'] ... - name: Do something else when: ansible_distribution in ['CentOS', 'Fedora', 'RedHat'] ... ``` Adding Rocky and AlmaLinux to these conditionals will have to be done separately. In order to simplify the task, some new variables are being introduced: ```yaml __$rolename_rh_distros: - AlmaLinux - CentOS - RedHat - Rocky __$rolename_rh_distros_fedora: "{{ __$rolename_rh_distros + ['Fedora'] }}" __$rolename_is_rh_distro: "{{ ansible_distribution in __$rolename_rh_distros }}" __$rolename_is_rh_distro_fedora: "{{ ansible_distribution in __$rolename_rh_distros_fedora }}" ``` Then the conditionals can be rewritten as: ```yaml some_var: "{{ 'some value' if __$rolename_is_rh_distro else 'other value' }}" another_var: "{{ 'some value' if __$rolename_is_rh_distro_fedora else 'other value' }}" ... - name: Do something when: __$rolename_is_rh_distro | bool ... - name: Do something else when: __$rolename_is_rh_distro_fedora | bool ... ``` For tests - tests that use such conditionals will need to use `vars_files` or `include_vars` to load the variables that are defined in `tests/vars/rh_distros_vars.yml`: ```yaml vars_files: - vars/rh_distros_vars.yml ``` We don't currently have CI testing for Rocky or Alma, so someone wanting to run tests on those platforms would need to change the test code to use these.
linux-system-roles · Oct 20, 2024 · 1a80ca4 · 1a80ca4
1 parent 349ab66
commit 1a80ca4
Show file tree

Hide file tree

Showing 5 changed files with 126 additions and 11 deletions.
diff --git a/inventory/group_vars/active_roles.yml b/inventory/group_vars/active_roles.yml
@@ -26,6 +26,7 @@ present_templates:
   - .github/workflows/weekly_ci.yml
   - .github/workflows/woke.yml
   - README-ansible.md
+  - tests/vars/rh_distros_vars.yml
 absent_files:
   - .github/workflows/commitlint.yml
   - .github/workflows/tox.yml
@@ -60,3 +61,4 @@ lsr_name: linux_system_roles
 lsr_role_namespace: linux_system_roles  # for ansible-lint
 gha_checkout_action: actions/checkout@v4
 tox_lsr_url: "git+https://github.com/linux-system-roles/[email protected]"
+lsr_rh_distros: "{{ ['AlmaLinux', 'CentOS', 'RedHat', 'Rocky'] + lsr_rh_distros_extra | d([]) }}"
diff --git a/inventory/host_vars/network.yml b/inventory/host_vars/network.yml
@@ -17,3 +17,4 @@ present_badges:
   - "[![Coverage Status](https://coveralls.io/repos/github/linux-system-roles/network/badge.svg)](https://coveralls.io/github/linux-system-roles/network)"
   - "[![Code Style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/ambv/black)"
   - "[![Language grade: Python](https://img.shields.io/lgtm/grade/python/g/linux-system-roles/network.svg?logo=lgtm&logoWidth=18)](https://lgtm.com/projects/g/linux-system-roles/network/context:python)"
+lsr_rh_distros_extra: [OracleLinux]
diff --git a/playbooks/scripts/update_meta_main.py b/playbooks/scripts/update_meta_main.py
@@ -53,6 +53,13 @@
     if role != "mssql":
         tags.add("el10")
 
+# uncomment when all of the roles have full support
+# if not "rocky" in tags:
+#     tags.add("rocky")
+
+# if not "almalinux" in tags:
+#     tags.add("almalinux")
+
 def normalize_key(key):
     ary = [key]
     match = re.match("([a-z]+)([0-9]+)", key)

diff --git a/playbooks/templates/tests/vars/rh_distros_vars.yml b/playbooks/templates/tests/vars/rh_distros_vars.yml
@@ -0,0 +1,14 @@
+# vars for handling conditionals for RedHat and clones
+# DO NOT EDIT - file is auto-generated
+---
+# Ansible distribution identifiers that the role treats like RHEL
+__{{ inventory_hostname }}_rh_distros:
+{{ lsr_rh_distros | sort | to_nice_yaml(indent=2) | indent(width=2, first=true) }}
+# Same as above but includes Fedora
+__{{ inventory_hostname }}_rh_distros_fedora: "{{ '{{' }} __{{ inventory_hostname }}_rh_distros + ['Fedora'] {{ '}}' }}"
+
+# Use this in conditionals to check if distro is Red Hat or clone
+__{{ inventory_hostname }}_is_rh_distro: "{{ '{{' }} ansible_distribution in __{{ inventory_hostname }}_rh_distros {{ '}}' }}"
+
+# Use this in conditionals to check if distro is Red Hat or clone, or Fedora
+__{{ inventory_hostname }}_is_rh_distro_fedora: "{{ '{{' }} ansible_distribution in __{{ inventory_hostname }}_rh_distros_fedora {{ '}}' }}"
diff --git a/playbooks/update_files.yml b/playbooks/update_files.yml
@@ -174,7 +174,7 @@
             path: "{{ git_dir }}/README.md"
             regex: \[\!.*
             firstmatch: true
-            line: "{%-for file in basenames -%}\
+            line: "{%- for file in basenames -%}\
               [![{{ file }}]\
               (https://github.com/linux-system-roles/{{ inventory_hostname }}/actions/workflows/{{ file }}/badge.svg)]\
               (https://github.com/linux-system-roles/{{ inventory_hostname }}/actions/workflows/{{ file }})\
@@ -203,6 +203,7 @@
                   fi
                 fi
               done
+          changed_when: true
 
         - name: Create ignore file for ansible-test 2.17
           shell:
@@ -221,6 +222,7 @@
                   rm .sanity-ansible-ignore-2.17.txt
                 fi
               fi
+          changed_when: true
 
         - name: Create vars, ostree files for el10
           shell:
@@ -241,6 +243,7 @@
                   git add "$dest"
                 fi
               done
+          changed_when: true
 
         - name: Add el10 to meta/main.yml
           script: scripts/update_meta_main.py {{ __meta_main | quote }}
@@ -249,6 +252,83 @@
           vars:
             __meta_main: "{{ git_dir }}/meta/main.yml"
 
+        # NOTE: Due to https://github.com/linux-system-roles/cockpit/issues/130
+        # we cannot automatically create symlinks for OracleLinux
+        - name: Create vars symlinks for CentOS, Rocky, Alma
+          shell:
+            chdir: "{{ git_dir }}"
+            cmd: |
+              set -euxo pipefail
+              same_vars_files() {
+                python -c 'import sys; import yaml
+              a = yaml.safe_load(open(sys.argv[1]))
+              b = yaml.safe_load(open(sys.argv[2]))
+              rv = 0 if a == b else 1
+              sys.exit(rv)' "$1" "$2"
+              }
+              for dir in vars roles/rsyslog/vars tests/vars; do
+                if [ ! -d "$dir" ]; then continue; fi
+                pushd "$dir"
+                for file in RedHat_*.yml RedHat-*.yml; do
+                  if [[ "$file" =~ ^RedHat([_-])([0-9]+).yml$ ]]; then
+                    sep="${BASH_REMATCH[1]}"
+                    ver="${BASH_REMATCH[2]}"
+                    for clone in CentOS Rocky AlmaLinux; do
+                      clone_file="${clone}${sep}$ver.yml"
+                      if [ ! -e "$clone_file" ]; then
+                        ln -s "$file" "$clone_file"
+                        git add "$clone_file"
+                      elif [ -f "$clone_file" ] && same_vars_files "$file" "$clone_file"; then
+                        git rm -f "$clone_file"
+                        ln -s "$file" "$clone_file"
+                        git add "$clone_file"
+                      else
+                        echo {{ inventory_hostname }} "$clone_file" exists and is different than "$file"
+                      fi
+                    done
+                  fi
+                done
+                popd
+              done
+          changed_when: true
+
+        - name: See if vars main exists
+          stat:
+            path: "{{ git_dir }}/vars/main.yml"
+          register: __vars_main
+
+        # some roles have no vars/ and use defaults/main.yml instead
+        - name: Add vars block to main
+          ansible.builtin.blockinfile:
+            path: "{{ git_dir ~ '/vars/main.yml' if __vars_main.stat.exists
+              else git_dir ~ '/defaults/main.yml' }}"
+            insertafter: EOF
+            marker: "# {mark} - DO NOT EDIT THIS BLOCK - rh distros variables"
+            prepend_newline: true
+            block: |
+              # Ansible distribution identifiers that the role treats like RHEL
+              __{{ inventory_hostname }}_rh_distros:
+              {{ lsr_rh_distros | sort | to_nice_yaml(indent=2) | indent(width=2, first=true) }}
+              # Same as above but includes Fedora
+              __{{ inventory_hostname }}_rh_distros_fedora: "{{ '{{' }} __{{ inventory_hostname }}_rh_distros + ['Fedora'] {{ '}}' }}"
+
+              # Use this in conditionals to check if distro is Red Hat or clone
+              __{{ inventory_hostname }}_is_rh_distro: "{{ '{{' }} ansible_distribution in __{{ inventory_hostname }}_rh_distros {{ '}}' }}"
+
+              # Use this in conditionals to check if distro is Red Hat or clone, or Fedora
+              __{{ inventory_hostname }}_is_rh_distro_fedora: "{{ '{{' }} ansible_distribution in __{{ inventory_hostname }}_rh_distros_fedora {{ '}}' }}"
+          register: __vars_main_block
+
+        - name: Add to git transaction
+          command:
+            cmd: git add {{ path | quote }}
+            chdir: "{{ git_dir }}"
+          when: __vars_main_block is changed
+          changed_when: true
+          vars:
+            path: "{{ 'vars/main.yml' if __vars_main.stat.exists
+              else 'defaults/main.yml' }}"
+
         - name: Create git commit, PR
           changed_when: false
           shell:
@@ -271,22 +351,30 @@
                 # nothing to do
                 echo No updates for role {{ inventory_hostname }}
                 exit 0
+              fi
+              commit_file={{ update_files_commit_file | quote }}
+              branch={{ update_files_branch | quote }}
+              if [ "{{ __is_new | ternary('true', 'false') }}" = true ]; then
+                git commit -s -F "$commit_file"
               else
-                git commit -s -F {{ update_files_commit_file | quote }}
-                force_flag="{{ __is_new | ternary('', '-f') }}"
-                if [ {{ lsr_dry_run | d('true') }} = false ]; then
-                  git push $force_flag -u origin {{ update_files_branch | quote }}
-                echo
-                  echo git push $force_flag -u origin {{ update_files_branch | quote }}
-                fi
+                # commit orig head, then amend, and ensure commit is signed
+                git commit -C ORIG_HEAD
+                git commit -s --amend -F "$commit_file"
+              fi
+              force_flag="{{ __is_new | ternary('', '-f') }}"
+              if [ {{ lsr_dry_run | d('true') }} = false ]; then
+                git push $force_flag -u origin "$branch"
+              echo
+                echo git push $force_flag -u origin "$branch"
               fi
-              prnum="$(gh pr list -H {{ update_files_branch | quote }} --json number --jq '.[].number')"
+              prnum="$(gh pr list -H "$branch" --json number --jq '.[].number')"
               if [ -z "$prnum" ]; then
+                cmd=(gh pr create --assignee @me --fill --base {{ __main_br | quote }})
                 if [ {{ lsr_dry_run | d('true') }} = false ]; then
                   retries=0
                   backoff=10
                   while [ "$retries" -lt 10 ]; do
-                    if gh pr create --fill --base "{{ __main_br }}"; then
+                    if "${cmd[@]}"; then
                       break
                     else
                       echo backing off "$backoff" seconds . . .
@@ -296,11 +384,14 @@
                     fi
                   done
                 else
-                  echo gh pr create --fill --base "{{ __main_br }}"
+                  echo "${cmd[@]}"
                 fi
               else
                 echo Existing PR "$prnum"
                 echo {{ github_url_prefix ~ github_org ~ '/' ~ inventory_hostname ~ '/pull/' }}$prnum
+                # update PR assignee, title, and body
+                tail -n +3 "$commit_file" | \
+                  gh pr edit "$prnum" -F - --add-assignee @me --title "$(head -1 "$commit_file")"
               fi
       always:
         - name: Clean up temp working dir