[BUG] openshift-tools-installer action takes too long when used through Github Enterprise

[urswiss]

Version

v1

Describe the bug

Since #58 was fixed we face the issue that the openshift-tools-installer runs around 3 minutes (compared to around 16 seconds before).

Steps to reproduce, workflow links, screenshots

On a Github Enterprise instance, create and run the below action:

name: Master Workflow

env:
  REPOSITORY_NAME: ${{ github.event.repository.name }}

on:
  push:
    branches:
      - 'master'

jobs:
  build-release:
    name: 'Build new release'
    runs-on: [ self-hosted, linux, x64 ]
    steps:
      - name: 'Setup OpenShift Tools'
        uses: redhat-actions/openshift-tools-installer@v1
        with:
          oc: 4

Output

Running this action results into the below output.

Setup Job (2s)

Current runner version: '2.284.0'
Runner name: 'github-actions-runner-basic-3-4r7q5'
Runner group name: 'Default'
Machine name: 'github-actions-runner-basic-3-4r7q5'
GITHUB_TOKEN Permissions
Actions: write
Checks: write
Contents: write
Deployments: write
Issues: write
Metadata: read
OrganizationPackages: write
Packages: write
PullRequests: write
RepositoryProjects: write
SecurityEvents: write
Statuses: write
Runner is running behind proxy server 'http://egress-http-proxy-external:8080' for all HTTP requests.
Runner is running behind proxy server 'http://egress-http-proxy-external:8080' for all HTTPS requests.
Prepare workflow directory
Prepare all required actions
Getting action download info
Download action repository 'redhat-actions/openshift-tools-installer@v1' (SHA:99744decfe48e62f0304d622c34bce5f1d045858)

Setup Openshift Tools (3m 2s)

Run redhat-actions/openshift-tools-installer@v1
with:
oc: 4
github_pat: ***
source: mirror
skip_cache: false
env:
REPOSITORY_NAME: test-openshift-tools-installer
Installing oc matching version "4"
ℹ️ Tools will be installed from "mirror".

🔎 Searching for a version of oc satisfying the range ">=4.0.0 <5.0.0-0" that was input as "4"
Download directory for oc is https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
Max oc version satisfying >=4.0.0 <5.0.0-0 is 4.9.15
Current operating system is linux
Selecting openshift-client-linux-4.9.15.tar.gz
⏩ GitHub enterprise detected; skipping cache. For more information, see actions/cache#505
⬇️ Downloading 47.12MB https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz ...
Downloaded openshift-client-linux-4.9.15.tar.gz in 1.8s
⬇️ Downloading hash file https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/sha256sum.txt
sha256 verification of openshift-client-linux-4.9.15.tar.gz succeeded.
/usr/bin/tar xz --warning=no-unknown-keyword -C /home/runner/_work/_temp/730744aa-ac4a-47d3-99d5-ab1953101224 -f /home/runner/_work/_temp/ef57516c22ae02527befdc776956cc11-openshift-client-linux-4.9.15.tar.gz
Removed /home/runner/_work/_temp/ef57516c22ae02527befdc776956cc11-openshift-client-linux-4.9.15.tar.gz
Current architecture is amd64
Using temporary directory for storage
📁 CLIs will be downloaded to /home/runner/_work/_temp/openshift-bin
Added /home/runner/_work/_temp/openshift-bin to PATH
oc installed into /home/runner/_work/_temp/openshift-bin/oc
/home/runner/_work/_temp/openshift-bin/oc version --client
Client Version: 4.9.15
⏩ GitHub enterprise detected; skipping cache. For more information, see actions/cache#505
✅ Successfully installed oc 4.9.15.

✅ Successfully installed 1/1 client:

• oc 4.9.15

Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json

Complete Job

Cleaning up orphan processes

Summary

As you can see the download itself is quite fast (1.8s) but a lot of time (around 3 minutes) is lost somewhere after. The statement Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json is actually appearing in less than a second, but afterwards the action is stuck for 3 minutes.
As mentioned, the action was running in less than 20 seconds on December 15th still, so I suspect the slow-down has something to do with the fix for #58 or the underlying problem.

[divyansh42]

Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json
It seems this is the last step of the tools installer. Not sure why it stuck there for more than 3 minutes 🤔
Will it be possible for you to record your screen after setting the secret ACTIONS_STEP_DEBUG to true

[urswiss]

Setup Job (2s)

##[debug]Starting: Set up job
Current runner version: '2.284.0'
Runner name: 'github-actions-runner-basic-3-4r7q5'
Runner group name: 'Default'
Machine name: 'github-actions-runner-basic-3-4r7q5'
GITHUB_TOKEN Permissions
Actions: write
Checks: write
Contents: write
Deployments: write
Issues: write
Metadata: read
OrganizationPackages: write
Packages: write
PullRequests: write
RepositoryProjects: write
SecurityEvents: write
Statuses: write
##[debug]Primary repository: esales/test-openshift-tools-installer
Runner is running behind proxy server 'http://egress-http-proxy-external:8080' for all HTTP requests.
Runner is running behind proxy server 'http://egress-http-proxy-external:8080' for all HTTPS requests.
Prepare workflow directory
##[debug]Update context data
##[debug]Evaluating job-level environment variables
##[debug]Evaluating: github.event.repository.name
##[debug]Evaluating Index:
##[debug]..Evaluating Index:
##[debug]....Evaluating Index:
##[debug]......Evaluating github:
##[debug]......=> Object
##[debug]......Evaluating String:
##[debug]......=> 'event'
##[debug]....=> Object
##[debug]....Evaluating String:
##[debug]....=> 'repository'
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'name'
##[debug]=> 'test-openshift-tools-installer'
##[debug]Result: 'test-openshift-tools-installer'
##[debug]Evaluating job container
##[debug]Evaluating job service containers
##[debug]Evaluating job defaults
Prepare all required actions
Getting action download info
Download action repository 'redhat-actions/openshift-tools-installer@v1' (SHA:99744decfe48e62f0304d622c34bce5f1d045858)
##[debug]Download 'https://api.github.com/repos/redhat-actions/openshift-tools-installer/tarball/99744decfe48e62f0304d622c34bce5f1d045858' to '/home/runner/_work/_actions/_temp_ca347818-31e1-42de-931e-cb0eae885c3b/1750b5dc-f336-4fb0-aa5a-e7fdca81b4b0.tar.gz'
##[debug]Unwrap 'redhat-actions-openshift-tools-installer-99744de' to '/home/runner/_work/_actions/redhat-actions/openshift-tools-installer/v1'
##[debug]Archive '/home/runner/_work/_actions/_temp_ca347818-31e1-42de-931e-cb0eae885c3b/1750b5dc-f336-4fb0-aa5a-e7fdca81b4b0.tar.gz' has been unzipped into '/home/runner/_work/_actions/redhat-actions/openshift-tools-installer/v1'.
##[debug]action.yml for action: '/home/runner/_work/_actions/redhat-actions/openshift-tools-installer/v1/action.yml'.
##[debug]Set step 'redhat-actionsopenshift-tools-installer' display name to: 'Setup OpenShift Tools'
##[debug]Collect running processes for tracking orphan processes.
##[debug]Finishing: Set up job

Setup Openshift Tools (3m 2s)

##[debug]Evaluating condition for step: 'Setup OpenShift Tools'
##[debug]Evaluating: success()
##[debug]Evaluating success:
##[debug]=> true
##[debug]Result: true
##[debug]Starting: Setup OpenShift Tools
##[debug]Loading inputs
##[debug]Evaluating: github.token
##[debug]Evaluating Index:
##[debug]..Evaluating github:
##[debug]..=> Object
##[debug]..Evaluating String:
##[debug]..=> 'token'
##[debug]=> ''
##[debug]Result: ''
##[debug]Loading env
Run redhat-actions/openshift-tools-installer@v1
with:
oc: 4
github_pat: ***
source: mirror
skip_cache: false
env:
REPOSITORY_NAME: test-openshift-tools-installer
Installing oc matching version "4"
ℹ️ Tools will be installed from "mirror".

🔎 Searching for a version of oc satisfying the range ">=4.0.0 <5.0.0-0" that was input as "4"
Download directory for oc is https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
##[debug]GET https://mirror.openshift.com/pub/openshift-v4/clients/ocp/
##[debug]Semantic versions of oc are 4.1.0, 4.1.11, 4.1.13, 4.1.14, 4.1.15, 4.1.16, 4.1.17, 4.1.18, 4.1.2, 4.1.20, 4.1.21, 4.1.22, 4.1.23, 4.1.24, 4.1.25, 4.1.26, 4.1.27, 4.1.28, 4.1.29, 4.1.3, 4.1.30, 4.1.31, 4.1.34, 4.1.38, 4.1.4, 4.1.41, 4.1.6, 4.1.7, 4.1.8, 4.1.9, 4.2.0, 4.2.10, 4.2.12, 4.2.13, 4.2.14, 4.2.16, 4.2.18, 4.2.19, 4.2.2, 4.2.20, 4.2.21, 4.2.22, 4.2.23, 4.2.24, 4.2.25, 4.2.26, 4.2.27, 4.2.28, 4.2.29, 4.2.30, 4.2.32, 4.2.33, 4.2.34, 4.2.36, 4.2.4, 4.2.7, 4.2.8, 4.2.9, 4.3.0, 4.3.1, 4.3.10, 4.3.11, 4.3.12, 4.3.13, 4.3.14, 4.3.15, 4.3.17, 4.3.18, 4.3.19, 4.3.2, 4.3.21, 4.3.22, 4.3.23, 4.3.24, 4.3.25, 4.3.26, 4.3.27, 4.3.28, 4.3.29, 4.3.3, 4.3.31, 4.3.32, 4.3.33, 4.3.35, 4.3.38, 4.3.40, 4.3.5, 4.3.8, 4.3.9, 4.4.10, 4.4.11, 4.4.12, 4.4.13, 4.4.14, 4.4.15, 4.4.16, 4.4.17, 4.4.18, 4.4.19, 4.4.20, 4.4.21, 4.4.22, 4.4.23, 4.4.24, 4.4.25, 4.4.26, 4.4.27, 4.4.28, 4.4.29, 4.4.3, 4.4.30, 4.4.31, 4.4.32, 4.4.33, 4.4.4, 4.4.5, 4.4.6, 4.4.7, 4.4.8, 4.4.9, 4.5.0, 4.5.1-rc.0, 4.5.1, 4.5.10, 4.5.11, 4.5.12, 4.5.13, 4.5.14, 4.5.15, 4.5.16, 4.5.17, 4.5.18, 4.5.19, 4.5.2, 4.5.20, 4.5.21, 4.5.22, 4.5.23, 4.5.24, 4.5.25, 4.5.27, 4.5.28, 4.5.29, 4.5.3, 4.5.30, 4.5.31, 4.5.32, 4.5.33, 4.5.34, 4.5.35, 4.5.36, 4.5.37, 4.5.38, 4.5.39, 4.5.4, 4.5.40, 4.5.41, 4.5.5, 4.5.6, 4.5.7, 4.5.8, 4.5.9, 4.6.0-rc.0, 4.6.0-rc.1, 4.6.0-rc.2, 4.6.0-rc.3, 4.6.0-rc.4, 4.6.0, 4.6.1, 4.6.10, 4.6.11, 4.6.12, 4.6.13, 4.6.15, 4.6.16, 4.6.17, 4.6.18, 4.6.19, 4.6.2, 4.6.20, 4.6.21, 4.6.22, 4.6.23, 4.6.24, 4.6.25, 4.6.26, 4.6.27, 4.6.28, 4.6.29, 4.6.3, 4.6.30, 4.6.31, 4.6.32, 4.6.33, 4.6.34, 4.6.35, 4.6.36, 4.6.37, 4.6.38, 4.6.39, 4.6.4, 4.6.40, 4.6.41, 4.6.42, 4.6.43, 4.6.44, 4.6.45, 4.6.46, 4.6.47, 4.6.48, 4.6.49, 4.6.5, 4.6.50, 4.6.51, 4.6.52, 4.6.53, 4.6.6, 4.6.7, 4.6.8, 4.6.9, 4.7.0-rc.0, 4.7.0-rc.1, 4.7.0-rc.2, 4.7.0-rc.3, 4.7.0, 4.7.1, 4.7.10, 4.7.11, 4.7.12, 4.7.13, 4.7.14, 4.7.15, 4.7.16, 4.7.17, 4.7.18, 4.7.19, 4.7.2, 4.7.20, 4.7.21, 4.7.22, 4.7.23, 4.7.24, 4.7.25, 4.7.26, 4.7.27, 4.7.28, 4.7.29, 4.7.3, 4.7.30, 4.7.31, 4.7.32, 4.7.33, 4.7.34, 4.7.35, 4.7.36, 4.7.37, 4.7.38, 4.7.39, 4.7.4, 4.7.40, 4.7.41, 4.7.5, 4.7.6, 4.7.7, 4.7.8, 4.7.9, 4.8.0-rc.0, 4.8.0-rc.1, 4.8.0-rc.2, 4.8.0-rc.3, 4.8.0, 4.8.1, 4.8.10, 4.8.11, 4.8.12, 4.8.13, 4.8.14, 4.8.15, 4.8.16, 4.8.17, 4.8.18, 4.8.19, 4.8.2, 4.8.20, 4.8.21, 4.8.22, 4.8.23, 4.8.24, 4.8.25, 4.8.26, 4.8.27, 4.8.3, 4.8.4, 4.8.5, 4.8.6, 4.8.7, 4.8.8, 4.8.9, 4.9.0-rc.0, 4.9.0-rc.1, 4.9.0-rc.3, 4.9.0-rc.4, 4.9.0-rc.5, 4.9.0-rc.6, 4.9.0-rc.7, 4.9.0-rc.8, 4.9.0, 4.9.1, 4.9.10, 4.9.11, 4.9.12, 4.9.13, 4.9.15, 4.9.2, 4.9.4, 4.9.5, 4.9.6, 4.9.7, 4.9.8, 4.9.9
##[debug]40 non-semantic versions were discarded
Max oc version satisfying >=4.0.0 <5.0.0-0 is 4.9.15
##[debug]GET https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/
##[debug]oc 4.9.15 files: .., changelog.html, changelog.md, openshift-client-linux-4.9.15.tar.gz, openshift-client-linux.tar.gz, openshift-client-mac-4.9.15.tar.gz, openshift-client-mac.tar.gz, openshift-client-windows-4.9.15.zip, openshift-client-windows.zip, openshift-install-linux-4.9.15.tar.gz, openshift-install-linux.tar.gz, openshift-install-mac-4.9.15.tar.gz, openshift-install-mac.tar.gz, opm-linux-4.9.15.tar.gz, opm-linux.tar.gz, opm-mac-4.9.15.tar.gz, opm-mac.tar.gz, opm-windows-4.9.15.tar.gz, opm-windows.tar.gz, release.txt, sha256sum.txt, sha256sum.txt.gpg
Current operating system is linux
##[debug]After filterByOS, 6 files remain.
##[debug]After bound filterByExecutable, 2 files remain.
##[debug]After bound filterByVersioned, 1 files remain.
##[debug]1 clients remaining; skipping filterByZipped.
Selecting openshift-client-linux-4.9.15.tar.gz
##[debug]File info for oc >=4.0.0 <5.0.0-0 resolved successfully to {"archiveFilename":"openshift-client-linux-4.9.15.tar.gz","archiveFileUrl":"https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz","clientName":"oc","version":"4.9.15","versionRange":{"options":{},"loose":false,"includePrerelease":false,"raw":"4","set":[[{"options":{},"loose":false,"operator":">=","semver":{"options":{},"loose":false,"includePrerelease":false,"raw":"4.0.0","major":4,"minor":0,"patch":0,"prerelease":[],"build":[],"version":"4.0.0"},"value":">=4.0.0"},{"options":{},"loose":false,"operator":"<","semver":{"options":{},"loose":false,"includePrerelease":false,"raw":"5.0.0-0","major":5,"minor":0,"patch":0,"prerelease":[0],"build":[],"version":"5.0.0-0"},"value":"<5.0.0-0"}]],"range":">=4.0.0 <5.0.0-0"},"mirrorDirectoryUrl":"https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/"}
⏩ GitHub enterprise detected; skipping cache. For more information, see actions/cache#505
⬇️ Downloading 47.12MB https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz ...
##[debug]Downloading https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz
##[debug]Destination /home/runner/_work/_temp/4cfac7f761abcdd906231673ce000d05-openshift-client-linux-4.9.15.tar.gz
##[debug]download complete
##[debug]Downloaded to /home/runner/_work/_temp/4cfac7f761abcdd906231673ce000d05-openshift-client-linux-4.9.15.tar.gz
Downloaded openshift-client-linux-4.9.15.tar.gz in 0.8s
##[debug]GET https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/
⬇️ Downloading hash file https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/sha256sum.txt
##[debug]sha256 hashing /home/runner/_work/_temp/4cfac7f761abcdd906231673ce000d05-openshift-client-linux-4.9.15.tar.gz...
##[debug]Correct hash for openshift-client-linux-4.9.15.tar.gz is af193ec89e7416e118fee275d6a7ab4cc1b7f690f82c1706df7614295db38d56
##[debug]Actual hash for openshift-client-linux-4.9.15.tar.gz is af193ec89e7416e118fee275d6a7ab4cc1b7f690f82c1706df7614295db38d56
sha256 verification of openshift-client-linux-4.9.15.tar.gz succeeded.
##[debug]Checking tar --version
##[debug]tar (GNU tar) 1.30
##[debug]Copyright (C) 2017 Free Software Foundation, Inc.
##[debug]License GPLv3+: GNU GPL version 3 or later https://gnu.org/licenses/gpl.html.
##[debug]This is free software: you are free to change and redistribute it.
##[debug]There is NO WARRANTY, to the extent permitted by law.
##[debug]
##[debug]Written by John Gilmore and Jay Fenlason.
/usr/bin/tar xz -v --warning=no-unknown-keyword -C /home/runner/_work/_temp/8994c8d4-1724-4c89-9f30-4e2444d4aeb7 -f /home/runner/_work/_temp/4cfac7f761abcdd906231673ce000d05-openshift-client-linux-4.9.15.tar.gz
README.md
oc
kubectl
Removed /home/runner/_work/_temp/4cfac7f761abcdd906231673ce000d05-openshift-client-linux-4.9.15.tar.gz
Current architecture is amd64
##[debug]Executable glob patterns are: oc oc-linux-amd64 linux-amd64-oc linux-amd64-oc
##[debug]followSymbolicLinks 'true'
##[debug]implicitDescendants 'true'
##[debug]omitBrokenSymbolicLinks 'true'
##[debug]Search path '/home/runner/_work/_temp/8994c8d4-1724-4c89-9f30-4e2444d4aeb7'
Using temporary directory for storage
📁 CLIs will be downloaded to /home/runner/_work/_temp/openshift-bin
Added /home/runner/_work/_temp/openshift-bin to PATH
##[debug]Move /home/runner/_work/_temp/8994c8d4-1724-4c89-9f30-4e2444d4aeb7/oc to /home/runner/_work/_temp/openshift-bin/oc
##[debug]chmod 755 /home/runner/_work/_temp/openshift-bin/oc
oc installed into /home/runner/_work/_temp/openshift-bin/oc
/home/runner/_work/_temp/openshift-bin/oc version --client
Client Version: 4.9.15
⏩ GitHub enterprise detected; skipping cache. For more information, see actions/cache#505
✅ Successfully installed oc 4.9.15.

✅ Successfully installed 1/1 client:

• oc 4.9.15

::set-output name=installed::{%0A "oc": {%0A "fromCache": false,%0A "installedPath": "/home/runner/_work/_temp/openshift-bin/oc",%0A "url": "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz",%0A "version": "4.9.15"%0A }%0A}
##[debug]='{
##[debug] "oc": {
##[debug] "fromCache": false,
##[debug] "installedPath": "/home/runner/_work/_temp/openshift-bin/oc",
##[debug] "url": "https://mirror.openshift.com/pub/openshift-v4/clients/ocp/4.9.15/openshift-client-linux-4.9.15.tar.gz",
##[debug] "version": "4.9.15"
##[debug] }
##[debug]}'
Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json
##[debug]Node Action run completed with exit code 0
##[debug]Finishing: Setup OpenShift Tools

Complete Job (0s)

##[debug]Starting: Complete job
Cleaning up orphan processes
##[debug]Finishing: Complete job

[divyansh42]

@urswiss won't it be possible for you to share the screen recording, as I'm not able to find out on which step it is taking time?

[urswiss]

I tried, but since I have a company Windows laptop the recording feature seems to be blocked. Anyway, around 90 % of the time is used after the statement Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json

[urswiss]

I just found out that redhat-actions/oc-installer@v1 runs in just 2 seconds. So we are going to use this action for now, even though it's not optimal, as the binaries are from December 2020 and are no longer updated (at least according to the README).

[JulianSchmidgall]

This is still an issue with redhat-actions/[email protected] and oc: 4.15.11. It takes on our side around 3 minutes. Are there any workarounds to get the latest oc version installed quickly?

[buehlerjochen]

Also got the same problem when using the action on GHES. Any news or any workaround to improve the situation?

We are using it like this (@v1 should pick the latest 1.x release):

      - name: Install OpenShift CLI
        uses: redhat-actions/openshift-tools-installer@v1
        with:
          oc: "latest"

[yannickbsaq]

We are observing the same 'slow' issue here using self-hosted github runners (github enterprise) and using it like this

    - name: Install oc and helm clients
      uses: redhat-actions/openshift-tools-installer@v1
      with:
        oc: 4
        helm: 3   

This step averages 2m30 to 3m long to execute and most of the execution seems to be used on this last line:

Wrote out installed versions to /home/runner/_work/_temp/openshift-bin/openshift-clients-installed.json