Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/loadtests9 #1583

Open
wants to merge 128 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
128 commits
Select commit Hold shift + click to select a range
533a7c2
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
3aa7dc6
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
ff664a1
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
6a99977
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
74ef010
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
8755509
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
35d137c
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 23, 2024
bac3bcc
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 24, 2024
cfd78b9
fix: Properly adding RESULTS_DIR as env var
cberg-aot Aug 29, 2024
d7694f2
fix: moving loadtests to own pipelines
cberg-aot Aug 29, 2024
17ba179
feat: split idir & bceid tests; testing new loadtest workflows
cberg-aot Sep 4, 2024
c0ec54f
ORV2-2732: updated get power units and get trailers token to bceid token
bcgov-brwang Sep 4, 2024
96670fa
ORV2-2732: updated dops test cases
bcgov-brwang Sep 5, 2024
3e26684
fix: split bceid and idir for frontend
cberg-aot Sep 5, 2024
480c36e
fix/loadtests9: updated frontend test samplers with selenium webdrive…
bcgov-brwang Sep 17, 2024
8411ca1
ORV2-2780: integrate selenium webdriver for login to get access token
bcgov-brwang Sep 27, 2024
b3ecd7f
Merge branch 'fix/loadtests9' of https://github.com/bcgov/onroutebc i…
bcgov-brwang Oct 8, 2024
c277fec
ORV2-2780: added idir test cases
bcgov-brwang Oct 8, 2024
4c38f78
ORV2-2780: added frontend test cases
bcgov-brwang Oct 8, 2024
d93c692
ORV2-2780: added crud vehicles test cases
bcgov-brwang Oct 9, 2024
1fbb8b6
Merge branch 'main' into fix/loadtests9
cberg-aot Oct 25, 2024
4ab656e
feat: integrating frontend loadtests w/ token into pipeline
cberg-aot Oct 25, 2024
e89d120
Merge branch 'main' into fix/loadtests9
cberg-aot Oct 25, 2024
7759473
Merge branch 'fix/loadtests9' of github.com:bcgov/onroutebc into fix/…
cberg-aot Oct 25, 2024
ced267a
ORV2-2780: added test scenarios for editing profile
bcgov-brwang Oct 28, 2024
0dc51ea
Merge branch 'fix/loadtests9' of https://github.com/bcgov/onroutebc i…
bcgov-brwang Oct 28, 2024
d760631
ORV2-2780: used environment variable to set up the path the driver
bcgov-brwang Oct 28, 2024
e509a19
Merge branch 'fix/loadtests9' of github.com:bcgov/onroutebc into fix/…
cberg-aot Oct 29, 2024
bc39ae8
ORV2-2780: added idir test cases
bcgov-brwang Oct 31, 2024
7b70c6e
Merge branch 'fix/loadtests9' of github.com:bcgov/onroutebc into fix/…
cberg-aot Nov 2, 2024
84b0038
iterating
cberg-aot Nov 3, 2024
1804f2d
iterating
cberg-aot Nov 3, 2024
2a3098e
iterating
cberg-aot Nov 3, 2024
1052b08
iterating
cberg-aot Nov 3, 2024
ffa1448
fix: iterating
cberg-aot Nov 3, 2024
05b9f37
iterating
cberg-aot Nov 4, 2024
f040e1e
iterating
cberg-aot Nov 4, 2024
f620270
iterating
cberg-aot Nov 4, 2024
e2ad0af
iterating
cberg-aot Nov 4, 2024
eb7cc0c
iterating
cberg-aot Nov 4, 2024
660ae75
iterating
cberg-aot Nov 4, 2024
c6ca88e
iterating
cberg-aot Nov 4, 2024
7118b7d
iterating
cberg-aot Nov 4, 2024
b3802f5
iterating
cberg-aot Nov 4, 2024
012832a
iterating
cberg-aot Nov 4, 2024
efa024b
iterating
cberg-aot Nov 4, 2024
84ae3d5
iterating
cberg-aot Nov 5, 2024
8c708c2
ORV2-2780: fixed the amend term oversize test case issue in pipeline
bcgov-brwang Nov 5, 2024
811367c
Merge branch 'fix/loadtests9' of https://github.com/bcgov/onroutebc i…
bcgov-brwang Nov 5, 2024
df5f31f
ORV2-2780: move the file to correct folder
bcgov-brwang Nov 5, 2024
43eb0ce
ORV2-2780: move the file to regression folder
bcgov-brwang Nov 5, 2024
1d77d26
DBC22-2780: amend term oversize updated
bcgov-brwang Nov 6, 2024
be600eb
iterating
cberg-aot Nov 7, 2024
e7a9884
iterating
cberg-aot Nov 7, 2024
ab7a540
Revert "iterating"
cberg-aot Nov 7, 2024
e17e190
iterating
cberg-aot Nov 7, 2024
6bd553d
iterating
cberg-aot Nov 7, 2024
3cb9877
iterating
cberg-aot Nov 7, 2024
e4990c4
iterating
cberg-aot Nov 7, 2024
1e64ce5
iterating
cberg-aot Nov 7, 2024
3f57282
iterating
cberg-aot Nov 7, 2024
86b2d65
iterating
cberg-aot Nov 7, 2024
59dbff0
iterating
cberg-aot Nov 7, 2024
f82cb9a
iterating
cberg-aot Nov 7, 2024
7083999
iterating
cberg-aot Nov 7, 2024
ba037fa
iterating
cberg-aot Nov 7, 2024
8c27f99
iterating
cberg-aot Nov 7, 2024
dcfe468
iterating
cberg-aot Nov 7, 2024
7b9cf2b
iterating
cberg-aot Nov 7, 2024
74b21f6
iterating
cberg-aot Nov 7, 2024
34d597f
iterating
cberg-aot Nov 7, 2024
c3a7cfd
iterating
cberg-aot Nov 7, 2024
1fe019c
iterating
cberg-aot Nov 7, 2024
b8f73f3
iterating
cberg-aot Nov 7, 2024
8b9cf96
iterating
cberg-aot Nov 7, 2024
b089db1
iterating
cberg-aot Nov 8, 2024
191f13d
iterating
cberg-aot Nov 8, 2024
8c4bee9
iterating
cberg-aot Nov 8, 2024
30f1706
iterating
cberg-aot Nov 8, 2024
62dd224
iterating
cberg-aot Nov 8, 2024
1104c42
iterating
cberg-aot Nov 8, 2024
b2c08cf
iterating
cberg-aot Nov 8, 2024
23fc016
iterating
cberg-aot Nov 8, 2024
e1b7a09
iterating
cberg-aot Nov 8, 2024
281d619
iterating
cberg-aot Nov 8, 2024
9c932f4
iterating
cberg-aot Nov 8, 2024
06c96ed
iterating
cberg-aot Nov 8, 2024
112ebca
iterating
cberg-aot Nov 8, 2024
bdaaf12
ORV2-2780: converted jmeter test case to cypress test case
bcgov-brwang Nov 12, 2024
bf8c47b
Merge branch 'fix/loadtests9' of https://github.com/bcgov/onroutebc i…
bcgov-brwang Nov 12, 2024
227b331
iterating
cberg-aot Nov 12, 2024
90b64f9
iterating
cberg-aot Nov 12, 2024
7a181db
iterating
cberg-aot Nov 12, 2024
f8444de
iterating
cberg-aot Nov 12, 2024
7bb2b5a
iterating
cberg-aot Nov 12, 2024
b159b2f
iterating
cberg-aot Nov 12, 2024
2ac2a95
iterating
cberg-aot Nov 13, 2024
1f04ec6
iterating
cberg-aot Nov 13, 2024
e2a684f
Added test case for bceid user update an application
bcgov-brwang Nov 13, 2024
f989594
iterating
cberg-aot Nov 14, 2024
1597178
iterating
cberg-aot Nov 14, 2024
69805ac
iterating
cberg-aot Nov 14, 2024
1d0c79c
iterating
cberg-aot Nov 14, 2024
47b1f07
iterating
cberg-aot Nov 14, 2024
63f2df5
iterating
cberg-aot Nov 14, 2024
89ede3f
iterating
cberg-aot Nov 14, 2024
062badb
iterating
cberg-aot Nov 14, 2024
9929018
iterating
cberg-aot Nov 14, 2024
d723581
iterating
cberg-aot Nov 14, 2024
850be2d
iterating
cberg-aot Nov 14, 2024
1ac164d
iterating
cberg-aot Nov 14, 2024
51efd8b
iterating
cberg-aot Nov 14, 2024
03bf6f0
iterating
cberg-aot Nov 14, 2024
bc8d187
iterating
cberg-aot Nov 14, 2024
adc65d2
iterating
cberg-aot Nov 14, 2024
95e6bab
iterating
cberg-aot Nov 14, 2024
091b237
iterating
cberg-aot Nov 14, 2024
40beecb
ORV2-3027: added new term oversize application for bceid user
bcgov-brwang Nov 14, 2024
e1a35c8
iterating
cberg-aot Nov 15, 2024
3a66001
ORV2-3027: added new term overweight test case
bcgov-brwang Nov 18, 2024
3cebda2
iterating
cberg-aot Nov 19, 2024
4fc9a0f
Revert "iterating"
cberg-aot Nov 19, 2024
49f4452
iterating
cberg-aot Nov 19, 2024
9ebcf98
Revert "iterating"
cberg-aot Nov 19, 2024
1d8d345
iterating
cberg-aot Nov 19, 2024
f5af7f3
ORV2-3027: added bceid user crud on vehicles
bcgov-brwang Nov 19, 2024
d782cb5
Merge branch 'fix/loadtests9' of https://github.com/bcgov/onroutebc i…
bcgov-brwang Nov 19, 2024
947653f
ORV2-3027: added idir void term oversize test case
bcgov-brwang Nov 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
259 changes: 219 additions & 40 deletions .github/workflows/.tests.yml
Original file line number Diff line number Diff line change
@@ -1,74 +1,253 @@
name: Test Workflow

on:
pull_request:
workflow_call:
inputs:
### Required
target:
description: 'PR number, test or prod.'
default: 'test'
required: true
type: string

jobs:
integration-tests:
name: Integration Tests
update-term-oversize:
name: (BCEID) Update Term Oversize
runs-on: ubuntu-22.04
timeout-minutes: 1
defaults:
run:
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- id: cache-npm
uses: actions/cache@v4
with:
path: ~/.npm
key: ${{ runner.os }}-build-cache-node-modules-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-cache-node-modules-
${{ runner.os }}-build-
${{ runner.os }}-

- name: Integration tests

- uses: cypress-io/[email protected]
name: Cypress run
env:
API_NAME: nest
BASE_URL: https://${{ github.event.repository.name }}-${{ inputs.target }}-frontend.apps.silver.devops.gov.bc.ca
run: |
cd integration-tests
npm ci
node src/main.js

cypress-e2e:
name: E2E Tests
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.BCEID_USERNAME}}
CYPRESS_bceid_password: ${{secrets.BCEID_PASSWORD}}
with:
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/bceid-update-term-oversize.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: always()
with:
name: (BCEID) Update Term Oversize
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
name: (BCEID) Update Term Oversize
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

new-term-oversize:
name: (BCEID) New Term Oversize
runs-on: ubuntu-22.04
defaults:
run:
working-directory: frontend
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- id: cache-npm
uses: actions/cache@v4

- uses: cypress-io/[email protected]
name: Cypress run
env:
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.BCEID_USERNAME}}
CYPRESS_bceid_password: ${{secrets.BCEID_PASSWORD}}
with:
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/bceid-new-term-oversize.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: always()
with:
name: (BCEID) New Term Oversize
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
path: ~/.npm
key: ${{ runner.os }}-build-cache-node-modules-${{ hashFiles('**/package-lock.json') }}
restore-keys: |
${{ runner.os }}-build-cache-node-modules-
${{ runner.os }}-build-
${{ runner.os }}-

- uses: cypress-io/github-action@v6
name: (BCEID) New Term Oversize
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

new-term-overweight:
name: (BCEID) New Term Overweight
runs-on: ubuntu-22.04
defaults:
run:
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4

- uses: cypress-io/[email protected]
name: Cypress run
env:
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.BCEID_USERNAME}}
CYPRESS_bceid_password: ${{secrets.BCEID_PASSWORD}}
with:
config: pageLoadTimeout=10000
working-directory: ./frontend
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/bceid-new-term-overweight.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: failure()
if: always()
with:
name: (BCEID) New Term Overweight
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-screenshots
path: ./cypress/screenshots
name: (BCEID) New Term Overweight
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

idir-update-term-oversize:
name: (IDIR) Update Term Oversize
runs-on: ubuntu-22.04
defaults:
run:
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4

- uses: cypress-io/[email protected]
name: Cypress run
env:
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.IDIR_USERNAME}}
CYPRESS_bceid_password: ${{secrets.IDIR_PASSWORD}}
with:
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/idir-amend-term-oversize.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-screenshots
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-videos
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

bceid-crud-vehicles:
name: (BCEID) Crud vehicles
runs-on: ubuntu-22.04
defaults:
run:
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4

- uses: cypress-io/[email protected]
name: Cypress run
env:
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.BCEID_USERNAME}}
CYPRESS_bceid_password: ${{secrets.BCEID_PASSWORD}}
with:
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/bceid-crud-vehicles.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-screenshots
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-videos
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

idir-void-term-oversize:
name: (IDIR) Void Term Oversize
runs-on: ubuntu-22.04
defaults:
run:
working-directory: tests/
strategy:
matrix:
browser: [chrome, firefox, edge]
timeout-minutes: 5
steps:
- uses: actions/checkout@v4

- uses: cypress-io/[email protected]
name: Cypress run
env:
CYPRESS_baseUrl: https://${{ github.event.repository.name }}-${{ inputs.target || 'test' }}-frontend.apps.silver.devops.gov.bc.ca/
CYPRESS_bceid_username: ${{vars.IDIR_USERNAME}}
CYPRESS_bceid_password: ${{secrets.IDIR_PASSWORD}}
with:
browser: ${{ matrix.browser }}
config: pageLoadTimeout=10000,video=true
spec: cypress/e2e/idir-void-term-oversize.cy.js
working-directory: tests

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-screenshots
path: tests/cypress/screenshots/
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`

- uses: actions/upload-artifact@v4
if: always()
with:
name: cypress-videos
path: tests/cypress/videos/
overwrite: true
if-no-files-found: ignore # 'warn' or 'error' are also available, defaults to `warn`
7 changes: 1 addition & 6 deletions .github/workflows/loadtest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,6 @@ on:
default: onroutebc-test-vehicles.apps.silver.devops.gov.bc.ca
required: true
type: string
bearer_token:
description: "The bearer token pertaining to the user running the tests"
required: true
type: string

jobs:
loadtests:
Expand All @@ -37,12 +33,11 @@ jobs:
- name: Run All JMeter Tests In tests Folder
uses: rbhadti94/[email protected]
with:
testFilePath: loadtests
testFilePath: loadtests/frontend/orbc_load_test_plan_frontend.jmx
outputReportsFolder: loadtests/results
args: |
--loglevel INFO
--jmeterlogconf=log.conf
-JBEARER_TOKEN=${{inputs.bearer_token}}
-JDOPS_API_URL=${{inputs.dops_api_url}}
-JFRONTEND_API_URL=${{inputs.frontend_api_url}}
-JVEHICLES_API_URL=${{inputs.vehicles_api_url}}
Expand Down
63 changes: 63 additions & 0 deletions .github/workflows/loadtest_dops.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
---
name: loadtest_dops

on:
workflow_dispatch:
inputs:
### Required
num_threads:
description: 'Number of concurrent threads (USERS) for each test in sequence'
default: 2
required: true
type: integer
release:
description: 'PR number, test or prod.'
default: test
required: true
type: string
bearer_token:
description: "The bearer token pertaining to the user running the tests"
required: true
type: string

jobs:
vars:
name: Set Variables
outputs:
dops_api_url: ${{ steps.vars.outputs.dops_api_url }}
frontend_api_url: ${{ steps.vars.outputs.frontend_api_url }}
vehicles_api_url: ${{ steps.vars.outputs.vehicles_api_url }}
runs-on: ubuntu-22.04
steps:
- name: vars
id: vars
run: |
echo "dops_api_url=onroutebc-${{inputs.environment}}-dops.apps.silver.devops.gov.bc.ca" >> GITHUB_ENV
echo "frontend_api_url=onroutebc-${{inputs.environment}}-frontend.apps.silver.devops.gov.bc.ca" >> GITHUB_ENV
echo "vehicles_api_url=onroutebc-${{inputs.environment}}-vehicles.apps.silver.devops.gov.bc.ca" >> GITHUB_ENV

orbc_load_test_plan_dops:
name: orbc_load_test_plan_dops
environment: ${{ inputs.environment }}
runs-on: ubuntu-22.04
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- uses: rbhadti94/[email protected]
with:
testFilePath: loadtests/dops/orbc_load_test_plan_dops.jmx
outputReportsFolder: results_dops
args: |
--loglevel INFO
--jmeterlogconf=log.conf
-JBEARER_TOKEN=${{inputs.bearer_token}}
-JNUM_THREADS=${{inputs.num_threads}}
-JFRONTEND_API_URL=${{vars.frontend_api_url}}
-JDOPS_API_URL=${{vars.dops_api_url}}
-JVEHICLES_API_URL=${{vars.vehicles_api_url}}
-JRESULTS_DIR=results_dops

- uses: actions/[email protected]
with:
name: jmeter-test-results-dops
path: results_dops
Loading
Loading