Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bug where Pester fails to run but workflow pipeline itself doesn't fail #1470

Open
wants to merge 161 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
161 commits
Select commit Hold shift + click to select a range
35f98e9
intentionally cause an error
james-garriss Dec 4, 2024
d4c5303
add try catch block
james-garriss Dec 4, 2024
ac0c901
add errorvariable and check count
james-garriss Dec 4, 2024
4ff2b5c
try the CI flag
james-garriss Dec 4, 2024
cb4a46f
add erroraction stop
james-garriss Dec 4, 2024
b091c7e
set ErrorActionPreference
james-garriss Dec 5, 2024
b9a2a47
write the error message
james-garriss Dec 5, 2024
c7f7c40
rethrow the err
james-garriss Dec 5, 2024
d9dbbcc
remove ErrorActionPreference
james-garriss Dec 5, 2024
52548d9
remove try catch
james-garriss Dec 5, 2024
337094f
use exit 1
james-garriss Dec 6, 2024
3e1942f
add write
james-garriss Dec 6, 2024
5f194a6
print result
james-garriss Dec 6, 2024
5fa2772
print just result
james-garriss Dec 6, 2024
e5be4cd
get type
james-garriss Dec 6, 2024
ae1e250
check result for null
james-garriss Dec 6, 2024
28d9175
fix typo
james-garriss Dec 6, 2024
f81e7d4
fix lint
james-garriss Dec 6, 2024
52764e9
should work
james-garriss Dec 6, 2024
d0da0c4
put result outside of try catch
james-garriss Dec 6, 2024
2e86f1e
only try invoke pester
james-garriss Dec 6, 2024
437c641
use configuration
james-garriss Dec 6, 2024
26c28fc
send to stream
james-garriss Dec 9, 2024
a40caa8
write the result
james-garriss Dec 9, 2024
73f0bd2
try passthru
james-garriss Dec 9, 2024
a384091
try passthru wo config
james-garriss Dec 9, 2024
549cbc7
fix lint
james-garriss Dec 9, 2024
8a13c24
try for failure with bad dir
james-garriss Dec 9, 2024
b3d723e
clean up code
james-garriss Dec 9, 2024
708ff04
i hate yaml lint
james-garriss Dec 9, 2024
e930e42
set correct dir
james-garriss Dec 9, 2024
c56c8b3
use function
james-garriss Dec 9, 2024
1aab9d0
fix path
james-garriss Dec 9, 2024
d6e0d25
test w bad dir
james-garriss Dec 9, 2024
b6817ad
update powershell unit tests to use function
james-garriss Dec 9, 2024
8ffcf46
fix linter
james-garriss Dec 9, 2024
2de374b
move test files
james-garriss Dec 9, 2024
fabde4f
debug path
james-garriss Dec 9, 2024
a00fb37
find name of files
james-garriss Dec 9, 2024
e05ea9f
ignore dummy file
james-garriss Dec 9, 2024
f6f63cd
more different remove
james-garriss Dec 9, 2024
9145f40
remove file from collection
james-garriss Dec 9, 2024
83d7c9a
find files
james-garriss Dec 9, 2024
747aed6
source functions
james-garriss Nov 15, 2024
69cb887
test pssa install
james-garriss Nov 15, 2024
8c73c10
add params to invoke call
james-garriss Nov 15, 2024
e4a4e5e
add results test
james-garriss Nov 15, 2024
e344a30
add write test
james-garriss Nov 15, 2024
cac5992
fix pssa error (oh the irony)
james-garriss Nov 15, 2024
a55d089
Remove afterall
james-garriss Nov 15, 2024
6087ae2
comment
james-garriss Nov 15, 2024
c27968e
all comms
james-garriss Nov 15, 2024
5b56337
remove alt test
james-garriss Nov 15, 2024
c81cfdf
test output
james-garriss Nov 15, 2024
414f30a
fix order
james-garriss Nov 15, 2024
cbf53e5
Fix typo
james-garriss Nov 20, 2024
ec5eca5
try write-host
james-garriss Nov 20, 2024
d39b7d2
sssh to PSSA
james-garriss Nov 20, 2024
e0d67f5
add before discovery
james-garriss Nov 20, 2024
849e248
fixed scope
james-garriss Nov 20, 2024
70472fc
Split describe
james-garriss Nov 20, 2024
91b935a
fix lint
james-garriss Nov 20, 2024
020c55a
fix typo
james-garriss Nov 20, 2024
0ca0f43
fix another typo grrrrr
james-garriss Nov 20, 2024
2d93e8f
try redirect
james-garriss Nov 20, 2024
2c993b3
remove write host
james-garriss Nov 20, 2024
d530107
fix the test to be a complete sentence
james-garriss Nov 20, 2024
0acafe5
Fix or
james-garriss Nov 22, 2024
b6cd533
FIx or
james-garriss Nov 22, 2024
83f9f9e
fix or
james-garriss Nov 22, 2024
aab0a61
Find last output
james-garriss Nov 22, 2024
adf0db7
Condense to one it
james-garriss Nov 22, 2024
7738f7b
set output to global
james-garriss Nov 22, 2024
98ce945
Suppress PSSA global
james-garriss Nov 22, 2024
968dbba
Split back into 2 describes
james-garriss Nov 22, 2024
35b778a
test concurrency
james-garriss Nov 22, 2024
60a7a6c
remove publish tests
james-garriss Nov 22, 2024
1f51d44
remove pssa
james-garriss Nov 22, 2024
ef9a22f
put publish back
james-garriss Nov 22, 2024
39f1de2
combine pssa check
james-garriss Nov 22, 2024
ea1cd53
fix lint
james-garriss Nov 22, 2024
24c06bc
add missing variable
james-garriss Nov 22, 2024
96d1efc
remove tests
james-garriss Nov 22, 2024
122188d
fix lint
james-garriss Nov 22, 2024
4c33edf
debug
james-garriss Nov 22, 2024
78bea91
debug
james-garriss Nov 22, 2024
e2c8cc8
fix lint
james-garriss Nov 22, 2024
10c4922
use write-warning
james-garriss Nov 22, 2024
43fd3b0
debug paths
james-garriss Nov 22, 2024
3e27fae
fix lint
james-garriss Nov 22, 2024
6c31eb5
Debug
james-garriss Nov 22, 2024
122deab
fix scriptpath
james-garriss Nov 22, 2024
f52b045
convert to string
james-garriss Nov 22, 2024
ac8181a
fix lint
james-garriss Nov 22, 2024
8a892db
invoke again
james-garriss Nov 22, 2024
7346658
fix path
james-garriss Nov 22, 2024
f0ae516
test invoke in bd
james-garriss Nov 22, 2024
36ad99b
test output
james-garriss Nov 22, 2024
06cf676
suppress PSSA
james-garriss Nov 22, 2024
65cfa71
tweak global
james-garriss Nov 22, 2024
56b32d4
just check install
james-garriss Nov 22, 2024
c14f885
fix typo in invoke
james-garriss Nov 22, 2024
73e4e4b
Change to Before All
james-garriss Nov 22, 2024
7172f8d
just run invoke pssa
james-garriss Dec 2, 2024
108b0a9
just run pssa
james-garriss Dec 2, 2024
455dc0f
tweak test
james-garriss Dec 2, 2024
6179116
fix source
james-garriss Dec 2, 2024
19922ae
remvoe suppress
james-garriss Dec 2, 2024
9c41409
remove before discovery
james-garriss Dec 2, 2024
a622563
list path
james-garriss Dec 2, 2024
f486c94
just invoke
james-garriss Dec 2, 2024
bc1278a
don't invoke
james-garriss Dec 2, 2024
a75d570
fix source
james-garriss Dec 2, 2024
b3021be
add suppress
james-garriss Dec 2, 2024
aa5b6c0
debug
james-garriss Dec 2, 2024
21118b4
add foreach
james-garriss Dec 2, 2024
56f6ad0
summarize
james-garriss Dec 2, 2024
7d31c0c
check exists
james-garriss Dec 2, 2024
a5794e4
drop param
james-garriss Dec 2, 2024
9a8fd85
change to warning
james-garriss Dec 2, 2024
d89fb19
change more to warning
james-garriss Dec 2, 2024
2cb026b
add debug mode
james-garriss Dec 2, 2024
eff9abb
finish warning
james-garriss Dec 2, 2024
0452695
remove exit 0
james-garriss Dec 2, 2024
e5b2284
test install
james-garriss Dec 2, 2024
38910bc
use output
james-garriss Dec 2, 2024
df9228d
use 3
james-garriss Dec 2, 2024
816aad3
check write warning
james-garriss Dec 2, 2024
4deb3f3
use warning variable
james-garriss Dec 2, 2024
5c918b9
write out the warnings
james-garriss Dec 2, 2024
6ea07d2
fix the test
james-garriss Dec 2, 2024
f807a92
split tests
james-garriss Dec 3, 2024
9a9a9ce
change location
james-garriss Dec 3, 2024
7ef907f
fix warning variable name
james-garriss Dec 3, 2024
2bc9a5c
time serial
james-garriss Dec 3, 2024
ccd125c
make serial for real
james-garriss Dec 3, 2024
9d6f8e6
Put in final form
james-garriss Dec 3, 2024
1ca834d
Put workflow pipeline back
james-garriss Dec 3, 2024
afca642
tweak docs
james-garriss Dec 9, 2024
74ca59e
remove unused
james-garriss Dec 9, 2024
d818a00
exclude dummy
james-garriss Dec 9, 2024
d9e7110
list file
james-garriss Dec 10, 2024
807b6da
fix bug in job name
james-garriss Dec 10, 2024
aee6bdb
put scriptblock inside {}
james-garriss Dec 10, 2024
a14337d
fix path
james-garriss Dec 10, 2024
295ab97
fix path.
james-garriss Dec 10, 2024
a6b2e0f
test for exit codes
james-garriss Dec 10, 2024
b28ba90
just run it.
james-garriss Dec 10, 2024
a72fbed
add missing }
james-garriss Dec 10, 2024
96b9509
write the result
james-garriss Dec 10, 2024
ebd9459
Run locally to see fail
james-garriss Dec 10, 2024
21a2d08
see the error
james-garriss Dec 10, 2024
aec09b7
Using Pester to test Pester is no
james-garriss Dec 10, 2024
56be848
add trivial pester pass
james-garriss Dec 10, 2024
e37f2ac
use trivial test
james-garriss Dec 10, 2024
27b6e93
There is not a dummy fail after all.
james-garriss Dec 12, 2024
667d97d
Fail intentionally
james-garriss Dec 12, 2024
3e70d6c
Set to pass, add comment
james-garriss Dec 12, 2024
228ab42
Undo changes
james-garriss Dec 12, 2024
78510e7
Merge branch 'main' into 1454-pipeline-does-not-fail-on-pester-fail
james-garriss Dec 19, 2024
2128511
fix typo
james-garriss Jan 17, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/unit_test_powershell.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ jobs:
- name: Check out repository code
uses: actions/checkout@v4
- name: Run Pester Tests
if: ${{ !cancelled() }}
shell: powershell
run: |
Invoke-Pester -Output 'Detailed' -Path 'PowerShell/ScubaGear/Testing/Unit/PowerShell'
# Source the function
. utils/workflow/Invoke-PesterTests.ps1
Invoke-PesterTests -Path 'PowerShell/ScubaGear/Testing/Unit/PowerShell'
7 changes: 4 additions & 3 deletions .github/workflows/unit_test_workflows.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,8 @@ jobs:
steps:
- name: Check out repository code
uses: actions/checkout@v4
- name: Run Pester Tests
- name: Run Invoke PSSA Pester Tests
run: |
# This should run all workflow unit tests
Invoke-Pester -Output 'Detailed' -Path 'Testing/workflow'
# Source the function
. utils/workflow/Invoke-PesterTests.ps1
Invoke-PesterTests -Path 'Testing/workflow'
7 changes: 7 additions & 0 deletions Testing/PesterTestFile/SimplePassFile.ps1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our test file process to date was usually to embed the files in a subdirectory under the place where the tests themselves were written. So would it make more sense to put the PesterTestFile directory under workflow?

Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Ultra trivial Pester test that should always pass.

Describe "Pester Check" {
It "Passing test should pass." {
$true | Should -Be $true
}
}
4 changes: 2 additions & 2 deletions Testing/workflow/Invoke-PSSA.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,8 @@ Describe "PSSA Check" {
It "PSSA should write output" {
# There should be write-warning statements
$global:Warnings | Should -Not -BeNullOrEmpty
# Note: This is a bit fragile. It only works as long as one of these two
# summary statements is the final output written by the Invoke function.
# Note: This is a little bit fragile. It only works as long as one of these two
# summary statements is the final output written.
$global:Warnings | Select-Object -Last 1 | Should -BeIn @("Problems were found in the PowerShell scripts.", "No problems were found in the PowerShell scripts.")
}
}
11 changes: 11 additions & 0 deletions Testing/workflow/Invoke-PesterTests.Tests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# The purpose of this test is to have a trivial Pester test that passes.
# This simply verifies that Pester is running.

Describe "Pester Check" {
It "Passing test should pass." {
$TestFile = Join-Path -Path $PSScriptRoot -ChildPath '../../Testing/PesterTestFile/SimplePassFile.ps1' -Resolve
# Source the function
. utils/workflow/Invoke-PesterTests.ps1
Invoke-PesterTests -Path $TestFile
}
}
30 changes: 30 additions & 0 deletions utils/workflow/Invoke-PesterTests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
function Invoke-PesterTests {
<#
.SYNOPSIS
Calls the Invoke-Pester command to test the PowerShell files in some location.
.PARAMETER Path
The path to the PowerShell. This can be a directory or a file.
#>
[CmdletBinding()]
param(
[Parameter(Mandatory = $true)]
[string]
$Path
)
Write-Warning "The path to test is ${Path}."
# The -PassThru parameter is what allows the output to be passed to the $result output.
# https://pester.dev/docs/commands/Invoke-Pester#-passthru
$result = Try {
Invoke-Pester -Output 'Detailed' -Path $Path -PassThru
} Catch {
# This catches an error with the Pester tests.
Write-Warning "An error occurred while running the Pester tests:"
Write-Warning $_
exit 1
}
# This catches an error that causes Pester not to run at all
# (e.g., if the -Path is set to a nonexistent directory).
if ($null -eq $result) {
throw "The Pester tests failed to run."
}
}
Loading