Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix remaining misuses of capturing the default file descriptors in flytectl unit tests #5950

Merged
merged 4 commits into from
Nov 4, 2024

Conversation

eapolinario
Copy link
Contributor

@eapolinario eapolinario commented Nov 1, 2024

Tracking issue

Closes #5937

Why are the changes needed?

#5325 didn't fix all the misuses of the capturing of stdout+stderr in unit tests. This PR fixes the remaining cases.

What changes were proposed in this pull request?

We add the teardown as a defer statement to all uses of testutils.Setup.
Notice that in most cases we use barebones tests, which don't have a central place for setup/teardowns, so even though there's a comment explaining that Teardown has to be invoked, there's no automated way, at least to my knowledge, to do that.

edit: as suggested by @Sovietaced , we can leverage the setup call to hookup the call to cleanup (to restore stdout and stderr).

How was this patch tested?

Confirmed that tests ran successfully Ran unit tests in a loop a few times:

❯ for i in {1..100}; do
  go test -count=1 -cover ./... -race || break;
done

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

@eapolinario eapolinario changed the title Search and replace remaining instances of testutils.Setup() Fix remaining misuses of capturing the default file descriptors in flytectl unit tests Nov 1, 2024
Copy link

codecov bot commented Nov 1, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 36.80%. Comparing base (f745030) to head (28c7a56).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #5950      +/-   ##
==========================================
- Coverage   36.81%   36.80%   -0.01%     
==========================================
  Files        1310     1310              
  Lines      131032   131034       +2     
==========================================
- Hits        48234    48230       -4     
- Misses      78612    78618       +6     
  Partials     4186     4186              
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.02% <ø> (-0.03%) ⬇️
unittests-flytecopilot 11.73% <ø> (ø)
unittests-flytectl 62.39% <100.00%> (+<0.01%) ⬆️
unittests-flyteidl 6.92% <ø> (ø)
unittests-flyteplugins 53.64% <ø> (ø)
unittests-flytepropeller 42.90% <ø> (ø)
unittests-flytestdlib 55.39% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@Sovietaced
Copy link
Contributor

I'm not super familiar with this style of unit tests but I feel that passing *testing.T to the setup function and then using t.Cleanup(...) is less error prone because it doesn't rely on anyone remembering to use defer teardown().

@eapolinario eapolinario merged commit 636cc23 into master Nov 4, 2024
54 checks passed
@eapolinario eapolinario deleted the fix-sandbox-cmd-unit-tests-flakiness branch November 4, 2024 20:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] github.com/flyteorg/flyte/flytectl/cmd/sandbox unit test is flaky
2 participants