Fix nfs test setup fail #2022
Merged
Fix nfs test setup fail #2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
phoebusm
force-pushed
the
fix/nfs_test_setup_fail
branch
3 times, most recently
from
be4da7e to
2d1247c
Compare
phoebusm
requested review from
alexowens90,
willdealtry and
poodlewars
as code owners
phoebusm
force-pushed
the
fix/nfs_test_setup_fail
branch
from
2d1247c to
11e075d
Compare
jamesmunro
requested changes
Dec 2, 2024
| @@ -339,7 +337,8 @@ def _start_server(self): | |||
| self.client_cert_file = "" | |||
| self.client_cert_dir = self.working_dir | |||
|
|
|||
| self._p = multiprocessing.Process( | |||
| spawn_context = multiprocessing.get_context('spawn') | |||
There was a problem hiding this comment.
Could we have a comment explaining what the spawn_context is for and how it avoids the issue?
poodlewars
approved these changes
Dec 2, 2024
jamesmunro
approved these changes
Dec 2, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Reference Issues/PRs
What does this implement or fix?
Seg fault in
motostart up during fixture creationTest affected: test_nfs_backed_s3_storage, test_read_path_with_dot
Any other comments?
For both test test_nfs_backed_s3_storage and test_read_path_with_dot , both needs to start up a new moto server respectively:
Sole user of nfs_backed_s3_storage
The moto server for nfs_backed_s3_storage will always be started at the spot
MotoNfsBackedS3StorageFixtureFactory and MotoS3StorageFixtureFactory will created moto server for themselves at the spot
Probably due to some underlying changes of the running image and version incompatibility, moto will seg fault at places related file descriptor, e.g. cannot read a json file packed in the moto wheel, or seg fault at importing werkzeug. I think it's due to file descriptor not being handled properly while using multiprocess fork to start up the server. The issue is gone once switched the startup method to spawn
The issue will be gone as well if the fixture is set autouse , to get the server started along with other storage fixture. e.g. s3_storage . I think that could be explained by different states of file descriptor at the time. So in another word, the issue should affect any late moto server start up.
As it only happens at py3.7, which is deprecated, I don't think it's worth opening a ticket for my observation.
Checklist
Checklist for code changes...