Thread-local Session Management and Cookie Reuse to Address EDL DSE issue #909
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request addresses a DSE issue reported by Earthdata Login (EDL) during data download requests involving large amount of granules through EarthAccess. One issue was traced to EarthAccess creating a new session for each download instead of reusing the existing session cookie, which is intended to reduce the number of requests to EDL by the DAAC data distribution endpoint (TEA). This PR introduces changes to enhance session cookie handling within each thread, avoiding EDL requests from TEA in subsequent authenticated data downloads.
Key modifications in this PR include:
_clone_session_in_local_thread
method to clone and store sessions in thread-local storage._download_file
method to utilize thread-local session cookies and reduce redundant session creation within each thread.get_requests_session
to retrieve an existing session or raise an error if none exists.Notes:
Pull Request (PR) draft checklist - click to expand
contributing documentation
before getting started.
title such as "Add testing details to the contributor section of the README".
Example PRs: #763
example
closes #1
. SeeGitHub docs - Linking a pull request to an issue.
CHANGELOG.md
with details about your change in a section titled## Unreleased
. If such a section does not exist, please create one. FollowCommon Changelog for your additions.
Example PRs: #763
README.md
with details of changes to theearthaccess interface, if any. Consider new environment variables, function names,
decorators, etc.
Click the "Ready for review" button at the bottom of the "Conversation" tab in GitHub
once these requirements are fulfilled. Don't worry if you see any test failures in
GitHub at this point!
Pull Request (PR) merge checklist - click to expand
Please do your best to complete these requirements! If you need help with any of these
requirements, you can ping the
@nsidc/earthaccess-support
team in a comment and wewill help you out!
Request containing "pre-commit.ci autofix" to automate this.
📚 Documentation preview 📚: https://earthaccess--909.org.readthedocs.build/en/909/