Fix/esa data ecosystem #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Running black (via `make format`) was failing because of an internal error where the second formatting pass produced different results than the first pass. Upgrading the relevant deps fixes the issue. Added formatting changes made by `make format` after fixing black error. Also, upgraded `responses` dep in downloader to latest version from version 0.12.1 because `responses.matchers` was added in version 0.14.0, and the downloader tests now make use of `responses.matchers`.
- Add some missing types to improve type checking - Fix some static typing warnings - Fix some "possibly unbound" warnings - Simplify DB session management - Eliminate some unnecessry mock patching in unit tests - Ignore layers/db/Pipfile (generated during unit testing) until migration away from pipenv is complete.
…CT/hls-sentinel2-downloader-serverless into fix/esa_data_ecosystem
There are nearly 19K accepted tile IDs, so using a set instead of a list makes containment tests much faster.
Replace custom date parsing logic with robust library that properly handles the problematic date formats that the custom logic was meant to address.
…CT/hls-sentinel2-downloader-serverless into fix/esa_data_ecosystem
CDK doesn't appear to support rules triggering `CfnStateMachine`s, only `StateMachines`. However, keeping `cdk/link-fetcher.asl.json` in place, as CDK v2 does support supplying definition to `StateMachine` via file, whereas CDK v1 does not (which is why using `CfnStateMachine` was tried). This reverts commit b90c1f8.
The OpenSearch API had a breaking change. In order to get a response containing a non-null value for the totalResults property, it is now necessary to include exactCount=1 as a query parameter. Previously, this was the default, but it is no longer the default.
chuckwondo
approved these changes
Nov 13, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What I am changing
A huge swath of functionality to support migration to the new Copernicus Data Ecosystem and the deprecation of the Inthub and Scihub APIs
How I did it
How you can test it