Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dockerfile: added airflow constrains #180

Merged
merged 1 commit into from
Nov 9, 2023
Merged

Dockerfile: added airflow constrains #180

merged 1 commit into from
Nov 9, 2023

Conversation

ErnestaP
Copy link
Contributor

@ErnestaP ErnestaP commented Nov 7, 2023

No description provided.

@ErnestaP ErnestaP marked this pull request as ready for review November 7, 2023 15:03
@ErnestaP ErnestaP force-pushed the add-contrains branch 2 times, most recently from 5436417 to 17fbb9e Compare November 8, 2023 10:45
@codecov-commenter
Copy link

codecov-commenter commented Nov 8, 2023

Codecov Report

Merging #180 (dd7eb4c) into main (cb6dd8f) will decrease coverage by 0.60%.
Report is 11 commits behind head on main.
The diff coverage is 86.88%.

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

@@            Coverage Diff             @@
##             main     #180      +/-   ##
==========================================
- Coverage   93.22%   92.62%   -0.60%     
==========================================
  Files          94      105      +11     
  Lines        4189     4598     +409     
==========================================
+ Hits         3905     4259     +354     
- Misses        284      339      +55     
Files Coverage Δ
dags/common/cleanup.py 100.00% <100.00%> (ø)
dags/common/parsing/parser.py 100.00% <ø> (ø)
dags/common/pull_ftp.py 100.00% <100.00%> (ø)
dags/common/utils.py 85.06% <100.00%> (+2.75%) ⬆️
dags/elsevier/sftp_service.py 100.00% <100.00%> (ø)
dags/iop/repository.py 100.00% <ø> (ø)
dags/springer/repository.py 92.10% <ø> (ø)
tests/integration/aps/test_utils.py 84.21% <ø> (-0.41%) ⬇️
...ests/integration/elsevier/test_files_harvesting.py 100.00% <100.00%> (ø)
tests/units/elsevier/test_elsevier_parser.py 100.00% <100.00%> (ø)
... and 11 more

@@ -1,2 +1,3 @@
psycopg2==2.9.6
psycopg2-binary==2.9.6
connexion==2.14.2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not needed. should be taken care of by the constraints..if it is not installed by the constraints, we should investigate which library needs it, and check if we should install differently

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Constraints files are requirements files that only control which version of a requirement is installed, not whether it is installed or not. Their syntax and contents is nearly identical to Requirements Files.
https://pip.pypa.io/en/stable/user_guide/#constraints-files

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok ..what i mean is:

for postgres, we shouldnt have psycopg2==2.9.6 or psycopg2-binary==2.9.6, but install it as apache-airflow[celery, postgres]==2.6.0

this way it will install all dependencies that are needed for postgres, and use the constraints for choosing a version

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the same way, we should try to see which package tries to install connexion and why the constraints dont work

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the same way, we should try to see which package tries to install connexion and why the constraints do not work

@ErnestaP ErnestaP marked this pull request as draft November 8, 2023 12:44
@ErnestaP ErnestaP marked this pull request as ready for review November 8, 2023 14:40
@ErnestaP ErnestaP requested a review from pamfilos November 8, 2023 14:40
@pamfilos pamfilos merged commit 4f71d05 into main Nov 9, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants