Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add transpiler to produce pip requirements file #345

Merged
merged 1 commit into from
Aug 29, 2023

Conversation

valentin-krasontovitsch
Copy link
Contributor

this adds a transpiler that can create a pip requirements file from a release matrix file (and a repo).

could for instance be used in workflows as input for checking dependency conflicts, checking for upgradable packages, or scanning for vulnerabilities.

Comment on lines 115 to 129
# 149: fail
# 180: fail
# 180:-25 - fail
# 200:-25 - fail
# 260:-25 - worked
# 200:261 - failed
# 230:261 - worked
# 200:230 - worked
# 220:261 - worked
# 200:241 - worked
# 200:251 - failed
# 210:251 - worked
# 205:251 - worked
# snyk_search_string = "\n".join(snyk_search_string.splitlines()[202:251])
# snyk_search_string = "soupsieve==2.3.2.post1"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the purpose of these comments?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh thank you that was me debugging a snyk failure - gonna remove those 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and all the other print statements ...

Comment on lines +59 to +120
for rhel_ver in ("rhel7",):
for py_ver in ("py38",):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an interesting way of writing:

rhel_ver = "rhel7"
py_ver = "py38"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed! i think it comes from a time when there were several potential versions of OS and python. if i scrap it here, i'd probably have to scrap it many places in the tests and/or code... what do you think? should i keep it consistent, do lots of changes, ... ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is only a matter of time before we add py310 ...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great! and i guess we will support both 3.8 and 3.10, at least for a while?

@kwinkunks
Copy link
Member

Wanted to ask if you've had a chance to look at #315 to see if/how they interact?

Question: is a "matrix file" a single release file for one 'element' in the matrix, or is it the big 'all platforms, all versions' file that combine produces? (The latter is what I think of as a matrix file, but the language we currently use in the repo is confusing / ambiguous).

Wondering if instead of adding a new subcommand, it could be clearer to add 'filtering' to the existing transpile command? That is, "give me the transpiled release file(s) but only for those packages with such-and-such a build system?

Lastly, I think the new functions need docstrings.

@valentin-krasontovitsch
Copy link
Contributor Author

thanks for bringing that issue / PR to my attention - i have not looked at it, but at first quick glance it seems orthogonal to what i try to achieve here, but would benefit from / have to be adjusted to the changes there.

i am so sorry, but i'm afraid i am not able to relieve you of your confusion, because i hold and cherish that same confusion myself : D
i suspect that the language is confusing because it doesn't accurately depict the current situation, and i try not to think about it too much, which of course works as long as i don't need to talk to anybody about details ^^

thing is, the transpilation as it stands does not produce "what i want", namely a pip requirements file of the form

pkg1==version1
pkg2==version2

rather i think it produces just some different yaml.

docstrings, yes, of course, i will look into that, thank you : )

@xjules
Copy link
Contributor

xjules commented Feb 27, 2023

thanks for bringing that issue / PR to my attention - i have not looked at it, but at first quick glance it seems orthogonal to what i try to achieve here, but would benefit from / have to be adjusted to the changes there.

i am so sorry, but i'm afraid i am not able to relieve you of your confusion, because i hold and cherish that same confusion myself : D i suspect that the language is confusing because it doesn't accurately depict the current situation, and i try not to think about it too much, which of course works as long as i don't need to talk to anybody about details ^^

thing is, the transpilation as it stands does not produce "what i want", namely a pip requirements file of the form

pkg1==version1
pkg2==version2

rather i think it produces just some different yaml.

docstrings, yes, of course, i will look into that, thank you : )

I guess regardless of this PR we should try to merge this first, where in the additional parameters (python and os versions) can be linked into your PR or?

@valentin-krasontovitsch
Copy link
Contributor Author

yes totally! gonna wait with this for now, and rebase / adjust once the other transpiler PR is merged

@valentin-krasontovitsch valentin-krasontovitsch force-pushed the add-pipfile-transpiler branch 2 times, most recently from f9f172d to 34bba01 Compare August 29, 2023 06:37
@valentin-krasontovitsch
Copy link
Contributor Author

rebasing is done, can i please get another review to get his merged? @yngve-sk

Copy link

@yngve-sk yngve-sk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM🚀

@valentin-krasontovitsch valentin-krasontovitsch merged commit 3272d79 into main Aug 29, 2023
@valentin-krasontovitsch valentin-krasontovitsch deleted the add-pipfile-transpiler branch August 29, 2023 07:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

6 participants