Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for extra pip install -e . for mac pip install #1570

Closed
wants to merge 2 commits into from

Conversation

Vargol
Copy link
Contributor

@Vargol Vargol commented Nov 26, 2022

#1521 added an extra step to the pip only install on do a -e . removing the same action from the requirements-*.txt files

This tries to get rid of it for the mac as I didn't need to do it. The only differences are I use python 3.10 and run with --prefer-binary.
I've put the prefer binary option back in into environments-and-requirements/requirements-mac-mps-cpu.txt
and reinstated the -e .

If this works its worth testing on the on other requirements files.

@Vargol Vargol mentioned this pull request Nov 26, 2022
@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

imho this change should also be done to the other environment's requirements files, not only to mac, to prevent having different docs for different environments again.

It then could also be removed from the action

@lstein
Copy link
Collaborator

lstein commented Nov 26, 2022

@mauwii Thanks for making this PR. A couple of days ago this line was causing the 100% CPU hang for you but not for me. Do you know what changed?

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

@mauwii Thanks for making this PR. A couple of days ago this line was causing the 100% CPU hang for you but not for me. Do you know what changed?

I only suggested this PR 🙈

imho it could be working again after this change

Edit by @lstein (not from me)
Ok, I'll test on Linux and Windows. (Actually I know it is working on Linux 3.9 because this is how I've got my local requirements set up)

@Vargol
Copy link
Contributor Author

Vargol commented Nov 26, 2022

Yep I mentioned it in the PR details, but I can only test on macOS , so see if this works with the macOS actions in case there's something about python 3.9 that causes the issue or something the actions are doing different (I looks like it does an upgrade -r rather than install -r) that might change the results I get when building locally.

@lstein
Copy link
Collaborator

lstein commented Nov 26, 2022

A previous PR has had its CI tests in the queue for nearly an hour. What determines the frequency with which a runner picks up a job? Are we in contention with other projects?

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

Yep I mentioned it in the PR details, but I can only test on macOS , so see if this works with the macOS actions in case there's something about python 3.9 that causes the issue or something the actions are doing different (I looks like it does an upgrade -r rather than install -r) that might change the results I get when building locally.

The Conda action does a upgrade and has absolutley nothing to do with the python version. The pip Action does exactly the same for python 3.9 and 3.10, no difference there at all.

A previous PR has had its CI tests in the queue for nearly an hour. What determines the frequency with which a runner picks up a job? Are we in contention with other projects?

I already started searching if there is something that the previous action gets canceled if a new commit comes in (in azure-pipelines you can do this, not so aware with github actions yet)

@Vargol
Copy link
Contributor Author

Vargol commented Nov 26, 2022

Yep I mentioned it in the PR details, but I can only test on macOS , so see if this works with the macOS actions in case there's something about python 3.9 that causes the issue or something the actions are doing different (I looks like it does an upgrade -r rather than install -r) that might change the results I get when building locally.

The Conda action does a upgrade and has absolutley nothing to do with the python version. The pip Action does exactly the same for python 3.9 and 3.10, no difference there at all.

The conda action does an upgrade, the pip action does an upgrade. I do an install
The difference between python 3.9's action and python 3.10's action is one uses python 3.9 and the other uses python 3.10. You'd hope that would make a difference but I wouldn't assume it.

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

Yep I mentioned it in the PR details, but I can only test on macOS , so see if this works with the macOS actions in case there's something about python 3.9 that causes the issue or something the actions are doing different (I looks like it does an upgrade -r rather than install -r) that might change the results I get when building locally.

The Conda action does a upgrade and has absolutley nothing to do with the python version. The pip Action does exactly the same for python 3.9 and 3.10, no difference there at all.

The conda does an upgrade, the action does an upgrade. I do an install The difference between python 3.9's action and python 3.10's action is one uses python 3.9 and the other uses python 3.10. You'd hope that would make a difference but I wouldn't assume it.

So if a package works on python 3.10 but not 3.9 there is no difference?!?!?!!?!

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

image

I dit not know that the owner can edit other`s comments :-o

@Vargol
Copy link
Contributor Author

Vargol commented Nov 26, 2022

The conda does an upgrade, the action does an upgrade. I do an install The difference between python 3.9's action and python 3.10's action is one uses python 3.9 and the other uses python 3.10. You'd hope that would make a difference but I wouldn't assume it.

So if a package works on python 3.10 but not 3.9 there is no difference?!?!?!!?!

Sorry thats a typo, mean to say 'You'd hope that wouldn't make a difference but I wouldn't assume it.'

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

Yep I mentioned it in the PR details, but I can only test on macOS , so see if this works with the macOS actions in case there's something about python 3.9 that causes the issue or something the actions are doing different (I looks like it does an upgrade -r rather than install -r) that might change the results I get when building locally.

Oh - good catch @Vargol

this slipped through since I changed Installation order (first did a install -e, then the install --upgrade -r).

So the --upgrade can then be removed as well as the -e . (as requested here)

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

The conda does an upgrade, the action does an upgrade. I do an install The difference between python 3.9's action and python 3.10's action is one uses python 3.9 and the other uses python 3.10. You'd hope that would make a difference but I wouldn't assume it.

So if a package works on python 3.10 but not 3.9 there is no difference?!?!?!!?!

Sorry thats a typo, mean to say 'You'd hope that wouldn't make a difference but I wouldn't assume it.'

this does not answer the question cause imho it makes a difference if you run invoke with python 3.9 or python 3.10

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

A previous PR has had its CI tests in the queue for nearly an hour. What determines the frequency with which a runner picks up a job? Are we in contention with other projects?

I already started searching if there is something that the previous action gets canceled if a new commit comes in (in azure-pipelines you can do this, not so aware with github actions yet)

Sorry, I didn't answer your question properly: As soon as there are free runners for your repository which are fitting the action (like os/arch), it will pick them up and start.

@lstein
Copy link
Collaborator

lstein commented Nov 26, 2022

I see the model loading failures in the CI that are originating in the model scanner and will investigate.

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

I see the model loading failures in the CI that are originating in the model scanner and will investigate.

I am not sure why suddenly the cache isn't available anymore:

image

and since the PR does not have access to the secret it cannot download the model itself.

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

@lstein: So maybe we should consider this if there are no legal problems by doing this

@Vargol
Copy link
Contributor Author

Vargol commented Nov 26, 2022

You where the one saying there was no difference..

"The Conda action does a upgrade and has absolutley nothing to do with the python version. The pip Action does exactly the same for python 3.9 and 3.10, no difference there at all."

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

"The Conda action does a upgrade and has absolutley nothing to do with the python version. The pip Action does exactly the same for python 3.9 and 3.10, no difference there at all."

No difference in what the action does between python 3.9 and python 3.10 (like using upgrade for python 3.9 but only install for 3.10). So yes, there is no difference in what the action does in terms of which commands it is running ....

@mauwii
Copy link
Contributor

mauwii commented Nov 26, 2022

The conda action does an upgrade, the pip action does an upgrade. I do an install The difference between python 3.9's action and python 3.10's action is one uses python 3.9 and the other uses python 3.10. You'd hope that would make a difference but I wouldn't assume it.

we just disabled python 3.9 since it is not working anymore, so yes, it makes a difference!

@mauwii mauwii self-requested a review November 27, 2022 00:09
Copy link
Contributor

@mauwii mauwii left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also update the other environment's requirement files, not only mac.
also update the action then by removing the line ${{ env.pythonLocation }}/bin/pip install -e .

-r environments-and-requirements/requirements-base.txt

grpcio<1.51.0
protobuf==3.19.6
torch<1.13.0
torchvision<0.14.0
# -e .
-e .
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please also update the other environment's requirement files, not only mac.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also update the action then by removing the line ${{ env.pythonLocation }}/bin/pip install -e .

@mauwii
Copy link
Contributor

mauwii commented Nov 27, 2022

unfortunatelly I cannot add the changes myself:

git push
Enumerating objects: 17, done.
Counting objects: 100% (17/17), done.
Delta compression using up to 8 threads
Compressing objects: 100% (9/9), done.
Writing objects: 100% (9/9), 1.19 KiB | 1.19 MiB/s, done.
Total 9 (delta 8), reused 0 (delta 0), pack-reused 0
remote: Resolving deltas: 100% (8/8), completed with 7 local objects.
To github.com:Vargol/InvokeAITesting
 ! [remote rejected]   pr/Vargol/1570 -> pr/Vargol/1570 (permission denied)
error: failed to push some refs to 'github.com:Vargol/InvokeAITesting'

@mauwii
Copy link
Contributor

mauwii commented Nov 27, 2022

closed since fixed in #1574

@mauwii mauwii closed this Nov 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants