-
-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create container-build.yml #200
Open
aryanbhosale
wants to merge
36
commits into
openclimatefix:main
Choose a base branch
from
aryanbhosale:main
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 9 commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
7fe5a6d
Create container-build.yml
aryanbhosale f749532
Update Dockerfile
aryanbhosale 46f3722
updated docker
aryanbhosale 3a4d998
qsf
aryanbhosale b0bb165
full app
aryanbhosale f053d0c
build
aryanbhosale 839d39e
rm poetry
aryanbhosale 3f55772
q
aryanbhosale 3354c7d
not everything
aryanbhosale b70e84e
Merge branch 'openclimatefix:main' into main
aryanbhosale b025399
copy pyproj
aryanbhosale ccd9030
Merge branch 'main' of https://github.com/aryanbhosale/open-source-qu…
aryanbhosale 441c002
namespace
aryanbhosale 2e95d0b
verbose
aryanbhosale 4c7c284
explicit
aryanbhosale 96de732
additional deps
aryanbhosale 669ab43
find
aryanbhosale eb92585
pandas add
aryanbhosale d33923a
rm unnc
aryanbhosale 07e9d63
h5py
aryanbhosale abe105e
revert frontend dockerisation
aryanbhosale 0f273a0
line
aryanbhosale 81c4a29
rm healthcheck
aryanbhosale c68a872
numcodecs
aryanbhosale 00f8952
explicit numcodecs
aryanbhosale 50b4623
no deps flag
aryanbhosale 48804e6
contraints
aryanbhosale 5fc13e7
from uk pvnet
aryanbhosale cd14498
conda
aryanbhosale 2f973d1
rm pytorch install numcodecs conda
aryanbhosale 0de6465
Merge branch 'openclimatefix:main' into main
aryanbhosale 6f7db85
Update Dockerfile
aryanbhosale aa5d012
Update docker-compose.yaml
aryanbhosale 1c8de2f
Update Dockerfile
aryanbhosale 2916fb8
arm and amd
aryanbhosale b548079
disable env
aryanbhosale File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,57 @@ | ||
name: Create and publish Container image | ||
|
||
on: | ||
push: | ||
branches: | ||
- main | ||
pull_request: | ||
branches: | ||
- main | ||
|
||
env: | ||
REGISTRY: ghcr.io | ||
IMAGE_NAME: ${{ github.repository }} | ||
|
||
jobs: | ||
build-and-push-image: | ||
runs-on: ubuntu-latest | ||
permissions: | ||
contents: read | ||
packages: write | ||
|
||
steps: | ||
- name: Checkout repository | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up Docker Buildx | ||
uses: docker/setup-buildx-action@v2 | ||
|
||
- name: Login to Container registry | ||
uses: docker/login-action@v2 | ||
with: | ||
registry: ${{ env.REGISTRY }} | ||
username: ${{ github.actor }} | ||
password: ${{ secrets.GITHUB_TOKEN }} | ||
|
||
- name: Docker meta | ||
id: meta | ||
uses: docker/metadata-action@v4 | ||
with: | ||
images: | | ||
${{ env.REGISTRY }}/${{ env.IMAGE_NAME }} | ||
tags: | | ||
type=raw,value=latest,enable={{is_default_branch}} | ||
type=ref,event=branch | ||
type=ref,event=pr | ||
type=edge | ||
type=sha | ||
|
||
- name: Build and push | ||
uses: docker/build-push-action@v4 | ||
with: | ||
platforms: linux/amd64,linux/arm64 | ||
context: . | ||
file: Dockerfile | ||
push: ${{ github.event_name != 'pull_request' }} | ||
tags: ${{ steps.meta.outputs.tags }} | ||
labels: ${{ steps.meta.outputs.labels }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
you need to copy the
pyproject.toml
file overThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Aren't we copying all the files in the directory anyway?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not currently. In general we try to only copy the files we need.
COPY pyproject.toml .
should do it.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah see, below we do, but we copy them to
app
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So simply COPY pyproject.toml . Should work?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#18 2.100 configuration error:
tool.setuptools.packages
must be valid exactly by one definition (0 matches found):this is the error, it means the packages section of pyproject.toml isnt correctly set
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this seems to be the point of failure:
Failed to build numcodecs h5py
220.5 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (numcodecs, h5py)
seems that h5py requires some additional dependencies , trying that
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
okay i tried to do a workaround here, tried installing h5py which was creating issues with numcodecs as seen in the error logs,
but i see this:
Dockerfile:14
12 |
13 | # Install h5py with no-binary flag
14 | >>> RUN pip install --no-binary h5py h5py
15 |
16 | # Copy the pyproject.toml file
ERROR: failed to solve: process "/bin/sh -c pip install --no-binary h5py h5py" did not complete successfully: exit code: 1
apparently everything fails at the first RUN pip install xyz(whatever dependency), are we missing anything @peterdudfield ?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you need
pip install --no-binary=h5py h5py
, I admit this is a weird problem thoughThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not really no, was just trying to see if installing it explicitly fixes the h5py, numcodec issue we see on docker, but rather it led us to another issue that the first pip install whatever always fails