Create container-build.yml

[aryanbhosale]

Pull Request

Description

github workflow for creating a docker image for the api

Checklist:

• My code follows OCF's coding style guidelines
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I have checked my code and corrected any misspellings

[aryanbhosale]

hey @peterdudfield , #197 removes requirements.txt and updates pyproject.toml, due to which the RUN pip install -r requirements.txt line had to be removed from the Dockerfile, but it's still throwing an error at RUN pip install -e .
any suggestions?

[aryanbhosale]

hey @peterdudfield , #197 removes requirements.txt and updates pyproject.toml, due to which the RUN pip install -r requirements.txt line had to be removed from the Dockerfile, but it's still throwing an error at RUN pip install -e . any suggestions?

was this tested?

[peterdudfield]

Try on the latest branch on main? Also what is the error?

[aryanbhosale]

Try on the latest branch on main? Also what is the error?

/tmp/pip-build-env-a5vcl7pi/overlay/lib/python3.11/site-packages/setuptools/command/build_py.py:218: _Warning: Package 'numcodecs.tests.package_with_entrypoint' is absent from the `packages` configuration.
#17 242.7       !!
#17 242.7       
#17 242.7               ********************************************************************************
#17 242.7               ############################
#17 242.7               # Package would be ignored #
#17 242.7               ############################
#17 242.7               Python recognizes 'numcodecs.tests.package_with_entrypoint' as an importable package[^1],
#17 242.7               but it is absent from setuptools' `packages` configuration.
#17 242.7       
#17 242.7               This leads to an ambiguous overall configuration. If you want to distribute this
#17 242.7               package, please make sure that 'numcodecs.tests.package_with_entrypoint' is explicitly added
#17 242.7               to the `packages` configuration field.
#17 242.7       
#17 242.7               Alternatively, you can also rely on setuptools' discovery methods
#17 242.7               (for example by using `find_namespace_packages(...)`/`find_namespace:`
#17 242.7               instead of `find_packages(...)`/`find:`).

[aryanbhosale]

Try on the latest branch on main? Also what is the error?

it works on my local machine, but on docker it can't find quartz_solar_forecast

[aryanbhosale]

Okayy so the issue is that pip install -e . works well in my local machine, but when it tries to build the container from the dockerfile, that same command fails, here's one not so helpful discussion i found https://stackoverflow.com/questions/78044103/pip-install-in-editable-mode-from-a-pyproject-in-a-docker-container-modulenotfo
i believe its something to do with the pyproject.toml, but i dont understand why it would work locally and not when building the container
any help on this would be appreciated

[peterdudfield]

you need to copy the pyproject.toml file over

[aryanbhosale]

you need to copy the pyproject.toml file over

Aren't we copying all the files in the directory anyway?

[peterdudfield]

Not currently. In general we try to only copy the files we need.
COPY pyproject.toml . should do it.

[peterdudfield]

Ah see, below we do, but we copy them to app

[aryanbhosale]

Ah see, below we do, but we copy them to app

So simply COPY pyproject.toml . Should work?

[aryanbhosale]

#18 2.100 configuration error: tool.setuptools.packages must be valid exactly by one definition (0 matches found):

this is the error, it means the packages section of pyproject.toml isnt correctly set

[aryanbhosale]

#18 2.100 configuration error: tool.setuptools.packages must be valid exactly by one definition (0 matches found):

this is the error, it means the packages section of pyproject.toml isnt correctly set

this seems to be the point of failure:
Failed to build numcodecs h5py
220.5 ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (numcodecs, h5py)

seems that h5py requires some additional dependencies , trying that

[aryanbhosale]

okay i tried to do a workaround here, tried installing h5py which was creating issues with numcodecs as seen in the error logs,
but i see this:

Dockerfile:14

12 |
13 | # Install h5py with no-binary flag
14 | >>> RUN pip install --no-binary h5py h5py
15 |
16 | # Copy the pyproject.toml file

ERROR: failed to solve: process "/bin/sh -c pip install --no-binary h5py h5py" did not complete successfully: exit code: 1

apparently everything fails at the first RUN pip install xyz(whatever dependency), are we missing anything @peterdudfield ?

[peterdudfield]

do you need pip install --no-binary=h5py h5py, I admit this is a weird problem though

[aryanbhosale]

do you need pip install --no-binary=h5py h5py, I admit this is a weird problem though

Not really no, was just trying to see if installing it explicitly fixes the h5py, numcodec issue we see on docker, but rather it led us to another issue that the first pip install whatever always fails

[aryanbhosale]

hey @peterdudfield , ive removed all the extra changes like adding the frontend to docker, even then i get the same error
locally i get:
failed to solve: process "/bin/sh -c pip install --no-cache-dir build && pip install --no-cache-dir ." did not complete successfully: exit code: 2

and on github i get:
#15 266.2 [numcodecs] command 'gcc' failed: No such file or directory
274.5 ERROR: Could not build wheels for numcodecs, ndindex, which is required to install pyproject.toml-based projects
1687

[peterdudfield]

Hmm, yea, you might have to install conda abd then install it. See uk-pvnet-app for an example - I think

[aryanbhosale]

Hmm, yea, you might have to install conda abd then install it. See uk-pvnet-app for an example - I think

trying this

[aryanbhosale]

Hmm, yea, you might have to install conda abd then install it. See uk-pvnet-app for an example - I think

same error :/
30 | >>> RUN pip install --no-cache-dir -e .

ERROR: failed to solve: process "conda run -n myenv /bin/bash -c pip install --no-cache-dir -e ." did not complete successfully: exit code: 1

[aryanbhosale]

Hmm, yea, you might have to install conda abd then install it. See uk-pvnet-app for an example - I think

same error :/ 30 | >>> RUN pip install --no-cache-dir -e .

ERROR: failed to solve: process "conda run -n myenv /bin/bash -c pip install --no-cache-dir -e ." did not complete successfully: exit code: 1

@peterdudfield it just refuses to install .(the current dir):
ERROR: failed to solve: process "conda run -n myenv /bin/bash -c pip install --no-cache-dir -e ." did not complete successfully: exit code: 1