Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Aarch runner #52

Merged
merged 60 commits into from
Mar 19, 2024
Merged
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
b2ec7eb
Release 0.1.8
zrezke Mar 1, 2024
b2c2c42
Bump crates
zrezke Mar 1, 2024
5359f83
Aarch runner testing
zrezke Mar 7, 2024
17dfd3d
Skip lints
zrezke Mar 7, 2024
b7b6fab
Skip lints
zrezke Mar 7, 2024
1bb3f3a
quotes
zrezke Mar 7, 2024
0556c6d
comma
zrezke Mar 7, 2024
5d11b61
aarch64
zrezke Mar 7, 2024
3a0a8c6
print ip
zrezke Mar 7, 2024
ab83ecf
ip
zrezke Mar 7, 2024
467d18e
nettolls
zrezke Mar 7, 2024
f9adc07
whoops
zrezke Mar 7, 2024
0647f06
Install rust.
zrezke Mar 7, 2024
6d08939
python already setup on aarch
zrezke Mar 7, 2024
5ca6e5d
alias pip
zrezke Mar 7, 2024
ac9f155
right pip
zrezke Mar 7, 2024
1d97e29
windows pip
zrezke Mar 7, 2024
30b5a81
Switch to docker image recomended by maturin
zrezke Mar 7, 2024
62898e9
before script linux
zrezke Mar 7, 2024
1788f99
probably wont work.
zrezke Mar 7, 2024
dee7c0c
Might work.
zrezke Mar 7, 2024
74214a5
Bump manylinux cross
zrezke Mar 8, 2024
550baa2
Maybe fix build.
zrezke Mar 8, 2024
d508b9d
Maybe fix build
zrezke Mar 8, 2024
37442e1
Probably fix build
zrezke Mar 8, 2024
607bc26
Switch to pypa container
zrezke Mar 11, 2024
0cb04ed
Don't install dependencies
zrezke Mar 11, 2024
fdccf5d
2_28
zrezke Mar 11, 2024
e2a283d
link python
zrezke Mar 11, 2024
9219b1a
replace py instead
zrezke Mar 11, 2024
64e39de
Correct python3.8 path
zrezke Mar 11, 2024
47f42e5
use moved python3
zrezke Mar 11, 2024
d188a5a
yum install some of the deps
zrezke Mar 11, 2024
1a4ffce
Use python3.11 as it comes with shared lib
zrezke Mar 11, 2024
6b56cc2
Perhaps fix manylinux wheels
zrezke Mar 14, 2024
124d31a
Perhaps fix build
zrezke Mar 14, 2024
3ca12b9
Try to use py3.8 maturin directly
zrezke Mar 18, 2024
8c962d0
No /opt(_internal/cpython-3.8.18/bin in path
zrezke Mar 18, 2024
c83fbaf
python3
zrezke Mar 18, 2024
7e098a7
Merge branch 'develop' of github.com:luxonis/depthai-viewer into aarc…
zrezke Mar 18, 2024
2028a8d
Don't setup default python
zrezke Mar 18, 2024
bdef0d7
pip python3.8
zrezke Mar 18, 2024
b61b0a2
Don't build linux and win rn
zrezke Mar 18, 2024
debebd1
py 3.8
zrezke Mar 18, 2024
0b61f5a
Upload wheels pre test. try enabling cargo cache
zrezke Mar 18, 2024
f7ca441
set up dnf cache
zrezke Mar 18, 2024
48eab81
Don't use restore keys
zrezke Mar 18, 2024
2921c90
echo opt
zrezke Mar 18, 2024
ed9ad9f
Unpack wheel
zrezke Mar 18, 2024
a2f1a83
aarch dnf cache rename
zrezke Mar 18, 2024
fda10aa
remove sudo
zrezke Mar 18, 2024
5fb0a81
Reset pytest-unpack-upload procedures
zrezke Mar 18, 2024
f4baf47
Missed a matrix pip
zrezke Mar 18, 2024
060ad90
echo path during tests
zrezke Mar 19, 2024
15e1d10
Perhaps fix pytest
zrezke Mar 19, 2024
f7e3455
quotes
zrezke Mar 19, 2024
2cdf530
fix
zrezke Mar 19, 2024
d3d809f
Should fix the remaining issues with aarch wheels
zrezke Mar 19, 2024
f517c49
Added back windows/linux builds and py lints
zrezke Mar 19, 2024
7d34e1c
Fix stuff I noticed in the PR
zrezke Mar 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 53 additions & 9 deletions .github/workflows/python.yml
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,11 @@ jobs:

if [[ $TAGGED_OR_MAIN ]]; then
# MacOS build is really slow (>30 mins); uses up a lot of CI minutes
matrix+=('{"platform": "macos", "runs_on": "macos-latest"},')
matrix+=('{"platform": "macos", "runs_on": "macos-latest", "pip": "pip", "python": "python3"},')
fi
matrix+=('{"platform": "windows", "runs_on": "windows-latest"},')
matrix+=('{"platform": "linux", "runs_on": "ubuntu-latest", container: {"image": "rerunio/ci_docker:0.5"}}')
matrix+=('{"platform": "windows", "runs_on": "windows-latest", "pip": "pip", "python": "python"},')
matrix+=('{"platform": "linux", "runs_on": "ubuntu-latest", container: {"image": "rerunio/ci_docker:0.5"}, "pip": "pip", "python": "python3"},')
matrix+=('{"platform": "aarch64", "runs_on": ["self-hosted", "linux", "ARM64"], container: {"image": "quay.io/pypa/manylinux_2_28_aarch64"}, "pip": "python3.8 -m pip", "python": "python3.8"}')

echo "Matrix values: ${matrix[@]}"

Expand All @@ -109,6 +110,10 @@ jobs:
container: ${{ matrix.container }}

steps:
- name: Install rust
if: matrix.platform == 'aarch64'
run: curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y && echo "$HOME/.cargo/bin" >> $GITHUB_PATH

- uses: actions/checkout@v3

# These should already be in the docker container, but run for good measure. A no-op install
Expand All @@ -122,6 +127,20 @@ jobs:
version: 2.0 # Increment this to pull newer packages
execute_install_scripts: true

- name: Cache DNF packages
if: matrix.platform == 'aarch64'
uses: actions/cache@v2
with:
path: |
/var/cache/dnf
key: ${{ runner.os }}-dnf-aarch64

- name: AARCH64 install packages
if: matrix.platform == 'aarch64'
run: |
dnf makecache --refresh
dnf install -y gtk3-devel openssl-devel

- name: Set up cargo cache
uses: Swatinem/rust-cache@v2
with:
Expand All @@ -134,7 +153,7 @@ jobs:
# The pip-cache setup logic doesn't work in the ubuntu docker container
# That's probably fine since we bake these deps into the container already
- name: Setup python
if: matrix.platform != 'linux'
if: matrix.platform != 'linux' && matrix.platform != 'aarch64'
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
Expand All @@ -144,7 +163,14 @@ jobs:
# These should already be in the docker container, but run for good measure. A no-op install
# should be fast, and this way things don't break if we add new packages without rebuilding
# docker
- run: pip install -r rerun_py/requirements-build.txt
- name: "Install build requirements"
run: |
if [ "${{ matrix.platform }}" = "aarch64" ]; then
${{ matrix.pip }} install -r rerun_py/requirements-build.txt && echo "/opt/_internal/cpython-3.8.16/bin" >> $GITHUB_PATH
else
${{ matrix.pip }} install -r rerun_py/requirements-build.txt
fi
shell: bash

# ----------------------------------------------------------------------------------
# Install prerequisites for building the web-viewer Wasm
Expand Down Expand Up @@ -196,6 +222,7 @@ jobs:
python3 scripts/version_util.py --check_version

- name: Build Wheel
if: matrix.platform != 'aarch64'
uses: PyO3/maturin-action@v1
with:
maturin-version: "0.14.10"
Expand All @@ -209,19 +236,32 @@ jobs:
--features pypi
--universal2
--out pre-dist
-i ${{ matrix.python }}
- name: Build Wheel (aarch64)
if: matrix.platform == 'aarch64'
run: python3.8 -m maturin build --manifest-path rerun_py/Cargo.toml --release --no-default-features --features pypi --universal2 --out pre-dist

- name: Install built wheel
run: |
pip install depthai_viewer --find-links pre-dist --force-reinstall
${{ matrix.pip }} install depthai_viewer --find-links pre-dist --force-reinstall

- name: Run tests
run: cd rerun_py/tests && pytest
run: |
if [ "${{ matrix.platform }}" = "aarch64" ]; then
cd rerun_py/tests && python3.8 -m pytest
else
cd rerun_py/tests && pytest
fi

- name: Unpack the wheel
shell: bash
run: |
mkdir unpack-dist
wheel unpack pre-dist/*.whl --dest unpack-dist
if [ "${{ matrix.platform }}" == "aarch64" ]; then
python3.8 -m wheel unpack pre-dist/*.whl --dest unpack-dist
else
wheel unpack pre-dist/*.whl --dest unpack-dist
fi

- name: Get the folder name
shell: bash
Expand All @@ -232,7 +272,11 @@ jobs:
shell: bash
run: |
mkdir dist
wheel pack unpack-dist/${{ env.pkg_folder }} --dest dist/
if [ "${{ matrix.platform }}" == "aarch64" ]; then
python3.8 -m wheel pack unpack-dist/${{ env.pkg_folder }} --dest dist/
else
wheel pack unpack-dist/${{ env.pkg_folder }} --dest dist/
fi

- name: Upload wheels
uses: actions/upload-artifact@v3
Expand Down
Loading