Skip to content

Commit

Permalink
explain a bit more about the build system
Browse files Browse the repository at this point in the history
  • Loading branch information
jo-m committed Nov 25, 2023
1 parent 6b0864c commit 3516928
Show file tree
Hide file tree
Showing 3 changed files with 80 additions and 61 deletions.
9 changes: 6 additions & 3 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: format lint test test_more bench check build_host build_arm64 docker_build docker_lint docker_test docker_test_more docker_bench clean run_confighelper run_camera run_videofile
.PHONY: format lint test test_more bench check build_host build_arm64 docker_build docker_lint docker_test docker_test_more docker_bench clean run_confighelper run_camera run_videofile list

# https://hub.docker.com/_/debian
DOCKER_BASE_IMAGE = debian:bullseye-20231120
Expand Down Expand Up @@ -119,10 +119,13 @@ run_videofile:
-X 0 -Y 0 -W 300 -H 300

deploy_trainbot: build_arm64
test -n "$(TRAINBOT_DEPLOY_TARGET_SSH_HOST_)" # missing env var
test -n "$(TRAINBOT_DEPLOY_TARGET_SSH_HOST_)" # missing env var, please set up env file from env.example and source it
scp env $(TRAINBOT_DEPLOY_TARGET_SSH_HOST_):
scp build/trainbot-arm64 $(TRAINBOT_DEPLOY_TARGET_SSH_HOST_):

deploy_confighelper: build_arm64
test -n "$(TRAINBOT_DEPLOY_TARGET_SSH_HOST_)" # missing env var
test -n "$(TRAINBOT_DEPLOY_TARGET_SSH_HOST_)" # missing env var, please set up env file from env.example and source it
scp build/confighelper-arm64 $(TRAINBOT_DEPLOY_TARGET_SSH_HOST_):

list:
@LC_ALL=C $(MAKE) -pRrq -f $(firstword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/(^|\n)# Files(\n|$$)/,/(^|\n)# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | grep -E -v -e '^[^[:alnum:]]' -e '^$@$$'
123 changes: 68 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,75 @@ The assumptions are (there might be more implicit ones):
1. Trains have a constant acceleration (might be 0) and do not stop and turn around while in front of the camera.
1. In reality, this is often not true, there happens to be a stop signal right in front of my balcony...

## Build system
## Web frontend

There is a helper `Makefile` which calls the standard Go build tools and an arm64 cross build inside Docker.
The web frontend is completely independent from the Go binary.
It consists of only static files.
The Go binary uploads Images and sqlite database to the web server via FTP.
Other transports are also possible, but not implemented.

## Deployment

There are two parts to deploy: First, the Go binary which detects trains, and second the web frontend. Only the former is explained here, for the latter check out the `frontend/` directory and its `Makefile`.

How to get binaries?
There are multiple options:

1. `go install github.com/jo-m/trainbot/cmd/trainbot@latest` - Let Go build and install the binary on your system.
2. Grab a binary from the latest CI run at https://github.com/jo-m/trainbot/actions
3. Build via tooling in this repo - see [#development](Development)

### Raspberry Pi

```bash
sudo usermod -a -G video pi

# confighelper
./confighelper-arm64 --log-pretty --input=picam3 --listen-addr=0.0.0.0:8080
```

The current production deployment is in a Tmux session... to be improved one day, but it has worked for 6 months now.

```bash
source ./env

while true; do \
./trainbot-arm64; \
done
```

Download latest data from Raspberry Pi:

```bash
ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST" sqlite3 data/db.sqlite3
.backup data/db.sqlite3.bak
# Ctrl+D
rsync --verbose --archive --rsh=ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST:data/" data/
rm data/db.sqlite3-shm data/db.sqlite3-wal
mv data/db.sqlite3.bak data/db.sqlite3
```

## Development

This repo is set up to compile for `x86_64` and `aarch64`.
There is support for building on your machine directly, or inside a Docker container.

Also, there is an extensive test suite.
Tests may also be executed locally, or inside Docker.

The single entrypoint for everything (incl. Docker) is the `Makefile`.
You can list available targets via `make list`.
Same is true for the frontend - check out [frontend/Makefile](frontend/Makefile).

Example:

```bash
git clone https://github.com/jo-m/trainbot
cd trainbot
make docker_build

# Find binaries in build/ after this has completed.
```

## V4L Settings

Expand Down Expand Up @@ -108,59 +174,6 @@ libcamera-vid \
mkvmerge -o test.mkv --timecodes 0:vid-timestamps.txt vid.h264
```

## Deployment

How to get binaries?
There are multiple options:

1. `go install github.com/jo-m/trainbot/cmd/trainbot@latest` - Let Go build the binary for your host system.
2. Grab a binary from the latest CI run at https://github.com/jo-m/trainbot/actions
3. Use the Docker setup in the repo. This will build for Linux `x86_64` and `arm64`:

```bash
git clone https://github.com/jo-m/trainbot
cd trainbot
make docker_build

# Find binaries in build/ after this has completed.
```

### Raspberry Pi

```bash
sudo usermod -a -G video pi

# confighelper
./confighelper-arm64 --log-pretty --input=picam3 --listen-addr=0.0.0.0:8080
```

The current production deployment is in a Tmux session... to be improved one day, but it has worked for 6 months now.

```bash
source ./env

while true; do \
./trainbot-arm64; \
done
```

Download latest data from Raspberry Pi:

```bash
ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST" sqlite3 data/db.sqlite3
.backup data/db.sqlite3.bak
# Ctrl+D
rsync --verbose --archive --rsh=ssh "$TRAINBOT_DEPLOY_TARGET_SSH_HOST:data/" data/
rm data/db.sqlite3-shm data/db.sqlite3-wal
mv data/db.sqlite3.bak data/db.sqlite3
```

### Web frontend

Images and database are uploaded to a web server via FTP.
The frontend served as a static HTML/JS bundle from the same server.
All database access happens in the browser via sql.js.

## Code notes

* Zerolog is used as logging framework
Expand Down
9 changes: 6 additions & 3 deletions frontend/Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.PHONY: run run_local format build deploy docker_build
.PHONY: run run_local format build deploy docker_build list

# https://hub.docker.com/_/node/
DOCKER_BASE_IMAGE = node:20.9.0-alpine3.18
Expand All @@ -22,14 +22,14 @@ format:
yarn run format

build:
test -n "$(VITE_FRONTEND_BASE_URL_)" # missing env var
test -n "$(VITE_FRONTEND_BASE_URL_)" # missing env var, please set up env file from env.example and source it
VITE_BASE_URL=$(VITE_FRONTEND_BASE_URL_) \
VITE_DB_URL=$(VITE_FRONTEND_BASE_URL_)/data/db.sqlite3 \
VITE_BLOBS_URL=$(VITE_FRONTEND_BASE_URL_)/data/blobs \
yarn run build

deploy: build
test -n "$(FRONTEND_DEPLOY_TARGET_SSH_HOST_)" # missing env var
test -n "$(FRONTEND_DEPLOY_TARGET_SSH_HOST_)" # missing env var, please set up env file from env.example and source it
rm -rf dist/_data
chmod 755 dist/
rsync --verbose --archive --compress --rsh=ssh dist/ $(FRONTEND_DEPLOY_TARGET_SSH_HOST_)
Expand All @@ -42,3 +42,6 @@ docker_build:
--target=export \
--output=dist \
.

list:
@LC_ALL=C $(MAKE) -pRrq -f $(firstword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/(^|\n)# Files(\n|$$)/,/(^|\n)# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | grep -E -v -e '^[^[:alnum:]]' -e '^$@$$'

0 comments on commit 3516928

Please sign in to comment.