Skip to content

Commit

Permalink
Improve documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
amotl committed Apr 19, 2024
1 parent a05b21e commit a607fb0
Show file tree
Hide file tree
Showing 6 changed files with 165 additions and 55 deletions.
100 changes: 63 additions & 37 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
A data logger specializing in reading an array of DS18B20 sensors.

A temperature sensor matrix with heatmap visualization for bee hive monitoring,
using Raspberry Pi, Python, DS18B20, MQTT, Kotori DAQ, and Grafana.
using Raspberry Pi, Linux, Python, DS18B20, MQTT, Kotori DAQ, and Grafana.


| View from outside | View from inside (sensor tip details) |
Expand Down Expand Up @@ -37,87 +37,113 @@ https://community.hiveeyes.org/t/laborprotokoll-4x5-temp-matrix-mit-ds18b20/5102
## What's Inside

- The `ds18b20-datalogger` program, reading DS18B20 sensors and
publishing readings to MQTT.
- JSON representation for a corresponding [Grafana Dashboard],
when acquired through [Kotori DAQ].
publishing readings to MQTT in JSON format.
- Configuration file in YAML format, like [`datalogger.yaml`].
- JSON representation for a corresponding Grafana Dashboard
[`grafana-dashboard.json`], when measurement data is submitted
and acquired through [Kotori DAQ].


## Synopsis
Acquire single reading, and echo it on STDOUT in JSON format.
```shell
ds18b20-datalogger read datalogger.yaml
```

Take a single reading, and publish it to the configured MQTT topic.
```shell
ds18b20-datalogger run etc/mois.yaml
ds18b20-datalogger run datalogger.yaml
```

## Live
In order to see a running system in action, please enjoy inspecting the
[Live Grafana Dashboard].

## Setup

We recommend to install the program into a Python virtualenv.
## Installation
Install the `ds18b20-datalogger` package from PyPI using `pip`.
```shell
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade ds18b20-datalogger
```
See also alternative installation methods and hands-on walkthroughs at
[development sandbox] and [production setup].

## Configuration
In order to operate the data logger successfully, you will need to configure
two important details:

In this spirit, you keep the installation separate from your system Python, so
you can easily nuke it and start from scratch in case anything goes south.
- Sensors: Map one-wire sensor sysfs paths to self-assigned sensor names.
- Telemetry: Adjust MQTT connection settings and MQTT topic.

You can create a blueprint configuration file by using the `make-config`
subcommand.
```shell
ds18b20-datalogger make-config > datalogger.yaml
```

## Operations
### Appliance: Sensor Wiring and Sensor Mapping

### Sensor Wiring
Be aware that you might have to adjust your resistors size.
With 30 sensors i had erratic sensor mapping using a 4.7k resistor.
I am getting valid mapping using a 2.2k resistor.

### Sensor Mapping
https://community.hiveeyes.org/t/ds18b20-temperatur-sensoren-am-one-wire-bus-anordnen/1399
Please read more about [sensor mapping] on our community forum. In practice,
just edit the `one-wire` section within the configuration file according to
your setup.

### Backend: Telemetry and Visualization

The data logger will publish measurements to an MQTT topic, where [Kotori DAQ]
can pick it up, in order to converge into a timeseries database, and displays
it on a Grafana Dashboard.

The package includes a corresponding Grafana Dashboard, which can be created
by invoking the `make-dashboard` subcommand.

### Data Publishing
```shell
ssh youruser@yourpi
screen
source /path/to/ds18b20-datalogger/.venv/bin/activate
wget https://github.com/hiveeyes/ds18b20-datalogger/raw/main/etc/mois.yaml
ds18b20-datalogger run mois.yaml
ds18b20-datalogger make-dashboard > dashboard.json
```

### MQTT data upload to Hiveeyes
https://community.hiveeyes.org/t/daten-per-mqtt-und-python-ans-backend-auf-swarm-hiveeyes-org-ubertragen/94/6

### Data visualization in Grafana
https://swarm.hiveeyes.org/grafana/d/Y9PcgE4Sz/mois-ex-wtf-test-ir-sensor-svg-pixmap-copy
On our community forum, you can find relevant discussions about this topic.

### Bonus: Sensor offsets
https://community.hiveeyes.org/t/temperatursensoren-justieren-kalibrieren/1744/2
- [MQTT data upload to Hiveeyes]
- [Data visualization in Grafana]
- [Sensor offsets] (optional)

### Cron Configuration
```
*/5 * * * * cd /path/to/data-directory && /path/to/ds18b20-datalogger/.venv/bin/ds18b20-datalogger
```

## Acknowledgements

Ursprünglicher Code zur Datenverarbeitung auf dem Pi:
https://community.element14.com/products/raspberry-pi/raspberrypi_projects/b/blog/posts/multiple-ds18b20-temp-sensors-interfacing-with-raspberry-pi?CommentId=9470e4e9-b054-4dd3-9a3f-ac9d1fe38087
The original code this implementation has been derived from has been discovered
on the element14 community forum at [Multiple DS18B20 Temp sensors interfacing
with Raspberry Pi], shared by [@laluha]. Thanks!


## Contributing

In order to learn how to start hacking on this program, please have a look at the
documentation about how to install a [development sandbox](./docs/sandbox.md).
documentation about how to install a [development sandbox].

Contributions of any kind are always welcome and appreciated. Thank you.



[Grafana Dashboard]: https://swarm.hiveeyes.org/grafana/d/T49wHSaIk/mois-ex-wtf-test-ds18b20-5x6-temp-matrix-svg-pixmap?orgId=2&from=1712771622514&to=1712807415379
[@laluha]: https://github.com/laluha
[`datalogger.yaml`]: https://github.com/hiveeyes/ds18b20-datalogger/raw/main/ds18b20_datalogger/datalogger.yaml
[Data visualization in Grafana]: https://swarm.hiveeyes.org/grafana/d/Y9PcgE4Sz/mois-ex-wtf-test-ir-sensor-svg-pixmap-copy
[`grafana-dashboard.json`]: https://github.com/hiveeyes/ds18b20-datalogger/raw/main/ds18b20_datalogger/grafana-dashboard.json
[Kotori DAQ]: https://kotori.readthedocs.io
[Live Grafana Dashboard]: https://swarm.hiveeyes.org/grafana/d/T49wHSaIk/mois-ex-wtf-test-ds18b20-5x6-temp-matrix-svg-pixmap?orgId=2&from=1712771622514&to=1712807415379
[MQTT data upload to Hiveeyes]: https://community.hiveeyes.org/t/daten-per-mqtt-und-python-ans-backend-auf-swarm-hiveeyes-org-ubertragen/94/6
[Multiple DS18B20 Temp sensors interfacing with Raspberry Pi]: https://community.element14.com/products/raspberry-pi/raspberrypi_projects/b/blog/posts/multiple-ds18b20-temp-sensors-interfacing-with-raspberry-pi?CommentId=9470e4e9-b054-4dd3-9a3f-ac9d1fe38087
[Sensor offsets]: https://community.hiveeyes.org/t/temperatursensoren-justieren-kalibrieren/1744/2
[sensor mapping]: https://community.hiveeyes.org/t/ds18b20-temperatur-sensoren-am-one-wire-bus-anordnen/1399

[Changelog]: https://github.com/hiveeyes/ds18b20-datalogger/blob/main/CHANGES.md
[development documentation]: https://ds18b20-datalogger.readthedocs.io/en/latest/sandbox.html
[development sandbox]: https://github.com/hiveeyes/ds18b20-datalogger/blob/main/docs/sandbox.md
[Documentation]: https://ds18b20-datalogger.readthedocs.io/
[Issues]: https://github.com/hiveeyes/ds18b20-datalogger/issues
[License]: https://github.com/hiveeyes/ds18b20-datalogger/blob/main/LICENSE
[production setup]: https://github.com/hiveeyes/ds18b20-datalogger/blob/main/docs/production.md
[PyPI]: https://pypi.org/project/ds18b20-datalogger/
[Source code]: https://github.com/hiveeyes/ds18b20-datalogger

Expand Down
11 changes: 9 additions & 2 deletions docs/backlog.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
# Backlog for ds18b20-datalogger

## Iteration +1
- Improve documentation
- Publish to PyPI

## Iteration +2
- By default, probe configuration at `/etc/ds18b20-datalogger/config.yaml`
- Add possibility to connect to MQTT/SSL

## Done
- Better software tests
- Break out sensor mapping configuration from code
to make it re-usable across different setups
- Publish to PyPI
- Subcommand `make-dashboard`
- Subcommand `read`
- Test coverage 100%?
- Improve documentation
9 changes: 9 additions & 0 deletions docs/mois.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Measurement reading and publishing for mois

```shell
ssh youruser@yourpi
screen
source /path/to/ds18b20-datalogger/.venv/bin/activate
wget https://github.com/hiveeyes/ds18b20-datalogger/raw/main/etc/mois.yaml
ds18b20-datalogger run mois.yaml
```
75 changes: 75 additions & 0 deletions docs/production.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,75 @@
# Production Setup

A walkthrough how to install the program on a Raspberry Pi machine running
Linux. In order to learn about more details, please also consult the [README]
document.

## Installation
```shell
sudo su -
python3 -m venv /opt/ds18b20-datalogger
/opt/ds18b20-datalogger/bin/pip install ds18b20-datalogger
/opt/ds18b20-datalogger/bin/ds18b20-datalogger make-config > /etc/ds18b20-datalogger/config.yaml
/opt/ds18b20-datalogger/bin/ds18b20-datalogger run /etc/ds18b20-datalogger/config.yaml
```

We recommend to install the program into a Python virtualenv.
In this spirit, you keep the installation separate from your system Python, so
you can easily nuke it and start from scratch in case anything goes south.

## Periodic Measurements
If you want to run the program periodically, for example each five minutes, use
Cron or, alternatively, Systemd Timers.

### Cron
This should go into `/etc/cron.d/ds18b20-datalogger`.
```
*/5 * * * * root /opt/ds18b20-datalogger/bin/ds18b20-datalogger run /etc/ds18b20-datalogger/config.yaml
```

### Systemd Timers

This should go into `/etc/systemd/system/ds18b20-datalogger.service`.
```ini
# Systemd service unit file for ds18b20-datalogger.
# https://github.com/hiveeyes/ds18b20-datalogger

[Unit]
Description=DS18B20 data logger
Wants=ds18b20-datalogger.timer

[Service]
Type=oneshot
ExecStart=/opt/ds18b20-datalogger/bin/ds18b20-datalogger run /etc/ds18b20-datalogger/config.yaml

[Install]
WantedBy=multi-user.target
```

This should go into `/etc/systemd/system/ds18b20-datalogger.timer`.
```ini
# Systemd timer unit file for ds18b20-datalogger.
# https://github.com/hiveeyes/ds18b20-datalogger

[Unit]
Description=DS18B20 data logger
Requires=ds18b20-datalogger.service

[Timer]
Unit=ds18b20-datalogger.service
OnCalendar=*-*-* *:0/5

[Install]
WantedBy=timers.target
```

Start and enable the service and the timer units.
```shell
systemctl start ds18b20-datalogger.service
systemctl enable ds18b20-datalogger.timer
systemctl list-timers
```


[README]: https://github.com/hiveeyes/ds18b20-datalogger/blob/main/README.md
[Systemd Timers]: https://opensource.com/article/20/7/systemd-timers
10 changes: 9 additions & 1 deletion docs/sandbox.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,21 @@
# Development Sandbox

## Install
Acquire the source code from the repository, install a Python virtualenv,
and install the package in development mode.
```shell
git clone https://github.com/hiveeyes/ds18b20-datalogger
cd ds18b20-datalogger
python3 -m venv .venv
source .venv/bin/activate
pip install --editable='.[develop,docs,release,test]'
pip install --upgrade --editable='.[develop,docs,release,test]'
```

We recommend to install the program into a Python virtualenv.
In this spirit, you keep the installation separate from your system Python, so
you can easily nuke it and start from scratch in case anything goes south.


## Software tests
Run all linters, and invoke the test suite.
```shell
Expand Down
15 changes: 0 additions & 15 deletions ds18b20_datalogger/core.py
Original file line number Diff line number Diff line change
@@ -1,18 +1,3 @@
#!/usr/bin/env python
"""
Configure:
- Insert your own sensor addresses (read_ds18b20_sensor_matrix).
- Adjust MQTT connection settings and credentials (send_measurement_mqtt).
Start:
```
ssh to your pi
screen
source /path/to/ds18b20-datalogger/.venv/bin/activate
ds18b20-datalogger
```
""" # noqa: E501

import json
import os

Expand Down

0 comments on commit a607fb0

Please sign in to comment.