diff --git a/tools/rpi/README.md b/tools/rpi/README.md index 79bc5cbbb..5d1a07503 100644 --- a/tools/rpi/README.md +++ b/tools/rpi/README.md @@ -89,17 +89,16 @@ python3 getting_started.py # to test and see whether RF24 class can be loaded as If there are no error messages on the last step, then the NRF24 Wrapper has been installed successfully. -Building RF24 Wrapper for Debian 11 (bullseye) 64 bit operating system +Building RF24 Wrapper on Debian 11 (bullseye) 64 bit operating system ---------------------------------------------------------------------- -The description above does not work on Debian 11 (bullseye) 64 bit operating system. +The description above does not work on Debian 11 (bullseye) 32 bit operating system. Please check first, if you have Debian 11 (bullseye) 64 bit operating system installed: - `uname -a` search for aarch64 - `lsb_release -d` - `cat /etc/debian_version` -There are 2 possible solutions to install the RF24 wrapper: +To install RF24 wrapper follow the instrauction: -**__1. Solution:__** ```code sudo apt install cmake git python3-dev libboost-python-dev python3-pip python3-rpi.gpio @@ -139,29 +138,60 @@ python3 -m pip list #watch for RF24 module - if its there its installed ``` -**__2. Solution:__** +Alternative: Install pyRF24 library on Debian 11 (bullseye) 64 bit operating system +----------------------------------------------------------------------------------- +The description above does not work on Debian 11 (bullseye) 32 bit operating system. +Please check first, if you have Debian 11 (bullseye) 64 bit operating system installed: + - `uname -a` search for aarch64 + - `lsb_release -d` + - `cat /etc/debian_version` + ```code -sudo apt install git python3-dev libboost-python-dev python3-pip python3-rpi.gpio +sudo apt install cmake git python3-dev libboost-python-dev python3-pip python3-rpi.gpio git clone --recurse-submodules https://github.com/nRF24/pyRF24.git cd pyRF24 python3 -m pip install . -v # this step takes about 5 minutes on my RPI-4 ! +cd ``` -If you have problems with your radio module from ahoi, e.g.: cannot interpret received data, -please try to reduce the speed of your radio module! -Add the following parameter to your ahoy.yml configuration file in "nrf" section: -`spispeed: 600000` (0.6 MHz) +Install pyRF24 library on Debian 12 (bookworm) 64 bit operating system +----------------------------------------------------------------------------------- +The description above does not work on Debian 11 (bullseye) 32 bit operating system. +Please check first, if you have Debian 11 (bullseye) 64 bit operating system installed: + - `uname -a` search for aarch64 + - `lsb_release -d` + - `cat /etc/debian_version` + +Important: Debian 12 follows the recommendation of [`PEP 668`] +(https://peps.python.org/pep-0668/) - now, PYTHON is configured as +"externally-managed-environment" ! +- You cann't install python libs via `pip`! +- You have to use a python virtual environment `https://docs.python.org/3/library/venv.html` +```code +sudo apt install cmake git python3-dev libboost-python-dev python3-pip python3-rpi.gpio + +cd ~ +python3 -m venv ahoyenv ## create python virtual environment +source ahoyenv/bin/activate ## activate the virtual environment + +git clone --recurse-submodules https://github.com/nRF24/pyRF24.git +cd pyRF24 + python3 -m pip install . -v + python3 -m pip list ## check: search for pyRF24 +cd ~ +``` + Required python modules ----------------------- Some modules are not installed by default on a RaspberryPi, therefore add them manually: ```code -pip install crcmod pyyaml paho-mqtt SunTimes +python3 -m pip install crcmod pyyaml paho-mqtt SunTimes ``` Configuration @@ -170,6 +200,12 @@ Configuration Local settings are read from ahoy.yml An example is provided as ahoy.yml.example +If you have any problems with your radio module, +e.g.: cannot interpret received data, +please try to reduce the speed of your radio module! +Add the following parameter to your `ahoy.yml` configuration file in section `nrf`: +`spispeed: 600000` (0.6 MHz) + Example Run ----------- @@ -178,13 +214,19 @@ The following command will run the communication tool, which will try to contact the inverter every second on channel 40, and listen for replies. Whenever it sees a reply, it will decoded and logged to the given log file. +```code + ~~$ sudo python3 -um hoymiles --log-transactions --verbose --config /home/dtu/ahoy.yml | tee -a log2.log~~ + ## when using PYTHON virtual environment only - see hint `PEP 668` + $ source /home/pi/ahoyenv/bin/activate - $ sudo python3 -um hoymiles --log-transactions --verbose --config /home/dtu/ahoy.yml | tee -a log2.log + $ tail -f RPI-AHOY-DTU.log & + $ python3 -um hoymiles --log-transactions --verbose --config /home/dtu/ahoy.yml +``` Python parameters - `-u` enables python's unbuffered mode - `-m hoymiles` tells python to load module 'hoymiles' as main app - +Do not forget to stop `tail -f ...` with `fg`(forground) and than `ctrl-c` The application describes itself ```code @@ -228,12 +270,20 @@ Example injects exactly the same as we normally use to poll data This allows for even faster hacking during runtime -Running it as a service + +Run as a service ----------------------- -If you want to run directly from the start, you might want to install it as a service. -Depending on if you want to run it once a user is logged in or as soon as the system is booted, two service examples are included. -ahoy.service allows you to start it as a user service upon login. -ahoy_system.service allows you to start it as a system service already before login without user interaction. +If you want to run directly at start, you have to install ahoy as a service. +Depending oni, if you want to run it once a user is logged in or as soon as the system is booted, +two service examples are included. +- `ahoy.service` allows you to start it as a user service upon login. +- `ahoy_system.service` allows you to start it as a system service already before login without user interaction. + +Run as a service on Debian 12 (bookworm) +---------------------------------------- +- `ahoy@bookworm.service` allows you to start it as a user service upon login. +- `ahoy@bookworm_system.service` allows you to start it as a system service already before login without user interaction. + Analysing the Logs ------------------ @@ -252,12 +302,10 @@ Use basic command line tools to get an idea what you recorded. For example: A brief example log is supplied in the `example-logs` folder. - - Todo ---- -- Ability to talk to multiple inverters +- Ability to talk to multiple inverters - implemented - please test - MQTT gateway - understand channel hopping - ~~configurable polling interval~~ done: interval ist configurable in ahoy.yml @@ -267,7 +315,6 @@ Todo - ... - References ---------- diff --git a/tools/rpi/ahoy@bookworm.service b/tools/rpi/ahoy@bookworm.service new file mode 100644 index 000000000..2dff36377 --- /dev/null +++ b/tools/rpi/ahoy@bookworm.service @@ -0,0 +1,37 @@ +###################################################################### +# systemd.service configuration for ahoy (lumapu) +# users can modify the lines: +# Description +# ExecStart (example: name of config file) +# WorkingDirectory (absolute path to your private ahoy dir) +# To change other config parameter, please consult systemd documentation +# +# To activate this service, enable and start ahoy.service +# $ systemctl --user enable /home/pi/ahoy/tools/rpi/ahoy@bookworm.service +# $ systemctl --user status ahoy@bookworm.service +# $ systemctl --user start ahoy@bookworm.service +# $ systemctl --user stop ahoy@bookworm.service +# $ systemctl --user disable ahoy@bookworm.service +# +# 2023.01 +# 2024.01 +###################################################################### + +[Unit] +Description=ahoy (lumapu) as Service + +[Service] +ExecStart=/bin/bash -c '\ + source /home/pi/ahoyenv/bin/activate; \ + python3 -um hoymiles --log-transactions --verbose --config ahoy.yml' +RestartSec=30 +Restart=on-failure +Type=simple + +# WorkingDirectory must be an absolute path - not relative path +WorkingDirectory=/home/pi/ahoy/tools/rpi +EnvironmentFile=/etc/environment + +[Install] +WantedBy=default.target + diff --git a/tools/rpi/ahoy@bookworm_system.service b/tools/rpi/ahoy@bookworm_system.service new file mode 100644 index 000000000..3bbfa647b --- /dev/null +++ b/tools/rpi/ahoy@bookworm_system.service @@ -0,0 +1,46 @@ +###################################################################### +# systemd.service configuration for ahoy (lumapu) +# users can modify the lines: +# Description +# ExecStart (example: name of config file) +# WorkingDirectory (absolute path to your private ahoy dir) +# To change other config parameter, please consult systemd documentation +# +# To activate this service, enable and start ahoy.service: +# - Create folder ahoy in /home/ and set owner to the user that the +# service should be executed for (e.g. pi) +# - Copy folder contents to new folder +# - Adjust the user that this service should be executed as, avoid root +# - Execute commands to setup, check and start/stop as wanted +# $ sudo systemctl enable /home/ahoy/tools/rpi/ahoy@bookworm_system.service +# $ sudo systemctl status ahoy@bookworm_system +# $ sudo systemctl start ahoy@bookworm_system +# $ sudo systemctl stop ahoy@bookworm_system +# $ sudo systemctl disable ahoy@bookworm_system +# +# 2023.01 +# 2023.03 +# 2024.01 +###################################################################### + +[Unit] + +Description=ahoy (lumapu) as Service +After=network.target local-fs.target time-sync.target + +[Service] +ExecStart=/bin/bash -c '\ + source /home/pi/ahoyenv/bin/activate; \ + python3 -um hoymiles --log-transactions --verbose --config ahoy.yml' +RestartSec=30 +Restart=on-failure +Type=simple +User=pi + +# WorkingDirectory must be an absolute path - not relative path +WorkingDirectory=/home/ahoy/tools/rpi +EnvironmentFile=/etc/environment + +[Install] +WantedBy=default.target + diff --git a/tools/rpi/hoymiles/__init__.py b/tools/rpi/hoymiles/__init__.py index 46a9275c2..33dfa5d98 100644 --- a/tools/rpi/hoymiles/__init__.py +++ b/tools/rpi/hoymiles/__init__.py @@ -13,33 +13,36 @@ import crcmod from .decoders import * from os import environ +from enum import IntEnum try: # OSI Layer 2 driver for nRF24L01 on Arduino & Raspberry Pi/Linux Devices # https://github.com/nRF24/RF24.git from RF24 import RF24, RF24_PA_MIN, RF24_PA_LOW, RF24_PA_HIGH, RF24_PA_MAX, RF24_250KBPS, RF24_CRC_DISABLED, RF24_CRC_8, RF24_CRC_16 if environ.get('TERM') is not None: - print('Using python Module: RF24') + print('Using python Module: "RF24"') except ModuleNotFoundError as e: if environ.get('TERM') is not None: - print(f'{e} - try to use module: RF24') + print(f"{e} - module not found, try to use 'pyRF24'") try: # Repo for pyRF24 package # https://github.com/nRF24/pyRF24.git from pyrf24 import RF24, RF24_PA_MIN, RF24_PA_LOW, RF24_PA_HIGH, RF24_PA_MAX, RF24_250KBPS, RF24_CRC_DISABLED, RF24_CRC_8, RF24_CRC_16 if environ.get('TERM') is not None: - print(f'{e} - Using python Module: pyrf24') + print(f"'pyrf24' found and used") except ModuleNotFoundError as e: if environ.get('TERM') is not None: print(f'{e} - exit') exit() +if environ.get('TERM') is not None: + print("run before starting AHOY: tail -f RPI-AHOY-DTU.log &") + +HOYMILES_TRANSACTION_LOGGING = False +HOYMILES_VERBOSE_LOGGING = False f_crc_m = crcmod.predefined.mkPredefinedCrcFun('modbus') f_crc8 = crcmod.mkCrcFun(0x101, initCrc=0, xorOut=0) -HOYMILES_TRANSACTION_LOGGING=False -HOYMILES_DEBUG_LOGGING=False - def ser_to_hm_addr(inverter_ser): """ Calculate the 4 bytes that the HM devices use in their internal messages to @@ -71,6 +74,27 @@ def ser_to_esb_addr(inverter_ser): air_order = ser_to_hm_addr(inverter_ser)[::-1] + b'\x01' return air_order[::-1] +class InfoCommands(IntEnum): + ''' compare to .../ahoy/src/hm/hmDefines.h ''' + InverterDevInform_Simple = 0 # 0x00 + InverterDevInform_All = 1 # 0x01 + GridOnProFilePara = 2 # 0x02 + HardWareConfig = 3 # 0x03 + SimpleCalibrationPara = 4 # 0x04 + SystemConfigPara = 5 # 0x05 + RealTimeRunData_Debug = 11 # 0x0b + RealTimeRunData_Reality = 12 # 0x0c + RealTimeRunData_A_Phase = 13 # 0x0d + RealTimeRunData_B_Phase = 14 # 0x0e + RealTimeRunData_C_Phase = 15 # 0x0f + AlarmData = 17 # 0x11, Alarm data - all unsent alarms + AlarmUpdate = 18 # 0x12, Alarm data - all pending alarms + RecordData = 19 # 0x13 + InternalData = 20 # 0x14 + GetLossRate = 21 # 0x15 + GetSelfCheckState = 30 # 0x1e + InitDataState = 0xff + class ResponseDecoderFactory: """ Prepare payload decoder @@ -175,50 +199,54 @@ def decode(self): :return: payload decoder instance :rtype: object """ - model = self.inverter_model - command = self.request_command + model = self.inverter_model + command = self.request_command + model_desc = str(InfoCommands(int(command, 16)).name) - if HOYMILES_DEBUG_LOGGING: - if command.upper() == '00': + if HOYMILES_VERBOSE_LOGGING: + if command.upper() == "00": ## 00 - 0x00 model_desc = "Inverter Dev Inform Simple" - elif command.upper() == '01': + elif command.upper() == "01": ## 01 - 0x01 model_desc = "Firmware version / date" - elif command.upper() == '02': + elif command.upper() == "02": ## 02 - 0x02 model_desc = "Inverter generic events log" - elif command.upper() == '03': ## HardWareConfig + elif command.upper() == "03": ## 03 - 0x03 model_desc = "Hardware configuration" - elif command.upper() == '04': ## SimpleCalibrationPara + elif command.upper() == "04": ## 04 - 0x04 model_desc = "Simple Calibration Parameter" - elif command.upper() == '05': ## SystemConfigPara - model_desc = "Inverter generic SystemConfigPara" - elif command.upper() == '0B': ## 11 - RealTimeRunData_Debug + elif command.upper() == "05": ## 05 - 0x05 + # model_desc = "Inverter generic SystemConfigPara" + model_desc = "SystemConfigPara create DebugDecodeAny" + elif command.upper() == "0B": ## 11 - 0x0b model_desc = "mirco-inverters status data" - elif command.upper() == '0C': ## 12 - RealTimeRunData_Reality + elif command.upper() == "0C": ## 12 - 0x0c model_desc = "mirco-inverters status data" - elif command.upper() == '0D': ## 13 - RealTimeRunData_A_Phase + elif command.upper() == "0D": ## 13 - 0x0d model_desc = "Real-Time Run Data A Phase " - elif command.upper() == '0E': ## 14 - RealTimeRunData_B_Phase + elif command.upper() == "0E": ## 14 - 0x0e model_desc = "Real-Time Run Data B Phase " - elif command.upper() == '0F': ## 15 - RealTimeRunData_C_Phase + elif command.upper() == "0F": ## 15 - 0x0f model_desc = "Real-Time Run Data C Phase " - elif command.upper() == '11': ## 17 - AlarmData - model_desc = "Inverter generic events log" - elif command.upper() == '12': ## 18 - AlarmUpdate + elif command.upper() == "11": ## 17 - 0x11 + # model_desc = "Inverter generic events log" + model_desc = "AlarmData create EventsResponse" + elif command.upper() == "12": ## 18 - 0x12 model_desc = "Inverter major events log" - elif command.upper() == '13': ## 19 - RecordData + elif command.upper() == "13": ## 19 - 0x13 model_desc = "Record Data" - elif command.upper() == '14': ## 20 - InternalData + elif command.upper() == "14": ## 20 - 0x14 model_desc = "Internal Data" - elif command.upper() == '15': ## 21 - GetLossRate + elif command.upper() == "15": ## 21 - 0x15 model_desc = "Get Loss Rate" - elif command.upper() == '1E': ## 30 - GetSelfCheckState + elif command.upper() == "1E": ## 30 model_desc = "Get Self Check State" - elif command.upper() == 'FF': ## 255 - InitDataState + elif command.upper() == "FF": ##255 - 0xff model_desc = "Initi Data State" - else: model_desc = "event not configured - check ahoy script" - logging.info(f'model_decoder: {model}Decode{command.upper()} - {model_desc}') + + logging.info(f'--> using model_decoder: {model}Decode{command.upper()}' + f' - {InfoCommands(int(command, 16)).name} [{command}] ({model_desc})') model_decoders = __import__('hoymiles.decoders') if hasattr(model_decoders, f'{model}Decode{command.upper()}'): @@ -318,8 +346,8 @@ def __str__(self): :rtype: str """ size = len(self.frame) - channel = f' channel {self.ch_rx}' if self.ch_rx else '' - return f"Received {size} bytes{channel}: {hexify_payload(self.frame)}" + channel = f' channel {self.ch_rx:>02}' if self.ch_rx else '' + return f"Received {size:>02} bytes{channel}: {hexify_payload(self.frame)}" class HoymilesNRF: """Hoymiles NRF24 Interface""" @@ -364,8 +392,7 @@ def transmit(self, packet, txpower=None): self.next_tx_channel() if HOYMILES_TRANSACTION_LOGGING: - c_datetime = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f") - logging.debug(f'{c_datetime} Transmit {len(packet)} bytes channel {self.tx_channel}: {hexify_payload(packet)}') + logging.debug(f'Transmit {len(packet):>02} bytes channel {self.tx_channel:>02}: {hexify_payload(packet)}') if not txpower: txpower = self.txpower diff --git a/tools/rpi/hoymiles/__main__.py b/tools/rpi/hoymiles/__main__.py index 581b5acb9..b04169ab4 100644 --- a/tools/rpi/hoymiles/__main__.py +++ b/tools/rpi/hoymiles/__main__.py @@ -7,69 +7,68 @@ import sys import struct -from enum import IntEnum -import re -import time import traceback -from datetime import datetime -from datetime import timedelta -from suntimes import SunTimes +import re + import argparse import yaml from yaml.loader import SafeLoader -import hoymiles import logging from logging.handlers import RotatingFileHandler +import time +from suntimes import SunTimes +from datetime import datetime, timedelta + +import hoymiles # import paket on this place, call once: "hoymiles/__init__.py" + ################################################################################ -""" Signal Handler """ +# SIGINT = Interrupt from keyboard (CTRL + C) +# SIGTERM = Signal Handler from terminating processes +# SIGHUP = Hangup detected on controlling terminal or death of controlling process +# SIGKILL = Signal Handler SIGKILL and SIGSTOP cannot be caught, blocked, or ignored!! ################################################################################ -# from signal import signal, Signals, SIGINT, SIGTERM, SIGKILL, SIGHUP -from signal import * +from signal import signal, Signals, SIGINT, SIGTERM, SIGHUP +from os import environ def signal_handler(sig_num, frame): - signame = Signals(sig_num).name - logging.info(f'Stop by Signal {signame} ({sig_num})') - print (f'Stop by Signal <{signame}> ({sig_num}) at: {time.strftime("%d.%m.%Y %H:%M:%S")}') + """ Signal Handler + + param: signal number [signal-name] + param: frame + """ + signame = Signals(sig_num).name + logging.info(f'Stop by Signal <{signame}> ({sig_num})') + if environ.get('TERM') is not None: + print (f'\nStop by Signal <{signame}> ({sig_num}) ' + f'at: {time.strftime("%d.%m.%Y %H:%M:%S")}\n') - if mqtt_client: - mqtt_client.disco() + if mqtt_client: + mqtt_client.disco() - if influx_client: - influx_client.disco() + if influx_client: + influx_client.disco() - if volkszaehler_client: - volkszaehler_client.disco() + if volkszaehler_client: + volkszaehler_client.disco() - sys.exit(0) + sys.exit(0) -signal(SIGINT, signal_handler) # Interrupt from keyboard (CTRL + C) -signal(SIGTERM, signal_handler) # Signal Handler from terminating processes -signal(SIGHUP, signal_handler) # Hangup detected on controlling terminal or death of controlling process -# signal(SIGKILL, signal_handler) # Signal Handler SIGKILL and SIGSTOP cannot be caught, blocked, or ignored!! +""" activate signal handler """ +signal(SIGINT, signal_handler) +signal(SIGTERM, signal_handler) +signal(SIGHUP, signal_handler) +# signal(SIGKILL, signal_handler) # not used ################################################################################ ################################################################################ -class InfoCommands(IntEnum): - InverterDevInform_Simple = 0 # 0x00 - InverterDevInform_All = 1 # 0x01 - GridOnProFilePara = 2 # 0x02 - HardWareConfig = 3 # 0x03 - SimpleCalibrationPara = 4 # 0x04 - SystemConfigPara = 5 # 0x05 - RealTimeRunData_Debug = 11 # 0x0b - RealTimeRunData_Reality = 12 # 0x0c - RealTimeRunData_A_Phase = 13 # 0x0d - RealTimeRunData_B_Phase = 14 # 0x0e - RealTimeRunData_C_Phase = 15 # 0x0f - AlarmData = 17 # 0x11, Alarm data - all unsent alarms - AlarmUpdate = 18 # 0x12, Alarm data - all pending alarms - RecordData = 19 # 0x13 - InternalData = 20 # 0x14 - GetLossRate = 21 # 0x15 - GetSelfCheckState = 30 # 0x1e - InitDataState = 0xff - class SunsetHandler: + """ Sunset class + to recognize the times of sunrise, sunset and to sleep at night time + + :param str inverter: inverter serial + :param retries: tx retry count if no inverter contact + :type retries: int + """ def __init__(self, sunset_config): self.suntimes = None if sunset_config and sunset_config.get('disabled', True) == False: @@ -78,9 +77,11 @@ def __init__(self, sunset_config): altitude = sunset_config.get('altitude') self.suntimes = SunTimes(longitude=longitude, latitude=latitude, altitude=altitude) self.nextSunset = self.suntimes.setutc(datetime.utcnow()) - logging.info (f'Todays sunset is at {self.nextSunset} UTC') + logging.info (f'Sunset today at: {self.nextSunset} UTC') + # send info to mqtt, if broker configured + self.sun_status2mqtt() else: - logging.info('Sunset disabled.') + logging.info('Sunset disabled!') def checkWaitForSunrise(self): if not self.suntimes: @@ -97,75 +98,88 @@ def checkWaitForSunrise(self): time_to_sleep = int((nextSunrise - datetime.utcnow()).total_seconds()) logging.info (f'Next sunrise is at {nextSunrise} UTC, next sunset is at {self.nextSunset} UTC, sleeping for {time_to_sleep} seconds.') if time_to_sleep > 0: - time.sleep(time_to_sleep) - logging.info (f'Woke up...') + time.sleep(time_to_sleep) + logging.info (f'Woke up...') - def sun_status2mqtt(self, dtu_ser, dtu_name): + def sun_status2mqtt(self): + """ send sunset information every day to MQTT broker """ if not mqtt_client or not self.suntimes: return if self.suntimes: local_sunrise = self.suntimes.riselocal(datetime.now()).strftime("%d.%m.%YT%H:%M") - local_sunset = self.suntimes.setlocal(datetime.now()).strftime("%d.%m.%YT%H:%M") - local_zone = self.suntimes.setlocal(datetime.now()).tzinfo.key - mqtt_client.info2mqtt({'topic' : f'{dtu_name}/{dtu_ser}'}, \ - {'dis_night_comm' : 'True', \ - 'local_sunrise' : local_sunrise, \ - 'local_sunset' : local_sunset, - 'local_zone' : local_zone}) + local_sunset = self.suntimes.setlocal(datetime.now()).strftime("%d.%m.%YT%H:%M") + local_zone = self.suntimes.setlocal(datetime.now()).tzinfo.key + + mqtt_client.info2mqtt(f'{dtu_name}/{dtu_serial}', + {'dis_night_comm' : 'True', + 'local_sunrise' : local_sunrise, + 'local_sunset' : local_sunset, + 'local_zone' : local_zone}) else: - mqtt_client.sun_info2mqtt({'sun_topic': f'{dtu_name}/{dtu_ser}'}, \ - {'dis_night_comm': 'False'}) + mqtt_client.info2mqtt(f'{dtu_name}/{dtu_serial}', {'dis_night_comm': 'False'}) def main_loop(ahoy_config): - """Main loop""" - inverters = [ - inverter for inverter in ahoy_config.get('inverters', []) - if not inverter.get('disabled', False)] + """ Main loop """ + # check 'interval' parameter in config-file + loop_interval = ahoy_config.get('interval', 15) + logging.info(f"AHOY-MAIN: loop interval : {loop_interval} sec.") + if (loop_interval <= 0): + logging.critical("Parameter 'loop_interval' must grater 0 - please check ahoy.yml.") + # print console message too + print("Parameter 'loop_interval' must be >0 - please check ahoy.yml - STOP(0)") + sys.exit(0) - sunset = SunsetHandler(ahoy_config.get('sunset')) - dtu_ser = ahoy_config.get('dtu', {}).get('serial', None) - dtu_name = ahoy_config.get('dtu', {}).get('name', 'hoymiles-dtu') - sunset.sun_status2mqtt(dtu_ser, dtu_name) - loop_interval = ahoy_config.get('interval', 1) + # check 'transmit_retries' parameter in config-file transmit_retries = ahoy_config.get('transmit_retries', 5) if (transmit_retries <= 0): - logging.critical('Parameter "transmit_retries" must be >0 - please check ahoy.yml.') - # print message to console too - print('Parameter "transmit_retries" must be >0 - please check ahoy.yml - STOP(0)x') + logging.critical("Parameter 'transmit_retries' must grater 0 - please check ahoy.yml.") + # print console message too + print("Parameter 'transmit_retries' must be >0 - please check ahoy.yml - STOP(0)") sys.exit(0) + # get parameter from config-file + inverters = [inverter for inverter in ahoy_config.get('inverters', []) + if not inverter.get('disabled', False)] + + # check all inverter names and serial numbers in config-file + for inverter in inverters: + if not 'name' in inverter: + inverter['name'] = 'hoymiles' + if not 'serial' in inverter: + logging.error("No inverter serial number found in ahoy.yml - exit") + sys.exit(999) + + # init Sunset-Handler object + sunset = SunsetHandler(ahoy_config.get('sunset')) + + if not hoymiles.HOYMILES_VERBOSE_LOGGING and not hoymiles.HOYMILES_TRANSACTION_LOGGING: + logging.info(f"MAIN LOOP starts now without any output") + try: do_init = True - while True: + while True: # MAIN endless LOOP + # check sunrise and sunset times and sleep in night time sunset.checkWaitForSunrise() - t_loop_start = time.time() for inverter in inverters: - if not 'name' in inverter: - inverter['name'] = 'hoymiles' - if not 'serial' in inverter: - logging.error("No inverter serial number found in ahoy.yml - exit") - sys.exit(999) - if hoymiles.HOYMILES_DEBUG_LOGGING: - logging.info(f'Poll inverter name={inverter["name"]} ser={inverter["serial"]}') - poll_inverter(inverter, dtu_ser, do_init, transmit_retries) + poll_inverter(inverter, do_init, transmit_retries) do_init = False - if loop_interval > 0: - time_to_sleep = loop_interval - (time.time() - t_loop_start) - if time_to_sleep > 0: - time.sleep(time_to_sleep) - + # calc time to pause main-loop + time_to_sleep = loop_interval - (time.time() - t_loop_start) + if time_to_sleep > 0: + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f'MAIN-LOOP: sleep for {time_to_sleep} sec.') + time.sleep(time_to_sleep) except Exception as e: logging.fatal('Exception catched: %s' % e) logging.fatal(traceback.print_exc()) raise - -def poll_inverter(inverter, dtu_ser, do_init, retries): +def poll_inverter(inverter, do_init, retries): """ Send/Receive command_queue, initiate status poll on inverter @@ -173,20 +187,34 @@ def poll_inverter(inverter, dtu_ser, do_init, retries): :param retries: tx retry count if no inverter contact :type retries: int """ - inverter_ser = inverter.get('serial') - inverter_name = inverter.get('name') + inverter_ser = inverter.get('serial') + inverter_name = inverter.get('name') inverter_strings = inverter.get('strings') + inv_str = str(inverter_ser) # Queue at least status data request - inv_str = str(inverter_ser) if do_init: - command_queue[inv_str].append(hoymiles.compose_send_time_payload(InfoCommands.InverterDevInform_All)) - #command_queue[inv_str].append(hoymiles.compose_send_time_payload(InfoCommands.SystemConfigPara)) - command_queue[inv_str].append(hoymiles.compose_send_time_payload(InfoCommands.RealTimeRunData_Debug)) + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.InverterDevInform_Simple)) # 00 + command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.InverterDevInform_All)) # 01 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.GridOnProFilePara)) # 02 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.HardWareConfig)) # 03 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.SimpleCalibrationPara)) # 04 + ##command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.SystemConfigPara)) # 05 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.RealTimeRunData_Reality)) # 0c + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.AlarmData)) # 11 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.AlarmUpdate)) # 12 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.RecordData)) # 13 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.InternalData)) # 14 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.GetLossRate)) # 15 + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.GetSelfCheckState)) # 1E + # command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.InitDataState)) # FF + command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.RealTimeRunData_Debug)) # 0b # Put all queued commands for current inverter on air while len(command_queue[inv_str]) > 0: - payload = command_queue[inv_str].pop(0) ## Sub.Cmd + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f'Poll inverter name={inverter_name} ser={inverter_ser} command={hoymiles.InfoCommands(command_queue[inv_str][0][0]).name}') + payload = command_queue[inv_str].pop(0) ## get first object from command queue # Send payload {ttl}-times until we get at least one reponse payload_ttl = retries @@ -196,12 +224,12 @@ def poll_inverter(inverter, dtu_ser, do_init, retries): com = hoymiles.InverterTransaction( radio=hmradio, txpower=inverter.get('txpower', None), - dtu_ser=dtu_ser, + dtu_ser=dtu_serial, inverter_ser=inverter_ser, request=next(hoymiles.compose_esb_packet( payload, seq=b'\x80', - src=dtu_ser, + src=dtu_serial, dst=inverter_ser ))) while com.rxtx(): @@ -213,51 +241,75 @@ def poll_inverter(inverter, dtu_ser, do_init, retries): logging.error(f'Error while retrieving data: {e_all}') pass - # Handle the response data if any + # Handle response data, if any if response: if hoymiles.HOYMILES_TRANSACTION_LOGGING: - logging.debug(f'Payload: ' + hoymiles.hexify_payload(response)) + logging.debug(f'Payload: {len(response)} bytes: {hoymiles.hexify_payload(response)}') - # prepare decoder object - decoder = hoymiles.ResponseDecoder(response, + # get a ResponseDecoder object to decode response-payload + decoder = hoymiles.ResponseDecoder(response, request=com.request, inverter_ser=inverter_ser, inverter_name=inverter_name, - dtu_ser=dtu_ser, strings=inverter_strings ) - # get decoder object - result = decoder.decode() - if hoymiles.HOYMILES_DEBUG_LOGGING: - logging.info(f'Decoded: {result.__dict__()}') + result = decoder.decode() # call decoder object + data = result.__dict__() # convert result into python-dict + if hoymiles.HOYMILES_TRANSACTION_LOGGING: + logging.debug(f'Decoded: {data}') - # check decoder object for output + # check result object for output if isinstance(result, hoymiles.decoders.StatusResponse): + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f"StatusResponse: payload contains {len(data)} elements " + f"(power={data['phases'][0]['power']} W - event_count={data['event_count']})") - data = result.__dict__() + # when 'event_count' is changed, add AlarmData-command to queue if data is not None and 'event_count' in data: - if event_message_index[inv_str] < data['event_count']: - event_message_index[inv_str] = data['event_count'] - command_queue[inv_str].append(hoymiles.compose_send_time_payload(InfoCommands.AlarmData, alarm_id=event_message_index[inv_str])) - + # if event_message_index[inv_str] < data['event_count']: + if event_message_index[inv_str] != data['event_count']: + event_message_index[inv_str] = data['event_count'] + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f"event_count changed to {data['event_count']} --> AlarmData requested") + # add AlarmData-command to queue + command_queue[inv_str].append(hoymiles.compose_send_time_payload(hoymiles.InfoCommands.AlarmData, alarm_id=event_message_index[inv_str])) + + # sent outputs if mqtt_client: - mqtt_client.store_status(result, topic=inverter.get('mqtt', {}).get('topic', None)) + mqtt_client.store_status(data) if influx_client: - influx_client.store_status(result) + influx_client.store_status(data) if volkszaehler_client: - volkszaehler_client.store_status(result) + volkszaehler_client.store_status(data) - # check decoder object for output + # check decoder object for different data types if isinstance(result, hoymiles.decoders.HardwareInfoResponse): - if mqtt_client: - mqtt_client.store_status(result, topic=inverter.get('mqtt', {}).get('topic', None)) + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f"Firmware version {data['FW_ver_maj']}.{data['FW_ver_min']}.{data['FW_ver_pat']}, " + f"build at {data['FW_build_dd']:>02}/{data['FW_build_mm']:>02}/{data['FW_build_yy']}T" + f"{data['FW_build_HH']:>02}:{data['FW_build_MM']:>02}, " + f"HW revision {data['FW_HW_ID']}") + if mqtt_client: + mqtt_client.store_status(data) + + if isinstance(result, hoymiles.decoders.EventsResponse): + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f"EventsResponse: {data['inv_stat_txt']} ({data['inv_stat_num']})") + + if isinstance(result, hoymiles.decoders.DebugDecodeAny): + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f"DebugDecodeAny: payload ({data['len_payload']} bytes): {data['payload']}") + +def mqtt_on_message(mqtt_client, userdata, message): + ''' + MQTT(PAHO) callcack method to handle receiving payload + ( run in thread: "paho-mqtt-client-" - important for signals and Exceptions !) + a) when receiving topic ends with "SENSOR" for privat electricity meter + b) when receiving topic ends with "command" for runtime faster debugging - -def mqtt_on_command(client, userdata, message): - """ Handle commands to topic hoymiles/{inverter_ser}/command frame a payload and put onto command_queue @@ -278,35 +330,45 @@ def mqtt_on_command(client, userdata, message): :param paho.mqtt.client.Client client: mqtt-client instance :param dict userdata: Userdata :param dict message: mqtt-client message object - """ - try: - inverter_ser = next( - item[0] for item in mqtt_command_topic_subs if item[1] == message.topic) - except StopIteration: - logging.warning('Unexpedtedly received mqtt message for {message.topic}') + ''' + # print(f"msg-topic: {message.topic} - QoS: {message.qos}") + # print(f"payload: ",str(message.payload.decode("utf-8")), "\n") + + # handle specific payload topic + if message.topic.endswith("SENSOR"): + if volkszaehler_client: + volkszaehler_client.store_status(yaml.safe_load(str(message.payload.decode("utf-8")))) - if inverter_ser: + if message.topic.endswith("command"): p_message = message.payload.decode('utf-8').lower() # Expand tttttttt to current time for use in hexlified payload expand_time = ''.join(f'{b:02x}' for b in struct.pack('>L', int(time.time()))) p_message = p_message.replace('tttttttt', expand_time) + logging.info (f"MQTT-command: {message.topic} - {p_message}") - if (len(p_message) < 2048 \ - and len(p_message) % 2 == 0 \ - and re.match(r'^[a-f0-9]+$', p_message)): + if (len(p_message) < 2048 and len(p_message) % 2 == 0 and re.match(r'^[a-f0-9]+$', p_message)): payload = bytes.fromhex(p_message) + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info (f"MQTT-command for {inv_str}: {payload}") + # commands must start with \x80 if payload[0] == 0x80: - command_queue[str(inverter_ser)].append( - hoymiles.frame_payload(payload[1:])) + # array "command_queue[inv_str]" will be shared to an other thread --> critical section + command_queue[inv_str].append(hoymiles.frame_payload(payload[1:])) + else: + logging.info (f"MQTT-command: must start with \x80: {payload}") + else: + logging.info (f"MQTT-command to long (max length: 2048 bytes) - or contains non hex char") def init_logging(ahoy_config): + """ init and prepare logging """ log_config = ahoy_config.get('logging') fn = 'hoymiles.log' lvl = logging.ERROR max_log_filesize = 1000000 max_log_files = 1 + if log_config: fn = log_config.get('filename', fn) level = log_config.get('level', 'ERROR') @@ -322,15 +384,23 @@ def init_logging(ahoy_config): lvl = logging.FATAL max_log_filesize = log_config.get('max_log_filesize', max_log_filesize) max_log_files = log_config.get('max_log_files', max_log_files) - if hoymiles.HOYMILES_TRANSACTION_LOGGING: + + # define log switches + if global_config.log_transactions: + hoymiles.HOYMILES_TRANSACTION_LOGGING = True lvl = logging.DEBUG + if global_config.verbose: + hoymiles.HOYMILES_VERBOSE_LOGGING = True + + # start configured logging logging.basicConfig(handlers=[RotatingFileHandler(fn, maxBytes=max_log_filesize, backupCount=max_log_files)], format='%(asctime)s %(levelname)s: %(message)s', datefmt='%Y-%m-%d %H:%M:%S.%s', level=lvl) - dtu_name = ahoy_config.get('dtu',{}).get('name','hoymiles-dtu') - logging.info(f'start logging for {dtu_name} with level: {logging.getLevelName(logging.root.level)}') + + logging.info(f'AHOY-logging started for "{dtu_name}" with level: {logging.getLevelName(logging.root.level)}') if __name__ == '__main__': + # read commandline parameter parser = argparse.ArgumentParser(description='Ahoy - Hoymiles solar inverter gateway', prog="hoymiles") parser.add_argument("-c", "--config-file", nargs="?", required=True, help="configuration file") @@ -340,7 +410,7 @@ def init_logging(ahoy_config): help="Enable detailed debug output (loglevel must be DEBUG)") global_config = parser.parse_args() - # Load ahoy.yml config file + # Load config file given in commandline parameter try: if isinstance(global_config.config_file, str): with open(global_config.config_file, 'r') as fh_yaml: @@ -355,27 +425,44 @@ def init_logging(ahoy_config): logging.error(f'Failed to load config file {global_config.config_file}: {e_yaml}') sys.exit(1) - if global_config.log_transactions: - hoymiles.HOYMILES_TRANSACTION_LOGGING=True - if global_config.verbose: - hoymiles.HOYMILES_DEBUG_LOGGING=True - - # read AHOY configuration file and prepare logging + # read all parameter from configuration file as 'ahoy_config' ahoy_config = dict(cfg.get('ahoy', {})) + + # extract 'DTU' parameter + dtu_serial = ahoy_config.get('dtu', {}).get('serial', None) + dtu_name = ahoy_config.get('dtu', {}).get('name', 'hoymiles-dtu') + + # init and prepare logging init_logging(ahoy_config) - # Prepare for multiple transceivers, makes them configurable + # Prepare for multiple transceivers (radio modules), makes them configurable for radio_config in ahoy_config.get('nrf', [{}]): hmradio = hoymiles.HoymilesNRF(**radio_config) - # create MQTT - client object - mqtt_client = None - mqtt_config = ahoy_config.get('mqtt', None) - if mqtt_config and not mqtt_config.get('disabled', False): + # create MQTT client object + # if: mqtt-disabled is "true" - only + # if: mqtt-disabled is "true" AND inverter-mqtt-send_raw_enabled is "true" + # if: mqtt topic is defined - only or with other functions + mqtt_c_obj = mqtt_client = None # create client-obj-placeholder + mqtt_config = ahoy_config.get('mqtt', None) # get mqtt-config, if available + + if mqtt_config and (not mqtt_config.get('disabled', False) or mqtt_topic): from .outputs import MqttOutputPlugin - mqtt_client = MqttOutputPlugin(mqtt_config) - # create INFLUX - client object + # MQTT_TOPIC array should contain QOS levels as well as topic names. + # MQTT_TOPIC = [("Server1/kpi1",0),("Server2/kpi2",0),("Server3/kpi3",0)] + mqtt_topic = mqtt_config.get('topic', None) # get topic, if available + mqtt_topic_array = [] # create empty array + if mqtt_topic: + mqtt_topic_array.append((mqtt_topic, mqtt_config.get('QoS',0))) + + # create MQTT(PAHO) client object with own callback funtion + mqtt_c_obj = MqttOutputPlugin(mqtt_config, mqtt_on_message) + + if mqtt_c_obj and not mqtt_config.get('disabled', False): + mqtt_client = mqtt_c_obj + + # create INFLUX client object influx_client = None influx_config = ahoy_config.get('influxdb', None) if influx_config and not influx_config.get('disabled', False): @@ -387,31 +474,35 @@ def init_logging(ahoy_config): bucket=influx_config.get('bucket', None), measurement=influx_config.get('measurement', 'hoymiles')) - # create VOLKSZAEHLER - client object + # create VOLKSZAEHLER client object volkszaehler_client = None volkszaehler_config = ahoy_config.get('volkszaehler', {}) if volkszaehler_config and not volkszaehler_config.get('disabled', False): from .outputs import VolkszaehlerOutputPlugin volkszaehler_client = VolkszaehlerOutputPlugin(volkszaehler_config) + # init important runtime variables event_message_index = {} command_queue = {} mqtt_command_topic_subs = [] - for g_inverter in ahoy_config.get('inverters', []): - g_inverter_ser = g_inverter.get('serial') - inv_str = str(g_inverter_ser) - command_queue[inv_str] = [] - event_message_index[inv_str] = 0 - - # Enables and subscribe inverter to mqtt /command-Topic - if mqtt_client and g_inverter.get('mqtt', {}).get('send_raw_enabled', False): - topic_item = ( - str(g_inverter_ser), - g_inverter.get('mqtt', {}).get('topic', f'hoymiles/{g_inverter_ser}') + '/command' - ) - mqtt_client.client.subscribe(topic_item[1]) - mqtt_command_topic_subs.append(topic_item) + for g_inverter in ahoy_config.get('inverters', []): # loop inverters in ahoy_config + inv_str = str(g_inverter.get('serial')) # inverter serial number as index + command_queue[inv_str] = [] # create empty command-queue + event_message_index[inv_str] = 0 # init event-queue with value=0 + + # if send_raw_enabled, add topic to subscribe command-queue + if mqtt_c_obj and g_inverter.get('mqtt', {}).get('send_raw_enabled', False): + mqtt_topic_array.append( + (g_inverter.get('mqtt', {}).get('topic', f'hoymiles/{inv_str}') + '/command', + mqtt_config.get('QoS',0) + )) + + # start subscribe mqtt broker, if requested 'topic' is available + if mqtt_c_obj and len(mqtt_topic_array) > 0: + if hoymiles.HOYMILES_VERBOSE_LOGGING: + logging.info(f'MQTT: subscribe for topic: {mqtt_topic_array}') + mqtt_c_obj.client.subscribe(mqtt_topic_array) # start main-loop main_loop(ahoy_config) diff --git a/tools/rpi/hoymiles/decoders/__init__.py b/tools/rpi/hoymiles/decoders/__init__.py index 0d370cea1..764aa2f26 100644 --- a/tools/rpi/hoymiles/decoders/__init__.py +++ b/tools/rpi/hoymiles/decoders/__init__.py @@ -64,7 +64,6 @@ class Response: """ All Response Shared methods """ inverter_ser = None inverter_name = None - dtu_ser = None response = None def __init__(self, *args, **params): @@ -73,7 +72,6 @@ def __init__(self, *args, **params): """ self.inverter_ser = params.get('inverter_ser', None) self.inverter_name = params.get('inverter_name', None) - self.dtu_ser = params.get('dtu_ser', None) self.response = args[0] strings = params.get('strings', None) @@ -89,7 +87,7 @@ def __dict__(self): return { 'inverter_ser': self.inverter_ser, 'inverter_name': self.inverter_name, - 'dtu_ser': self.dtu_ser} + } class StatusResponse(Response): """Inverter StatusResponse object""" @@ -250,74 +248,74 @@ class EventsResponse(UnknownResponse): alarm_codes = { # HM Error Codes - 1: 'Inverter start', # 0x01 - 2: 'DTU command failed', # 0x02 - 121: 'Over temperature protection', # 0x79 + 1: 'Inverter start', # 0x01 + 2: 'DTU command failed', # 0x02 + 121: 'Over temperature protection', # 0x79 125: 'Grid configuration parameter error', # 0x7D - 126: 'Software error code 126', # 0x7E - 127: 'Firmware error', # 0x7F - 128: 'Software error code 128', # 0x80 - 129: 'Software error code 129', # 0x81 - 130: 'Offline', # 0x82 - 141: 'Grid overvoltage', # 0x8D - 142: 'Average grid overvoltage', # 0x8E - 143: 'Grid undervoltage', # 0x8F - 144: 'Grid overfrequency', # 0x90 - 145: 'Grid underfrequency', # 0x91 - 146: 'Rapid grid frequency change', # 0x92 - 147: 'Power grid outage', # 0x93 - 148: 'Grid disconnection', # 0x94 - 149: 'Island detected', # 0x95 - 205: 'Input port 1 & 2 overvoltage', # 0xCD - 206: 'Input port 3 & 4 overvoltage', # 0xCE - 207: 'Input port 1 & 2 undervoltage', # 0xCF - 208: 'Input port 3 & 4 undervoltage', # 0xD0 - 209: 'Port 1 no input', # 0xD1 - 210: 'Port 2 no input', # 0xD2 - 211: 'Port 3 no input', # 0xD3 - 212: 'Port 4 no input', # 0xD4 - 213: 'PV-1 & PV-2 abnormal wiring', # 0xD5 - 214: 'PV-3 & PV-4 abnormal wiring', # 0xD6 - 215: 'PV-1 Input overvoltage', # 0xD7 - 216: 'PV-1 Input undervoltage', # 0xD8 - 217: 'PV-2 Input overvoltage', # 0xD9 - 218: 'PV-2 Input undervoltage', # 0xDA - 219: 'PV-3 Input overvoltage', # 0xDB - 220: 'PV-3 Input undervoltage', # 0xDC - 221: 'PV-4 Input overvoltage', # 0xDD - 222: 'PV-4 Input undervoltage', # 0xDE - 301: 'Hardware error code 301', # 0x012D - 302: 'Hardware error code 302', # 0x012E - 303: 'Hardware error code 303', # 0x012F - 304: 'Hardware error code 304', # 0x0130 - 305: 'Hardware error code 305', # 0x0131 - 306: 'Hardware error code 306', # 0x0132 - 307: 'Hardware error code 307', # 0x0133 - 308: 'Hardware error code 308', # 0x0134 - 309: 'Hardware error code 309', # 0x0135 - 310: 'Hardware error code 310', # 0x0136 - 311: 'Hardware error code 311', # 0x0137 - 312: 'Hardware error code 312', # 0x0138 - 313: 'Hardware error code 313', # 0x0139 - 314: 'Hardware error code 314', # 0x013A + 126: 'Software error code 126', # 0x7E + 127: 'Firmware error', # 0x7F + 128: 'Software error code 128', # 0x80 + 129: 'Software error code 129', # 0x81 + 130: 'Offline', # 0x82 + 141: 'Grid overvoltage', # 0x8D + 142: 'Average grid overvoltage', # 0x8E + 143: 'Grid undervoltage', # 0x8F + 144: 'Grid overfrequency', # 0x90 + 145: 'Grid underfrequency', # 0x91 + 146: 'Rapid grid frequency change', # 0x92 + 147: 'Power grid outage', # 0x93 + 148: 'Grid disconnection', # 0x94 + 149: 'Island detected', # 0x95 + 205: 'Input port 1 & 2 overvoltage', # 0xCD + 206: 'Input port 3 & 4 overvoltage', # 0xCE + 207: 'Input port 1 & 2 undervoltage', # 0xCF + 208: 'Input port 3 & 4 undervoltage', # 0xD0 + 209: 'Port 1 no input', # 0xD1 + 210: 'Port 2 no input', # 0xD2 + 211: 'Port 3 no input', # 0xD3 + 212: 'Port 4 no input', # 0xD4 + 213: 'PV-1 & PV-2 abnormal wiring', # 0xD5 + 214: 'PV-3 & PV-4 abnormal wiring', # 0xD6 + 215: 'PV-1 Input overvoltage', # 0xD7 + 216: 'PV-1 Input undervoltage', # 0xD8 + 217: 'PV-2 Input overvoltage', # 0xD9 + 218: 'PV-2 Input undervoltage', # 0xDA + 219: 'PV-3 Input overvoltage', # 0xDB + 220: 'PV-3 Input undervoltage', # 0xDC + 221: 'PV-4 Input overvoltage', # 0xDD + 222: 'PV-4 Input undervoltage', # 0xDE + 301: 'Hardware error code 301', # 0x012D + 302: 'Hardware error code 302', # 0x012E + 303: 'Hardware error code 303', # 0x012F + 304: 'Hardware error code 304', # 0x0130 + 305: 'Hardware error code 305', # 0x0131 + 306: 'Hardware error code 306', # 0x0132 + 307: 'Hardware error code 307', # 0x0133 + 308: 'Hardware error code 308', # 0x0134 + 309: 'Hardware error code 309', # 0x0135 + 310: 'Hardware error code 310', # 0x0136 + 311: 'Hardware error code 311', # 0x0137 + 312: 'Hardware error code 312', # 0x0138 + 313: 'Hardware error code 313', # 0x0139 + 314: 'Hardware error code 314', # 0x013A # MI Error Codes - 5041: 'Error code-04 Port 1', # 0x13B1 - 5042: 'Error code-04 Port 2', # 0x13B2 - 5043: 'Error code-04 Port 3', # 0x13B3 - 5044: 'Error code-04 Port 4', # 0x13B4 + 5041: 'Error code-04 Port 1', # 0x13B1 + 5042: 'Error code-04 Port 2', # 0x13B2 + 5043: 'Error code-04 Port 3', # 0x13B3 + 5044: 'Error code-04 Port 4', # 0x13B4 5051: 'PV Input 1 Overvoltage/Undervoltage', # 0x13BB 5052: 'PV Input 2 Overvoltage/Undervoltage', # 0x13BC 5053: 'PV Input 3 Overvoltage/Undervoltage', # 0x13BD 5054: 'PV Input 4 Overvoltage/Undervoltage', # 0x13BE - 5060: 'Abnormal bias', # 0x13C4 - 5070: 'Over temperature protection', # 0x13CE - 5080: 'Grid Overvoltage/Undervoltage', # 0x13D8 - 5090: 'Grid Overfrequency/Underfrequency', # 0x13E2 - 5100: 'Island detected', # 0x13EC - 5120: 'EEPROM reading and writing error', # 0x1400 - 5150: '10 min value grid overvoltage', # 0x141E - 5200: 'Firmware error', # 0x1450 - 8310: 'Shut down', # 0x2076 + 5060: 'Abnormal bias', # 0x13C4 + 5070: 'Over temperature protection', # 0x13CE + 5080: 'Grid Overvoltage/Undervoltage', # 0x13D8 + 5090: 'Grid Overfrequency/Underfrequency', # 0x13E2 + 5100: 'Island detected', # 0x13EC + 5120: 'EEPROM reading and writing error', # 0x1400 + 5150: '10 min value grid overvoltage', # 0x141E + 5200: 'Firmware error', # 0x1450 + 8310: 'Shut down', # 0x2076 9000: 'Microinverter is suspected of being stolen' # 0x2328 } @@ -331,7 +329,6 @@ def __init__(self, *args, **params): self.status = struct.unpack('>H', self.response[:2])[0] self.a_text = self.alarm_codes.get(self.status, 'N/A') - logging.info (f'Inverter status: {self.a_text} ({self.status})') chunk_size = 12 for i_chunk in range(2, len(self.response), chunk_size): @@ -387,7 +384,7 @@ def __dict__(self): logging.error(f'HardwareInfoResponse: data: {self.response}') return data - logging.info(f'HardwareInfoResponse: {struct.unpack(">HHHHHHHH", self.response[0:16])}') + logging.debug(f'HardwareInfoResponse: {struct.unpack(">HHHHHHHH", self.response[0:16])}') fw_version, fw_build_yyyy, fw_build_mmdd, fw_build_hhmm, hw_id = struct.unpack('>HHHHH', self.response[0:10]) fw_version_maj = int((fw_version / 10000)) @@ -397,9 +394,6 @@ def __dict__(self): fw_build_dd = int(fw_build_mmdd % 100) fw_build_HH = int(fw_build_hhmm / 100) fw_build_MM = int(fw_build_hhmm % 100) - logging.info(f'Firmware: {fw_version_maj}.{fw_version_min}.{fw_version_pat} '\ - f'build at {fw_build_dd:>02}/{fw_build_mm:>02}/{fw_build_yyyy}T{fw_build_HH:>02}:{fw_build_MM:>02}, '\ - f'HW revision {hw_id}') data['FW_ver_maj'] = fw_version_maj data['FW_ver_min'] = fw_version_min @@ -428,9 +422,6 @@ def __init__(self, *args, **params): logging.debug(' payload has valid modbus crc') self.response = self.response[:-2] - l_payload = len(self.response) - logging.debug(f' payload has {l_payload} bytes') - logging.debug('') logging.debug('Field view: int') print_table_unpack('>B', self.response) @@ -455,6 +446,13 @@ def __init__(self, *args, **params): except UnicodeDecodeError: logging.debug(' type ascii : ascii decode error') + def __dict__(self): + """ Base values, availabe in each __dict__ call """ + data = super().__dict__() + + data['len_payload'] = len(self.response) + data['payload'] = self.response + return data # 1121-Series Intervers, 1 MPPT, 1 Phase class Hm300Decode01(HardwareInfoResponse): diff --git a/tools/rpi/hoymiles/outputs.py b/tools/rpi/hoymiles/outputs.py index aa574f2d5..8f916be70 100644 --- a/tools/rpi/hoymiles/outputs.py +++ b/tools/rpi/hoymiles/outputs.py @@ -9,7 +9,7 @@ import logging from datetime import datetime, timezone from hoymiles.decoders import StatusResponse, HardwareInfoResponse -from hoymiles import HOYMILES_TRANSACTION_LOGGING, HOYMILES_DEBUG_LOGGING +from hoymiles import HOYMILES_TRANSACTION_LOGGING, HOYMILES_VERBOSE_LOGGING class OutputPluginFactory: def __init__(self, **params): @@ -22,7 +22,7 @@ def __init__(self, **params): :type inverter_name: str """ - self.inverter_ser = params.get('inverter_ser', '') + self.inverter_ser = params.get('inverter_ser', '') self.inverter_name = params.get('inverter_name', None) def store_status(self, response, **params): @@ -64,7 +64,7 @@ def __init__(self, url, token, **params): print(ErrorText1, ErrorText2) logging.error(ErrorText1) logging.error(ErrorText2) - exit() + exit(1) self._bucket = params.get('bucket', 'hoymiles/autogen') self._org = params.get('org', '') @@ -72,12 +72,15 @@ def __init__(self, url, token, **params): with InfluxDBClient(url, token, bucket=self._bucket) as self.client: self.api = self.client.write_api() + if HOYMILES_VERBOSE_LOGGING: + logging.info(f"Influx: connect to DB {url} initialized") def disco(self, **params): self.client.close() # Shutdown the client return - def store_status(self, response, **params): + # def store_status(self, response, **params): + def store_status(self, data, **params): """ Publish StatusResponse object @@ -89,10 +92,12 @@ def store_status(self, response, **params): :raises ValueError: when response is not instance of StatusResponse """ - if not isinstance(response, StatusResponse): - raise ValueError('Data needs to be instance of StatusResponse') + # if not isinstance(response, StatusResponse): + # raise ValueError('Data needs to be instance of StatusResponse') + if not 'phases' in data or not 'strings' in data: + raise ValueError('DICT need key "inverter_ser" and "inverter_name"') - data = response.__dict__() + # data = response.__dict__() # convert response-parameter into python-dict measurement = self._measurement + f',location={data["inverter_ser"]}' @@ -108,7 +113,7 @@ def store_status(self, response, **params): # InfluxDB requires nanoseconds ctime = int(utctime.timestamp() * 1e9) - if HOYMILES_DEBUG_LOGGING: + if HOYMILES_VERBOSE_LOGGING: logging.info(f'InfluxDB: utctime: {utctime}') # AC Data @@ -144,8 +149,8 @@ def store_status(self, response, **params): data_stack.append(f'{measurement},type=YieldToday value={data["yield_today"]/1000:.3f} {ctime}') data_stack.append(f'{measurement},type=Efficiency value={data["efficiency"]:.2f} {ctime}') - if HOYMILES_DEBUG_LOGGING: - #logging.debug(f'INFLUX data to DB: {data_stack}') + if HOYMILES_VERBOSE_LOGGING: + logging.debug(f'INFLUX data to DB: {data_stack}') pass self.api.write(self._bucket, self._org, data_stack) @@ -153,7 +158,7 @@ class MqttOutputPlugin(OutputPluginFactory): """ Mqtt output plugin """ client = None - def __init__(self, config, **params): + def __init__(self, config, cb_message, **params): """ Initialize MqttOutputPlugin @@ -177,34 +182,51 @@ def __init__(self, config, **params): super().__init__(**params) try: - import paho.mqtt.client + import paho.mqtt.client as mqtt except ModuleNotFoundError: ErrorText1 = f'Module "paho.mqtt.client" for MQTT-output necessary.' ErrorText2 = f'Install module with command: python3 -m pip install paho-mqtt' print(ErrorText1, ErrorText2) logging.error(ErrorText1) logging.error(ErrorText2) - exit() + exit(1) + + # For paho-mqtt 2.0.0, you need to set callback_api_version. + # self.client = mqtt.Client(callback_api_version=mqtt.CallbackAPIVersion.VERSION1) + self.client = mqtt.Client(mqtt.CallbackAPIVersion.VERSION2) - mqtt_client = paho.mqtt.client.Client() if config.get('useTLS',False): - mqtt_client.tls_set() - mqtt_client.tls_insecure_set(config.get('insecureTLS',False)) - mqtt_client.username_pw_set(config.get('user', None), config.get('password', None)) + self.client.tls_set() + self.client.tls_insecure_set(config.get('insecureTLS',False)) + self.client.username_pw_set(config.get('user', None), config.get('password', None)) last_will = config.get('last_will', None) if last_will: lw_topic = last_will.get('topic', 'last will hoymiles') lw_payload = last_will.get('payload', 'last will') - mqtt_client.will_set(str(lw_topic), str(lw_payload)) + self.client.will_set(str(lw_topic), str(lw_payload)) - mqtt_client.connect(config.get('host', '127.0.0.1'), config.get('port', 1883)) - mqtt_client.loop_start() + self.client.connect(config.get('host', '127.0.0.1'), config.get('port', 1883)) + self.client.loop_start() - self.client = mqtt_client self.qos = config.get('QoS', 0) # Quality of Service self.ret = config.get('Retain', True) # Retain Message + # connect own (PAHO) callback functions + self.client.on_connect = self.mqtt_on_connect + self.client.on_message = cb_message + + # MQTT(PAHO) callcack method to inform about connection to mqtt broker + def mqtt_on_connect(self, client, userdata, flags, reason_code, properties): + if flags.session_present: + logging.info("flags.session_present") + if reason_code == 0: # success connect + if HOYMILES_VERBOSE_LOGGING: + logging.info(f"MQTT: Connected to Broker: {self.client.host}:{self.client.port} as user {self.client.username}") + if reason_code > 0: # error processing + logging.error(f'Connect failed: {reason_code}') # error message + + def disco(self, **params): self.client.loop_stop() # Stop loop self.client.disconnect() # disconnect @@ -212,10 +234,11 @@ def disco(self, **params): def info2mqtt(self, mqtt_topic, mqtt_data): for mqtt_key in mqtt_data: - self.client.publish(f'{mqtt_topic["topic"]}/{mqtt_key}', mqtt_data[mqtt_key], self.qos, self.ret) + self.client.publish(f'{mqtt_topic}/{mqtt_key}', mqtt_data[mqtt_key], self.qos, self.ret) return - def store_status(self, response, **params): + # def store_status(self, response, **params): + def store_status(self, data, **params): """ Publish StatusResponse object @@ -226,20 +249,20 @@ def store_status(self, response, **params): :raises ValueError: when response is not instance of StatusResponse """ - data = response.__dict__() + # data = response.__dict__() # convert response-parameter into python-dict if data is None: - logging.warn("received data object is empty") + logging.warn("OUTPUT-MQTT: received data object is empty") return - topic = params.get('topic', None) - if not topic: - topic = f'{data.get("inverter_name", "hoymiles")}/{data.get("inverter_ser", None)}' + topic = f'{data.get("inverter_name", "hoymiles")}/{data.get("inverter_ser", None)}' - if HOYMILES_DEBUG_LOGGING: - logging.info(f'MQTT-topic: {topic} data-type: {type(response)}') + if HOYMILES_TRANSACTION_LOGGING: + logging.info(f'MQTT topic : {topic}') + logging.info(f'MQTT payload: {data}') - if isinstance(response, StatusResponse): + # if isinstance(response, StatusResponse): + if 'phases' in data and 'strings' in data: # Global Head if data['time'] is not None: @@ -290,38 +313,35 @@ def store_status(self, response, **params): self.client.publish(f'{topic}/Efficiency', data['efficiency'], self.qos, self.ret) - elif isinstance(response, HardwareInfoResponse): - if data["FW_ver_maj"] is not None and data["FW_ver_min"] is not None and data["FW_ver_pat"] is not None: - self.client.publish(f'{topic}/Firmware/Version',\ - f'{data["FW_ver_maj"]}.{data["FW_ver_min"]}.{data["FW_ver_pat"]}', self.qos, self.ret) + # elif isinstance(response, HardwareInfoResponse): + elif 'FW_ver_maj' in data and 'FW_ver_min' in data and 'FW_ver_pat' in data: + payload = f'{data["FW_ver_maj"]}.{data["FW_ver_min"]}.{data["FW_ver_pat"]}' + self.client.publish(f'{topic}/Firmware/Version', payload , self.qos, self.ret) - if data["FW_build_dd"] is not None and data["FW_build_mm"] is not None and data["FW_build_yy"] is not None and data["FW_build_HH"] is not None and data["FW_build_MM"] is not None: - self.client.publish(f'{topic}/Firmware/Build_at',\ - f'{data["FW_build_dd"]}/{data["FW_build_mm"]}/{data["FW_build_yy"]}T{data["FW_build_HH"]}:{data["FW_build_MM"]}',\ - self.qos, self.ret) + payload = f'{data["FW_build_dd"]}/{data["FW_build_mm"]}/{data["FW_build_yy"]}T{data["FW_build_HH"]}:{data["FW_build_MM"]}' + self.client.publish(f'{topic}/Firmware/Build_at', payload, self.qos, self.ret) - if data["FW_HW_ID"] is not None: - self.client.publish(f'{topic}/Firmware/HWPartId',\ - f'{data["FW_HW_ID"]}', self.qos, self.ret) + payload = f'{data["FW_HW_ID"]}' + self.client.publish(f'{topic}/Firmware/Build_at', payload, self.qos, self.ret) else: raise ValueError('Data needs to be instance of StatusResponse or a instance of HardwareInfoResponse') class VzInverterOutput: - def __init__(self, config, session): - self.session = session - self.serial = config.get('serial') - self.baseurl = config.get('url', 'http://localhost/middleware/') + def __init__(self, vz_inverter_config, session): + self.session = session + self.serial = vz_inverter_config.get('serial') + self.baseurl = vz_inverter_config.get('url', 'http://localhost/middleware/') self.channels = dict() - for channel in config.get('channels', []): - uid = channel.get('uid', None) + for channel in vz_inverter_config.get('channels', []): ctype = channel.get('type') + uid = channel.get('uid', None) # if uid and ctype: if ctype: self.channels[ctype] = uid - def store_status(self, data, session): + def store_status(self, data): """ Publish StatusResponse object @@ -329,63 +349,74 @@ def store_status(self, data, session): :raises ValueError: when response is not instance of StatusResponse """ + if len(self.channels) == 0: - return + logging.debug('no channels configured - no data to send') + return ts = int(round(data['time'].timestamp() * 1000)) - if HOYMILES_DEBUG_LOGGING: - logging.info(f'Volkszaehler-Timestamp: {ts}') - # AC Data phase_id = 0 - for phase in data['phases']: - self.try_publish(ts, f'ac_voltage{phase_id}', phase['voltage']) - self.try_publish(ts, f'ac_current{phase_id}', phase['current']) - self.try_publish(ts, f'ac_power{phase_id}', phase['power']) + if 'phases' in data: + for phase in data['phases']: + self.try_publish(ts, f'ac_voltage{phase_id}', phase['voltage']) + self.try_publish(ts, f'ac_current{phase_id}', phase['current']) + self.try_publish(ts, f'ac_power{phase_id}', phase['power']) self.try_publish(ts, f'ac_reactive_power{phase_id}', phase['reactive_power']) - self.try_publish(ts, f'ac_frequency{phase_id}', phase['frequency']) + self.try_publish(ts, f'ac_frequency{phase_id}', phase['frequency']) phase_id = phase_id + 1 # DC Data string_id = 0 - for string in data['strings']: - self.try_publish(ts, f'dc_voltage{string_id}', string['voltage']) - self.try_publish(ts, f'dc_current{string_id}', string['current']) - self.try_publish(ts, f'dc_power{string_id}', string['power']) + if 'strings' in data: + for string in data['strings']: + self.try_publish(ts, f'dc_voltage{string_id}', string['voltage']) + self.try_publish(ts, f'dc_current{string_id}', string['current']) + self.try_publish(ts, f'dc_power{string_id}', string['power']) self.try_publish(ts, f'dc_energy_daily{string_id}', string['energy_daily']) self.try_publish(ts, f'dc_energy_total{string_id}', string['energy_total']) - self.try_publish(ts, f'dc_irradiation{string_id}', string['irradiation']) + self.try_publish(ts, f'dc_irradiation{string_id}', string['irradiation']) string_id = string_id + 1 # Global - if data['event_count'] is not None: + if 'event_count' in data: self.try_publish(ts, f'event_count', data['event_count']) - if data['powerfactor'] is not None: + if 'powerfactor' in data: self.try_publish(ts, f'powerfactor', data['powerfactor']) - self.try_publish(ts, f'temperature', data['temperature']) - if data['yield_total'] is not None: + if 'temperature' in data: + self.try_publish(ts, f'temperature', data['temperature']) + if 'yield_total' in data: self.try_publish(ts, f'yield_total', data['yield_total']) - if data['yield_today'] is not None: + if 'yield_today' in data: self.try_publish(ts, f'yield_today', data['yield_today']) - self.try_publish(ts, f'efficiency', data['efficiency']) + if 'efficiency' in data: + self.try_publish(ts, f'efficiency', data['efficiency']) + + # eBZ = elektronischer Basiszähler (Stromzähler) + if '1_8_0' in data: + self.try_publish(ts, f'eBZ-import', data['1_8_0']) + if '2_8_0' in data: + self.try_publish(ts, f'eBZ-export', data['2_8_0']) + if '16_7_0' in data: + self.try_publish(ts, f'eBZ-power', data['16_7_0']) + return def try_publish(self, ts, ctype, value): if not ctype in self.channels: - if HOYMILES_DEBUG_LOGGING: - logging.warning(f'ctype \"{ctype}\" not found in ahoy.yml') + logging.debug(f'ctype \"{ctype}\" not found in ahoy.yml') return uid = self.channels[ctype] url = f'{self.baseurl}/data/{uid}.json?operation=add&ts={ts}&value={value}' if uid == None: - if HOYMILES_DEBUG_LOGGING: - logging.debug(f'ctype \"{ctype}\" has no configured uid-value in ahoy.yml') + logging.debug(f'ctype \"{ctype}\" has no configured uid-value in ahoy.yml') return - if HOYMILES_DEBUG_LOGGING: - logging.debug(f'VZ-url: {url}') + # if HOYMILES_VERBOSE_LOGGING: + if HOYMILES_TRANSACTION_LOGGING: + logging.info(f'VZ-url: {url}') try: r = self.session.get(url) @@ -400,36 +431,44 @@ def try_publish(self, ts, ctype, value): return class VolkszaehlerOutputPlugin(OutputPluginFactory): - def __init__(self, config, **params): + def __init__(self, vz_config, **params): """ - Initialize VolkszaehlerOutputPlugin + Initialize VolkszaehlerOutputPlugin with VZ-config + + Python Requests Module: + Make a request to a web page, and print the response text + https://requests.readthedocs.io/en/latest/user/advanced/ """ super().__init__(**params) try: import requests - import time except ModuleNotFoundError: - ErrorText1 = f'Module "requests" and "time" for VolkszaehlerOutputPlugin necessary.' + # ErrorText1 = f'Module "requests" and "time" for VolkszaehlerOutputPlugin necessary.' + ErrorText1 = f'Module "requests" for VolkszaehlerOutputPlugin necessary.' ErrorText2 = f'Install module with command: python3 -m pip install requests' print(ErrorText1, ErrorText2) logging.error(ErrorText1) logging.error(ErrorText2) exit(1) + # The Session object allows you to persist certain parameters across requests. self.session = requests.Session() - self.inverters = dict() - for inverterconfig in config.get('inverters', []): - serial = inverterconfig.get('serial') - output = VzInverterOutput(inverterconfig, self.session) - self.inverters[serial] = output + self.vz_inverters = dict() + for inverter_in_vz_config in vz_config.get('inverters', []): + url = inverter_in_vz_config.get('url') + serial = inverter_in_vz_config.get('serial') + # create class object with parameter "inverter_in_vz_config" and "requests.Session" object + self.vz_inverters[serial] = VzInverterOutput(inverter_in_vz_config, self.session) + if HOYMILES_VERBOSE_LOGGING: + logging.info(f"Volkszaehler: init connection object to host: {url}/{serial}") def disco(self, **params): self.session.close() # closing the connection return - def store_status(self, response, **params): + def store_status(self, data, **params): """ Publish StatusResponse object @@ -437,20 +476,44 @@ def store_status(self, response, **params): :raises ValueError: when response is not instance of StatusResponse """ - - # check decoder object for output - if not isinstance(response, StatusResponse): - raise ValueError('Data needs to be instance of StatusResponse') - - if len(self.inverters) == 0: + + if len(self.vz_inverters) == 0: # check list of inverters + logging.error('VolkszaehlerOutputPlugin:store_status: No inverters configured') + return + + # prep variables for output + if 'phases' in data and 'strings' in data: + serial = data["inverter_ser"] # extract "inverter-serial-number" from "response-data" + + elif 'Time' in data: + __data = dict() # create empty dict + for key in data: + if key == "Time": + __data['time'] = datetime.strptime(data[key], '%Y-%m-%dT%H:%M:%S') + elif isinstance(data[key], dict): + __data |= {'key' : key} + __data |= data[key] + + if not 'key' in __data: + raise ValueError(f"no 'key' in data - no output is sent: {__data}") + return + + data = __data + if HOYMILES_VERBOSE_LOGGING: + # eBZ = elektronischer Basiszähler (Stromzähler) + serial = data['96_1_0'] + logging.info(f"{data['key']}: {serial}" + f" - import:{data['1_8_0']:>8} kWh" + f" - export:{data['2_8_0']:>5} kWh" + f" - power:{data['16_7_0']:>8} W") + else: + raise ValueError(f"Unknown instance type - no output is sent: {data}") return - data = response.__dict__() - serial = data["inverter_ser"] - if serial in self.inverters: - output = self.inverters[serial] - try: - output.store_status(data, self.session) - except ValueError as e: - logging.warning('Could not send data to volkszaehler instance: %s' % e) - return + if serial in self.vz_inverters: # check, if inverter-serial-number in list of vz_inverters + try: + # call method VzInverterOutput.store_status with parameter "data" + self.vz_inverters[serial].store_status(data) + except ValueError as e: + logging.warning('Could not send data to volkszaehler instance: %s' % e) +