Merge pull request #69 from semuconsulting/RC-1.0.43

Rc 1.0.43

.github/workflows/checkpr.yml

@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.8, 3.9, "3.10", "3.11", "3.12", "3.13.0-beta.3"]
+        python-version: [3.9, "3.10", "3.11", "3.12", "3.13.0-rc.3"]
 
     steps:
       - uses: actions/checkout@v4

.github/workflows/main.yml

@@ -10,7 +10,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        python-version: [3.8, 3.9, "3.10", "3.11", "3.12", "3.13.0-beta.3"]
+        python-version: [3.9, "3.10", "3.11", "3.12", "3.13.0-rc.3"]
 
     steps:
       - uses: actions/checkout@v4

.vscode/settings.json

@@ -4,5 +4,5 @@
     "editor.formatOnSave": true,
     "modulename": "${workspaceFolderBasename}",
     "distname": "${workspaceFolderBasename}",
-    "moduleversion": "1.0.42",
+    "moduleversion": "1.0.43",
 }

CONTRIBUTING.md

@@ -15,7 +15,7 @@ Please help us keep our issue list small by adding fixes: #{$ISSUE_NO} to the co
 ## Coding conventions
 
 * This is open source software. Code should be as simple and transparent as possible. Favour clarity over brevity.
-* The code should be compatible with Python >= 3.8.
+* The code should be compatible with Python >= 3.9.
 * The core code should be as generic and reusable as possible. We endeavour to limit the amount of processing dedicated to specific UBX message types, though this is sometimes unavoidable.
 * Avoid external library dependencies unless there's a compelling reason not to.
 * We use and recommend [Visual Studio Code](https://code.visualstudio.com/) with the [Python Extension](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for development and testing.

README.md

@@ -52,7 +52,7 @@ Contributions welcome - please refer to [CONTRIBUTING.MD](https://github.com/sem
 [![PyPI version](https://img.shields.io/pypi/v/pynmeagps.svg?style=flat)](https://pypi.org/project/pynmeagps/)
 ![PyPI downloads](https://img.shields.io/pypi/dm/pynmeagps.svg?style=flat)
 
-`pynmeagps` is compatible with Python 3.8 - 3.13 and has no third-party library dependencies.
+`pynmeagps` is compatible with Python 3.9 - 3.13 and has no third-party library dependencies.
 
 In the following, `python3` & `pip` refer to the Python 3 executables. You may need to substitute `python` for `python3`, depending on your particular environment (*on Windows it's generally `python`*).
 
@@ -101,6 +101,7 @@ The constructor accepts the following optional keyword arguments:
 * `nmeaonly`: True = raise error if stream contains non-NMEA data, False = ignore non-NMEA data (default)
 * `validate`: validation flags `VALCKSUM` (0x01) = validate checksum (default), `VALMSGID` (0x02) = validate msgId (i.e. raise error if unknown NMEA message is received)
 * `quitonerror`: `ERR_IGNORE` (0) = ignore errors,  `ERR_LOG` (1) = log continue, `ERR_RAISE` (2) = (re)raise (1)
+* `userdefined`: An optional user-defined payload definition dictionary, supplementing the existing `NMEA_PAYLOADS_GET` and `NMEA_PAYLOADS_GET_PROP` dictionaries (None).
 
 Examples:
 
@@ -152,6 +153,7 @@ The `parse()` function accepts the following optional keyword arguments:
 * `msgmode`: 0 = GET (default), 1 = SET, 2 = POLL
 * `validate`: validation flags `VALCKSUM` (0x01) = validate checksum (default), `VALMSGID` (0x02) = validate msgId (i.e. raise error if unknown NMEA message is received)
 * `quitonerror`: `ERR_IGNORE` (0) = ignore errors,  `ERR_LOG` (1) = log continue, `ERR_RAISE` (2) = (re)raise (1)
+* `userdefined`: An optional user-defined payload definition dictionary, supplementing the existing `NMEA_PAYLOADS_GET` and `NMEA_PAYLOADS_GET_PROP` dictionaries (None).
 
 Example:
 
@@ -207,9 +209,12 @@ class pynmeagps.nmeamessage.NMEAMessage(talker: str, msgID: str, msgmode: int, *
 
 You can create an `NMEAMessage` object by calling the constructor with the following parameters:
 1. talker (must be a valid talker from `pynmeagps.NMEA_TALKERS`)
-2. message id (must be a valid id from `pynmeagps.NMEA_MSGIDS` or `pynmeagps.NMEA_MSGIDS_PROP`)
-3. msgmode (0=GET, 1=SET, 2=POLL)
-4. (optional) a series of keyword parameters representing the message payload
+1. message id (must be a valid id from `pynmeagps.NMEA_MSGIDS` or `pynmeagps.NMEA_MSGIDS_PROP`)
+1. msgmode (0=GET, 1=SET, 2=POLL)
+1. hpnmeamode - boolean flag to signify high-precision NMEA mode (7 dp rather than 5) (False)
+1. validate - integer flag for checksum and/or message type validation (0=VALNONE, 1=VALCKSUM, 2=VALMSGID) (1)
+1. userdefined - an optional user-defined payload definition dictionary (None)
+1. (optional) a series of keyword parameters representing the message payload
 
 The 'msgmode' parameter signifies whether the message payload refers to a:
 

RELEASE_NOTES.md

@@ -1,5 +1,12 @@
 # pynmeagps Release Notes
 
+### RELEASE 1.0.43
+
+ENHANCEMENTS:
+
+1. Add provision for user-defined payload definition dictionary, supplementing the existing `NMEA_PAYLOADS_GET` and `NMEA_PAYLOADS_GET_PROP` dictionaries. Format should mirror that used in `NMEA_PAYLOADS_GET`. Can be used for proprietary product development purposes or for standard payload definitions which are not yet in the public domain.
+1. Drop active support for Python 3.8 - now End of Life as at October 2024.
+
 ### RELEASE 1.0.42
 
 ENHANCEMENTS:

pyproject.toml

@@ -7,10 +7,10 @@ name = "pynmeagps"
 authors = [{ name = "semuadmin", email = "[email protected]" }]
 maintainers = [{ name = "semuadmin", email = "[email protected]" }]
 description = "NMEA protocol parser and generator"
-version = "1.0.42"
+version = "1.0.43"
 license = { file = "LICENSE" }
 readme = "README.md"
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 classifiers = [
     "Operating System :: OS Independent",
     "Development Status :: 5 - Production/Stable",
@@ -22,7 +22,6 @@ classifiers = [
     "Intended Audience :: Science/Research",
     "Intended Audience :: End Users/Desktop",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.8",
     "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
@@ -56,10 +55,10 @@ test = [
 ]
 
 [tool.black]
-target-version = ['py38']
+target-version = ['py39']
 
 [tool.isort]
-py_version = 38
+py_version = 39
 profile = "black"
 
 [tool.bandit]
@@ -70,7 +69,7 @@ skips = []
 jobs = 0
 recursive = "y"
 reports = "y"
-py-version = "3.8"
+py-version = "3.9"
 fail-under = "9.8"
 fail-on = "E,F"
 clear-cache-post-run = "y"

src/pynmeagps/_version.py

@@ -8,4 +8,4 @@
 :license: BSD 3-Clause
 """
 
-__version__ = "1.0.42"
+__version__ = "1.0.43"

src/pynmeagps/nmeamessage.py

@@ -8,7 +8,7 @@
 :license: BSD 3-Clause
 """
 
-# pylint: disable=invalid-name, too-many-instance-attributes
+# pylint: disable=invalid-name, too-many-instance-attributes, too-many-positional-arguments
 
 import struct
 from datetime import datetime, timezone
@@ -41,6 +41,7 @@ def __init__(
         msgmode: int,
         hpnmeamode: bool = False,
         validate: int = nmt.VALCKSUM,
+        userdefined: dict = None,
         **kwargs,
     ):
         """Constructor.
@@ -55,7 +56,9 @@ def __init__(
         :param str msgID: message ID e.g. "GGA"
         :param int msgmode: mode (0=GET, 1=SET, 2=POLL)
         :param bool hpnmeamode: high precision lat/lon mode (7dp rather than 5dp) (False)
-        :param int validate: validation flags - VALNONE (0), VALCKSUM (1), VALMSGID (2) (1)
+        :param int validate: VALNONE (0), VALCKSUM (1), VALMSGID (2),
+            (can be OR'd) (1)
+        :param dict userdefined: user-defined payload definition dictionary (None)
         :param kwargs: keyword arg(s) representing all or some payload attributes
         :raises: NMEAMessageError
         """
@@ -64,20 +67,23 @@ def __init__(
         super().__setattr__("_immutable", False)
         self._logger = getLogger(__name__)
         self._validate = validate
-        self._nominal = False  # flag for unrecognised NMEA sentence types
-        self._proprietary = False  # proprietary message definition
+        self._userdefined = {} if userdefined is None else userdefined
 
         if msgmode not in (0, 1, 2):
             raise nme.NMEAMessageError(
                 f"Invalid msgmode {msgmode} - must be 0, 1 or 2."
             )
         if talker not in nmt.NMEA_TALKERS:
-            raise nme.NMEAMessageError(f"Unknown talker {talker}.")
+            if self._validate & nmt.VALMSGID:
+                raise nme.NMEAMessageError(f"Unknown talker {talker}.")
         if msgID in nmt.NMEA_MSGIDS:
-            self._proprietary = False
-        elif msgID in nmt.NMEA_MSGIDS_PROP or msgID in nmt.PROP_MSGIDS:
-            self._proprietary = True
+            self._defsource = nmt.DEF_STND  # standard
+        elif msgID in nmt.NMEA_MSGIDS_PROP or msgID in nmt.NMEA_PREFIX_PROP:
+            self._defsource = nmt.DEF_PROP  # proprietary
+        elif msgID in self._userdefined:
+            self._defsource = nmt.DEF_USER  # user-defined
         else:
+            self._defsource = nmt.DEF_UNKN  # unrecognised
             if self._validate & nmt.VALMSGID:
                 raise nme.NMEAMessageError(
                     f"Unknown msgID {talker}{msgID}, msgmode {('GET','SET','POLL')[msgmode]}."
@@ -263,7 +269,6 @@ def _set_attribute_nominal(self, payload: list):
         :param list payload: payload as list
         """
 
-        self._nominal = True
         for i, fld in enumerate(payload):
             setattr(self, f"field_{i+1:02d}", fld)
 
@@ -277,7 +282,7 @@ def _get_dict(self, **kwargs) -> dict:
 
         try:
             key = self.msgID
-            if key in nmt.PROP_MSGIDS:  # proprietary, first element is msgId
+            if key in nmt.NMEA_PREFIX_PROP:  # proprietary, first element is msgId
                 if "payload" in kwargs:
                     if key == "ASHR" and self._payload[0][1].isdigit():
                         pass  # exception for PASHR pitch and roll sentence without msgId
@@ -288,16 +293,18 @@ def _get_dict(self, **kwargs) -> dict:
                 else:
                     raise nme.NMEAMessageError(
                         f"P{key} message definitions must "
-                        + "include payload or msgId keyword arguments."
+                        "include payload or msgId keyword arguments."
                     )
             key = key.upper()
             if self._mode == nmt.POLL:
                 return nmp.NMEA_PAYLOADS_POLL[key]
             if self._mode == nmt.SET:
                 return nms.NMEA_PAYLOADS_SET[key]
-            if self._proprietary:
+            if self._defsource == nmt.DEF_PROP:  # proprietary
                 return nmgp.NMEA_PAYLOADS_GET_PROP[key]
-            return nmg.NMEA_PAYLOADS_GET[key]
+            if self._defsource == nmt.DEF_USER:  # user defined
+                return self._userdefined[key]
+            return nmg.NMEA_PAYLOADS_GET[key]  # standard
         except KeyError as err:
             erm = f"Unknown msgID {key} msgmode {('GET', 'SET', 'POLL')[self._mode]}."
             if self._validate & nmt.VALMSGID:
@@ -332,7 +339,7 @@ def __str__(self) -> str:
 
         stg = f"<NMEA({self.identity}"
         stg += ", "
-        if self._nominal:
+        if self._defsource == nmt.DEF_UNKN:
             stg += "NOMINAL, "
         for i, att in enumerate(self.__dict__):
             if att[0] != "_":  # only show public attributes
@@ -402,7 +409,7 @@ def identity(self) -> str:
 
         if (
             self._talker == "P"
-            and self._msgID in nmt.PROP_MSGIDS
+            and self._msgID in nmt.NMEA_PREFIX_PROP
             and hasattr(self, "msgId")
         ):
             return self._talker + self._msgID + self.msgId

src/pynmeagps/nmeareader.py

@@ -7,7 +7,9 @@
 Can also read from socket via SocketStream wrapper.
 
 Returns both the raw binary data (as bytes) and the parsed
-data (as a NMEAMessage object).
+data (as an NMEAMessage object).
+
+Implements an iterator: `for raw, parsed in NMEAReader(stream):`
 
 If the 'nmeaonly' kwarg is set to 'True', the reader
 will raise a NMEAParseError if it encounters any non-NMEA
@@ -21,6 +23,8 @@
 :license: BSD 3-Clause
 """
 
+# pylint: disable=too-many-positional-arguments
+
 from logging import getLogger
 from socket import socket
 
@@ -52,17 +56,20 @@ def __init__(
         quitonerror: int = ERR_LOG,
         bufsize: int = 4096,
         errorhandler: object = None,
+        userdefined: dict = None,
     ):
         """Constructor.
 
         :param stream stream: input data stream (e.g. Serial or binary File)
         :param int msgmode: 0=GET, 1=SET, 2=POLL (0)
-        :param int validate: validation flags - VALNONE (0), VALCKSUM (1), VALMSGID (2) (1)
+        :param int validate: VALNONE (0), VALCKSUM (1), VALMSGID (2),
+            (can be OR'd) (1)
         :param bool nmeaonly: True = error on non-NMEA data, False = ignore non-NMEA data
         :param int quitonerror: ERR_IGNORE (0) = ignore errors,  ERR_LOG (1) = log continue,
             ERR_RAISE (2) = (re)raise (1)
         :param int bufsize: socket recv buffer size (4096)
         :param object errorhandler: error handling object or function (None)
+        :param dict userdefined: user-defined payload definition dictionary (None)
         :raises: NMEAParseError (if mode is invalid)
         """
         # pylint: disable=too-many-arguments
@@ -80,6 +87,7 @@ def __init__(
         self._nmea_only = nmeaonly
         self._validate = validate
         self._mode = msgmode
+        self._userdefined = userdefined
         self._logger = getLogger(__name__)
 
     def __iter__(self):
@@ -130,6 +138,7 @@ def read(self) -> tuple:
                         raw_data,
                         msgmode=self._mode,
                         validate=self._validate,
+                        userdefined=self._userdefined,
                     )
                     parsing = False
                 else:  # it's not a NMEA message (UBX or something else)
@@ -222,16 +231,19 @@ def parse(
         message: bytes,
         msgmode: int = GET,
         validate: int = VALCKSUM,
+        userdefined: dict = None,
     ) -> object:
         """
         Parse NMEA byte stream to NMEAMessage object.
 
         :param bytes message: bytes message to parse
         :param int msgmode: 0=GET, 1=SET, 2=POLL (0)
-        :param int validate: 1 VALCKSUM (default), 2 VALMSGID (can be OR'd)
+        :param int validate: VALNONE (0), VALCKSUM (1), VALMSGID (2),
+            (can be OR'd) (1)
+        :param dict userdefined: user-defined payload definition dictionary (None)
         :return: NMEAMessage object (or None if unknown message and VALMSGID is not set)
         :rtype: NMEAMessage
-        :raises: Exception (if data stream contains invalid data or unknown message type)
+        :raises: NMEAParseError (if data stream contains invalid data or unknown message type)
 
         """
 
@@ -256,6 +268,7 @@ def parse(
                 payload=payload,
                 checksum=checksum,
                 validate=validate,
+                userdefined=userdefined,
             )
 
         except nme.NMEAMessageError as err: