New feature/improve demo (#1)

* clean-up sub-module git

* make demo value more intersting

* able to build functional wheel using python setup.py bdist_wheel

* ready to publish to pypi before clean-up

* working setup.py for src structure, able to create tar, so, whl

* testpypi enable, note: using stand-alone setup.py

* re-anchored deps/dnp3

* changed to forked repo for deps/dnp3 sub-module with ownership

* cleaned up for package release

* added docs on building wheel

* updated notes_on_packaging.md

* updated requirements.txt

* clean-up, deleted wheel

* use local (relative) import for dnp3demo to prevent circular import

* cleaned-up dnp3demo import

* Resolved master and outstation not shutdown gracefully issue.

* resolved master outstation not able to shutdown gracefully

* allowed master and outstation to shutdown gracefully

.gitmodules

@@ -3,4 +3,5 @@
 	url = https://github.com/Kisensum/pybind11.git
 [submodule "deps/dnp3"]
 	path = deps/dnp3
-	url = https://github.com/automatak/dnp3.git
+	# url = https://github.com/automatak/dnp3.git
+	url = https://github.com/kefeimo/opendnp3.git

README.md

@@ -1,56 +1,67 @@
-# pydnp3
+# dnp3-python
 Python bindings for the [opendnp3](https://github.com/automatak/dnp3) library,  an open source
 implementation of the [DNP3](http://ww.dnp.org) protocol stack written in C++14.
 
-Note:  This is a work in progress.  See [Issues](http://github.com/Kisensum/pydnp3/issues) for things we know about and feel free to add your own.
+Note:  This is a redesign of [pydnp3](https://github.com/ChargePoint/pydnp3) and work in progress.  
 
-**Supported Platforms:** Linux, MacOS
 
-## Dependencies
-To build the library from source, you must have:
+**Supported Platforms:** Linux
 
-* A toolchain with a C++14 compiler
-* CMake >= 2.8.12 (https://cmake.org/download/)
+## Install
+Support Python >= 3.8, using pip
+```
+    $ pip install dnp3-python
+```
+#### Validate Installation
+After installing the package, run the following command to validate the installation.
+```
+    $ python -m dnp3demo
+```
+Expected output
+```
+ms(1666217818743) INFO    manager - Starting thread (0)
+channel state change: OPENING
+ms(1666217818744) INFO    tcpclient - Connecting to: 127.0.0.1
+ms(1666217818744) WARN    tcpclient - Error Connecting: Connection refused
+2022-10-19 17:16:58,744 dnp3demo.data_retrieval_demo    DEBUG   Initialization complete. Master Station in command loop.
+ms(1666217818746) INFO    manager - Starting thread (0)
+ms(1666217818746) INFO    server - Listening on: 127.0.0.1:20000
+2022-10-19 17:16:58,746 dnp3demo.data_retrieval_demo    DEBUG   Initialization complete. OutStation in command loop.
+ms(1666217819745) INFO    tcpclient - Connecting to: 127.0.0.1
+ms(1666217819745) INFO    tcpclient - Connected to: 127.0.0.1
+channel state change: OPEN
+ms(1666217819745) INFO    server - Accepted connection from: 127.0.0.1
 
-This repository includes two repositories as submodules (under `deps/`):
+...
 
-* dnp3 (https://github.com/automatak/dnp3)
-* pybind11 (https://github.com/Kisensum/pybind11) - This is a fork containing a minor patch
-required to compile some of the pydnp3 wrapper code. It will be replaced with pybind11 proper
-when the issue is resolved.
+===important log: case6 get_db_by_group_variation ==== 2 2022-10-19 17:17:01.157129 {GroupVariation.Group30Var6: {0: 5.588852790313346, 1: 17.7138169198775, 2: 22.456219616993142, 3: 0.0, 4: 0.0, 5: 0.0, 6: 0.0, 7: 0.0, 8: 0.0, 9: 0.0}}
+===important log: case6b get_db_by_group_variation ==== 2 2022-10-19 17:17:01.157407 {GroupVariation.Group1Var2: {0: True, 1: True, 2: True, 3: False, 4: False, 5: False, 6: False, 7: False, 8: False, 9: False}}
+===important log: case6c get_db_by_group_variation ==== 2 2022-10-19 17:17:01.157559 {GroupVariation.Group30Var1: {0: 5, 1: 17, 2: 22, 3: 0, 4: 0, 5: 0, 6: 0, 7: 0, 8: 0, 9: 0}}
+===important log: case7 get_db_by_group_variation_index ==== 2 2022-10-19 17:17:01.157661 {GroupVariation.Group30Var6: {0: 5.588852790313346}}
+===important log: case7b get_db_by_group_variation_index ==== 2 2022-10-19 17:17:01.157878 17.7138169198775
+===important log: case7c get_db_by_group_variation_index ==== 2 2022-10-19 17:17:01.157974 0.0
 
-## Build & Install
-At the moment, this library must be built from source:
-```
-    $ clone --recursive http://github.com/Kisensum/pydnp3
-    $ cd pydnp3
-    $ python setup.py install
 ```
 
 
-## Documentation
+## For Developers
 
-pydnp3 is a thin wrapper around most all of the opendnp3 classes.  Documentation for the opendnp3
+pydnp3 is a thin wrapper around opendnp3 classes. Documentation for the opendnp3
 classes is available at [automatak](https://www.automatak.com/opendnp3/#documentation).
 
-Use python's help to discover the available wrapper classes and functions.  For example,
+#### Dependencies
+To build the library from source, you must have:
 
-```
-> import pydnp3
-> help (pydnp3.opendnp3)
-Help on module pydnp3.opendnp3 in pydnp3:
+* A toolchain with a C++14 compiler
+* CMake >= 2.8.12 (https://cmake.org/download/)
 
-NAME
-    pydnp3.opendnp3 - Bindings for opendnp3 namespace
+This repository includes two repositories as submodules (under `deps/`):
 
-FILE
-    (built-in)
+* dnp3 (https://github.com/automatak/dnp3)
+* pybind11 (https://github.com/Kisensum/pybind11) - This is a fork containing a minor patch
+required to compile some of the pydnp3 wrapper code. It will be replaced with pybind11 proper
+when the issue is resolved.
 
-CLASSES
-    pybind11_builtins.pybind11_object(__builtin__.object)
-        AnalogCommandEvent
-        AnalogInfo
-            AnalogSpec
-...
-```
+Please find more info in the /docs folder about packaging process, e.g., building from the C++ source code, 
+packaging native Python code with C++ binding code, etc.
 

docs/Notes_on_packaging.md

@@ -0,0 +1,94 @@
+# Notes on Packaging
+
+## Background
+* The dnp3-python version 0.2.0 is an extension and repackaging of [pydnp3](https://github.com/ChargePoint/pydnp3) version 0.1.0
+* The wrapper provides more user-friendly and out-of-the-box examples (under /src/dnp3demo)
+* The wrapper enables features to install from [The Python Package Index (Pypi)](https://pypi.org/), 
+  while the original package requires installing from the source code.
+
+## Overview
+* While, building package from the source code is NOT needed when using the dnp3-python package,
+  it is still required for packing for distribution (i.e., building wheel and publishing to pypi.)
+* The dependencies to build c++ binding code are cmake and pybind11.
+  * CMake >= 2.8.12 is required.
+  * a special version of pybind11 is used and located at /deps/pybind11.
+* The c++ source code is based on [opendnp3](https://github.com/automatak/dnp3), 
+  forked and version-pinned at /deps/dnp3.
+  the binding code are located at /src and its sub-folders.
+* setup.py describes the packaging configuration. (tested with with python==3.8.13, setuptools==63.4.1)
+  * to build and install the package locally, run `python setup.py install`
+  * to build wheel, run `python setup.py bdist_wheel [--plat-name=manylinux1_x86_64]`
+
+## Notes on Building native Python + CPP-binding Python Package
+#### Intro
+* Need to build + package a python project with cpp-binding code.
+* Desired result
+  * pip install <package_name:dnp3_python>
+  * able to import cpp-binding modules, e.g., “from pydnp3 import opendnp3”
+  * able to import native python code, e.g., “from dnp3_python.dnp3station import master_new” 
+* Note:
+  * the package is a wrapper on “pydnp3” project by repackaging its cpp-binding functionality 
+    and extended/redesigned API in Python.
+  * the package inherited “pydnp3” project’s root module naming, 
+    i.e., “pydnp3” for cpp-binding related functionality.
+  * the extended method adopted root module naming, “dnp3_python”. 
+    (There is a discussion at the end of this memo about the reason for using different root module name.)
+#### Ingredients
+* Reference: [Package Discovery and Namespace Packages](https://setuptools.pypa.io/en/latest/userguide/package_discovery.html)
+* Key configuration in `setup.py`
+    ```
+    packages=find_namespace_packages(
+            where='src',
+            include=['dnp3_python*', 'dnp3demo']  # to include sub-packages as well.
+        ),
+    package_dir={"": "src"},
+    ```
+* Codebase structure (under /src)
+  ```
+    ./src
+    ├── asiodnp3
+    │   ...
+    ├── asiopal
+    │ 	...
+    ├── dnp3demo
+    │   ├── data_retrieval_demo.py
+    │   ...
+    ├── dnp3_python
+    │   ├── dnp3station
+    │   │   ├── __init__.py
+    │   │   ├── master_new.py
+    │   │   ├── outstation_new.py
+    │   │   ├── outstation_utils.py
+    │   │   ├── station_utils.py
+    │   │   └── visitors.py
+    │   └── __init__.py
+    ├── opendnp3
+    │  	...
+    ├── openpal
+    │  	...
+    ├── pydnp3asiodnp3.cpp
+    ├── pydnp3asiopal.cpp
+    ├── pydnp3.cpp
+    ├── pydnp3opendnp3.cpp
+    └── pydnp3openpal.cpp
+    ```
+
+#### Key takeaways
+* Using find_namespace_packages to “automatically” find packages 
+  (assuming defined sub-modules properly, i.e., with `__init__.py`)
+* Using trailing * to include submodules 
+  (e.g., dnp3_python* will include dnp3_python and dnp3_python/dnp3station)
+* Verifying with artifact structure (e.g., whl structure or tar structure)
+
+#### Discussion:
+* dnp3_python is a package mixed with cpp binding binary and native Python source code.
+    * the cpp binding path is resolved by using dynamic binary (i.e., *.so file)
+    * the name space is called “pydnp3”
+* To avoid namespace conflict, use different root namespace for python source code package.
+  * e.g., at one point, the pacakge adopted the structure /src/pydnp3/dnp3station, 
+    with the attempt to achieve `from pydnp3.dnp3station.master_new import *`. 
+    As a result, it will create a “pydnp3/dnp3station” dir at the site-package path.
+  * However, under the aforementioned structure, the cpp binding submodules are not resolvable, 
+    e.g., not able to achieve “from pydnp3 import opendnp3”. 
+    python will find the native “pydnp3/” first and ignore the package path linked to `*.so` file.
+