Merge branch 'develop' into 2030_5_anew

docs/source/deploying-volttron/platform-configuration.rst

@@ -6,11 +6,11 @@ Platform Configuration
 
 Each instance of the VOLTTRON platform includes a `config` file which is used to configure the platform instance on
 startup.  This file is kept in :term:`VOLTTRON_HOME` and is created using the `volttron-cfg` (`vcfg`) command, or will
-be created with default values on start up of the platform otherwise.
+be created with default values on start up of the platform otherwise. `vcfg` also provides commands to configure and
+install few frequently used agents and update configuration-store entries
 
 Following is helpful information about the `config` file and the `vcfg` command.
 
-
 VOLTTRON_HOME
 =============
 
@@ -75,9 +75,13 @@ The example consists of the following entries:
 VOLTTRON Config
 ===============
 
-The `volttron-cfg` or `vcfg` command allows for an easy configuration of the VOLTTRON environment.  The command includes
-the ability to set up the platform configuration, an instance of the platform historian, VOLTTRON Central UI, and
-VOLTTRON Central Platform agent.
+The `volttron-cfg` or `vcfg` command allows for an easy configuration of the VOLTTRON environment.
+The `vcfg` command can be run without any arguments to go through all available configuration steps sequentially and
+pick what is appropriate for a given instance of VOLTTRON. This is useful when starting with a new VOLTTRON instance.
+
+`vcfg` command can also be used with specific subcommands to configure and install specific agents such as
+listener agent, platform historian, VOLTTRON Central UI, and VOLTTRON Central Platform agent
+or add one or more agent configurations to VOLTTRON's configuration store.
 
 Running `vcfg` will create a `config` file in `VOLTTRON_HOME` which will be populated according to the answers to
 prompts.  This process should be repeated for each platform instance, and can be re-run to reconfigure a platform
@@ -158,12 +162,52 @@ Optional Arguments
   - **-v, --verbose** - Enables verbose output in standard-output (PIP output, etc.)
   - **--vhome VHOME** - Provide a path to set `VOLTTRON_HOME` for this instance
   - **--instance-name INSTANCE_NAME** - Provide a name for this instance.  Required for running secure agents mode
-  - **--list-agents** - Display a list of configurable agents (Listener, Platform Driver, Platform Historian, VOLTTRON
-    Central, VOLTTRON Central Platform)
-  - **--agent AGENT [AGENT ...]** - Configure listed agents
   - **--agent-isolation-mode** - Require that agents run as their own Unix users (this requires running
     `scripts/secure_user_permissions.sh` as `sudo`)
 
+Sub commands
+------------
+**--list-agents**
+~~~~~~~~~~~~~~~~~~
+    Display a list of configurable agents (Listener, Platform Driver, Platform Historian, VOLTTRON
+    Central, VOLTTRON Central Platform)
+
+**--agent AGENT [AGENT ...]**
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+Configure and install one of the listed agents for a specific VOLTTRON instance
+
+**update-config-store --metadata-file METADATA_FILE_OR_DIR [METADATA_FILE_OR_DIR ...]**
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Command to bulk add/update multiple agent configurations to VOLTTRON's configuration store. This is especially useful
+for automated deployment use cases. For example, a automated deployment setup where devices in a network are
+detected, and configurations for platform driver and control agents such as ILCAgent are auto generated using semantic
+tags. In such as scenario, adding all the generated configurations to the configuration store using the below command
+is more efficient than running the "vctl config store" command once for each configuration
+
+This command can only be used when volttron instance is not running.
+
+Usage:
+
+.. code-block:: bash
+
+    vcfg update-config-store --metadata-file <one or more metadata file or directories containing metadata files separated by space>
+
+
+Format for Metadata file:
+
+.. code-block::
+
+    { "vip-id": [
+     {
+         "config-name": "optional. name. defaults to config
+         "config": "json config or string config or config file name",
+         "config-type": "optional. type of config - csv or json or raw. defaults to json"
+     }, ...
+     ],...
+    }
+
+
 RabbitMQ Arguments
 ------------------
 vcfg command to configure a single RabbitMQ instance of VOLTTRON.

volttron/platform/agent/utils.py

@@ -151,7 +151,8 @@ def load_config(config_path):
 
     if not os.path.exists(config_path):
         raise ValueError(
-            f"Config file specified by AGENT_CONFIG path {config_path} does not exist.")
+             f"Config file specified by path {config_path} does not exist."
+        )
 
     # First attempt parsing the file with a yaml parser (allows comments natively)
     # Then if that fails we fallback to our modified json parser.
@@ -166,7 +167,7 @@ def load_config(config_path):
             with open(config_path) as f:
                 return parse_json_config(f.read())
         except Exception as e:
-            _log.error("Problem parsing agent configuration")
+            _log.error(f"Problem parsing configuration {config_path}: {e}")
             raise
 
 

volttron/platform/control/control_utils.py

@@ -36,23 +36,20 @@
 # under Contract DE-AC05-76RL01830
 # }}}
 import collections
-import itertools
 import sys
 import re
 from volttron.platform import jsonapi
 from volttron.platform.agent.utils import is_secure_mode
+import os
 
 _stdout = sys.stdout
 _stderr = sys.stderr
 
 
 def _calc_min_uuid_length(agents):
-    n = 0
-    for agent1, agent2 in itertools.combinations(agents, 2):
-        common_len = sum(1 for a, b in zip(agent1.uuid, agent2.uuid) if a == b)
-        if common_len > n:
-            n = common_len
-    return n + 1
+    agent_ids = [agent.uuid for agent in agents]
+    common_len = len(os.path.commonprefix(agent_ids))
+    return common_len + 1
 
 
 def _list_agents(aip):