Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the management hub Ansible role #199

Open
wants to merge 2 commits into
base: new-mongodb
Choose a base branch
from
Open

Update the management hub Ansible role #199

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
bfb9916
ansible: Add feature to upgrade deployments
loudonlune Oct 29, 2024
648b1ef
ansible: Enable user management
loudonlune Nov 29, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 10 additions & 3 deletions ansible/Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@ ifeq ($(HZN_MGMT_HUB_EXPORT_VARS),Y)
ANSIBLE_PLAY_ARGS +=-e "export_data=true"
endif

ANSIBLE_PLAY_ARGS=-i "$(HZN_MGMT_HUB_ANSIBLE_INVENTORY)"
ANSIBLE_PLAY_ARGS?=" "
ANSIBLE_PLAY_ARGS += -i "$(HZN_MGMT_HUB_ANSIBLE_INVENTORY)"

ifdef HZN_MGMT_HUB_EXTRA_VARS_FILE
ANSIBLE_PLAY_ARGS +=-e "@$(HZN_MGMT_HUB_EXTRA_VARS_FILE)"
Expand All @@ -41,16 +42,22 @@ endif
dependencies: check_for_ansible
ansible-galaxy install $(ANSIBLE_GALAXY_ARGS)

# Install the playbook.
# Install the management hub.
install: dependencies
$(eval ANSIBLE_PLAY_ARGS +=-e "install=true")
ansible-playbook $(ANSIBLE_PLAY_ARGS) mgmt_hub.yml

# Uninstall the management hub (this is a destructive operation)
uninstall: dependencies
$(eval ANSIBLE_PLAY_ARGS +=-e "uninstall=true")
ansible-playbook $(ANSIBLE_PLAY_ARGS) mgmt_hub.yml

# Upgrade the management hub's containers
upgrade: dependencies
$(eval ANSIBLE_PLAY_ARGS +=-e "upgrade=true")
ansible-playbook $(ANSIBLE_PLAY_ARGS) mgmt_hub.yml

# Sync an existing deployment.
# Intended for updating users and groups
sync: dependencies
ansible-playbook $(ANSIBLE_PLAY_ARGS) mgmt_hub.yml
ansible-playbook $(ANSIBLE_PLAY_ARGS) mgmt_hub.yml
6 changes: 4 additions & 2 deletions ansible/roles/custom/hzn_mgmt_hub/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,12 @@ The role has several variables used to control the behavior of the role and the

|Variable|Type|Description|
|-|-|-|
|agent_install|str|A path in this repository to a legacy agent install config, as you would pass into the install script. Used when running in install mode.|
|agent_install|str|A path in this repository to an agent install config, as you would pass into the install script.|
|administrator_env|str|A path to additional environment variables that are more confidential. e.g. credentials, passwords, API tokens.|
|export_data|boolean|Writes the detected management hub deployment settings (including secrets) to a YAML file on the remote system.|
|install|boolean|Runs the playbook in installation mode (installs and syncs settings).|
|uninstall|boolean|Runs the playbook in uninstallation mode.|
|uninstall|boolean|Runs the playbook in uninstallation mode. **This is a destructive operation.**|
|update|boolean|Runs the playbook in update mode (updates the container images).|
|hzn_mgmt_hub|dict|A structured representation of the management hub deployment settings. This is automatically generated by the role when installing.|
|exchange_config|dict|Specifies a list of users and orgs which should exist on the target system.|

Expand Down
67 changes: 9 additions & 58 deletions ansible/roles/custom/hzn_mgmt_hub/defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,12 @@

# This is a default vars file based off of the behavior of the original installer script.

# Set this variable to the absolute path of a local configuration file.
# Set this variable to the path of your local agent-install.cfg
# It will be copied onto the remote system, and then loaded into the Ansible SSH session env.
# Accomplished using this: https://docs.ansible.com/ansible/latest/collections/ansible/builtin/raw_module.html
agent_install: test.env
agent_install: null

# Set this variable to the path
administrator_env: null

# If true, exports this config with populated secrets using the imported vault secret.
export_data: false
Expand All @@ -17,61 +19,10 @@ uninstall: false
# If true, installs the management hub.
install: false

# If true, upgrades the management hub.
upgrade: false

# This is an additional place to define the configuration for the management hub.
# If fields are not given, then the playbook will generate defaults.
# For now, that is handled by the deploy-mgmt-hub script.
hzn_mgmt_hub:
config: {}

fdo:
config: {}
secrets: {}
net: {}

agent:
config: {}

vault:
config: {}
secrets: {}
net: {}

mongo:
config: {}
secrets: {}
net: {}

postgres:
config: {}
secrets: {}
net: {}

css:
config: {}
secrets: {}
net: {}

exchange:
# General settings for the exchange container.
config: {}

# Secrets for the exchange container. When set to null, they will be generated.
# These keys can not be generated by Ansible.
# Either set them to known values, or do not provide them.
# root_pw: null
# root_pw_bcrypted: null
secrets: {}

# Network settings for the exchange container.
net: {}

hzn:
config: {}
secrets: {}
net: {}
agbot:
config: {}
secrets: {}
net: {}

agbot2: {}
# hzn_mgmt_hub: {}
120 changes: 99 additions & 21 deletions ansible/roles/custom/hzn_mgmt_hub/library/load_env_config.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,9 @@
# The management hub config.
"mgmt_hub": {"type": dict, "required": True},
"env_file": {"type": str, "required": False},
"agent_install": {"type": str, "required": False},
}


def key_to_env(key: str, prefix: Optional[str]) -> str:
if prefix:
return f"{prefix.upper()}_{key.upper()}"
else:
return key.upper()


@unique
class ConfFieldType(Enum):
secrets = "secrets"
Expand Down Expand Up @@ -106,7 +99,17 @@ def __init__(self, ekey: str):
self.field_type = ConfFieldType.get_field_type(tokens)
self.key = "_".join(tokens)


"""
Object for manipulating the OpenHorizon Management Hub configuration variables.
It can produce a structured configuration from environment variables,
and turn a structured configuration back into environment variables, providing
some compatibility between the Ansible way of storing configuration and
the existing method for configuring OpenHorizon via the all-in-one script.

It uses some established, de facto patterns about the naming convention for
creating a structured document out of the environment variables. Most importantly, the fact that
environment variables are preceded with the name of the component they are related to.
"""
class ConfigLoader(object):
COMPONENTS: List[str] = [
"agbot",
Expand All @@ -128,10 +131,30 @@ class ConfigLoader(object):
arch: str
changed: bool

"""
Get a configuration value located at a path.
Returns None if the value could not be found.
"""
def get(self, path: ConfPath) -> Optional[Any]:
sec = self.config.get(path.component) if path.component else self.config

if type(sec) == dict \
and (cmp := sec.get(path.field_type.value)) \
and (val := cmp.get(path.key)):
return val
else:
return None

"""
Insert a value into the configuration.
"""
def insert(self, path: ConfPath, value: Any):
idict = self.config

if path.component:
if not path.component in idict:
idict[path.component] = {}

idict = idict[path.component]

if path.field_type.value not in idict:
Expand Down Expand Up @@ -181,7 +204,7 @@ def __init__(self, facts: Dict, config: Dict):
self.changed = False

def settings_into_env(
self, name: Optional[str], settings: Dict[str, Any]
self, name: Optional[str], settings: Dict[str, Any], export: bool = False
) -> List[str]:
if name:
prefix = f"{name.upper()}_"
Expand All @@ -190,31 +213,76 @@ def settings_into_env(

return list(
map(
lambda i: f'{prefix}{i[0].upper()}="{i[1]}"',
lambda i: f"export {prefix}{i[0].upper()}='{i[1]}'" if export else f"{prefix}{i[0].upper()}='{i[1]}'",
filter(lambda i: not i[1] is None, settings.items()),
)
)

def component_into_env(self, name: str, config: Dict) -> List[str]:
def component_into_env(self, name: str, config: Dict, export: bool = False) -> List[str]:
lines: List[str] = []

for value in config.values():
if type(value) is dict:
lines.extend(self.settings_into_env(name, value))
lines.extend(self.settings_into_env(name, value, export))

return lines

def into_env(self) -> str:
lines: List[str] = []
"""
Make the administrator's environment using the entire configuration.
Returns the contents of a generated environment file.
"""
def make_administrator_environment(self) -> str:
lines: List[str] = [
"# Administrator's environment file used to run deploy-mgmt-hub.sh",
"# Usage: source <this-file>",
"# MANAGED BY ANSIBLE! DO NOT EDIT! Edits applied to this file will not be persistent.",
"",
]

for key, value in self.config.items():
if not type(value) is dict:
continue

if key in self.COMPONENTS:
lines.extend(self.component_into_env(key, value))
lines.extend(self.component_into_env(key, value, True))
else:
lines.extend(self.settings_into_env(None, value))
lines.extend(self.settings_into_env(None, value, True))

lines.append("") # Add newline at end of file

return "\n".join(lines)

"""
Make the agent-install.cfg using only a subset of the configuration.
Returns the contents of a generated environment file.
"""
def make_agent_install(self) -> str:
lines: List[str] = [
"# agent-install.cfg",
"# MANAGED BY ANSIBLE! DO NOT EDIT! Edits applied to this file will not be persistent.",
"",
]

# These environment variable names will be included in the agent-install.cfg only if they are present in the config.
AGENT_INSTALL_KEYS = [
"HZN_LISTEN_IP",
"HZN_LISTEN_PUBLIC_IP",
"HZN_ORG_ID",
"HZN_EXCHANGE_URL",
"HZN_FSS_CSSURL",
"HZN_AGBOT_URL",
"HZN_FDO_SVC_URL",
]

# Include only needed keys in agent-install.cfg
def cond_append(ekey: str):
if org_id := self.get(ConfPath(ekey)):
lines.append(f"{ekey}='{org_id}'")

for key in AGENT_INSTALL_KEYS:
cond_append(key)

lines.append("") # Add newline at end of file

return "\n".join(lines)

Expand All @@ -225,6 +293,7 @@ def run_module():
config: Dict = module.params["mgmt_hub"]
facts: Dict = module.params["ansible_facts"]
env_file: Optional[str] = module.params.get("env_file")
agent_install: Optional[str] = module.params.get("agent_install")

if "architecture" not in facts:
module.fail_json("System arch not in facts.")
Expand All @@ -235,18 +304,27 @@ def run_module():
if val_msgs:
module.fail_json("Configuration validation failed", validation_errors=val_msgs)

# If the env_file is present,
# load it into the environment and update the config.
# Load the environment file.
if env_file and os.path.isfile(env_file):
loader.insert_environment_keys(env_file)

env_str = loader.into_env()
# Load the agent-install (another source of environment variables).
# The provided agent-install takes highest precedence.
if agent_install and os.path.isfile(agent_install):
loader.insert_environment_keys(agent_install)

# This environment is used to run the installer.
env_str = loader.make_administrator_environment()

# This environment is the agent-install.cfg, safe for distribution.
agent_install = loader.make_agent_install()

module.exit_json(
**{
"changed": loader.changed,
"hzn_mgmt_hub": loader.config,
"env_script": env_str,
"administrator_env": env_str,
"agent_install": agent_install,
}
)

Expand Down
Loading