Support for JSON and YAML configuration files

[NeodymiumFerBore]

Desired Behavior

.json and/or .yaml/.yml files in /etc/netbox/config/ are loaded and used for configuration (in addition of Python files).

Contrast to Current Behavior

Only python is supported to define configuration.

Required Changes

Add support for other file suffixes in /docker/configuration.docker.py.

Discussion: Benefits and Drawbacks

If you use templating tools like Helm to manage netbox configuration, templating Python files can be tricky with nested dicts (PLUGINS_CONFIG is the main use case). yaml and json resolve booleans to true/false, while a valid Python boolean is True/False (capitalized), so we cannot just convert some conf to json. Also, it is pretty hard to handle unquoted booleans, lists and dicts, while keeping strings quoted. Covering all cases in Helm was a huge pain.

Benefits:

• define any part of netbox configuration in JSON and/or YAML format
• may or may not break old configurations:
  • if no json/yaml file is present, the process will remain the same (nothing breaks)
  • if for any reason, people were storing json/yaml files in /etc/netbox/config, then it may break

Drawbacks:

• may lose typing subtleties (example: json/yaml don't have tuple). However, it's still possible to have an extra.py file.

I don't know if other people have such use case, or if I just do it wrong. It's an open discussion. If this feature is accepted, I can submit a PR.

[tobiasge]

Netbox itself is using Python files for the configuration. I don't think it's a good idea to rewrite this system in the Docker image.
This would need to be a FR for Netbox.

You can mount configuration files as ConfigMaps and populate those directly from Python files. See --set-file parameter in Helm.

[NeodymiumFerBore]

We cannot leverage Helm deep merge with --set-file . For example, If you have 1 plugin with 10 config keys (with nested dicts), and need one to be changed in all your environments, then you only override this one key in your env-specific Values, and keep the 9 others DRY in a common Values file. Now, scale this to 10 plugins: this is another use case.

I considered using --set-file, but that was too much code repetition. I could have implemented deep merge for plugins config, but I found it easier to maintain to just accept yaml as config input.

I understand your point of view about this FR being for Netbox more than netbox-docker. I don't think it will be accepted though.

[cimnine]

I believe the following should work just fine:
As the config files are Python, you can create your own config file with Python code which loads values from a specific yaml/json file or deserializes values from a json/yaml string. That string could be helm-managed. Just put that special config file in the regular config directory and off you go. No need for anything special on our end.

[NeodymiumFerBore]

I believe the following should work just fine: As the config files are Python, you can create your own config file with Python code which loads values from a specific yaml/json file or deserializes values from a json/yaml string. That string could be helm-managed. Just put that special config file in the regular config directory and off you go. No need for anything special on our end.

That's what I do already. Wanted to propose it so the community can benefit from it. Was also interested in people feedback on how they handle such case.

[cimnine]

Wanted to propose it so the community can benefit from it.

That's kind of you. It seems like this could become a Wiki page for those interested. Or you could ask whether the netbox-chart project of @bootc is interested in such a contribution.

From a maintainer's perspective, we'd rather not add more custom functionality to the image itself. The reason is that we have to maintain the feature and test it with every release (our releases, and releases of NetBox). We're not keen on committing to more work. (It's why we extracted the initializers to become their own plugin, for example.)

[bootc]

My chart already does this and has from the start. It's really trivially done and all the dependencies are already there. It seems a shame to miss something so simple out of netbox-docker itself tbh.