Flagged #10
Flagged #10
Conversation
- Changes in vault_backend.rb to support skipping reading from vault, unless it is flagged to do so using the override parameter - Added special hiera_vault and hiera_vault_array parser functions, hiera_vault_hash will be added soon.
Conflicts: lib/hiera/backend/vault_backend.rb
Now the Flagged functionality is complete. And with the HieraVault module all 3 specific function use the HieraVault module since most code is the same.
after merging in Flagged branch
- Test vault availabality with every lookup, not only at initialize. Puppet runs will now fail consistently when vault is unavailable. No need to restart puppetmasterd (or apache2 or nginx), after vault has become available anymore. - Fix: in case default was empty or nil, the specific functions would still return the default in case lookup in vault and/or other backends would not find anything, while an Exception should be raised. - Added auto-generate feature, see README.md and code for more details.
- The previous fix was incomplete: When a value was found in vault, but not in the rest of the backends, it would still raise an exception when default was empty or nil.
Conflicts: README.md : fixed lib/hiera/backend/vault_backend.rb : fixed
This reverts commit c8a3498.
This reverts commit ed25fbd.
This reverts commit cf0f62a.
- While HieraVault module retrieved hiera config from HieraPuppet, the logger was reset to console_logger. This caused logs to easily get filled up with a lot of debug messages, since console_logger just writes to stdout. - Very often, when exceptions occurred, it was not clear at which position of the code.
This makes the vault backend compliant with other backends. WARNING: using this with a decently long `:hierarchy` list will result in a lot of connections to vault. This can be solved using the 'Flagged' behavior. See jsok#10
Conflicts: lib/hiera/backend/vault_backend.rb
|
+1 |
|
The last commit contains a bug. Investigating and fixing it. |
It was already correct. Did not think of Ruby passing variables by reference.
When using the vault backend, we do not want to skip it if any error occurs with connecting or reading. For example when permission is denied.
|
Is it possible to perform a fallback to vault from eg. hiera-yaml? Like in hiera-yaml you can do a new lookup with first_key: "%{hiera('other_key')}"Then you could specify from the yaml file that some secret value should be found in vault: the_password: "%{hiera_vault('the_password')}"Any thoughts? Is this already possible in another way? |
|
As far as I have seen in the hiera code, the hiera() function as used in the data sources is one of a few supported functions. See also https://docs.puppet.com/hiera/1/variables.html |
|
I'm aware of the existing functions, I have no clue about implementing a custom one... |
|
interpolation functions are defined here: https://github.com/puppetlabs/hiera/blob/master/lib/hiera/interpolate.rb#L22:L26 |
|
+1 pretty please merge this PR. I'd like to start using Vault+Puppet but seeing that every |
|
I tried your PR. Unfortunately, I get: with the original code, I get: so there's a regression somewhere.. |
|
@FransUrbo the error clearly shows you still need to provide the token. It is a while ago that I have been working on this code. It might be that this PR requires the token to be provided via environment variable. Not sure. |
This changeset (use_hierarchy branch #9 merged into it) adds the feature to read from vault, only when using custom
hiera_vault,hiera_vault_array, andhiera_vault_hashfunctions in the puppet code.Normal
hiera*function calls will cause thevaultbackend to skip reading from vault.Why? Because always reading from vault is pretty heavy and easily can exhaust the number of source ports on the puppet master. Found this when doing test runs in a relatively small environment.
Additional features:
hiera_vault*calls to tell it to look in vault only. This can even be the default behavior.hiera_vault*functions.hiera_vault*functions. Actually a side effect of making it possible to specify a third parameter without a valid fallback default.More info in the README and the code.