Skip to content

Commit

Permalink
DRAFT: terraform: add var.content, see nix-community#413.
Browse files Browse the repository at this point in the history
This PR adds a Terraform input variable named `content`.
This allows passing in a string from Terraform to expose to
NixOS's run-time to make available as a file (default:
`/etc/nixos-vars.json`) as suggested by @Mic92 at nix-community#414.

This third implementation wraps the original `lib.nixosSystem`
call to allow passing info without either use of `--impure`
or having to stage to Git.

Example usage:

```nix
let
  servers = ...;
  variable = ...;
  data = ...;
  resource = ...;
in
{
  inherit variable data resource;
  module =
    lib.mapAttrs (server_name: _server_config: let
    in {
      # pin module version by nix flake inputs
      source =
"github.com/numtide/nixos-anywhere?ref=${inputs.nixos-anywhere.sourceInfo.rev}/terraform/all-in-one";
      ...
      special_args = lib.tfRef "jsonencode(${lib.strings.toJSON {
        tf = {
          inherit server_name;
          # all variables
          # var = lib.mapAttrs (k: _: lib.tfRef
  "jsonencode(var.${k})") variable;
          # non-sensitive variables
          var = lib.mapAttrs (k: _: lib.tfRef "var.${k}")
  (lib.filterAttrs (_k: v: !(v ? sensitive && v.sensitive)) variable);
          data = lib.mapAttrs (type: instances: lib.mapAttrs (k: _:
  tfRef "data.${type}.${k}") instances) data;
          resource = lib.mapAttrs (type: instances: lib.mapAttrs (k:
  _: tfRef "resource.${type}.${k}") instances) resource;
          server = lib.tfRef "resource.hcloud_server.${server_name}";
        };
      }})";
    })
    servers;
}
```

You can then use these in your `nixosConfigurations` thru the `tf` argument.

Status:

- [ ] resolve occasional NAR hash mismatches
- [ ] Test putting the content file into `.gitignore`.
  • Loading branch information
KiaraGrouwstra committed Oct 30, 2024
1 parent 51d347d commit f352ca0
Show file tree
Hide file tree
Showing 11 changed files with 152 additions and 9 deletions.
3 changes: 3 additions & 0 deletions terraform/all-in-one.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,7 @@ No providers.
| -------------------------------------------------------------------------------------- | ---------------- | ------- |
| <a name="module_install"></a> [install](#module_install) | ../install | n/a |
| <a name="module_nixos-rebuild"></a> [nixos-rebuild](#module_nixos-rebuild) | ../nixos-rebuild | n/a |
| <a name="module_nixos-vars"></a> [nixos-vars](#module_nixos-vars) | ../nixos-vars | n/a |
| <a name="module_partitioner-build"></a> [partitioner-build](#module_partitioner-build) | ../nix-build | n/a |
| <a name="module_system-build"></a> [system-build](#module_system-build) | ../nix-build | n/a |

Expand All @@ -107,12 +108,14 @@ No resources.

| Name | Description | Type | Default | Required |
| --------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------- | ----------------------------------------------------------------------- | :------: |
| <a name="input_content"></a> [content](#input_content) | Content to expose to the NixOS build as a file. | `string` | `"{}"` | no |
| <a name="input_debug_logging"></a> [debug\_logging](#input_debug_logging) | Enable debug logging | `bool` | `false` | no |
| <a name="input_deployment_ssh_key"></a> [deployment\_ssh\_key](#input_deployment_ssh_key) | Content of private key used to deploy to the target\_host after initial installation. To ensure maximum security, it is advisable to connect to your host using ssh-agent instead of relying on this variable | `string` | `null` | no |
| <a name="input_disk_encryption_key_scripts"></a> [disk\_encryption\_key\_scripts](#input_disk_encryption_key_scripts) | Each script will be executed locally. Output of each will be created at the given path to disko during installation. The keys will be not copied to the final system | <pre>list(object({<br> path = string<br> script = string<br> }))</pre> | `[]` | no |
| <a name="input_extra_environment"></a> [extra\_environment](#input_extra_environment) | Extra environment variables to be set during installation. This can be useful to set extra variables for the extra\_files\_script or disk\_encryption\_key\_scripts | `map(string)` | `{}` | no |
| <a name="input_extra_files_script"></a> [extra\_files\_script](#input_extra_files_script) | A script that should place files in the current directory that will be copied to the targets / directory | `string` | `null` | no |
| <a name="input_file"></a> [file](#input_file) | Nix file containing the nixos\_system\_attr and nixos\_partitioner\_attr. Use this if you are not using flake | `string` | `null` | no |
| <a name="input_filename"></a> [filename](#input_filename) | Name of the file to which to dump `content`. Defaults to `nixos-vars.json`. | `string` | `"./nixos-vars.json"` | no |
| <a name="input_install_port"></a> [install\_port](#input_install_port) | SSH port used to connect to the target\_host, before installing NixOS. If null than the value of `target_port` is used | `string` | `null` | no |
| <a name="input_install_ssh_key"></a> [install\_ssh\_key](#input_install_ssh_key) | Content of private key used to connect to the target\_host during initial installation | `string` | `null` | no |
| <a name="input_install_user"></a> [install\_user](#input_install_user) | SSH user used to connect to the target\_host, before installing NixOS. If null than the value of `target_host` is used | `string` | `null` | no |
Expand Down
10 changes: 10 additions & 0 deletions terraform/all-in-one/main.tf
Original file line number Diff line number Diff line change
@@ -1,15 +1,25 @@
module "nixos-vars" {
source = "../nixos-vars"
content = var.special_args
filename = var.filename
}

module "system-build" {
source = "../nix-build"
attribute = var.nixos_system_attr
file = var.file
nix_options = var.nix_options
content_file = var.filename
content_nar = module.nixos-vars.result.out
}

module "partitioner-build" {
source = "../nix-build"
attribute = var.nixos_partitioner_attr
file = var.file
nix_options = var.nix_options
content_file = var.filename
content_nar = module.nixos-vars.result.out
}

locals {
Expand Down
12 changes: 12 additions & 0 deletions terraform/all-in-one/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -131,3 +131,15 @@ variable "nixos_facter_path" {
description = "Path to which to write a `facter.json` generated by `nixos-facter`."
default = ""
}

variable "special_args" {
type = string
default = "{}"
description = "A map exposed as NixOS's `specialArgs` thru a file."
}

variable "filename" {
type = string
default = "./nixos-vars.json"
description = "Name of the file to which to dump `content`. Defaults to `nixos-vars.json`."
}
3 changes: 3 additions & 0 deletions terraform/nix-build/main.tf
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,12 @@ locals {
data "external" "nix-build" {
program = [ "${path.module}/nix-build.sh" ]
query = {
wrapper_path = "${path.module}/wrapper.tmpl.nix"
attribute = var.attribute
file = var.file
nix_options = local.nix_options
content_file = var.content_file
content_nar = var.content_nar
}
}
output "result" {
Expand Down
43 changes: 34 additions & 9 deletions terraform/nix-build/nix-build.sh
Original file line number Diff line number Diff line change
@@ -1,15 +1,40 @@
#!/usr/bin/env bash
set -efu
set -xefu

declare file attribute nix_options
eval "$(jq -r '@sh "attribute=\(.attribute) file=\(.file) nix_options=\(.nix_options)"')"
options=$(echo "${nix_options}" | jq -r '.options | to_entries | map("--option \(.key) \(.value)") | join(" ")')
if [[ -n ${file-} ]] && [[ -e ${file-} ]]; then
declare file attribute nix_options content_file content_nar wrapper_path
eval "$(jaq -r '@sh "attribute=\(.attribute) file=\(.file) nix_options=\(.nix_options) content_file=\(.content_file) content_nar=\(.content_nar) wrapper_path=\(.wrapper_path)"')"
if [ "${nix_options}" = '{"options":{}}' ]; then
options=""
else
options=$(echo "${nix_options}" | jaq -r '.options | to_entries | map("--option \(.key) \(.value)") | join(" ")')
fi
if [[ -n ${file-} ]] && [[ -e ${file-} ]] && [ "${file}" != "null" ]; then
# shellcheck disable=SC2086
out=$(nix build --no-link --json $options -f "$file" "$attribute")
printf '%s' "$out" | jq -c '.[].outputs'
else
# shellcheck disable=SC2086
out=$(nix build --no-link --json $options "$attribute")
printf '%s' "$out" | jq -c '.[].outputs'
# flakes want files to be staged to git, which is annoying, so hack around that
if [[ -n ${content_file-} ]] && [[ -e ${content_file-} ]] && [[ -n ${content_nar-} ]]; then
# default to saving the content file under the same name
content_name="$(basename "$content_file")"
rest="$(echo "${attribute}" | cut -d "#" -f 2)"
# e.g. config_path=nixosConfigurations.aarch64-linux.combined
config_path="${rest%.config.*}"
# e.g. config_attribute=config.system.build.toplevel
config_attribute="config.${rest#*.config.}"

# grab flake nar from error message
flake_rel="$(echo "${attribute}" | cut -d "#" -f 1)"
flake_dir="$(readlink -f "${flake_rel}")"
content_file="$(readlink -f "${content_file}")"
flake_nar="$(nix build --expr "builtins.getFlake ''git+file://${flake_dir}?narHash=sha256-0000000000000000000000000000000000000000000=''" 2>&1 | grep -Po "(?<=got ')sha256-[^']*(?=')")"
# substitute variables into the template
nix_expr="$(sed -e "s%\$flake_dir%${flake_dir}%g" -e "s%\$flake_nar%${flake_nar}%g" -e "s%\$content_name%${content_name}%g" -e "s%\$content_file%${content_file}%g" -e "s%\$content_nar%${content_nar}%g" -e "s%\$config_path%${config_path}%g" "${wrapper_path}")"
# inject content file into nixos config's `/etc/`
# shellcheck disable=SC2086
out=$(nix build --no-link --json ${options} --expr "${nix_expr}" "${config_attribute}")
else
# shellcheck disable=SC2086
out=$(nix build --no-link --json ${options} "$attribute")
fi
fi
printf '%s' "$out" | jaq -c '.[].outputs'
12 changes: 12 additions & 0 deletions terraform/nix-build/variables.tf
Original file line number Diff line number Diff line change
Expand Up @@ -14,3 +14,15 @@ variable "nix_options" {
description = "the options of nix"
default = {}
}

variable "content_file" {
type = string
description = "name of the file to be passed, if any"
default = null
}

variable "content_nar" {
type = string
description = "nar of the content file, if applicable"
default = null
}
10 changes: 10 additions & 0 deletions terraform/nix-build/wrapper.tmpl.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
let
cfg = (builtins.getFlake "file://$flake_dir/flake.nix?narHash=$flake_nar").$config_path;
in cfg.extendModules {
# pass the content thru fetchTree (over directly passing content) to prevent stack overflows
specialArgs = builtins.fromJSON (builtins.readFile (builtins.fetchTree {
type = "file";
url = (if (builtins.compareVersions builtins.nixVersion "2.19") == -1 then "" else "file:") + "$content_file";
narHash = "$content_nar";
}).outPath);
}
34 changes: 34 additions & 0 deletions terraform/nixos-vars.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
<!-- BEGIN_TF_DOCS -->
## Requirements

No requirements.

## Providers

| Name | Version |
|------|---------|
| <a name="provider_external"></a> [external](#provider\_external) | n/a |

## Modules

No modules.

## Resources

| Name | Type |
|------|------|
| [external_external.nixos-vars](https://registry.terraform.io/providers/hashicorp/external/latest/docs/data-sources/external) | data source |

## Inputs

| Name | Description | Type | Default | Required |
|------|-------------|------|---------|:--------:|
| <a name="input_content"></a> [content](#input\_content) | Content to expose to the NixOS build as a file. | `string` | `"{}"` | no |
| <a name="input_filename"></a> [filename](#input\_filename) | Name of the file to which to dump `content`. Defaults to `nixos-vars.json`. | `string` | `"./nixos-vars.json"` | no |

## Outputs

| Name | Description |
|------|-------------|
| <a name="output_result"></a> [result](#output\_result) | n/a |
<!-- END_TF_DOCS -->
10 changes: 10 additions & 0 deletions terraform/nixos-vars/main.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
data "external" "nixos-vars" {
program = [ "${path.module}/nixos-vars.sh" ]
query = {
content = var.content
filename = var.filename
}
}
output "result" {
value = data.external.nixos-vars.result
}
13 changes: 13 additions & 0 deletions terraform/nixos-vars/nixos-vars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/env bash
set -efu

declare content filename
eval "$(jq -r '@sh "content=\(.content) filename=\(.filename)"')"

if [ "${content}" != "{}" ]; then
echo "${content}" > "${filename}"
nar=$(nix hash path "${filename}")
else
nar=""
fi
printf "{\"out\":\"%s\"}" "${nar}"
11 changes: 11 additions & 0 deletions terraform/nixos-vars/variables.tf
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
variable "content" {
type = string
default = "{}"
description = "Content to expose to the NixOS build as a file."
}

variable "filename" {
type = string
default = "./nixos-vars.json"
description = "Name of the file to which to dump `content`. Defaults to `nixos-vars.json`."
}

0 comments on commit f352ca0

Please sign in to comment.