providers/vmware: Process guestinfo.metadata netplan configuration

The network environment can be dynamic and thus needs to be provided as
VM metadata. Since the format should not depend on whether the VM runs
uses Ignition and Afterburn or Cloud-Init, the idea is to also support
the guestinfo.metadata variable as used by Cloud-Init which contains
Netplan YAML/JSON network configuration.
Add a new command to write out Netplan configs to a given directory,
similar as we do with networkd units. (While this is currently just used
for VMware, other providers could also construct the netplan data type
from the netplan-types crate to provide Netplan configurations if the OS
rather wants to use NetworkManager than systemd-networkd. For backwards
compatibility and to not need Netplan it would be nice to keep the
systemd-networkd support as long as its used.)

References:
https://cloudinit.readthedocs.io/en/latest/reference/datasources/vmware.html#walkthrough-of-guestinfo-keys-transport
https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html
https://netplan.io/reference/
https://linux-on-z.blogspot.com/p/using-netplan-on-ibm-z.html

Cargo.toml

@@ -35,9 +35,11 @@ debug = true
 
 [dependencies]
 anyhow = "1.0"
+base64 = "0.21"
 cfg-if = "1.0"
 clap = { version = "4", "default_features" = false, "features" = ["std", "cargo", "derive", "error-context", "help", "suggestions", "usage", "wrap_help"] }
 ipnetwork = ">= 0.17, < 0.21"
+libflate = "1.3"
 libsystemd = ">= 0.2.1, < 0.7.0"
 mailparse = ">= 0.13, < 0.15"
 maplit = "1.0"
@@ -49,7 +51,7 @@ pnet_datalink = ">= 0.26, < 0.35"
 reqwest = { version = ">= 0.10, < 0.12", features = [ "blocking" ] }
 serde =  { version = "1.0", features = [ "derive" ] }
 serde-xml-rs = ">= 0.4, < 0.7"
-serde_json = "1.0"
+serde_json = { version = "1.0", features = [ "preserve_order" ] }
 serde_yaml = ">= 0.8, < 0.10"
 slog = { version = "2.7", features = ["max_level_trace", "release_max_level_info"] }
 slog-async = ">= 2.5, < 3"

docs/development/distro.md

@@ -14,3 +14,16 @@ Alternatively, sshd can be configured to read the fragment file directly:
 ```
 AuthorizedKeysFile .ssh/authorized_keys .ssh/authorized_keys.d/afterburn
 ```
+
+## VMware Netplan guestinfo metadata
+
+The `guestinfo.metadata` and `guestinfo.metadata.encoding` fields can contain a Netplan configuration provided by the VM provisioning logic.
+Netplan is required on the OS to render the Netplan format to either NetworkManager or systemd-networkd configuration files. By default, Netplan generates systemd-networkd units. Since the renderer backend is defined in the Netplan config itself, requiring NetworkManager in the config would rule out support for systems that don't use it (unless they would ship a drop-in file with later lexicographical ordering to force it to `networkd`). As systemd-networkd can work in parallel with NetworkManager, it's expected that the renderer field is left to its default but systems can also add a default drop-in file with early lexicographical ordering to prefer NetworkManager.
+
+The Afterburn invocation is as follows, where `FOLDER` could be `/run/netplan/`:
+
+```
+afterburn multi --netplan-configs FOLDER --provider vmware
+```
+
+Afterwards, `netplan generate` can be used to render the config files. If that is done before `systemd-networkd` runs, this is enough, but if the network already is up, `netplan apply` should be used instead.

docs/index.md

@@ -17,6 +17,7 @@ It comprises several modules which may run at different times during the lifecyc
 Depending on the specific platform, the following services may run in the [initramfs](https://github.com/coreos/afterburn/tree/main/dracut/30afterburn) on first boot:
  * setting local hostname
  * injecting [network command-line arguments](usage/initrd-network-cmdline.md)
+ * configuring the network with [Netplan guestinfo metadata on VMware](usage/vmware-netplan-guestinfo-metadata.md)
 
 The following features are conditionally available on some platforms as [systemd service units](https://github.com/coreos/afterburn/tree/main/systemd):
  * installing public SSH keys for local system users

docs/release-notes.md

@@ -9,6 +9,7 @@ nav_order: 8
 Major changes:
 
 - Add support for Scaleway
+- Add Netplan guestinfo support on VMware
 
 Minor changes:
 

docs/usage.md

@@ -10,6 +10,10 @@ has_toc: false
 
 See [Initrd first-boot network arguments](usage/initrd-network-cmdline.md).
 
+## VMware Netplan guestinfo metadata
+
+See [VMware Netplan guestinfo metadata](usage/vmware-netplan-guestinfo-metadata.md).
+
 ## Metadata attributes
 
 See [Metadata attributes](usage/attributes.md).

docs/usage/vmware-netplan-guestinfo-metadata.md

@@ -0,0 +1,27 @@
+---
+nav_order: 2
+parent: Usage
+---
+
+# VMware Netplan guestinfo metadata
+
+The network environment can vary between VMware servers and instead of leaking these requirements into userdata snippets, a well-known guestinfo metadata field can be used.
+The guestinfo metadata field is OS-independent and supported by cloud-init (spec [here](https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html), example [here](https://cloudinit.readthedocs.io/en/latest/reference/datasources/vmware.html#walkthrough-of-guestinfo-keys-transport)) and Afterburn. When the OS supports this mechanism the user can provide Netplan configs which the OS renders using the backend of choice.
+
+## Specifying the guestinfo metadata
+
+The guestinfo keys are named `guestinfo.metadata` for the content and `guestinfo.metadata.encoding` to specify the encoding of the content.
+The value of the encoding field can be empty to indicate raw string data, or one of `base64` or `b64` to indicate an base64 encoding, or one of `gzip+base64` or `gz+b64` to indicate base64-encoded gzip data.
+
+An example for raw string data is the following:
+```
+network:
+  version: 2
+  ethernets:
+    nics:
+      match:
+        name: ens*
+      dhcp4: yes
+```
+
+The supported config format with examples can be found in the [Netplan specification](https://netplan.readthedocs.io/en/latest/netplan-yaml/).

src/cli/multi.rs

@@ -26,6 +26,9 @@ pub struct CliMulti {
     /// The directory into which network units are written
     #[arg(long = "network-units", value_name = "path")]
     network_units_dir: Option<String>,
+    /// The directory into which netplan configs are written
+    #[arg(long = "netplan-configs", value_name = "path")]
+    netplan_config_dir: Option<String>,
     /// Update SSH keys for the given user
     #[arg(long = "ssh-keys", value_name = "username")]
     ssh_keys_user: Option<String>,
@@ -41,6 +44,7 @@ impl CliMulti {
 
         if self.attributes_file.is_none()
             && self.network_units_dir.is_none()
+            && self.netplan_config_dir.is_none()
             && !self.check_in
             && self.ssh_keys_user.is_none()
             && self.hostname_file.is_none()
@@ -72,6 +76,11 @@ impl CliMulti {
             .map_or(Ok(()), |x| metadata.write_network_units(x))
             .context("writing network units")?;
 
+        // write netplan configs if configured to do so
+        self.netplan_config_dir
+            .map_or(Ok(()), |x| metadata.write_netplan_configs(x))
+            .context("writing network units")?;
+
         // perform boot check-in.
         if self.check_in {
             metadata

src/providers/mod.rs

@@ -200,6 +200,10 @@ pub trait MetadataProvider {
         Ok(vec![])
     }
 
+    fn netplan_config(&self) -> Result<Option<String>> {
+        Ok(None)
+    }
+
     fn boot_checkin(&self) -> Result<()> {
         warn!("boot check-in requested, but not supported on this platform");
         Ok(())
@@ -295,6 +299,22 @@ pub trait MetadataProvider {
         }
         Ok(())
     }
+
+    fn write_netplan_configs(&self, netplan_configs_dir: String) -> Result<()> {
+        let dir_path = Path::new(&netplan_configs_dir);
+        fs::create_dir_all(dir_path)
+            .with_context(|| format!("failed to create directory {dir_path:?}"))?;
+
+        // Write a single afterburn `.yaml` netplan config.
+        if let Some(netplan_config) = &self.netplan_config()? {
+            let file_path = dir_path.join("50-afterburn.yaml");
+            let mut config_file = File::create(&file_path)
+                .with_context(|| format!("failed to create file {file_path:?}"))?;
+            write!(&mut config_file, "{netplan_config}")
+                .with_context(|| format!("failed to write netplan config file {config_file:?}"))?;
+        }
+        Ok(())
+    }
 }
 
 #[cfg(test)]