From 08527f13c511f19c96ecd003c191c8d744df9164 Mon Sep 17 00:00:00 2001 From: Castor Sky Date: Wed, 4 Dec 2024 23:39:51 +0300 Subject: [PATCH 1/4] add: new datasource and documentation --- .web-docs/README.md | 6 + .../data-source/virtual_machine/README.md | 155 +++++++++++++ .web-docs/metadata.hcl | 5 + datasource/virtual_machine/data.go | 182 +++++++++++++++ datasource/virtual_machine/data.hcl2spec.go | 115 ++++++++++ datasource/virtual_machine/data_test.go | 210 ++++++++++++++++++ datasource/virtual_machine/driver.go | 63 ++++++ datasource/virtual_machine/filters.go | 132 +++++++++++ .../virtual_machine/testing/call_restapi.go | 62 ++++++ .../virtual_machine/testing/simulator.go | 161 ++++++++++++++ .../virtual_machine/Config-not-required.mdx | 26 +++ .../virtual_machine/DatasourceOutput.mdx | 5 + .../virtual_machine/Tag-required.mdx | 7 + .../datasource/virtual_machine/Tag.mdx | 17 ++ docs/README.md | 6 + docs/datasources/virtual_machine.mdx | 92 ++++++++ main.go | 2 + 17 files changed, 1246 insertions(+) create mode 100644 .web-docs/components/data-source/virtual_machine/README.md create mode 100644 datasource/virtual_machine/data.go create mode 100644 datasource/virtual_machine/data.hcl2spec.go create mode 100644 datasource/virtual_machine/data_test.go create mode 100644 datasource/virtual_machine/driver.go create mode 100644 datasource/virtual_machine/filters.go create mode 100644 datasource/virtual_machine/testing/call_restapi.go create mode 100644 datasource/virtual_machine/testing/simulator.go create mode 100644 docs-partials/datasource/virtual_machine/Config-not-required.mdx create mode 100644 docs-partials/datasource/virtual_machine/DatasourceOutput.mdx create mode 100644 docs-partials/datasource/virtual_machine/Tag-required.mdx create mode 100644 docs-partials/datasource/virtual_machine/Tag.mdx create mode 100644 docs/datasources/virtual_machine.mdx diff --git a/.web-docs/README.md b/.web-docs/README.md index 2057cd076..d6da08e1e 100644 --- a/.web-docs/README.md +++ b/.web-docs/README.md @@ -49,6 +49,12 @@ packer plugins install github.com/hashicorp/vsphere This builder deploys and publishes new virtual machine to a vSphere Supervisor cluster using VM Service. +#### Data Sources + +- [vsphere-virtual_machine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtual_machine) - + This datasource returns name of existing virtual machine that matches all defined filters to use + it as a builder source for `vsphere-clone`. + #### Post-Processors - [vsphere](/packer/integrations/hashicorp/vsphere/latest/components/post-processor/vsphere) - diff --git a/.web-docs/components/data-source/virtual_machine/README.md b/.web-docs/components/data-source/virtual_machine/README.md new file mode 100644 index 000000000..a24f7bca0 --- /dev/null +++ b/.web-docs/components/data-source/virtual_machine/README.md @@ -0,0 +1,155 @@ +Type: `vsphere-virtual_machine` +Artifact BuilderId: `vsphere.virtual_machine` + +This datasource is able to get information about existing virtual machines from vSphere +and return name of one virtual machine that matches all specified filters. This virtual +machine can later be used in the vSphere Clone builder to select template. + +## Configuration Reference + +### Filters Configuration + +**Optional:** + + + +- `name` (string) - Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. + Using strict globs will not reduce execution time because vSphere API returns the full inventory. + But can be used for better readability over regular expressions. + +- `name_regex` (string) - Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. + The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. + E.g. the `^[^_]+$` filter will search names without any underscores. + The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + +- `template` (bool) - Filter to return only objects that are virtual machine templates. + Defaults to `false` and returns all VMs. + +- `node` (string) - Filter to search virtual machines only on the specified node. + +- `vm_tags` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. + Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. + Should work since vCenter 6.7. To avoid incompatibility, REST client is being + initialized only when at least one tag has been defined in the config. + +- `latest` (bool) - This filter determines how to handle multiple machines that were matched with all + previous filters. Machine creation time is being used to find latest. + By default, multiple matching machines results in an error. + + + + +### Tags Filter Configuration + + + +Example of multiple vm_tags blocks in HCL format: +``` + + vm_tags { + category = "team" + name = "operations" + } + vm_tags { + category = "SLA" + name = "gold" + } + +``` + + + + +**Required:** + + + +- `name` (string) - Tag with this name must be attached to virtual machine which should pass the Tags Filter. + +- `category` (string) - Name of the category that contains this tag. Both tag and category must be specified. + + + + +### Connection Configuration + +**Optional:** + + + +- `vcenter_server` (string) - The fully qualified domain name or IP address of the vCenter Server + instance. + +- `username` (string) - The username to authenticate with the vCenter Server instance. + +- `password` (string) - The password to authenticate with the vCenter Server instance. + +- `insecure_connection` (bool) - Do not validate the certificate of the vCenter Server instance. + Defaults to `false`. + + -> **Note:** This option is beneficial in scenarios where the certificate + is self-signed or does not meet standard validation criteria. + +- `datacenter` (string) - The name of the datacenter object in the vSphere inventory. + + -> **Note:** Required if more than one datacenter object exists in the + vSphere inventory. + + + + +## Output + + + +- `vm_name` (string) - Name of the found virtual machine. + + + + +## Example Usage + +This is a very basic example that connects to vSphere cluster and tries to search +the latest virtual machine that matches all filters. The machine name is then printed +to console as output variable. +```hcl +data "vsphere-virtual_machine" "default" { + vcenter_server = "vcenter.example.org" + insecure_connection = true + username = "administrator@example.org" + password = "St4ongPa$$w0rd" + datacenter = "AZ1" + latest = true + vm_tags { + category = "team" + name = "operations" + } + vm_tags { + category = "SLA" + name = "gold" + } + +} + +locals { + vm_name = data.vsphere-virtual_machine.default.vm_name +} + +source "null" "basic-example" { + communicator = "none" +} + +build { + sources = [ + "source.null.basic-example" + ] + + provisioner "shell-local" { + inline = [ + "echo vm_name: ${local.vm_name}", + ] + } +} + + +``` diff --git a/.web-docs/metadata.hcl b/.web-docs/metadata.hcl index 017cc2ad3..7e7e55368 100644 --- a/.web-docs/metadata.hcl +++ b/.web-docs/metadata.hcl @@ -33,4 +33,9 @@ integration { name = "vSphere Template" slug = "vsphere-template" } + component { + type = "data-source" + name = "vSphere Virtual Machine" + slug = "vsphere-virtual_machine" + } } diff --git a/datasource/virtual_machine/data.go b/datasource/virtual_machine/data.go new file mode 100644 index 000000000..30e457a91 --- /dev/null +++ b/datasource/virtual_machine/data.go @@ -0,0 +1,182 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +//go:generate packer-sdc struct-markdown +//go:generate packer-sdc mapstructure-to-hcl2 -type Config,Tag,DatasourceOutput +package virtual_machine + +import ( + "github.com/hashicorp/hcl/v2/hcldec" + "github.com/hashicorp/packer-plugin-sdk/common" + "github.com/hashicorp/packer-plugin-sdk/hcl2helper" + packersdk "github.com/hashicorp/packer-plugin-sdk/packer" + "github.com/hashicorp/packer-plugin-sdk/template/config" + vsCommon "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" + "github.com/pkg/errors" + "github.com/zclconf/go-cty/cty" +) + +// Example of multiple vm_tags blocks in HCL format: +// ``` +// +// vm_tags { +// category = "team" +// name = "operations" +// } +// vm_tags { +// category = "SLA" +// name = "gold" +// } +// +// ``` +type Tag struct { + // Tag with this name must be attached to virtual machine which should pass the Tags Filter. + Name string `mapstructure:"name" required:"true"` + // Name of the category that contains this tag. Both tag and category must be specified. + Category string `mapstructure:"category" required:"true"` +} + +type Config struct { + common.PackerConfig `mapstructure:",squash"` + vsCommon.ConnectConfig `mapstructure:",squash"` + + // Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. + // Using strict globs will not reduce execution time because vSphere API returns the full inventory. + // But can be used for better readability over regular expressions. + Name string `mapstructure:"name"` + // Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. + // The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. + // E.g. the `^[^_]+$` filter will search names without any underscores. + // The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + NameRegex string `mapstructure:"name_regex"` + // Filter to return only objects that are virtual machine templates. + // Defaults to `false` and returns all VMs. + Template bool `mapstructure:"template"` + // Filter to search virtual machines only on the specified node. + Node string `mapstructure:"node"` + // Filter to return only that virtual machines that have attached all specifies tags. + // Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. + // Should work since vCenter 6.7. To avoid incompatibility, REST client is being + // initialized only when at least one tag has been defined in the config. + VmTags []Tag `mapstructure:"vm_tags"` + // This filter determines how to handle multiple machines that were matched with all + // previous filters. Machine creation time is being used to find latest. + // By default, multiple matching machines results in an error. + Latest bool `mapstructure:"latest"` +} + +type Datasource struct { + config Config +} + +type DatasourceOutput struct { + // Name of the found virtual machine. + VmName string `mapstructure:"vm_name"` +} + +func (d *Datasource) ConfigSpec() hcldec.ObjectSpec { + return d.config.FlatMapstructure().HCL2Spec() +} + +func (d *Datasource) Configure(raws ...interface{}) error { + err := config.Decode(&d.config, nil, raws...) + if err != nil { + return err + } + + if d.config.Name == "" { + d.config.Name = "*" + } + + var errs *packersdk.MultiError + if d.config.VCenterServer == "" { + errs = packersdk.MultiErrorAppend(errs, errors.New("'vcenter_server' is required")) + } + if d.config.Username == "" { + errs = packersdk.MultiErrorAppend(errs, errors.New("'username' is required")) + } + if d.config.Password == "" { + errs = packersdk.MultiErrorAppend(errs, errors.New("'password' is required")) + } + if len(d.config.VmTags) > 0 { + for _, tag := range d.config.VmTags { + if tag.Name == "" || tag.Category == "" { + errs = packersdk.MultiErrorAppend(errs, errors.New("both name and category are required for tag")) + } + } + } + + if errs != nil && len(errs.Errors) > 0 { + return errs + } + + return nil +} + +func (d *Datasource) OutputSpec() hcldec.ObjectSpec { + return (&DatasourceOutput{}).FlatMapstructure().HCL2Spec() +} + +func (d *Datasource) Execute() (cty.Value, error) { + driver, err := newDriver(d.config) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to initialize driver") + } + + // This is the first level of filters + // (the finder with glob will return filtered list or drop an error if found nothing). + filteredVms, err := driver.finder.VirtualMachineList(driver.ctx, d.config.Name) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to retrieve virtual machines list") + } + + // Chain of other filters that will be executed only when defined + // and previous filter in chain left some virtual machines in the list. + if d.config.NameRegex != "" { + filteredVms = filterByNameRegex(filteredVms, d.config.NameRegex) + } + + if len(filteredVms) > 0 && d.config.Template { + filteredVms, err = filterByTemplate(driver, filteredVms) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by template attribute") + } + } + + if len(filteredVms) > 0 && d.config.Node != "" { + filteredVms, err = filterByNode(driver, d.config, filteredVms) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by node attribute") + } + } + + if len(filteredVms) > 0 && d.config.VmTags != nil { + filteredVms, err = filterByTags(driver, d.config.VmTags, filteredVms) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by tags") + } + } + + // No VMs passed the filter chain. Nothing to return. + if len(filteredVms) == 0 { + return cty.NullVal(cty.EmptyObject), errors.New("not a single VM matches the configured filters") + } + + if len(filteredVms) > 1 { + if d.config.Latest { + filteredVms, err = filterByLatest(driver, filteredVms) + if err != nil { + return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to find the latest VM") + } + } else { + // Too many machines passed the filter chain. Cannot decide which machine to return. + return cty.NullVal(cty.EmptyObject), errors.New("multiple VMs match the configured filters") + } + } + + output := DatasourceOutput{ + VmName: filteredVms[0].Name(), + } + + return hcl2helper.HCL2ValueFromConfig(output, d.OutputSpec()), nil +} diff --git a/datasource/virtual_machine/data.hcl2spec.go b/datasource/virtual_machine/data.hcl2spec.go new file mode 100644 index 000000000..abba4cc6b --- /dev/null +++ b/datasource/virtual_machine/data.hcl2spec.go @@ -0,0 +1,115 @@ +// Code generated by "packer-sdc mapstructure-to-hcl2"; DO NOT EDIT. + +package virtual_machine + +import ( + "github.com/hashicorp/hcl/v2/hcldec" + "github.com/zclconf/go-cty/cty" +) + +// FlatConfig is an auto-generated flat version of Config. +// Where the contents of a field with a `mapstructure:,squash` tag are bubbled up. +type FlatConfig struct { + PackerBuildName *string `mapstructure:"packer_build_name" cty:"packer_build_name" hcl:"packer_build_name"` + PackerBuilderType *string `mapstructure:"packer_builder_type" cty:"packer_builder_type" hcl:"packer_builder_type"` + PackerCoreVersion *string `mapstructure:"packer_core_version" cty:"packer_core_version" hcl:"packer_core_version"` + PackerDebug *bool `mapstructure:"packer_debug" cty:"packer_debug" hcl:"packer_debug"` + PackerForce *bool `mapstructure:"packer_force" cty:"packer_force" hcl:"packer_force"` + PackerOnError *string `mapstructure:"packer_on_error" cty:"packer_on_error" hcl:"packer_on_error"` + PackerUserVars map[string]string `mapstructure:"packer_user_variables" cty:"packer_user_variables" hcl:"packer_user_variables"` + PackerSensitiveVars []string `mapstructure:"packer_sensitive_variables" cty:"packer_sensitive_variables" hcl:"packer_sensitive_variables"` + VCenterServer *string `mapstructure:"vcenter_server" cty:"vcenter_server" hcl:"vcenter_server"` + Username *string `mapstructure:"username" cty:"username" hcl:"username"` + Password *string `mapstructure:"password" cty:"password" hcl:"password"` + InsecureConnection *bool `mapstructure:"insecure_connection" cty:"insecure_connection" hcl:"insecure_connection"` + Datacenter *string `mapstructure:"datacenter" cty:"datacenter" hcl:"datacenter"` + Name *string `mapstructure:"name" cty:"name" hcl:"name"` + NameRegex *string `mapstructure:"name_regex" cty:"name_regex" hcl:"name_regex"` + Template *bool `mapstructure:"template" cty:"template" hcl:"template"` + Node *string `mapstructure:"node" cty:"node" hcl:"node"` + VmTags []FlatTag `mapstructure:"vm_tags" cty:"vm_tags" hcl:"vm_tags"` + Latest *bool `mapstructure:"latest" cty:"latest" hcl:"latest"` +} + +// FlatMapstructure returns a new FlatConfig. +// FlatConfig is an auto-generated flat version of Config. +// Where the contents a fields with a `mapstructure:,squash` tag are bubbled up. +func (*Config) FlatMapstructure() interface{ HCL2Spec() map[string]hcldec.Spec } { + return new(FlatConfig) +} + +// HCL2Spec returns the hcl spec of a Config. +// This spec is used by HCL to read the fields of Config. +// The decoded values from this spec will then be applied to a FlatConfig. +func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec { + s := map[string]hcldec.Spec{ + "packer_build_name": &hcldec.AttrSpec{Name: "packer_build_name", Type: cty.String, Required: false}, + "packer_builder_type": &hcldec.AttrSpec{Name: "packer_builder_type", Type: cty.String, Required: false}, + "packer_core_version": &hcldec.AttrSpec{Name: "packer_core_version", Type: cty.String, Required: false}, + "packer_debug": &hcldec.AttrSpec{Name: "packer_debug", Type: cty.Bool, Required: false}, + "packer_force": &hcldec.AttrSpec{Name: "packer_force", Type: cty.Bool, Required: false}, + "packer_on_error": &hcldec.AttrSpec{Name: "packer_on_error", Type: cty.String, Required: false}, + "packer_user_variables": &hcldec.AttrSpec{Name: "packer_user_variables", Type: cty.Map(cty.String), Required: false}, + "packer_sensitive_variables": &hcldec.AttrSpec{Name: "packer_sensitive_variables", Type: cty.List(cty.String), Required: false}, + "vcenter_server": &hcldec.AttrSpec{Name: "vcenter_server", Type: cty.String, Required: false}, + "username": &hcldec.AttrSpec{Name: "username", Type: cty.String, Required: false}, + "password": &hcldec.AttrSpec{Name: "password", Type: cty.String, Required: false}, + "insecure_connection": &hcldec.AttrSpec{Name: "insecure_connection", Type: cty.Bool, Required: false}, + "datacenter": &hcldec.AttrSpec{Name: "datacenter", Type: cty.String, Required: false}, + "name": &hcldec.AttrSpec{Name: "name", Type: cty.String, Required: false}, + "name_regex": &hcldec.AttrSpec{Name: "name_regex", Type: cty.String, Required: false}, + "template": &hcldec.AttrSpec{Name: "template", Type: cty.Bool, Required: false}, + "node": &hcldec.AttrSpec{Name: "node", Type: cty.String, Required: false}, + "vm_tags": &hcldec.BlockListSpec{TypeName: "vm_tags", Nested: hcldec.ObjectSpec((*FlatTag)(nil).HCL2Spec())}, + "latest": &hcldec.AttrSpec{Name: "latest", Type: cty.Bool, Required: false}, + } + return s +} + +// FlatDatasourceOutput is an auto-generated flat version of DatasourceOutput. +// Where the contents of a field with a `mapstructure:,squash` tag are bubbled up. +type FlatDatasourceOutput struct { + VmName *string `mapstructure:"vm_name" cty:"vm_name" hcl:"vm_name"` +} + +// FlatMapstructure returns a new FlatDatasourceOutput. +// FlatDatasourceOutput is an auto-generated flat version of DatasourceOutput. +// Where the contents a fields with a `mapstructure:,squash` tag are bubbled up. +func (*DatasourceOutput) FlatMapstructure() interface{ HCL2Spec() map[string]hcldec.Spec } { + return new(FlatDatasourceOutput) +} + +// HCL2Spec returns the hcl spec of a DatasourceOutput. +// This spec is used by HCL to read the fields of DatasourceOutput. +// The decoded values from this spec will then be applied to a FlatDatasourceOutput. +func (*FlatDatasourceOutput) HCL2Spec() map[string]hcldec.Spec { + s := map[string]hcldec.Spec{ + "vm_name": &hcldec.AttrSpec{Name: "vm_name", Type: cty.String, Required: false}, + } + return s +} + +// FlatTag is an auto-generated flat version of Tag. +// Where the contents of a field with a `mapstructure:,squash` tag are bubbled up. +type FlatTag struct { + Name *string `mapstructure:"name" required:"true" cty:"name" hcl:"name"` + Category *string `mapstructure:"category" required:"true" cty:"category" hcl:"category"` +} + +// FlatMapstructure returns a new FlatTag. +// FlatTag is an auto-generated flat version of Tag. +// Where the contents a fields with a `mapstructure:,squash` tag are bubbled up. +func (*Tag) FlatMapstructure() interface{ HCL2Spec() map[string]hcldec.Spec } { + return new(FlatTag) +} + +// HCL2Spec returns the hcl spec of a Tag. +// This spec is used by HCL to read the fields of Tag. +// The decoded values from this spec will then be applied to a FlatTag. +func (*FlatTag) HCL2Spec() map[string]hcldec.Spec { + s := map[string]hcldec.Spec{ + "name": &hcldec.AttrSpec{Name: "name", Type: cty.String, Required: false}, + "category": &hcldec.AttrSpec{Name: "category", Type: cty.String, Required: false}, + } + return s +} diff --git a/datasource/virtual_machine/data_test.go b/datasource/virtual_machine/data_test.go new file mode 100644 index 000000000..97be44d45 --- /dev/null +++ b/datasource/virtual_machine/data_test.go @@ -0,0 +1,210 @@ +package virtual_machine + +import ( + "testing" + "time" + + vsCommon "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" + "github.com/vmware/govmomi/simulator" + + dsTesting "github.com/hashicorp/packer-plugin-vsphere/datasource/virtual_machine/testing" +) + +func TestExecute(t *testing.T) { + machinesToPrepare := []dsTesting.SimulatedVMConfig{ + { + Name: "first-vm", + Tags: []dsTesting.Tag{ + { + Category: "operating-system-class", + Name: "Linux", + }, + }, + }, { + Name: "second-vm", + Tags: []dsTesting.Tag{ + { + Category: "operating-system-class", + Name: "Linux", + }, + { + Category: "security-team", + Name: "red", + }, + { + Category: "security-team", + Name: "blue", + }, + }, + Template: true, + }, { + Name: "machine-three", + Tags: []dsTesting.Tag{ + { + Category: "operating-system-class", + Name: "Linux", + }, + { + Category: "security-team", + Name: "blue", + }, + }, + CreationTime: time.Now().AddDate(0, 0, 1), + }, + } + + model := simulator.VPX() + model.Datacenter = 2 + model.Machine = 8 + + vcSim, err := dsTesting.NewVCenterSimulator(model) + if err != nil { + t.Fatalf("error creating vCenter simulator: %s", err) + } + defer vcSim.Stop() + + err = vcSim.CustomizeSimulator(machinesToPrepare) + if err != nil { + t.Fatalf("error customizing simulator: %s", err) + } + + simulatorPassword, _ := vcSim.Server.URL.User.Password() + connectConfig := vsCommon.ConnectConfig{ + VCenterServer: vcSim.Server.URL.Host, + Username: vcSim.Server.URL.User.Username(), + Password: simulatorPassword, + InsecureConnection: true, + Datacenter: vcSim.Datacenter.Name(), + } + + dsTestConfigs := []struct { + name string + expectFailure bool + expectVmName string + config Config + }{ + { + name: "first-vm was found by name, no error", + expectFailure: false, + expectVmName: "first-vm", + config: Config{ + Name: "first-vm", + }, + }, + { + name: "no machines match the filter, error", + expectFailure: true, + expectVmName: "", + config: Config{ + Name: "firstest-vm", + }, + }, + { + name: "second-vm was found by the regex, no error", + expectFailure: false, + expectVmName: "second-vm", + config: Config{ + NameRegex: "^seco.*m$", + }, + }, + { + name: "multiple machines match the regex, but latest not used, error", + expectFailure: true, + expectVmName: "", + config: Config{ + NameRegex: ".*-vm", + }, + }, + { + name: "multiple guests match the regex and latest used, no error", + expectFailure: false, + expectVmName: "machine-three", + config: Config{ + NameRegex: "^[^_]+$", + Latest: true, + }, + }, + { + name: "found machine that is a template, no error", + expectFailure: false, + expectVmName: "second-vm", + config: Config{ + Template: true, + }, + }, + { + name: "found multiple machines at the node, error", + expectFailure: true, + expectVmName: "", + config: Config{ + Node: "DC0_H0", + }, + }, + { + name: "cluster node not found, error", + expectFailure: true, + expectVmName: "", + config: Config{ + Node: "unexpected_node", + }, + }, + { + name: "found machine with defined set of tags, no error", + expectFailure: false, + expectVmName: "second-vm", + config: Config{ + VmTags: []Tag{ + { + Category: "security-team", + Name: "blue", + }, + { + Category: "security-team", + Name: "red", + }, + }, + }, + }, + { + name: "found multiple machines with defined set of tags, error", + expectFailure: true, + expectVmName: "", + config: Config{ + VmTags: []Tag{ + { + Category: "operating-system-class", + Name: "Linux", + }, + }, + }, + }, + } + + for _, testConfig := range dsTestConfigs { + t.Run(testConfig.name, func(t *testing.T) { + testConfig.config.ConnectConfig = connectConfig + + ds := Datasource{ + config: testConfig.config, + } + err := ds.Configure() + if err != nil { + t.Fatalf("Failed to configure datasource: %s", err) + } + + result, err := ds.Execute() + if err != nil && !testConfig.expectFailure { + t.Fatalf("unexpected failure: %s", err) + } + if err == nil && testConfig.expectFailure { + t.Errorf("expected failure, but execution succeeded") + } + if err == nil { + vmName := result.GetAttr("vm_name").AsString() + if vmName != testConfig.expectVmName { + t.Errorf("expected vm name `%s`, but got `%s`", testConfig.expectVmName, vmName) + } + } + }) + } +} diff --git a/datasource/virtual_machine/driver.go b/datasource/virtual_machine/driver.go new file mode 100644 index 000000000..82070cb34 --- /dev/null +++ b/datasource/virtual_machine/driver.go @@ -0,0 +1,63 @@ +package virtual_machine + +import ( + "context" + "fmt" + + "net/url" + + "github.com/pkg/errors" + "github.com/vmware/govmomi" + "github.com/vmware/govmomi/find" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/vapi/rest" +) + +type VCenterDriver struct { + ctx context.Context + client *govmomi.Client + restClient *rest.Client + finder *find.Finder + datacenter *object.Datacenter +} + +func newDriver(config Config) (*VCenterDriver, error) { + ctx := context.Background() + + vcenterUrl, err := url.Parse(fmt.Sprintf("https://%v/sdk", config.VCenterServer)) + if err != nil { + return nil, errors.Wrap(err, "failed to parse URL") + } + vcenterUrl.User = url.UserPassword(config.Username, config.Password) + + client, err := govmomi.NewClient(ctx, vcenterUrl, true) + if err != nil { + return nil, errors.Wrap(err, "failed to create govmomi client") + } + + var restClient *rest.Client + if config.VmTags != nil { + // REST client is only needed when the plugin has to retrieve tags from VMs. + // Skip initialization if not needed (there is additional risk of fail on old vCenter versions). + restClient = rest.NewClient(client.Client) + err = restClient.Login(ctx, vcenterUrl.User) + if err != nil { + return nil, errors.Wrap(err, "failed to login to REST API endpoint") + } + } + + finder := find.NewFinder(client.Client, true) + datacenter, err := finder.DatacenterOrDefault(ctx, config.Datacenter) + if err != nil { + return nil, errors.Wrap(err, "failed to find datacenter") + } + finder.SetDatacenter(datacenter) + + return &VCenterDriver{ + ctx: ctx, + client: client, + restClient: restClient, + finder: finder, + datacenter: datacenter, + }, nil +} diff --git a/datasource/virtual_machine/filters.go b/datasource/virtual_machine/filters.go new file mode 100644 index 000000000..671fa6bb7 --- /dev/null +++ b/datasource/virtual_machine/filters.go @@ -0,0 +1,132 @@ +package virtual_machine + +import ( + "regexp" + "time" + + "github.com/pkg/errors" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/property" + "github.com/vmware/govmomi/vapi/tags" + "github.com/vmware/govmomi/vim25/mo" +) + +// Filter machines by matching their names against defined regular expression. +func filterByNameRegex(vmList []*object.VirtualMachine, nameRegex string) []*object.VirtualMachine { + re, _ := regexp.Compile(nameRegex) + result := make([]*object.VirtualMachine, 0) + for _, i := range vmList { + if re.MatchString(i.Name()) { + result = append(result, i) + } + } + return result +} + +// Filter machines by template attribute. Only templates will pass the filter. +func filterByTemplate(driver *VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + result := make([]*object.VirtualMachine, 0) + for _, i := range vmList { + isTemplate, err := i.IsTemplate(driver.ctx) + if err != nil { + return nil, errors.Wrap(err, "error checking if VM is a tempalte") + } + + if isTemplate { + result = append(result, i) + } + } + return result, nil +} + +// Filter machines by node placement. Only machines that are stored on the defined node will pass the filter. +func filterByNode(driver *VCenterDriver, config Config, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + pc := property.DefaultCollector(driver.client.Client) + obj, err := driver.finder.HostSystem(driver.ctx, config.Node) + if err != nil { + return nil, errors.Wrap(err, "error finding defined host system") + } + + var host mo.HostSystem + err = pc.RetrieveOne(driver.ctx, obj.Reference(), []string{"vm"}, &host) + if err != nil { + return nil, errors.Wrap(err, "error retrieving properties of host system") + } + + var nodeVms []mo.VirtualMachine + err = pc.Retrieve(driver.ctx, host.Vm, []string{"name"}, &nodeVms) + if err != nil { + return nil, errors.Wrap(err, "failed to get properties for VM") + } + + result := make([]*object.VirtualMachine, 0) + for _, filteredVm := range vmList { + vmName := filteredVm.Name() + for _, nodeVm := range nodeVms { + if vmName == nodeVm.Name { + result = append(result, filteredVm) + } + } + } + + return result, nil +} + +// Filter machines by tags. Tags are stored in the driver as list of flatTag elements. +// Only machines that has all the tags from list will pass the filter. +func filterByTags(driver *VCenterDriver, vmTags []Tag, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + result := make([]*object.VirtualMachine, 0) + tagMan := tags.NewManager(driver.restClient) + for _, filteredVm := range vmList { + realTagsList, err := tagMan.GetAttachedTags(driver.ctx, filteredVm.Reference()) + if err != nil { + return nil, errors.Wrap(err, "failed to get attached tags for vm") + } + matchedTagsCount := 0 + for _, configTag := range vmTags { + configTagMatched := false + for _, realTag := range realTagsList { + if configTag.Name == realTag.Name { + category, err := tagMan.GetCategory(driver.ctx, realTag.CategoryID) + if err != nil { + return nil, errors.Wrap(err, "failed to get attached category for tag") + } + if configTag.Category == category.Name { + configTagMatched = true + break + } + } + } + if configTagMatched { + matchedTagsCount++ + } else { + // If a single requested tag from config not matched then no need to proceed. + // Fail early. + break + } + } + if matchedTagsCount == len(vmTags) { + result = append(result, filteredVm) + } + } + + return result, nil +} + +func filterByLatest(driver *VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + var latestVM *object.VirtualMachine + var latestTimestamp time.Time + for _, elementVM := range vmList { + var vmConfig mo.VirtualMachine + err := elementVM.Properties(driver.ctx, elementVM.Reference(), []string{"config"}, &vmConfig) + if err != nil { + return nil, errors.Wrap(err, "error retrieving config properties of VM") + } + if vmConfig.Config.CreateDate.After(latestTimestamp) { + latestVM = elementVM + latestTimestamp = *vmConfig.Config.CreateDate + } + } + result := []*object.VirtualMachine{latestVM} + return result, nil +} diff --git a/datasource/virtual_machine/testing/call_restapi.go b/datasource/virtual_machine/testing/call_restapi.go new file mode 100644 index 000000000..b5ad58002 --- /dev/null +++ b/datasource/virtual_machine/testing/call_restapi.go @@ -0,0 +1,62 @@ +package testing + +import ( + "context" + + "github.com/pkg/errors" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/vapi/tags" +) + +func MarkSimulatedVmAsTemplate(ctx context.Context, vm *object.VirtualMachine) error { + task, err := vm.PowerOff(ctx) + if err != nil { + return errors.Wrap(err, "failed to issue powering off command to the machine") + } + err = task.Wait(ctx) + if err != nil { + return errors.Wrap(err, "failed to power off the machine") + } + err = vm.MarkAsTemplate(ctx) + if err != nil { + return errors.Wrap(err, "failed to mark VM as a template") + } + return nil +} + +// Try to find category passed by name, create category if not found and return category ID. +// Category will be created with "MULTIPLE" constraint. +func FindOrCreateCategory(ctx context.Context, man *tags.Manager, catName string) (string, error) { + categoryList, err := man.GetCategories(ctx) + if err != nil { + return "", errors.Wrap(err, "cannot get categories from cluster") + } + for _, category := range categoryList { + if category.Name == catName { + return category.ID, nil + } + } + newCategoryID, err := man.CreateCategory(ctx, &tags.Category{Name: catName, Cardinality: "MULTIPLE"}) + if err != nil { + return "", errors.Wrap(err, "cannot create category") + } + return newCategoryID, nil +} + +// Try to find the tagName in category with catID, create if not found and return tag ID. +func FindOrCreateTag(ctx context.Context, man *tags.Manager, catID string, tagName string) (string, error) { + tagsInCategory, err := man.GetTagsForCategory(ctx, catID) + if err != nil { + return "", errors.Wrap(err, "cannot get tags for category") + } + for _, tag := range tagsInCategory { + if tag.Name == tagName { + return tag.ID, nil + } + } + newTagID, err := man.CreateTag(ctx, &tags.Tag{Name: tagName, CategoryID: catID}) + if err != nil { + return "", errors.Wrap(err, "cannot create tag") + } + return newTagID, nil +} diff --git a/datasource/virtual_machine/testing/simulator.go b/datasource/virtual_machine/testing/simulator.go new file mode 100644 index 000000000..988e347dd --- /dev/null +++ b/datasource/virtual_machine/testing/simulator.go @@ -0,0 +1,161 @@ +package testing + +import ( + "context" + "crypto/tls" + "net/url" + "time" + + "github.com/pkg/errors" + "github.com/vmware/govmomi" + "github.com/vmware/govmomi/find" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/simulator" + "github.com/vmware/govmomi/vapi/rest" + _ "github.com/vmware/govmomi/vapi/simulator" + "github.com/vmware/govmomi/vapi/tags" + "github.com/vmware/govmomi/vim25/types" +) + +type Tag struct { + Category string + Name string +} + +type SimulatedVMConfig struct { + Name string + Tags []Tag + Template bool + CreationTime time.Time +} + +type VCenterSimulator struct { + Model *simulator.Model + Server *simulator.Server + Ctx context.Context + Client *govmomi.Client + RestClient *rest.Client + Finder *find.Finder + Datacenter *object.Datacenter +} + +// NewVCenterSimulator creates simulator object with model passed as argument. +func NewVCenterSimulator(model *simulator.Model) (*VCenterSimulator, error) { + ctx := context.Background() + if model == nil { + return nil, errors.New("model has not been initialized") + } + + err := model.Create() + if err != nil { + return nil, errors.Wrap(err, "failed to create simulator model") + } + model.Service.RegisterEndpoints = true + model.Service.TLS = new(tls.Config) + + server := model.Service.NewServer() + + u, err := url.Parse(server.URL.String()) + if err != nil { + return nil, errors.Wrap(err, "failed to parse simulator URL") + } + password, _ := simulator.DefaultLogin.Password() + u.User = url.UserPassword(simulator.DefaultLogin.Username(), password) + + client, err := govmomi.NewClient(ctx, u, true) + if err != nil { + return nil, errors.Wrap(err, "failed to connect to SOAP simulator") + } + + restClient := rest.NewClient(client.Client) + err = restClient.Login(ctx, simulator.DefaultLogin) + if err != nil { + return nil, errors.Wrap(err, "failed to login to REST simulator") + } + + finder := find.NewFinder(client.Client, false) + dcs, err := finder.DatacenterList(ctx, "*") + if err != nil { + return nil, errors.Wrap(err, "failed to list datacenters") + } + if len(dcs) == 0 { + return nil, errors.Wrap(err, "datacenters were not found in the simulator") + } + finder.SetDatacenter(dcs[0]) + + return &VCenterSimulator{ + Ctx: ctx, + Server: server, + Model: model, + Client: client, + Finder: finder, + RestClient: restClient, + Datacenter: dcs[0], + }, nil +} + +func (sim *VCenterSimulator) Stop() { + if sim.Model != nil { + sim.Model.Remove() + } + if sim.Server != nil { + sim.Server.Close() + } +} + +// CustomizeSimulator configures virtual machines in order that was retrieved from simulator according to +// list of machine configs in `vmsConfig`. Available options can be found in SimulatedVMConfig type. +func (sim *VCenterSimulator) CustomizeSimulator(vmsConfig []SimulatedVMConfig) error { + tagMan := tags.NewManager(sim.RestClient) + + vms, err := sim.Finder.VirtualMachineList(sim.Ctx, "*") + if err != nil { + return errors.Wrap(err, "failed to list VMs in cluster") + } + + for i := 0; i < len(vmsConfig); i++ { + vmConfig := types.VirtualMachineConfigSpec{ + Name: vmsConfig[i].Name, + } + + if !vmsConfig[i].CreationTime.IsZero() { + vmConfig.CreateDate = &vmsConfig[i].CreationTime + } + + if vmsConfig[i].Name != "" { + task, err := vms[i].Reconfigure(sim.Ctx, vmConfig) + if err != nil { + return errors.Wrap(err, "failed to issue rename of VM command") + } + if err = task.Wait(sim.Ctx); err != nil { + return errors.Wrap(err, "failed to rename VM") + } + } + + if vmsConfig[i].Template { + err = MarkSimulatedVmAsTemplate(sim.Ctx, vms[i]) + if err != nil { + return errors.Wrap(err, "failed to convert VMs to templates") + } + } + + if vmsConfig[i].Tags != nil { + for _, tag := range vmsConfig[i].Tags { + catID, err := FindOrCreateCategory(sim.Ctx, tagMan, tag.Category) + if err != nil { + return errors.Wrap(err, "failed to find/create category") + } + tagID, err := FindOrCreateTag(sim.Ctx, tagMan, catID, tag.Name) + if err != nil { + return errors.Wrap(err, "failed to find/create tag") + } + err = tagMan.AttachTag(sim.Ctx, tagID, vms[i].Reference()) + if err != nil { + return errors.Wrap(err, "failed to attach tag to VM") + } + } + } + } + + return nil +} diff --git a/docs-partials/datasource/virtual_machine/Config-not-required.mdx b/docs-partials/datasource/virtual_machine/Config-not-required.mdx new file mode 100644 index 000000000..7c3973d1f --- /dev/null +++ b/docs-partials/datasource/virtual_machine/Config-not-required.mdx @@ -0,0 +1,26 @@ + + +- `name` (string) - Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. + Using strict globs will not reduce execution time because vSphere API returns the full inventory. + But can be used for better readability over regular expressions. + +- `name_regex` (string) - Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. + The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. + E.g. the `^[^_]+$` filter will search names without any underscores. + The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + +- `template` (bool) - Filter to return only objects that are virtual machine templates. + Defaults to `false` and returns all VMs. + +- `node` (string) - Filter to search virtual machines only on the specified node. + +- `vm_tags` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. + Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. + Should work since vCenter 6.7. To avoid incompatibility, REST client is being + initialized only when at least one tag has been defined in the config. + +- `latest` (bool) - This filter determines how to handle multiple machines that were matched with all + previous filters. Machine creation time is being used to find latest. + By default, multiple matching machines results in an error. + + diff --git a/docs-partials/datasource/virtual_machine/DatasourceOutput.mdx b/docs-partials/datasource/virtual_machine/DatasourceOutput.mdx new file mode 100644 index 000000000..30c8a0950 --- /dev/null +++ b/docs-partials/datasource/virtual_machine/DatasourceOutput.mdx @@ -0,0 +1,5 @@ + + +- `vm_name` (string) - Name of the found virtual machine. + + diff --git a/docs-partials/datasource/virtual_machine/Tag-required.mdx b/docs-partials/datasource/virtual_machine/Tag-required.mdx new file mode 100644 index 000000000..65f82b0e9 --- /dev/null +++ b/docs-partials/datasource/virtual_machine/Tag-required.mdx @@ -0,0 +1,7 @@ + + +- `name` (string) - Tag with this name must be attached to virtual machine which should pass the Tags Filter. + +- `category` (string) - Name of the category that contains this tag. Both tag and category must be specified. + + diff --git a/docs-partials/datasource/virtual_machine/Tag.mdx b/docs-partials/datasource/virtual_machine/Tag.mdx new file mode 100644 index 000000000..cfe63d631 --- /dev/null +++ b/docs-partials/datasource/virtual_machine/Tag.mdx @@ -0,0 +1,17 @@ + + +Example of multiple vm_tags blocks in HCL format: +``` + + vm_tags { + category = "team" + name = "operations" + } + vm_tags { + category = "SLA" + name = "gold" + } + +``` + + diff --git a/docs/README.md b/docs/README.md index 2057cd076..d6da08e1e 100644 --- a/docs/README.md +++ b/docs/README.md @@ -49,6 +49,12 @@ packer plugins install github.com/hashicorp/vsphere This builder deploys and publishes new virtual machine to a vSphere Supervisor cluster using VM Service. +#### Data Sources + +- [vsphere-virtual_machine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtual_machine) - + This datasource returns name of existing virtual machine that matches all defined filters to use + it as a builder source for `vsphere-clone`. + #### Post-Processors - [vsphere](/packer/integrations/hashicorp/vsphere/latest/components/post-processor/vsphere) - diff --git a/docs/datasources/virtual_machine.mdx b/docs/datasources/virtual_machine.mdx new file mode 100644 index 000000000..d493c4dfd --- /dev/null +++ b/docs/datasources/virtual_machine.mdx @@ -0,0 +1,92 @@ +--- +modeline: | + vim: set ft=pandoc: +description: | + This datasource is able to get information about existing virtual machines from vSphere + and return name of one virtual machine that matches all specified filters. This virtual + machine can later be used in the vSphere Clone builder to select template. +page_title: vSphere VM - Datasources +sidebar_title: Virtual Machine +--- + +# VMware vSphere Virtual Machine Datasource + +Type: `vsphere-virtual_machine` +Artifact BuilderId: `vsphere.virtual_machine` + +This datasource is able to get information about existing virtual machines from vSphere +and return name of one virtual machine that matches all specified filters. This virtual +machine can later be used in the vSphere Clone builder to select template. + +## Configuration Reference + +### Filters Configuration + +**Optional:** + +@include 'datasource/virtual_machine/Config-not-required.mdx' + +### Tags Filter Configuration + +@include 'datasource/virtual_machine/Tag.mdx' + +**Required:** + +@include 'datasource/virtual_machine/Tag-required.mdx' + +### Connection Configuration + +**Optional:** + +@include 'builder/vsphere/common/ConnectConfig-not-required.mdx' + +## Output + +@include 'datasource/virtual_machine/DatasourceOutput.mdx' + +## Example Usage + +This is a very basic example that connects to vSphere cluster and tries to search +the latest virtual machine that matches all filters. The machine name is then printed +to console as output variable. +```hcl +data "vsphere-virtual_machine" "default" { + vcenter_server = "vcenter.example.org" + insecure_connection = true + username = "administrator@example.org" + password = "St4ongPa$$w0rd" + datacenter = "AZ1" + latest = true + vm_tags { + category = "team" + name = "operations" + } + vm_tags { + category = "SLA" + name = "gold" + } + +} + +locals { + vm_name = data.vsphere-virtual_machine.default.vm_name +} + +source "null" "basic-example" { + communicator = "none" +} + +build { + sources = [ + "source.null.basic-example" + ] + + provisioner "shell-local" { + inline = [ + "echo vm_name: ${local.vm_name}", + ] + } +} + + +``` diff --git a/main.go b/main.go index 790b807b0..121f768ec 100644 --- a/main.go +++ b/main.go @@ -12,6 +12,7 @@ import ( "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/clone" "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/iso" "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/supervisor" + "github.com/hashicorp/packer-plugin-vsphere/datasource/virtual_machine" "github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere" vsphereTemplate "github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere-template" "github.com/hashicorp/packer-plugin-vsphere/version" @@ -22,6 +23,7 @@ func main() { pps.RegisterBuilder("iso", new(iso.Builder)) pps.RegisterBuilder("clone", new(clone.Builder)) pps.RegisterBuilder("supervisor", new(supervisor.Builder)) + pps.RegisterDatasource("virtual_machine", new(virtual_machine.Datasource)) pps.RegisterPostProcessor(plugin.DEFAULT_NAME, new(vsphere.PostProcessor)) pps.RegisterPostProcessor("template", new(vsphereTemplate.PostProcessor)) pps.SetVersion(version.PluginVersion) From 6fdd9945cfb71e6f3fef8250fa48ba8d46f12648 Mon Sep 17 00:00:00 2001 From: Castor Sky Date: Sun, 8 Dec 2024 01:14:04 +0300 Subject: [PATCH 2/4] review: code fixed - Made minor changes to documentation. - Standard `errors` and `fmt` were used for custom errors. - field `Node` was renamed to `Host`. - field `VmTags` was renamed to `Tags` - package `virtual_machine` was renamed to `virtualmachine` - `driver` and `testing` were moved to `common` location --- .web-docs/README.md | 5 +- .../data-source/virtual_machine/README.md | 155 ----------------- .../data-source/virtualmachine/README.md | 160 ++++++++++++++++++ .web-docs/metadata.hcl | 2 +- builder/vsphere/driver/vm.go | 1 - datasource/common/driver/driver.go | 60 +++++++ .../testing/call_restapi.go | 25 +-- .../testing/simulator.go | 33 ++-- datasource/virtual_machine/driver.go | 63 ------- datasource/virtual_machine/filters.go | 132 --------------- .../data.go | 107 ++++++------ .../data.hcl2spec.go | 10 +- .../data_test.go | 33 ++-- datasource/virtualmachine/filters.go | 137 +++++++++++++++ .../virtual_machine/Config-not-required.mdx | 26 --- .../virtual_machine/Tag-required.mdx | 7 - .../virtualmachine/Config-not-required.mdx | 29 ++++ .../DatasourceOutput.mdx | 4 +- .../virtualmachine/Tag-required.mdx | 11 ++ .../Tag.mdx | 15 +- docs/README.md | 5 +- docs/datasources/virtual_machine.mdx | 92 ---------- docs/datasources/virtualmachine.mdx | 89 ++++++++++ main.go | 4 +- 24 files changed, 618 insertions(+), 587 deletions(-) delete mode 100644 .web-docs/components/data-source/virtual_machine/README.md create mode 100644 .web-docs/components/data-source/virtualmachine/README.md create mode 100644 datasource/common/driver/driver.go rename datasource/{virtual_machine => common}/testing/call_restapi.go (56%) rename datasource/{virtual_machine => common}/testing/simulator.go (75%) delete mode 100644 datasource/virtual_machine/driver.go delete mode 100644 datasource/virtual_machine/filters.go rename datasource/{virtual_machine => virtualmachine}/data.go (52%) rename datasource/{virtual_machine => virtualmachine}/data.hcl2spec.go (94%) rename datasource/{virtual_machine => virtualmachine}/data_test.go (85%) create mode 100644 datasource/virtualmachine/filters.go delete mode 100644 docs-partials/datasource/virtual_machine/Config-not-required.mdx delete mode 100644 docs-partials/datasource/virtual_machine/Tag-required.mdx create mode 100644 docs-partials/datasource/virtualmachine/Config-not-required.mdx rename docs-partials/datasource/{virtual_machine => virtualmachine}/DatasourceOutput.mdx (60%) create mode 100644 docs-partials/datasource/virtualmachine/Tag-required.mdx rename docs-partials/datasource/{virtual_machine => virtualmachine}/Tag.mdx (51%) delete mode 100644 docs/datasources/virtual_machine.mdx create mode 100644 docs/datasources/virtualmachine.mdx diff --git a/.web-docs/README.md b/.web-docs/README.md index d6da08e1e..20127e6fe 100644 --- a/.web-docs/README.md +++ b/.web-docs/README.md @@ -51,9 +51,8 @@ packer plugins install github.com/hashicorp/vsphere #### Data Sources -- [vsphere-virtual_machine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtual_machine) - - This datasource returns name of existing virtual machine that matches all defined filters to use - it as a builder source for `vsphere-clone`. +- [vsphere-virtualmachine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtualmachine) - + This data source returns the name of a virtual machine that matches all defined filters. #### Post-Processors diff --git a/.web-docs/components/data-source/virtual_machine/README.md b/.web-docs/components/data-source/virtual_machine/README.md deleted file mode 100644 index a24f7bca0..000000000 --- a/.web-docs/components/data-source/virtual_machine/README.md +++ /dev/null @@ -1,155 +0,0 @@ -Type: `vsphere-virtual_machine` -Artifact BuilderId: `vsphere.virtual_machine` - -This datasource is able to get information about existing virtual machines from vSphere -and return name of one virtual machine that matches all specified filters. This virtual -machine can later be used in the vSphere Clone builder to select template. - -## Configuration Reference - -### Filters Configuration - -**Optional:** - - - -- `name` (string) - Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. - Using strict globs will not reduce execution time because vSphere API returns the full inventory. - But can be used for better readability over regular expressions. - -- `name_regex` (string) - Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. - The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. - E.g. the `^[^_]+$` filter will search names without any underscores. - The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). - -- `template` (bool) - Filter to return only objects that are virtual machine templates. - Defaults to `false` and returns all VMs. - -- `node` (string) - Filter to search virtual machines only on the specified node. - -- `vm_tags` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. - Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. - Should work since vCenter 6.7. To avoid incompatibility, REST client is being - initialized only when at least one tag has been defined in the config. - -- `latest` (bool) - This filter determines how to handle multiple machines that were matched with all - previous filters. Machine creation time is being used to find latest. - By default, multiple matching machines results in an error. - - - - -### Tags Filter Configuration - - - -Example of multiple vm_tags blocks in HCL format: -``` - - vm_tags { - category = "team" - name = "operations" - } - vm_tags { - category = "SLA" - name = "gold" - } - -``` - - - - -**Required:** - - - -- `name` (string) - Tag with this name must be attached to virtual machine which should pass the Tags Filter. - -- `category` (string) - Name of the category that contains this tag. Both tag and category must be specified. - - - - -### Connection Configuration - -**Optional:** - - - -- `vcenter_server` (string) - The fully qualified domain name or IP address of the vCenter Server - instance. - -- `username` (string) - The username to authenticate with the vCenter Server instance. - -- `password` (string) - The password to authenticate with the vCenter Server instance. - -- `insecure_connection` (bool) - Do not validate the certificate of the vCenter Server instance. - Defaults to `false`. - - -> **Note:** This option is beneficial in scenarios where the certificate - is self-signed or does not meet standard validation criteria. - -- `datacenter` (string) - The name of the datacenter object in the vSphere inventory. - - -> **Note:** Required if more than one datacenter object exists in the - vSphere inventory. - - - - -## Output - - - -- `vm_name` (string) - Name of the found virtual machine. - - - - -## Example Usage - -This is a very basic example that connects to vSphere cluster and tries to search -the latest virtual machine that matches all filters. The machine name is then printed -to console as output variable. -```hcl -data "vsphere-virtual_machine" "default" { - vcenter_server = "vcenter.example.org" - insecure_connection = true - username = "administrator@example.org" - password = "St4ongPa$$w0rd" - datacenter = "AZ1" - latest = true - vm_tags { - category = "team" - name = "operations" - } - vm_tags { - category = "SLA" - name = "gold" - } - -} - -locals { - vm_name = data.vsphere-virtual_machine.default.vm_name -} - -source "null" "basic-example" { - communicator = "none" -} - -build { - sources = [ - "source.null.basic-example" - ] - - provisioner "shell-local" { - inline = [ - "echo vm_name: ${local.vm_name}", - ] - } -} - - -``` diff --git a/.web-docs/components/data-source/virtualmachine/README.md b/.web-docs/components/data-source/virtualmachine/README.md new file mode 100644 index 000000000..c976eea10 --- /dev/null +++ b/.web-docs/components/data-source/virtualmachine/README.md @@ -0,0 +1,160 @@ +Type: `vsphere-virtualmachine` +Artifact BuilderId: `vsphere.virtualmachine` + +This data source retrieves information about existing virtual machines from vSphere +and return name of one virtual machine that matches all specified filters. This virtual +machine can be used in the vSphere Clone builder to select a template. + +## Configuration Reference + +### Filters Configuration + +**Optional:** + + + +- `name` (string) - Basic filter with glob support (e.g. `ubuntu_basic*`). Defaults to `*`. + Using strict globs will not reduce execution time because vSphere API + returns the full inventory. But can be used for better readability over + regular expressions. + +- `name_regex` (string) - Extended name filter with regular expressions support + (e.g. `ubuntu[-_]basic[0-9]*`). Default is empty. The match of the + regular expression is checked by substring. Use `^` and `$` to define a + full string. For example, the `^[^_]+$` filter will search names + without any underscores. The expression must use + [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + +- `template` (bool) - Filter to return only objects that are virtual machine templates. + Defaults to `false` and returns all virtual machines. + +- `host` (string) - Filter to search virtual machines only on the specified ESX host. + +- `tags` ([]Tag) - Filter to return only that virtual machines that have attached all + specifies tags. Specify one or more `tags` blocks to define list of tags + for the filter. + +- `latest` (bool) - This filter determines how to handle multiple machines that were + matched with all previous filters. Machine creation time is being used + to find latest. By default, multiple matching machines results in an + error. + + + + +### Tags Filter Configuration + + + +HCL Example: + +```hcl + + tags { + category = "team" + name = "operations" + } + tags { + category = "sla" + name = "gold" + } + +``` + + + + +**Required:** + + + +- `name` (string) - Name of the tag added to virtual machine which must pass the `tags` + filter. + +- `category` (string) - Name of the tag category that contains the tag. + + -> **Note:** Both `name` and `category` must be specified in the `tags` + filter. + + + + +### Connection Configuration + +**Optional:** + + + +- `vcenter_server` (string) - The fully qualified domain name or IP address of the vCenter Server + instance. + +- `username` (string) - The username to authenticate with the vCenter Server instance. + +- `password` (string) - The password to authenticate with the vCenter Server instance. + +- `insecure_connection` (bool) - Do not validate the certificate of the vCenter Server instance. + Defaults to `false`. + + -> **Note:** This option is beneficial in scenarios where the certificate + is self-signed or does not meet standard validation criteria. + +- `datacenter` (string) - The name of the datacenter object in the vSphere inventory. + + -> **Note:** Required if more than one datacenter object exists in the + vSphere inventory. + + + + +## Output + + + +- `vm_name` (string) - Name of the found virtual machine. + + + + +## Example Usage + +This example demonstrates how to connect to vSphere cluster and search for the latest virtual machine +that matches the filters. The name of the machine is then output to the console as an output variable. +```hcl +data "vsphere-virtualmachine" "default" { + vcenter_server = "vcenter.example.com" + insecure_connection = true + username = "administrator@vsphere.local" + password = "VMware1!" + datacenter = "dc-01" + latest = true + tags { + category = "team" + name = "operations" + } + tags { + category = "sla" + name = "gold" + } + +} + +locals { + vm_name = data.vsphere-virtualmachine.default.vm_name +} + +source "null" "example" { + communicator = "none" +} + +build { + sources = [ + "source.null.example" + ] + + provisioner "shell-local" { + inline = [ + "echo vm_name: ${local.vm_name}", + ] + } +} +``` diff --git a/.web-docs/metadata.hcl b/.web-docs/metadata.hcl index 7e7e55368..1a7e5589f 100644 --- a/.web-docs/metadata.hcl +++ b/.web-docs/metadata.hcl @@ -36,6 +36,6 @@ integration { component { type = "data-source" name = "vSphere Virtual Machine" - slug = "vsphere-virtual_machine" + slug = "vsphere-virtualmachine" } } diff --git a/builder/vsphere/driver/vm.go b/builder/vsphere/driver/vm.go index e3a6c413d..20b69751f 100644 --- a/builder/vsphere/driver/vm.go +++ b/builder/vsphere/driver/vm.go @@ -449,7 +449,6 @@ func (vm *VirtualMachineDriver) Clone(ctx context.Context, config *CloneConfig) Device: adapter.(types.BaseVirtualDevice), Operation: types.VirtualDeviceConfigSpecOperationEdit, } - configSpec.DeviceChange = append(configSpec.DeviceChange, config) } diff --git a/datasource/common/driver/driver.go b/datasource/common/driver/driver.go new file mode 100644 index 000000000..f4e342dfd --- /dev/null +++ b/datasource/common/driver/driver.go @@ -0,0 +1,60 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package driver + +import ( + "context" + "fmt" + "net/url" + + "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" + "github.com/vmware/govmomi" + "github.com/vmware/govmomi/find" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/vapi/rest" +) + +type VCenterDriver struct { + Ctx context.Context + Client *govmomi.Client + RestClient *rest.Client + Finder *find.Finder + Datacenter *object.Datacenter +} + +func NewDriver(config common.ConnectConfig) (*VCenterDriver, error) { + ctx := context.Background() + + vcenterUrl, err := url.Parse(fmt.Sprintf("https://%v/sdk", config.VCenterServer)) + if err != nil { + return nil, fmt.Errorf("failed to parse URL: %w", err) + } + vcenterUrl.User = url.UserPassword(config.Username, config.Password) + + client, err := govmomi.NewClient(ctx, vcenterUrl, true) + if err != nil { + return nil, fmt.Errorf("failed to create govmomi Client: %w", err) + } + + restClient := rest.NewClient(client.Client) + err = restClient.Login(ctx, vcenterUrl.User) + if err != nil { + return nil, fmt.Errorf("failed to login to REST API endpoint: %w", err) + } + + finder := find.NewFinder(client.Client, true) + datacenter, err := finder.DatacenterOrDefault(ctx, config.Datacenter) + if err != nil { + return nil, fmt.Errorf("failed to find datacenter: %w", err) + } + finder.SetDatacenter(datacenter) + + return &VCenterDriver{ + Ctx: ctx, + Client: client, + RestClient: restClient, + Finder: finder, + Datacenter: datacenter, + }, nil +} diff --git a/datasource/virtual_machine/testing/call_restapi.go b/datasource/common/testing/call_restapi.go similarity index 56% rename from datasource/virtual_machine/testing/call_restapi.go rename to datasource/common/testing/call_restapi.go index b5ad58002..57d840115 100644 --- a/datasource/virtual_machine/testing/call_restapi.go +++ b/datasource/common/testing/call_restapi.go @@ -1,35 +1,40 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + package testing import ( "context" + "fmt" - "github.com/pkg/errors" "github.com/vmware/govmomi/object" "github.com/vmware/govmomi/vapi/tags" ) +// MarkSimulatedVmAsTemplate powers off the virtual machine before converting it to a template (because the simulator +// creates all virtual machines in an online state). func MarkSimulatedVmAsTemplate(ctx context.Context, vm *object.VirtualMachine) error { task, err := vm.PowerOff(ctx) if err != nil { - return errors.Wrap(err, "failed to issue powering off command to the machine") + return fmt.Errorf("failed to issue powering off command to the machine: %w", err) } err = task.Wait(ctx) if err != nil { - return errors.Wrap(err, "failed to power off the machine") + return fmt.Errorf("failed to power off the machine: %w", err) } err = vm.MarkAsTemplate(ctx) if err != nil { - return errors.Wrap(err, "failed to mark VM as a template") + return fmt.Errorf("failed to mark virtual machine as a template: %w", err) } return nil } -// Try to find category passed by name, create category if not found and return category ID. +// FindOrCreateCategory tries to find category passed by name, creates category if not found and returns category ID. // Category will be created with "MULTIPLE" constraint. func FindOrCreateCategory(ctx context.Context, man *tags.Manager, catName string) (string, error) { categoryList, err := man.GetCategories(ctx) if err != nil { - return "", errors.Wrap(err, "cannot get categories from cluster") + return "", fmt.Errorf("cannot return categories from cluster: %w", err) } for _, category := range categoryList { if category.Name == catName { @@ -38,16 +43,16 @@ func FindOrCreateCategory(ctx context.Context, man *tags.Manager, catName string } newCategoryID, err := man.CreateCategory(ctx, &tags.Category{Name: catName, Cardinality: "MULTIPLE"}) if err != nil { - return "", errors.Wrap(err, "cannot create category") + return "", fmt.Errorf("cannot create category: %w", err) } return newCategoryID, nil } -// Try to find the tagName in category with catID, create if not found and return tag ID. +// FindOrCreateTag tries to find the tagName in category with catID, creates if not found and returns tag ID. func FindOrCreateTag(ctx context.Context, man *tags.Manager, catID string, tagName string) (string, error) { tagsInCategory, err := man.GetTagsForCategory(ctx, catID) if err != nil { - return "", errors.Wrap(err, "cannot get tags for category") + return "", fmt.Errorf("cannot return tags for category: %w", err) } for _, tag := range tagsInCategory { if tag.Name == tagName { @@ -56,7 +61,7 @@ func FindOrCreateTag(ctx context.Context, man *tags.Manager, catID string, tagNa } newTagID, err := man.CreateTag(ctx, &tags.Tag{Name: tagName, CategoryID: catID}) if err != nil { - return "", errors.Wrap(err, "cannot create tag") + return "", fmt.Errorf("cannot create tag: %w", err) } return newTagID, nil } diff --git a/datasource/virtual_machine/testing/simulator.go b/datasource/common/testing/simulator.go similarity index 75% rename from datasource/virtual_machine/testing/simulator.go rename to datasource/common/testing/simulator.go index 988e347dd..ba1b5a906 100644 --- a/datasource/virtual_machine/testing/simulator.go +++ b/datasource/common/testing/simulator.go @@ -1,12 +1,15 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + package testing import ( "context" "crypto/tls" + "fmt" "net/url" "time" - "github.com/pkg/errors" "github.com/vmware/govmomi" "github.com/vmware/govmomi/find" "github.com/vmware/govmomi/object" @@ -43,12 +46,12 @@ type VCenterSimulator struct { func NewVCenterSimulator(model *simulator.Model) (*VCenterSimulator, error) { ctx := context.Background() if model == nil { - return nil, errors.New("model has not been initialized") + return nil, fmt.Errorf("model has not been initialized") } err := model.Create() if err != nil { - return nil, errors.Wrap(err, "failed to create simulator model") + return nil, fmt.Errorf("failed to create simulator model: %w", err) } model.Service.RegisterEndpoints = true model.Service.TLS = new(tls.Config) @@ -57,29 +60,29 @@ func NewVCenterSimulator(model *simulator.Model) (*VCenterSimulator, error) { u, err := url.Parse(server.URL.String()) if err != nil { - return nil, errors.Wrap(err, "failed to parse simulator URL") + return nil, fmt.Errorf("failed to parse simulator URL: %w", err) } password, _ := simulator.DefaultLogin.Password() u.User = url.UserPassword(simulator.DefaultLogin.Username(), password) client, err := govmomi.NewClient(ctx, u, true) if err != nil { - return nil, errors.Wrap(err, "failed to connect to SOAP simulator") + return nil, fmt.Errorf("failed to connect to SOAP simulator: %w", err) } restClient := rest.NewClient(client.Client) err = restClient.Login(ctx, simulator.DefaultLogin) if err != nil { - return nil, errors.Wrap(err, "failed to login to REST simulator") + return nil, fmt.Errorf("failed to login to REST simulator: %w", err) } finder := find.NewFinder(client.Client, false) dcs, err := finder.DatacenterList(ctx, "*") if err != nil { - return nil, errors.Wrap(err, "failed to list datacenters") + return nil, fmt.Errorf("failed to list datacenters: %w", err) } if len(dcs) == 0 { - return nil, errors.Wrap(err, "datacenters were not found in the simulator") + return nil, fmt.Errorf("datacenters were not found in the simulator: %w", err) } finder.SetDatacenter(dcs[0]) @@ -110,7 +113,7 @@ func (sim *VCenterSimulator) CustomizeSimulator(vmsConfig []SimulatedVMConfig) e vms, err := sim.Finder.VirtualMachineList(sim.Ctx, "*") if err != nil { - return errors.Wrap(err, "failed to list VMs in cluster") + return fmt.Errorf("failed to list virtual machines in cluster: %w", err) } for i := 0; i < len(vmsConfig); i++ { @@ -125,17 +128,17 @@ func (sim *VCenterSimulator) CustomizeSimulator(vmsConfig []SimulatedVMConfig) e if vmsConfig[i].Name != "" { task, err := vms[i].Reconfigure(sim.Ctx, vmConfig) if err != nil { - return errors.Wrap(err, "failed to issue rename of VM command") + return fmt.Errorf("failed to issue rename of virtual machine command: %w", err) } if err = task.Wait(sim.Ctx); err != nil { - return errors.Wrap(err, "failed to rename VM") + return fmt.Errorf("failed to rename virtual machine: %w", err) } } if vmsConfig[i].Template { err = MarkSimulatedVmAsTemplate(sim.Ctx, vms[i]) if err != nil { - return errors.Wrap(err, "failed to convert VMs to templates") + return fmt.Errorf("failed to convert to templates: %w", err) } } @@ -143,15 +146,15 @@ func (sim *VCenterSimulator) CustomizeSimulator(vmsConfig []SimulatedVMConfig) e for _, tag := range vmsConfig[i].Tags { catID, err := FindOrCreateCategory(sim.Ctx, tagMan, tag.Category) if err != nil { - return errors.Wrap(err, "failed to find/create category") + return fmt.Errorf("failed to find/create category: %w", err) } tagID, err := FindOrCreateTag(sim.Ctx, tagMan, catID, tag.Name) if err != nil { - return errors.Wrap(err, "failed to find/create tag") + return fmt.Errorf("failed to find/create tag: %w", err) } err = tagMan.AttachTag(sim.Ctx, tagID, vms[i].Reference()) if err != nil { - return errors.Wrap(err, "failed to attach tag to VM") + return fmt.Errorf("failed to attach tag to virtual machine: %w", err) } } } diff --git a/datasource/virtual_machine/driver.go b/datasource/virtual_machine/driver.go deleted file mode 100644 index 82070cb34..000000000 --- a/datasource/virtual_machine/driver.go +++ /dev/null @@ -1,63 +0,0 @@ -package virtual_machine - -import ( - "context" - "fmt" - - "net/url" - - "github.com/pkg/errors" - "github.com/vmware/govmomi" - "github.com/vmware/govmomi/find" - "github.com/vmware/govmomi/object" - "github.com/vmware/govmomi/vapi/rest" -) - -type VCenterDriver struct { - ctx context.Context - client *govmomi.Client - restClient *rest.Client - finder *find.Finder - datacenter *object.Datacenter -} - -func newDriver(config Config) (*VCenterDriver, error) { - ctx := context.Background() - - vcenterUrl, err := url.Parse(fmt.Sprintf("https://%v/sdk", config.VCenterServer)) - if err != nil { - return nil, errors.Wrap(err, "failed to parse URL") - } - vcenterUrl.User = url.UserPassword(config.Username, config.Password) - - client, err := govmomi.NewClient(ctx, vcenterUrl, true) - if err != nil { - return nil, errors.Wrap(err, "failed to create govmomi client") - } - - var restClient *rest.Client - if config.VmTags != nil { - // REST client is only needed when the plugin has to retrieve tags from VMs. - // Skip initialization if not needed (there is additional risk of fail on old vCenter versions). - restClient = rest.NewClient(client.Client) - err = restClient.Login(ctx, vcenterUrl.User) - if err != nil { - return nil, errors.Wrap(err, "failed to login to REST API endpoint") - } - } - - finder := find.NewFinder(client.Client, true) - datacenter, err := finder.DatacenterOrDefault(ctx, config.Datacenter) - if err != nil { - return nil, errors.Wrap(err, "failed to find datacenter") - } - finder.SetDatacenter(datacenter) - - return &VCenterDriver{ - ctx: ctx, - client: client, - restClient: restClient, - finder: finder, - datacenter: datacenter, - }, nil -} diff --git a/datasource/virtual_machine/filters.go b/datasource/virtual_machine/filters.go deleted file mode 100644 index 671fa6bb7..000000000 --- a/datasource/virtual_machine/filters.go +++ /dev/null @@ -1,132 +0,0 @@ -package virtual_machine - -import ( - "regexp" - "time" - - "github.com/pkg/errors" - "github.com/vmware/govmomi/object" - "github.com/vmware/govmomi/property" - "github.com/vmware/govmomi/vapi/tags" - "github.com/vmware/govmomi/vim25/mo" -) - -// Filter machines by matching their names against defined regular expression. -func filterByNameRegex(vmList []*object.VirtualMachine, nameRegex string) []*object.VirtualMachine { - re, _ := regexp.Compile(nameRegex) - result := make([]*object.VirtualMachine, 0) - for _, i := range vmList { - if re.MatchString(i.Name()) { - result = append(result, i) - } - } - return result -} - -// Filter machines by template attribute. Only templates will pass the filter. -func filterByTemplate(driver *VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { - result := make([]*object.VirtualMachine, 0) - for _, i := range vmList { - isTemplate, err := i.IsTemplate(driver.ctx) - if err != nil { - return nil, errors.Wrap(err, "error checking if VM is a tempalte") - } - - if isTemplate { - result = append(result, i) - } - } - return result, nil -} - -// Filter machines by node placement. Only machines that are stored on the defined node will pass the filter. -func filterByNode(driver *VCenterDriver, config Config, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { - pc := property.DefaultCollector(driver.client.Client) - obj, err := driver.finder.HostSystem(driver.ctx, config.Node) - if err != nil { - return nil, errors.Wrap(err, "error finding defined host system") - } - - var host mo.HostSystem - err = pc.RetrieveOne(driver.ctx, obj.Reference(), []string{"vm"}, &host) - if err != nil { - return nil, errors.Wrap(err, "error retrieving properties of host system") - } - - var nodeVms []mo.VirtualMachine - err = pc.Retrieve(driver.ctx, host.Vm, []string{"name"}, &nodeVms) - if err != nil { - return nil, errors.Wrap(err, "failed to get properties for VM") - } - - result := make([]*object.VirtualMachine, 0) - for _, filteredVm := range vmList { - vmName := filteredVm.Name() - for _, nodeVm := range nodeVms { - if vmName == nodeVm.Name { - result = append(result, filteredVm) - } - } - } - - return result, nil -} - -// Filter machines by tags. Tags are stored in the driver as list of flatTag elements. -// Only machines that has all the tags from list will pass the filter. -func filterByTags(driver *VCenterDriver, vmTags []Tag, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { - result := make([]*object.VirtualMachine, 0) - tagMan := tags.NewManager(driver.restClient) - for _, filteredVm := range vmList { - realTagsList, err := tagMan.GetAttachedTags(driver.ctx, filteredVm.Reference()) - if err != nil { - return nil, errors.Wrap(err, "failed to get attached tags for vm") - } - matchedTagsCount := 0 - for _, configTag := range vmTags { - configTagMatched := false - for _, realTag := range realTagsList { - if configTag.Name == realTag.Name { - category, err := tagMan.GetCategory(driver.ctx, realTag.CategoryID) - if err != nil { - return nil, errors.Wrap(err, "failed to get attached category for tag") - } - if configTag.Category == category.Name { - configTagMatched = true - break - } - } - } - if configTagMatched { - matchedTagsCount++ - } else { - // If a single requested tag from config not matched then no need to proceed. - // Fail early. - break - } - } - if matchedTagsCount == len(vmTags) { - result = append(result, filteredVm) - } - } - - return result, nil -} - -func filterByLatest(driver *VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { - var latestVM *object.VirtualMachine - var latestTimestamp time.Time - for _, elementVM := range vmList { - var vmConfig mo.VirtualMachine - err := elementVM.Properties(driver.ctx, elementVM.Reference(), []string{"config"}, &vmConfig) - if err != nil { - return nil, errors.Wrap(err, "error retrieving config properties of VM") - } - if vmConfig.Config.CreateDate.After(latestTimestamp) { - latestVM = elementVM - latestTimestamp = *vmConfig.Config.CreateDate - } - } - result := []*object.VirtualMachine{latestVM} - return result, nil -} diff --git a/datasource/virtual_machine/data.go b/datasource/virtualmachine/data.go similarity index 52% rename from datasource/virtual_machine/data.go rename to datasource/virtualmachine/data.go index 30e457a91..3847b88a3 100644 --- a/datasource/virtual_machine/data.go +++ b/datasource/virtualmachine/data.go @@ -3,65 +3,76 @@ //go:generate packer-sdc struct-markdown //go:generate packer-sdc mapstructure-to-hcl2 -type Config,Tag,DatasourceOutput -package virtual_machine +package virtualmachine import ( + "errors" + "fmt" + "github.com/hashicorp/hcl/v2/hcldec" "github.com/hashicorp/packer-plugin-sdk/common" "github.com/hashicorp/packer-plugin-sdk/hcl2helper" packersdk "github.com/hashicorp/packer-plugin-sdk/packer" "github.com/hashicorp/packer-plugin-sdk/template/config" - vsCommon "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" - "github.com/pkg/errors" + vsphere "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" + "github.com/hashicorp/packer-plugin-vsphere/datasource/common/driver" "github.com/zclconf/go-cty/cty" ) -// Example of multiple vm_tags blocks in HCL format: -// ``` +// HCL Example: +// +// ```hcl // -// vm_tags { +// tags { // category = "team" // name = "operations" // } -// vm_tags { -// category = "SLA" +// tags { +// category = "sla" // name = "gold" // } // // ``` type Tag struct { - // Tag with this name must be attached to virtual machine which should pass the Tags Filter. + // Name of the tag added to virtual machine which must pass the `tags` + // filter. Name string `mapstructure:"name" required:"true"` - // Name of the category that contains this tag. Both tag and category must be specified. + // Name of the tag category that contains the tag. + // + // -> **Note:** Both `name` and `category` must be specified in the `tags` + // filter. Category string `mapstructure:"category" required:"true"` } type Config struct { - common.PackerConfig `mapstructure:",squash"` - vsCommon.ConnectConfig `mapstructure:",squash"` + common.PackerConfig `mapstructure:",squash"` + vsphere.ConnectConfig `mapstructure:",squash"` - // Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. - // Using strict globs will not reduce execution time because vSphere API returns the full inventory. - // But can be used for better readability over regular expressions. + // Basic filter with glob support (e.g. `ubuntu_basic*`). Defaults to `*`. + // Using strict globs will not reduce execution time because vSphere API + // returns the full inventory. But can be used for better readability over + // regular expressions. Name string `mapstructure:"name"` - // Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. - // The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. - // E.g. the `^[^_]+$` filter will search names without any underscores. - // The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + // Extended name filter with regular expressions support + // (e.g. `ubuntu[-_]basic[0-9]*`). Default is empty. The match of the + // regular expression is checked by substring. Use `^` and `$` to define a + // full string. For example, the `^[^_]+$` filter will search names + // without any underscores. The expression must use + // [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). NameRegex string `mapstructure:"name_regex"` // Filter to return only objects that are virtual machine templates. - // Defaults to `false` and returns all VMs. + // Defaults to `false` and returns all virtual machines. Template bool `mapstructure:"template"` - // Filter to search virtual machines only on the specified node. - Node string `mapstructure:"node"` - // Filter to return only that virtual machines that have attached all specifies tags. - // Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. - // Should work since vCenter 6.7. To avoid incompatibility, REST client is being - // initialized only when at least one tag has been defined in the config. - VmTags []Tag `mapstructure:"vm_tags"` - // This filter determines how to handle multiple machines that were matched with all - // previous filters. Machine creation time is being used to find latest. - // By default, multiple matching machines results in an error. + // Filter to search virtual machines only on the specified ESX host. + Host string `mapstructure:"host"` + // Filter to return only that virtual machines that have attached all + // specifies tags. Specify one or more `tags` blocks to define list of tags + // for the filter. + Tags []Tag `mapstructure:"tags"` + // This filter determines how to handle multiple machines that were + // matched with all previous filters. Machine creation time is being used + // to find latest. By default, multiple matching machines results in an + // error. Latest bool `mapstructure:"latest"` } @@ -98,8 +109,8 @@ func (d *Datasource) Configure(raws ...interface{}) error { if d.config.Password == "" { errs = packersdk.MultiErrorAppend(errs, errors.New("'password' is required")) } - if len(d.config.VmTags) > 0 { - for _, tag := range d.config.VmTags { + if len(d.config.Tags) > 0 { + for _, tag := range d.config.Tags { if tag.Name == "" || tag.Category == "" { errs = packersdk.MultiErrorAppend(errs, errors.New("both name and category are required for tag")) } @@ -118,16 +129,16 @@ func (d *Datasource) OutputSpec() hcldec.ObjectSpec { } func (d *Datasource) Execute() (cty.Value, error) { - driver, err := newDriver(d.config) + dr, err := driver.NewDriver(d.config.ConnectConfig) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to initialize driver") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to initialize driver: %w", err) } // This is the first level of filters // (the finder with glob will return filtered list or drop an error if found nothing). - filteredVms, err := driver.finder.VirtualMachineList(driver.ctx, d.config.Name) + filteredVms, err := dr.Finder.VirtualMachineList(dr.Ctx, d.config.Name) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to retrieve virtual machines list") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to retrieve virtual machines list: %w", err) } // Chain of other filters that will be executed only when defined @@ -137,40 +148,40 @@ func (d *Datasource) Execute() (cty.Value, error) { } if len(filteredVms) > 0 && d.config.Template { - filteredVms, err = filterByTemplate(driver, filteredVms) + filteredVms, err = filterByTemplate(dr, filteredVms) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by template attribute") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to filter by template attribute: %w", err) } } - if len(filteredVms) > 0 && d.config.Node != "" { - filteredVms, err = filterByNode(driver, d.config, filteredVms) + if len(filteredVms) > 0 && d.config.Host != "" { + filteredVms, err = filterByHost(dr, d.config, filteredVms) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by node attribute") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to filter by host attribute: %w", err) } } - if len(filteredVms) > 0 && d.config.VmTags != nil { - filteredVms, err = filterByTags(driver, d.config.VmTags, filteredVms) + if len(filteredVms) > 0 && d.config.Tags != nil { + filteredVms, err = filterByTags(dr, d.config.Tags, filteredVms) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to filter by tags") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to filter by tags: %w", err) } } // No VMs passed the filter chain. Nothing to return. if len(filteredVms) == 0 { - return cty.NullVal(cty.EmptyObject), errors.New("not a single VM matches the configured filters") + return cty.NullVal(cty.EmptyObject), errors.New("no virtual machine matches the filters") } if len(filteredVms) > 1 { if d.config.Latest { - filteredVms, err = filterByLatest(driver, filteredVms) + filteredVms, err = filterByLatest(dr, filteredVms) if err != nil { - return cty.NullVal(cty.EmptyObject), errors.Wrap(err, "failed to find the latest VM") + return cty.NullVal(cty.EmptyObject), fmt.Errorf("failed to find the latest virtual machine: %w", err) } } else { // Too many machines passed the filter chain. Cannot decide which machine to return. - return cty.NullVal(cty.EmptyObject), errors.New("multiple VMs match the configured filters") + return cty.NullVal(cty.EmptyObject), errors.New("more than one virtual machine matched the filters") } } diff --git a/datasource/virtual_machine/data.hcl2spec.go b/datasource/virtualmachine/data.hcl2spec.go similarity index 94% rename from datasource/virtual_machine/data.hcl2spec.go rename to datasource/virtualmachine/data.hcl2spec.go index abba4cc6b..9ddbfb915 100644 --- a/datasource/virtual_machine/data.hcl2spec.go +++ b/datasource/virtualmachine/data.hcl2spec.go @@ -1,6 +1,6 @@ // Code generated by "packer-sdc mapstructure-to-hcl2"; DO NOT EDIT. -package virtual_machine +package virtualmachine import ( "github.com/hashicorp/hcl/v2/hcldec" @@ -26,8 +26,8 @@ type FlatConfig struct { Name *string `mapstructure:"name" cty:"name" hcl:"name"` NameRegex *string `mapstructure:"name_regex" cty:"name_regex" hcl:"name_regex"` Template *bool `mapstructure:"template" cty:"template" hcl:"template"` - Node *string `mapstructure:"node" cty:"node" hcl:"node"` - VmTags []FlatTag `mapstructure:"vm_tags" cty:"vm_tags" hcl:"vm_tags"` + Host *string `mapstructure:"host" cty:"host" hcl:"host"` + Tags []FlatTag `mapstructure:"tags" cty:"tags" hcl:"tags"` Latest *bool `mapstructure:"latest" cty:"latest" hcl:"latest"` } @@ -59,8 +59,8 @@ func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec { "name": &hcldec.AttrSpec{Name: "name", Type: cty.String, Required: false}, "name_regex": &hcldec.AttrSpec{Name: "name_regex", Type: cty.String, Required: false}, "template": &hcldec.AttrSpec{Name: "template", Type: cty.Bool, Required: false}, - "node": &hcldec.AttrSpec{Name: "node", Type: cty.String, Required: false}, - "vm_tags": &hcldec.BlockListSpec{TypeName: "vm_tags", Nested: hcldec.ObjectSpec((*FlatTag)(nil).HCL2Spec())}, + "host": &hcldec.AttrSpec{Name: "host", Type: cty.String, Required: false}, + "tags": &hcldec.BlockListSpec{TypeName: "tags", Nested: hcldec.ObjectSpec((*FlatTag)(nil).HCL2Spec())}, "latest": &hcldec.AttrSpec{Name: "latest", Type: cty.Bool, Required: false}, } return s diff --git a/datasource/virtual_machine/data_test.go b/datasource/virtualmachine/data_test.go similarity index 85% rename from datasource/virtual_machine/data_test.go rename to datasource/virtualmachine/data_test.go index 97be44d45..8a5ccad7c 100644 --- a/datasource/virtual_machine/data_test.go +++ b/datasource/virtualmachine/data_test.go @@ -1,20 +1,23 @@ -package virtual_machine +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package virtualmachine import ( "testing" "time" - vsCommon "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" + "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/common" "github.com/vmware/govmomi/simulator" - dsTesting "github.com/hashicorp/packer-plugin-vsphere/datasource/virtual_machine/testing" + commonT "github.com/hashicorp/packer-plugin-vsphere/datasource/common/testing" ) func TestExecute(t *testing.T) { - machinesToPrepare := []dsTesting.SimulatedVMConfig{ + machinesToPrepare := []commonT.SimulatedVMConfig{ { Name: "first-vm", - Tags: []dsTesting.Tag{ + Tags: []commonT.Tag{ { Category: "operating-system-class", Name: "Linux", @@ -22,7 +25,7 @@ func TestExecute(t *testing.T) { }, }, { Name: "second-vm", - Tags: []dsTesting.Tag{ + Tags: []commonT.Tag{ { Category: "operating-system-class", Name: "Linux", @@ -39,7 +42,7 @@ func TestExecute(t *testing.T) { Template: true, }, { Name: "machine-three", - Tags: []dsTesting.Tag{ + Tags: []commonT.Tag{ { Category: "operating-system-class", Name: "Linux", @@ -57,7 +60,7 @@ func TestExecute(t *testing.T) { model.Datacenter = 2 model.Machine = 8 - vcSim, err := dsTesting.NewVCenterSimulator(model) + vcSim, err := commonT.NewVCenterSimulator(model) if err != nil { t.Fatalf("error creating vCenter simulator: %s", err) } @@ -69,7 +72,7 @@ func TestExecute(t *testing.T) { } simulatorPassword, _ := vcSim.Server.URL.User.Password() - connectConfig := vsCommon.ConnectConfig{ + connectConfig := common.ConnectConfig{ VCenterServer: vcSim.Server.URL.Host, Username: vcSim.Server.URL.User.Username(), Password: simulatorPassword, @@ -133,19 +136,19 @@ func TestExecute(t *testing.T) { }, }, { - name: "found multiple machines at the node, error", + name: "found multiple machines at the host, error", expectFailure: true, expectVmName: "", config: Config{ - Node: "DC0_H0", + Host: "DC0_H0", }, }, { - name: "cluster node not found, error", + name: "cluster host not found, error", expectFailure: true, expectVmName: "", config: Config{ - Node: "unexpected_node", + Host: "unexpected_host", }, }, { @@ -153,7 +156,7 @@ func TestExecute(t *testing.T) { expectFailure: false, expectVmName: "second-vm", config: Config{ - VmTags: []Tag{ + Tags: []Tag{ { Category: "security-team", Name: "blue", @@ -170,7 +173,7 @@ func TestExecute(t *testing.T) { expectFailure: true, expectVmName: "", config: Config{ - VmTags: []Tag{ + Tags: []Tag{ { Category: "operating-system-class", Name: "Linux", diff --git a/datasource/virtualmachine/filters.go b/datasource/virtualmachine/filters.go new file mode 100644 index 000000000..ec9c1c655 --- /dev/null +++ b/datasource/virtualmachine/filters.go @@ -0,0 +1,137 @@ +// Copyright (c) HashiCorp, Inc. +// SPDX-License-Identifier: MPL-2.0 + +package virtualmachine + +import ( + "fmt" + "regexp" + "time" + + "github.com/hashicorp/packer-plugin-vsphere/datasource/common/driver" + "github.com/vmware/govmomi/object" + "github.com/vmware/govmomi/property" + "github.com/vmware/govmomi/vapi/tags" + "github.com/vmware/govmomi/vim25/mo" +) + +// filterByNameRegex filters machines by matching their names against defined regular expression. +func filterByNameRegex(vmList []*object.VirtualMachine, nameRegex string) []*object.VirtualMachine { + re, _ := regexp.Compile(nameRegex) + result := make([]*object.VirtualMachine, 0) + for _, i := range vmList { + if re.MatchString(i.Name()) { + result = append(result, i) + } + } + return result +} + +// filterByTemplate filters machines by template attribute. Only templates will pass the filter. +func filterByTemplate(driver *driver.VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + result := make([]*object.VirtualMachine, 0) + for _, i := range vmList { + isTemplate, err := i.IsTemplate(driver.Ctx) + if err != nil { + return nil, fmt.Errorf("error checking if virtual machine is a template: %w", err) + } + + if isTemplate { + result = append(result, i) + } + } + return result, nil +} + +// filterByHost filters machines by ESX host placement. +// Only machines that are stored on the defined host will pass the filter. +func filterByHost(driver *driver.VCenterDriver, config Config, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + pc := property.DefaultCollector(driver.Client.Client) + obj, err := driver.Finder.HostSystem(driver.Ctx, config.Host) + if err != nil { + return nil, fmt.Errorf("error finding defined host system: %w", err) + } + + var host mo.HostSystem + err = pc.RetrieveOne(driver.Ctx, obj.Reference(), []string{"vm"}, &host) + if err != nil { + return nil, fmt.Errorf("error retrieving properties of host system: %w", err) + } + + var hostVms []mo.VirtualMachine + err = pc.Retrieve(driver.Ctx, host.Vm, []string{"name"}, &hostVms) + if err != nil { + return nil, fmt.Errorf("failed to get properties for the virtual machine: %w", err) + } + + result := make([]*object.VirtualMachine, 0) + for _, filteredVm := range vmList { + vmName := filteredVm.Name() + for _, hostVm := range hostVms { + if vmName == hostVm.Name { + result = append(result, filteredVm) + } + } + } + + return result, nil +} + +// filterByTags filters machines by tags. Only machines that has all the tags from list will pass the filter. +func filterByTags(driver *driver.VCenterDriver, vmTags []Tag, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + result := make([]*object.VirtualMachine, 0) + tagMan := tags.NewManager(driver.RestClient) + for _, filteredVm := range vmList { + realTagsList, err := tagMan.GetAttachedTags(driver.Ctx, filteredVm.Reference()) + if err != nil { + return nil, fmt.Errorf("failed return tags for the virtual machine: %w", err) + } + matchedTagsCount := 0 + for _, configTag := range vmTags { + configTagMatched := false + for _, realTag := range realTagsList { + if configTag.Name == realTag.Name { + category, err := tagMan.GetCategory(driver.Ctx, realTag.CategoryID) + if err != nil { + return nil, fmt.Errorf("failed to return tag category for tag: %w", err) + } + if configTag.Category == category.Name { + configTagMatched = true + break + } + } + } + if configTagMatched { + matchedTagsCount++ + } else { + // If a single requested tag from config not matched then no need to proceed. + // Fail early. + break + } + } + if matchedTagsCount == len(vmTags) { + result = append(result, filteredVm) + } + } + + return result, nil +} + +// filterByLatest filters machines by creation date. This filter returns list with one element. +func filterByLatest(driver *driver.VCenterDriver, vmList []*object.VirtualMachine) ([]*object.VirtualMachine, error) { + var latestVM *object.VirtualMachine + var latestTimestamp time.Time + for _, elementVM := range vmList { + var vmConfig mo.VirtualMachine + err := elementVM.Properties(driver.Ctx, elementVM.Reference(), []string{"config"}, &vmConfig) + if err != nil { + return nil, fmt.Errorf("error retrieving config properties for the virtual machine: %w", err) + } + if vmConfig.Config.CreateDate.After(latestTimestamp) { + latestVM = elementVM + latestTimestamp = *vmConfig.Config.CreateDate + } + } + result := []*object.VirtualMachine{latestVM} + return result, nil +} diff --git a/docs-partials/datasource/virtual_machine/Config-not-required.mdx b/docs-partials/datasource/virtual_machine/Config-not-required.mdx deleted file mode 100644 index 7c3973d1f..000000000 --- a/docs-partials/datasource/virtual_machine/Config-not-required.mdx +++ /dev/null @@ -1,26 +0,0 @@ - - -- `name` (string) - Basic filter with glob support (e.g. `nginx_basic*`). Defaults to `*`. - Using strict globs will not reduce execution time because vSphere API returns the full inventory. - But can be used for better readability over regular expressions. - -- `name_regex` (string) - Extended name filter with regular expressions support (e.g. `nginx[-_]basic[0-9]*`). Default is empty. - The match of the regular expression is checked by substring. Use `^` and `$` to define a full string. - E.g. the `^[^_]+$` filter will search names without any underscores. - The expression must use [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). - -- `template` (bool) - Filter to return only objects that are virtual machine templates. - Defaults to `false` and returns all VMs. - -- `node` (string) - Filter to search virtual machines only on the specified node. - -- `vm_tags` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. - Specify one or more `vm_tags` blocks to define list of tags that will make up the filter. - Should work since vCenter 6.7. To avoid incompatibility, REST client is being - initialized only when at least one tag has been defined in the config. - -- `latest` (bool) - This filter determines how to handle multiple machines that were matched with all - previous filters. Machine creation time is being used to find latest. - By default, multiple matching machines results in an error. - - diff --git a/docs-partials/datasource/virtual_machine/Tag-required.mdx b/docs-partials/datasource/virtual_machine/Tag-required.mdx deleted file mode 100644 index 65f82b0e9..000000000 --- a/docs-partials/datasource/virtual_machine/Tag-required.mdx +++ /dev/null @@ -1,7 +0,0 @@ - - -- `name` (string) - Tag with this name must be attached to virtual machine which should pass the Tags Filter. - -- `category` (string) - Name of the category that contains this tag. Both tag and category must be specified. - - diff --git a/docs-partials/datasource/virtualmachine/Config-not-required.mdx b/docs-partials/datasource/virtualmachine/Config-not-required.mdx new file mode 100644 index 000000000..7c27292f3 --- /dev/null +++ b/docs-partials/datasource/virtualmachine/Config-not-required.mdx @@ -0,0 +1,29 @@ + + +- `name` (string) - Basic filter with glob support (e.g. `ubuntu_basic*`). Defaults to `*`. + Using strict globs will not reduce execution time because vSphere API + returns the full inventory. But can be used for better readability over + regular expressions. + +- `name_regex` (string) - Extended name filter with regular expressions support + (e.g. `ubuntu[-_]basic[0-9]*`). Default is empty. The match of the + regular expression is checked by substring. Use `^` and `$` to define a + full string. For example, the `^[^_]+$` filter will search names + without any underscores. The expression must use + [Go Regex Syntax](https://pkg.go.dev/regexp/syntax). + +- `template` (bool) - Filter to return only objects that are virtual machine templates. + Defaults to `false` and returns all virtual machines. + +- `host` (string) - Filter to search virtual machines only on the specified ESX host. + +- `tags` ([]Tag) - Filter to return only that virtual machines that have attached all + specifies tags. Specify one or more `tags` blocks to define list of tags + for the filter. + +- `latest` (bool) - This filter determines how to handle multiple machines that were + matched with all previous filters. Machine creation time is being used + to find latest. By default, multiple matching machines results in an + error. + + diff --git a/docs-partials/datasource/virtual_machine/DatasourceOutput.mdx b/docs-partials/datasource/virtualmachine/DatasourceOutput.mdx similarity index 60% rename from docs-partials/datasource/virtual_machine/DatasourceOutput.mdx rename to docs-partials/datasource/virtualmachine/DatasourceOutput.mdx index 30c8a0950..ff3002449 100644 --- a/docs-partials/datasource/virtual_machine/DatasourceOutput.mdx +++ b/docs-partials/datasource/virtualmachine/DatasourceOutput.mdx @@ -1,5 +1,5 @@ - + - `vm_name` (string) - Name of the found virtual machine. - + diff --git a/docs-partials/datasource/virtualmachine/Tag-required.mdx b/docs-partials/datasource/virtualmachine/Tag-required.mdx new file mode 100644 index 000000000..551e1fa24 --- /dev/null +++ b/docs-partials/datasource/virtualmachine/Tag-required.mdx @@ -0,0 +1,11 @@ + + +- `name` (string) - Name of the tag added to virtual machine which must pass the `tags` + filter. + +- `category` (string) - Name of the tag category that contains the tag. + + -> **Note:** Both `name` and `category` must be specified in the `tags` + filter. + + diff --git a/docs-partials/datasource/virtual_machine/Tag.mdx b/docs-partials/datasource/virtualmachine/Tag.mdx similarity index 51% rename from docs-partials/datasource/virtual_machine/Tag.mdx rename to docs-partials/datasource/virtualmachine/Tag.mdx index cfe63d631..fd7a77560 100644 --- a/docs-partials/datasource/virtual_machine/Tag.mdx +++ b/docs-partials/datasource/virtualmachine/Tag.mdx @@ -1,17 +1,18 @@ - + -Example of multiple vm_tags blocks in HCL format: -``` +HCL Example: + +```hcl - vm_tags { + tags { category = "team" name = "operations" } - vm_tags { - category = "SLA" + tags { + category = "sla" name = "gold" } ``` - + diff --git a/docs/README.md b/docs/README.md index d6da08e1e..20127e6fe 100644 --- a/docs/README.md +++ b/docs/README.md @@ -51,9 +51,8 @@ packer plugins install github.com/hashicorp/vsphere #### Data Sources -- [vsphere-virtual_machine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtual_machine) - - This datasource returns name of existing virtual machine that matches all defined filters to use - it as a builder source for `vsphere-clone`. +- [vsphere-virtualmachine](/packer/integrations/hashicorp/vsphere/latest/components/data-source/vsphere-virtualmachine) - + This data source returns the name of a virtual machine that matches all defined filters. #### Post-Processors diff --git a/docs/datasources/virtual_machine.mdx b/docs/datasources/virtual_machine.mdx deleted file mode 100644 index d493c4dfd..000000000 --- a/docs/datasources/virtual_machine.mdx +++ /dev/null @@ -1,92 +0,0 @@ ---- -modeline: | - vim: set ft=pandoc: -description: | - This datasource is able to get information about existing virtual machines from vSphere - and return name of one virtual machine that matches all specified filters. This virtual - machine can later be used in the vSphere Clone builder to select template. -page_title: vSphere VM - Datasources -sidebar_title: Virtual Machine ---- - -# VMware vSphere Virtual Machine Datasource - -Type: `vsphere-virtual_machine` -Artifact BuilderId: `vsphere.virtual_machine` - -This datasource is able to get information about existing virtual machines from vSphere -and return name of one virtual machine that matches all specified filters. This virtual -machine can later be used in the vSphere Clone builder to select template. - -## Configuration Reference - -### Filters Configuration - -**Optional:** - -@include 'datasource/virtual_machine/Config-not-required.mdx' - -### Tags Filter Configuration - -@include 'datasource/virtual_machine/Tag.mdx' - -**Required:** - -@include 'datasource/virtual_machine/Tag-required.mdx' - -### Connection Configuration - -**Optional:** - -@include 'builder/vsphere/common/ConnectConfig-not-required.mdx' - -## Output - -@include 'datasource/virtual_machine/DatasourceOutput.mdx' - -## Example Usage - -This is a very basic example that connects to vSphere cluster and tries to search -the latest virtual machine that matches all filters. The machine name is then printed -to console as output variable. -```hcl -data "vsphere-virtual_machine" "default" { - vcenter_server = "vcenter.example.org" - insecure_connection = true - username = "administrator@example.org" - password = "St4ongPa$$w0rd" - datacenter = "AZ1" - latest = true - vm_tags { - category = "team" - name = "operations" - } - vm_tags { - category = "SLA" - name = "gold" - } - -} - -locals { - vm_name = data.vsphere-virtual_machine.default.vm_name -} - -source "null" "basic-example" { - communicator = "none" -} - -build { - sources = [ - "source.null.basic-example" - ] - - provisioner "shell-local" { - inline = [ - "echo vm_name: ${local.vm_name}", - ] - } -} - - -``` diff --git a/docs/datasources/virtualmachine.mdx b/docs/datasources/virtualmachine.mdx new file mode 100644 index 000000000..b616f7a4e --- /dev/null +++ b/docs/datasources/virtualmachine.mdx @@ -0,0 +1,89 @@ +--- +modeline: | + vim: set ft=pandoc: +description: | + This data source retrieves information about existing virtual machines from vSphere + and return name of one virtual machine that matches all specified filters. This virtual + machine can be used in the vSphere Clone builder to select a template. +page_title: vSphere Virtual Machine - Data Source +sidebar_title: vSphere Virtual Machine +--- + +# Virtual Machine Data Source + +Type: `vsphere-virtualmachine` +Artifact BuilderId: `vsphere.virtualmachine` + +This data source retrieves information about existing virtual machines from vSphere +and return name of one virtual machine that matches all specified filters. This virtual +machine can be used in the vSphere Clone builder to select a template. + +## Configuration Reference + +### Filters Configuration + +**Optional:** + +@include 'datasource/virtualmachine/Config-not-required.mdx' + +### Tags Filter Configuration + +@include 'datasource/virtualmachine/Tag.mdx' + +**Required:** + +@include 'datasource/virtualmachine/Tag-required.mdx' + +### Connection Configuration + +**Optional:** + +@include 'builder/vsphere/common/ConnectConfig-not-required.mdx' + +## Output + +@include 'datasource/virtualmachine/DatasourceOutput.mdx' + +## Example Usage + +This example demonstrates how to connect to vSphere cluster and search for the latest virtual machine +that matches the filters. The name of the machine is then output to the console as an output variable. +```hcl +data "vsphere-virtualmachine" "default" { + vcenter_server = "vcenter.example.com" + insecure_connection = true + username = "administrator@vsphere.local" + password = "VMware1!" + datacenter = "dc-01" + latest = true + tags { + category = "team" + name = "operations" + } + tags { + category = "sla" + name = "gold" + } + +} + +locals { + vm_name = data.vsphere-virtualmachine.default.vm_name +} + +source "null" "example" { + communicator = "none" +} + +build { + sources = [ + "source.null.example" + ] + + provisioner "shell-local" { + inline = [ + "echo vm_name: ${local.vm_name}", + ] + } +} +``` diff --git a/main.go b/main.go index 121f768ec..ddfbe2d68 100644 --- a/main.go +++ b/main.go @@ -12,7 +12,7 @@ import ( "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/clone" "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/iso" "github.com/hashicorp/packer-plugin-vsphere/builder/vsphere/supervisor" - "github.com/hashicorp/packer-plugin-vsphere/datasource/virtual_machine" + "github.com/hashicorp/packer-plugin-vsphere/datasource/virtualmachine" "github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere" vsphereTemplate "github.com/hashicorp/packer-plugin-vsphere/post-processor/vsphere-template" "github.com/hashicorp/packer-plugin-vsphere/version" @@ -23,7 +23,7 @@ func main() { pps.RegisterBuilder("iso", new(iso.Builder)) pps.RegisterBuilder("clone", new(clone.Builder)) pps.RegisterBuilder("supervisor", new(supervisor.Builder)) - pps.RegisterDatasource("virtual_machine", new(virtual_machine.Datasource)) + pps.RegisterDatasource("virtualmachine", new(virtualmachine.Datasource)) pps.RegisterPostProcessor(plugin.DEFAULT_NAME, new(vsphere.PostProcessor)) pps.RegisterPostProcessor("template", new(vsphereTemplate.PostProcessor)) pps.SetVersion(version.PluginVersion) From 6d299a9ef52a9343384370d4cd3584b3320ba810 Mon Sep 17 00:00:00 2001 From: Castor Sky Date: Mon, 9 Dec 2024 22:52:54 +0300 Subject: [PATCH 3/4] review: renamed `tags` config field to `tag` --- .../data-source/virtualmachine/README.md | 14 +++++++------- datasource/virtualmachine/data.go | 14 +++++++------- datasource/virtualmachine/data.hcl2spec.go | 4 ++-- .../virtualmachine/Config-not-required.mdx | 6 +++--- .../datasource/virtualmachine/Tag-required.mdx | 4 ++-- docs-partials/datasource/virtualmachine/Tag.mdx | 4 ++-- 6 files changed, 23 insertions(+), 23 deletions(-) diff --git a/.web-docs/components/data-source/virtualmachine/README.md b/.web-docs/components/data-source/virtualmachine/README.md index c976eea10..199cdf5be 100644 --- a/.web-docs/components/data-source/virtualmachine/README.md +++ b/.web-docs/components/data-source/virtualmachine/README.md @@ -30,9 +30,9 @@ machine can be used in the vSphere Clone builder to select a template. - `host` (string) - Filter to search virtual machines only on the specified ESX host. -- `tags` ([]Tag) - Filter to return only that virtual machines that have attached all - specifies tags. Specify one or more `tags` blocks to define list of tags - for the filter. +- `tag` ([]Tag) - Filter to return only that virtual machines that have attached all + specifies tags. Specify one or more `tag` blocks to define list of tags + for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). - `latest` (bool) - This filter determines how to handle multiple machines that were matched with all previous filters. Machine creation time is being used @@ -50,11 +50,11 @@ HCL Example: ```hcl - tags { + tag { category = "team" name = "operations" } - tags { + tag { category = "sla" name = "gold" } @@ -68,12 +68,12 @@ HCL Example: -- `name` (string) - Name of the tag added to virtual machine which must pass the `tags` +- `name` (string) - Name of the tag added to virtual machine which must pass the `tag` filter. - `category` (string) - Name of the tag category that contains the tag. - -> **Note:** Both `name` and `category` must be specified in the `tags` + -> **Note:** Both `name` and `category` must be specified in the `tag` filter. diff --git a/datasource/virtualmachine/data.go b/datasource/virtualmachine/data.go index 3847b88a3..a280518b7 100644 --- a/datasource/virtualmachine/data.go +++ b/datasource/virtualmachine/data.go @@ -23,23 +23,23 @@ import ( // // ```hcl // -// tags { +// tag { // category = "team" // name = "operations" // } -// tags { +// tag { // category = "sla" // name = "gold" // } // // ``` type Tag struct { - // Name of the tag added to virtual machine which must pass the `tags` + // Name of the tag added to virtual machine which must pass the `tag` // filter. Name string `mapstructure:"name" required:"true"` // Name of the tag category that contains the tag. // - // -> **Note:** Both `name` and `category` must be specified in the `tags` + // -> **Note:** Both `name` and `category` must be specified in the `tag` // filter. Category string `mapstructure:"category" required:"true"` } @@ -66,9 +66,9 @@ type Config struct { // Filter to search virtual machines only on the specified ESX host. Host string `mapstructure:"host"` // Filter to return only that virtual machines that have attached all - // specifies tags. Specify one or more `tags` blocks to define list of tags - // for the filter. - Tags []Tag `mapstructure:"tags"` + // specifies tags. Specify one or more `tag` blocks to define list of tags + // for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). + Tags []Tag `mapstructure:"tag"` // This filter determines how to handle multiple machines that were // matched with all previous filters. Machine creation time is being used // to find latest. By default, multiple matching machines results in an diff --git a/datasource/virtualmachine/data.hcl2spec.go b/datasource/virtualmachine/data.hcl2spec.go index 9ddbfb915..31aa2da37 100644 --- a/datasource/virtualmachine/data.hcl2spec.go +++ b/datasource/virtualmachine/data.hcl2spec.go @@ -27,7 +27,7 @@ type FlatConfig struct { NameRegex *string `mapstructure:"name_regex" cty:"name_regex" hcl:"name_regex"` Template *bool `mapstructure:"template" cty:"template" hcl:"template"` Host *string `mapstructure:"host" cty:"host" hcl:"host"` - Tags []FlatTag `mapstructure:"tags" cty:"tags" hcl:"tags"` + Tags []FlatTag `mapstructure:"tag" cty:"tag" hcl:"tag"` Latest *bool `mapstructure:"latest" cty:"latest" hcl:"latest"` } @@ -60,7 +60,7 @@ func (*FlatConfig) HCL2Spec() map[string]hcldec.Spec { "name_regex": &hcldec.AttrSpec{Name: "name_regex", Type: cty.String, Required: false}, "template": &hcldec.AttrSpec{Name: "template", Type: cty.Bool, Required: false}, "host": &hcldec.AttrSpec{Name: "host", Type: cty.String, Required: false}, - "tags": &hcldec.BlockListSpec{TypeName: "tags", Nested: hcldec.ObjectSpec((*FlatTag)(nil).HCL2Spec())}, + "tag": &hcldec.BlockListSpec{TypeName: "tag", Nested: hcldec.ObjectSpec((*FlatTag)(nil).HCL2Spec())}, "latest": &hcldec.AttrSpec{Name: "latest", Type: cty.Bool, Required: false}, } return s diff --git a/docs-partials/datasource/virtualmachine/Config-not-required.mdx b/docs-partials/datasource/virtualmachine/Config-not-required.mdx index 7c27292f3..193b6eae5 100644 --- a/docs-partials/datasource/virtualmachine/Config-not-required.mdx +++ b/docs-partials/datasource/virtualmachine/Config-not-required.mdx @@ -17,9 +17,9 @@ - `host` (string) - Filter to search virtual machines only on the specified ESX host. -- `tags` ([]Tag) - Filter to return only that virtual machines that have attached all - specifies tags. Specify one or more `tags` blocks to define list of tags - for the filter. +- `tag` ([]Tag) - Filter to return only that virtual machines that have attached all + specifies tags. Specify one or more `tag` blocks to define list of tags + for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). - `latest` (bool) - This filter determines how to handle multiple machines that were matched with all previous filters. Machine creation time is being used diff --git a/docs-partials/datasource/virtualmachine/Tag-required.mdx b/docs-partials/datasource/virtualmachine/Tag-required.mdx index 551e1fa24..0b9d544d1 100644 --- a/docs-partials/datasource/virtualmachine/Tag-required.mdx +++ b/docs-partials/datasource/virtualmachine/Tag-required.mdx @@ -1,11 +1,11 @@ -- `name` (string) - Name of the tag added to virtual machine which must pass the `tags` +- `name` (string) - Name of the tag added to virtual machine which must pass the `tag` filter. - `category` (string) - Name of the tag category that contains the tag. - -> **Note:** Both `name` and `category` must be specified in the `tags` + -> **Note:** Both `name` and `category` must be specified in the `tag` filter. diff --git a/docs-partials/datasource/virtualmachine/Tag.mdx b/docs-partials/datasource/virtualmachine/Tag.mdx index fd7a77560..df29348e3 100644 --- a/docs-partials/datasource/virtualmachine/Tag.mdx +++ b/docs-partials/datasource/virtualmachine/Tag.mdx @@ -4,11 +4,11 @@ HCL Example: ```hcl - tags { + tag { category = "team" name = "operations" } - tags { + tag { category = "sla" name = "gold" } From a6e04cdcc063cb63c8fd12b88c4e16ea9392c5b8 Mon Sep 17 00:00:00 2001 From: Castor Sky Date: Tue, 10 Dec 2024 04:55:37 +0300 Subject: [PATCH 4/4] review: reformat documentation And drop info about dynamic blocks in datasource. --- .../data-source/virtualmachine/README.md | 35 ++++++++----------- datasource/virtualmachine/data.go | 29 ++++++++------- .../virtualmachine/Config-not-required.mdx | 15 +++++++- .../datasource/virtualmachine/Tag.mdx | 18 ---------- docs/datasources/virtualmachine.mdx | 2 -- 5 files changed, 42 insertions(+), 57 deletions(-) delete mode 100644 docs-partials/datasource/virtualmachine/Tag.mdx diff --git a/.web-docs/components/data-source/virtualmachine/README.md b/.web-docs/components/data-source/virtualmachine/README.md index 199cdf5be..d877bd1d5 100644 --- a/.web-docs/components/data-source/virtualmachine/README.md +++ b/.web-docs/components/data-source/virtualmachine/README.md @@ -32,7 +32,20 @@ machine can be used in the vSphere Clone builder to select a template. - `tag` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. Specify one or more `tag` blocks to define list of tags - for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). + for the filter. + + HCL Example: + + ```hcl + tag { + category = "team" + name = "operations" + } + tag { + category = "sla" + name = "gold" + } + ``` - `latest` (bool) - This filter determines how to handle multiple machines that were matched with all previous filters. Machine creation time is being used @@ -44,26 +57,6 @@ machine can be used in the vSphere Clone builder to select a template. ### Tags Filter Configuration - - -HCL Example: - -```hcl - - tag { - category = "team" - name = "operations" - } - tag { - category = "sla" - name = "gold" - } - -``` - - - - **Required:** diff --git a/datasource/virtualmachine/data.go b/datasource/virtualmachine/data.go index a280518b7..adb9839bc 100644 --- a/datasource/virtualmachine/data.go +++ b/datasource/virtualmachine/data.go @@ -19,20 +19,6 @@ import ( "github.com/zclconf/go-cty/cty" ) -// HCL Example: -// -// ```hcl -// -// tag { -// category = "team" -// name = "operations" -// } -// tag { -// category = "sla" -// name = "gold" -// } -// -// ``` type Tag struct { // Name of the tag added to virtual machine which must pass the `tag` // filter. @@ -67,7 +53,20 @@ type Config struct { Host string `mapstructure:"host"` // Filter to return only that virtual machines that have attached all // specifies tags. Specify one or more `tag` blocks to define list of tags - // for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). + // for the filter. + // + // HCL Example: + // + // ```hcl + // tag { + // category = "team" + // name = "operations" + // } + // tag { + // category = "sla" + // name = "gold" + // } + // ``` Tags []Tag `mapstructure:"tag"` // This filter determines how to handle multiple machines that were // matched with all previous filters. Machine creation time is being used diff --git a/docs-partials/datasource/virtualmachine/Config-not-required.mdx b/docs-partials/datasource/virtualmachine/Config-not-required.mdx index 193b6eae5..03779212d 100644 --- a/docs-partials/datasource/virtualmachine/Config-not-required.mdx +++ b/docs-partials/datasource/virtualmachine/Config-not-required.mdx @@ -19,7 +19,20 @@ - `tag` ([]Tag) - Filter to return only that virtual machines that have attached all specifies tags. Specify one or more `tag` blocks to define list of tags - for the filter. Multiple blocks can be created programmatically in HCL2 with the [`dynamic_block`](/packer/docs/templates/hcl_templates/expressions#dynamic-blocks). + for the filter. + + HCL Example: + + ```hcl + tag { + category = "team" + name = "operations" + } + tag { + category = "sla" + name = "gold" + } + ``` - `latest` (bool) - This filter determines how to handle multiple machines that were matched with all previous filters. Machine creation time is being used diff --git a/docs-partials/datasource/virtualmachine/Tag.mdx b/docs-partials/datasource/virtualmachine/Tag.mdx deleted file mode 100644 index df29348e3..000000000 --- a/docs-partials/datasource/virtualmachine/Tag.mdx +++ /dev/null @@ -1,18 +0,0 @@ - - -HCL Example: - -```hcl - - tag { - category = "team" - name = "operations" - } - tag { - category = "sla" - name = "gold" - } - -``` - - diff --git a/docs/datasources/virtualmachine.mdx b/docs/datasources/virtualmachine.mdx index b616f7a4e..22e9ff4ed 100644 --- a/docs/datasources/virtualmachine.mdx +++ b/docs/datasources/virtualmachine.mdx @@ -28,8 +28,6 @@ machine can be used in the vSphere Clone builder to select a template. ### Tags Filter Configuration -@include 'datasource/virtualmachine/Tag.mdx' - **Required:** @include 'datasource/virtualmachine/Tag-required.mdx'