From b20d94c88b7fc3abb4f3426a320c98966c4c6201 Mon Sep 17 00:00:00 2001 From: Lucas Bajolet Date: Mon, 27 Nov 2023 16:55:43 -0500 Subject: [PATCH 1/3] docs: reorg plugins installation tabs In order to better document packer plugins installation methods, and since `packer plugins install' is not really manual, we split in two sections the "manual installation". As this is legacy, we stop documenting how to install single-component plugins, and reword the documentation for manually installing a plugin. --- .../content/docs/plugins/install-plugins.mdx | 80 +++++-------------- 1 file changed, 19 insertions(+), 61 deletions(-) diff --git a/website/content/docs/plugins/install-plugins.mdx b/website/content/docs/plugins/install-plugins.mdx index 0460e6c561c..c885d53665c 100644 --- a/website/content/docs/plugins/install-plugins.mdx +++ b/website/content/docs/plugins/install-plugins.mdx @@ -19,12 +19,8 @@ This page explains how to install custom external plugins. Refer to [External Pl ## Installation Guides -Choose the tab that corresponds to the type of plugin you want to install. If you are not sure, check the plugin's name. -- Multi-component plugin names have the prefix `packer-plugin-`. -- Single-component plugin names have a prefix containing the component type, like `packer-provisioner-` or `packer-builder`. - - + -> **Note:** Only _multi-component plugin binaries_ -- plugins named packer-plugin-\*, like the `packer-plugin-amazon` -- are expected to work with @@ -198,72 +194,34 @@ Packer will error if you set the `packer-plugin-` prefix in a `source`. This will avoid conflicting with other plugins for other tools, like Terraform. - - --> The [`packer plugins`](/packer/docs/commands/plugins), available from Packer v1.8.0, command allows -you to install plugins without going through `init`. + -```shell -packer plugins install github.com/hashicorp/vagrant -``` - -## Plugin Installation Workflow Plugin installation via `packer plugins install` works similar to that of the `packer init` command, with the following exceptions no `required_plugins` block required and can be used with both legacy JSON and HCL2 templates. -* [`packer plugins install`](/packer/docs/commands/plugins) will install plugins in the **last** directory -in the following numbered list. - -1. `PACKER_PLUGINS_PATH` if set will be the sole location for installing plugins. All other -plugin directories will be ignored. -1. `PACKER_CONFIG_DIR`\plugins on Windows systems, or `PACKER_CONFIG_DIR`/plugins on all other systems. - - -For manual installation of plugin binaries, without the `packer plugins` command, please continue reading. +-> The [`packer plugins`](/packer/docs/commands/plugins), available from Packer v1.8.0, command allows +you to install plugins without going through `init`. -The easiest way to manually install a plugin is to name it correctly, then place -it in the proper directory. To name a plugin correctly, make sure the binary is -named `packer-plugin-NAME`. For example, `packer-plugin-amazon` for a "plugin" -binary named "amazon". This binary will make one or more components available to -use. Valid types for plugins are down this page. - -The valid types for plugins are: - -- `plugin` - A plugin binary that can contain one or more of each Packer component - type. - -- `builder` - Plugins responsible for building images for a specific - platform. - -- `post-processor` - A post-processor responsible for taking an artifact from - a builder and turning it into something else. - -- `provisioner` - A provisioner to install software on images created by a - builder. +```shell +packer plugins install github.com/hashicorp/vagrant +``` +The command will install the plugin in the `PACKER_CONFIG_DIR` set, or its +default location, which depends on the OS/environment, as documented in +[Configuring Packer](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory). - - -The easiest way to manually install a plugin is to name it correctly, then place -it in the proper directory. To name a plugin correctly, make sure the binary is -named `packer-COMPONENT-NAME`. For example, `packer-provisioner-comment` for a "plugin" -binary named "comment". This binary will make a single provisioner named `comment` available to -use. Valid types for plugins are down this page. - -The valid types for plugins are: - -- `plugin` - A plugin binary that can contain one or more of each Packer component - type. - -- `builder` - Plugins responsible for building images for a specific - platform. + -- `post-processor` - A post-processor responsible for taking an artifact from - a builder and turning it into something else. +If you have obtained or built a plugin binary for your OS/Architecture and want to +use it with Packer, you can install it manually. For Packer to load the plugin, +it must be named with the convention `packer-plugin-NAME`, and placed in Packer's plugin +directory, as documented in +[Configuring Packer](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory). -- `provisioner` - A provisioner to install software on images created by a - builder. +For example, if your configuration directory is located in `~/.config/packer`, +you can copy the binary to `~/.config/plugins/packer-plugin-NAME`, and +Packer will be able to load it afterwards. From facc138088b4af9dcc2ea70f5d5fa1159a7a62f7 Mon Sep 17 00:00:00 2001 From: Lucas Bajolet Date: Tue, 28 Nov 2023 10:49:17 -0500 Subject: [PATCH 2/3] docs: rm implicit github urls from install guide This section is redundant with the explanations given above, and can therefore be removed from the docs. --- .../content/docs/plugins/install-plugins.mdx | 24 ++----------------- 1 file changed, 2 insertions(+), 22 deletions(-) diff --git a/website/content/docs/plugins/install-plugins.mdx b/website/content/docs/plugins/install-plugins.mdx index c885d53665c..fad795bbb59 100644 --- a/website/content/docs/plugins/install-plugins.mdx +++ b/website/content/docs/plugins/install-plugins.mdx @@ -149,8 +149,8 @@ various outputs, like error messages. * [`packer init`](/packer/docs/commands/init) will install plugins in the **last** directory in the following numbered list. -1. `PACKER_PLUGIN_PATH` if set will be the sole location for installing plugins. All other -plugin directories will be ignored. +1. `PACKER_PLUGIN_PATH` if set will be the sole location for installing plugins. All other +plugin directories will be ignored. 1. `PACKER_CONFIG_DIR`\plugins on Windows systems, or `PACKER_CONFIG_DIR`/plugins on all other systems. * During the initialization of Packer, any plugin required in the @@ -173,26 +173,6 @@ The SHA256SUM file will be automatically generated when a plugin is installed vi was installed manually into `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/` then the file `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64_SHA256SUM` must be generated manually as well. -## Implicit Github urls - -Using the following example : - -```hcl - required_plugins { - happycloud = { - version = ">= 2.7.0" - source = "github.com/azr/happycloud" - } - } -``` - -The plugin getter will look for plugins located at: - -- github.com/azr/packer-plugin-happycloud - -Packer will error if you set the `packer-plugin-` prefix in a `source`. This -will avoid conflicting with other plugins for other tools, like Terraform. - From e5bc3dd1c3bb0af5e05f96cefbf1dd89b257c900 Mon Sep 17 00:00:00 2001 From: Lucas Bajolet Date: Wed, 29 Nov 2023 09:55:58 -0500 Subject: [PATCH 3/3] docs: reorganise plugin installation docs The current documentation about installing plugins does not explain (outside of the `packer init' section) how Packer discovers plugins, what the expected file system hierarchy should be, and the quirk of how this takes precedence over the rest when `required_plugins' is specified. This commit addresses that by reorganising the page to highlight general usage questions on sources and plugins, and simplifies the tabs below to only highlight installation methods. --- .../content/docs/plugins/install-plugins.mdx | 202 ++++++++++-------- 1 file changed, 113 insertions(+), 89 deletions(-) diff --git a/website/content/docs/plugins/install-plugins.mdx b/website/content/docs/plugins/install-plugins.mdx index fad795bbb59..1c2b602823f 100644 --- a/website/content/docs/plugins/install-plugins.mdx +++ b/website/content/docs/plugins/install-plugins.mdx @@ -8,34 +8,113 @@ page_title: Install Plugins Packer plugins are separate, standalone applications that perform tasks during each build. -You do not need to install the builder, provisioner, or -post-processor components that ship with the Packer binary. Packer automatically knows how to find and launch these built-in plugins. +You do not need to install the components that ship with the Packer binary. +Packer automatically knows how to find and launch these built-in plugins. This page explains how to install custom external plugins. Refer to [External Plugins](/packer/plugins) for a list of available plugins and their documentation. -## Plugin Loading Order +Depending on the template type you're using (HCL2 or legacy JSON), the methods for installing plugins may differ. -@include "plugins/plugin-location.mdx" +If you're using HCL2, `packer init` is recommended as you can install all your requirements with one +command, and those requirements are explicitly documented in the template. + +`packer plugins install` is also used to automate the installation from a source, and will need to +be repeated for as many plugins as you need. +We recommend this for JSON as the template cannot contain the information about the required plugins. + +Finally, you can manually install any plugin. This is mostly useful if you're in an environment with +restricted internet access, or if you're installing non-final versions of plugins. + +Refer to the [Installation Guides](#installation-guides) section of this page for information about +each, including usage examples. + +The remainder of this document will serve as documentation on how Packer interacts with plugins. +We encourage you to read this to get familiar with this process, as it will help you troubleshoot +your builds if you encounter problems with that. + +## Source Addresses + +A plugin's source address is its global identifier. It also tells Packer where +to download it. + +Source addresses consist of three parts delimited by slashes (`/`), as +follows: + +`//` + +- **Hostname:** The hostname of the location/service that + distributes the plugin. Currently, the only valid "hostname" is github.com, + but we plan to eventually support plugins downloaded from other domains. + +- **Namespace:** An organizational namespace within the specified host. + This often is the organization that publishes the plugin. + +- **Type:** A short name for the platform or system the plugin manages. The + type is usually the plugin's preferred local name. + +For example, the fictional `myawesomecloud` plugin could belong to the +`hashicorp` namespace on `github.com`, so its `source` could be +`github.com/hashicorp/myawesomecloud`, + +-> Note: the actual _repository_ that myawesomecloud comes from must always have +the name format `github.com/hashicorp/packer-plugin-myawesomecloud`, but the +`required_plugins` block omits the redundant `packer-plugin-` repository prefix +for brevity. + +The source address with all three components given explicitly is called the +plugin's _fully-qualified address_. You will see fully-qualified address in +various outputs, like error messages. + +## Plugin Loading Workflow + +At initialization, Packer attempts to discover the plugins installed locally. The +logic follows what's described in Configuring Packer's +[plugin directory](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory) +section. + +While Packer is not verbose during this step, you can peek into what it is discovering +with `PACKER_LOG=1` enabled, where you can find log lines similar to the following: + +```shell +[TRACE] discovering plugins in [...] +[INFO] Discovered potential plugin: [...] +``` + +This logic however is ignored when plugins are defined in `required_plugins` blocks; +instead, for every plugin required in this way, Packer will only consider them if they're +installed in Packer's plugin directory, under a directory hierarchy that matches the +source, with the plugin name respecting a convention. + +For example, if we install the `github.com/hashicorp/amazon` plugin in version 1.2.8 through +either `packer init` or `packer plugins install`, this will yield the following (in a +Linux x86_64 environment): + +```shell + +└── github.com + └── hashicorp + └── amazon + ├── packer-plugin-amazon_v1.2.8_x5.0_linux_amd64 + └── packer-plugin-amazon_v1.2.8_x5.0_linux_amd64_SHA256SUM +``` + +Both the plugin's binary, and the related SHA256SUM file must be placed alongside +each other for Packer to consider them for a `required_plugins` constraint. ## Installation Guides --> **Note:** Only _multi-component plugin binaries_ -- plugins named -packer-plugin-\*, like the `packer-plugin-amazon` -- are expected to work with -Packer init. The legacy `builder`, `post-processor` and `provisioner` plugin -types will continue to be detected but Packer cannot install them automatically. -If a plugin you use has not been upgraded to use the multi-component plugin -architecture, contact your maintainer to request an upgrade. +In order to use `packer init` for managing installation of your plugins, there are +two steps required. -## Create a required_plugins block +First, add a [`required_plugins`](/packer/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements) +block to your [packer block](/packer/docs/templates/hcl_templates/blocks/packer). -1. Add a - [`required_plugins`](/packer/docs/templates/hcl_templates/blocks/packer#specifying-plugin-requirements) - block to your [packer block](/packer/docs/templates/hcl_templates/blocks/packer). Each block will tell Packer what version(s) of a - particular plugin can be installed. Make sure to set a valid [version - constraint string](/packer/docs/templates/hcl_templates/blocks/packer#version-constraints). +Each block will tell Packer what version(s) of a particular plugin can be installed. +Make sure to set a valid +[version constraint string](/packer/docs/templates/hcl_templates/blocks/packer#version-constraints). Here is an example `required_plugins` block: @@ -54,17 +133,18 @@ packer { } ``` -2. Run [`packer init`](/packer/docs/commands/init) from your project directory (the - directory containing your Packer templates) to install all missing plugin - binaries. Given the above example, Packer will try to look for a GitHub - repository owned by user or organization `azr` named - `packer-plugin-myawesomecloud` and `packer-plugin-happycloud`. +Once your template contains those `required_plugins`, run +[`packer init`](/packer/docs/commands/init) to install all missing plugin +binaries. +Given the above example, Packer will try to look for a GitHub +repository owned by user or organization `azr` named +`packer-plugin-myawesomecloud` and `packer-plugin-happycloud`. ## Names and Addresses Each plugin has two identifiers: -- A `source` address, which is necessary when requiring a plugin not bundled with the Packer binary. +- A `source` address, which is where the plugin is downloaded from. - A unique **local name**, which is used everywhere else in a Packer configuration. ## Local Names @@ -111,81 +191,21 @@ source "foo-ebs" "example" { } ``` -## Source Addresses - -A plugin's source address is its global identifier. It also tells Packer where -to download it. - -Source addresses consist of three parts delimited by slashes (`/`), as -follows: - -`//` - -- **Hostname:** The hostname of the location/service that - distributes the plugin. Currently, the only valid "hostname" is github.com, - but we plan to eventually support plugins downloaded from other domains. - -- **Namespace:** An organizational namespace within the specified host. - This often is the organization that publishes the plugin. - -- **Type:** A short name for the platform or system the plugin manages. The - type is usually the plugin's preferred local name. - -For example, the fictional `myawesomecloud` plugin could belong to the -`hashicorp` namespace on `github.com`, so its `source` could be -`github.com/hashicorp/myawesomecloud`, - --> Note: the actual _repository_ that myawesomecloud comes from must always have -the name format `github.com/hashicorp/packer-plugin-myawesomecloud`, but the -`required_plugins` block omits the redundant `packer-plugin-` repository prefix -for brevity. - -The source address with all three components given explicitly is called the -plugin's _fully-qualified address_. You will see fully-qualified address in -various outputs, like error messages. - -## Plugin Installation Workflow - -* [`packer init`](/packer/docs/commands/init) will install plugins in the **last** directory -in the following numbered list. - -1. `PACKER_PLUGIN_PATH` if set will be the sole location for installing plugins. All other -plugin directories will be ignored. -1. `PACKER_CONFIG_DIR`\plugins on Windows systems, or `PACKER_CONFIG_DIR`/plugins on all other systems. - -* During the initialization of Packer, any plugin required in the -**`required_plugins`** section will be looked up in all entries of the following -list. **First** plugin found takes precedence. Two binaries of the same plugin -with two different version will be considered as two different plugins. Highest -found version matching `required_plugins` will be taken into consideration. - -During initialization, on a `darwin_amd64` system, Packer will look-up for the -following files: - -* `PACKER_PLUGIN_PATH/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64` -* `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64` - -The first plugin-name/version files found will take precedence. - -For plugins located under the `github.com/azr/happycloud/` directory structure an accompanying SHA256SUM file -will be required in order for `packer init` to ensure the plugin being loaded has not been tampered with. -The SHA256SUM file will be automatically generated when a plugin is installed via `packer init` if the plugin -was installed manually into `PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/` then the file -`PACKER_CONFIG_DIR/plugins/github.com/azr/happycloud/packer-plugin-happycloud_*_x5.0_darwin_amd64_SHA256SUM` must be generated manually as well. - -Plugin installation via `packer plugins install` works similar to that of the `packer init` command, with the following -exceptions no `required_plugins` block required and can be used with both legacy JSON and HCL2 templates. - --> The [`packer plugins`](/packer/docs/commands/plugins), available from Packer v1.8.0, command allows -you to install plugins without going through `init`. +Plugin installation via `packer plugins install` works similar to that of the `packer init` command, but +no `required_plugins` block are required, and thus can be used with both legacy JSON and HCL2 templates. ```shell packer plugins install github.com/hashicorp/vagrant ``` +-> You can only install one plugin per invocation of the command. If you need to install + a specific version of a plugin, you can specify a version to install as an optional + argument to the command line. + e.g.: `packer plugins install "github.com/hashicorp/vagrant" "v1.0.1"` + The command will install the plugin in the `PACKER_CONFIG_DIR` set, or its default location, which depends on the OS/environment, as documented in [Configuring Packer](https://developer.hashicorp.com/packer/docs/configure#packer-s-plugin-directory). @@ -203,5 +223,9 @@ For example, if your configuration directory is located in `~/.config/packer`, you can copy the binary to `~/.config/plugins/packer-plugin-NAME`, and Packer will be able to load it afterwards. +If you have a `required_plugins` for the plugin you're manually installing, make sure +it respects the constraints described in the [Plugin loading workflow](#plugin-loading-workflow) +section, otherwise Packer will not be able to load it. +