From 72bc6c811a6ecb808bc5a746eea08fbbea37d3db Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Mon, 20 Nov 2023 14:07:07 -0500 Subject: [PATCH 1/6] Document process for registering a plugin as a Packer Integration --- .github/ISSUE_TEMPLATE/plugin_integration.md | 21 +++++ .../content/docs/plugins/creation/index.mdx | 94 +++++-------------- 2 files changed, 42 insertions(+), 73 deletions(-) create mode 100644 .github/ISSUE_TEMPLATE/plugin_integration.md diff --git a/.github/ISSUE_TEMPLATE/plugin_integration.md b/.github/ISSUE_TEMPLATE/plugin_integration.md new file mode 100644 index 00000000000..c84da673895 --- /dev/null +++ b/.github/ISSUE_TEMPLATE/plugin_integration.md @@ -0,0 +1,21 @@ +--- +name: Plugin Integration Request +about: Open request to add your plugin as a Packer integration (https://developer.hashicorp.com/packer/integrations) +labels: integration-request +--- + +#### Description + +A written description of your plugin along with a link to the plugin repository. + +#### Integration Tier + + +#### Checklist +- [] Has valid [`metadata.hcl`](https://github.com/hashicorp/integration-template) file in plugin repository. +- [] Has added integration scripts [packer-plugin-scaffolding](https://github.com/hashicorp/packer-plugin-scoffolding) to plugin repository. +- [] Has added top-level integration README.md file to plugin `docs` directory. +- [] All plugins components have one README.md describing their usage. +- [] Has a fully synced `.web-docs` directory ready for publishing to the integrations portal. + diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 32537f160d3..3ba19fee600 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -214,87 +214,35 @@ Here's what you need to create releases using GitHub Actions: to install a plugin using `packer init`, see the [init docs](/packer/docs/commands/init). -## Registering Plugin Documentation +## Registering Plugins -~> Note: Registering a remote plugin's plugin documentation requires the use of [Packer's plugin docs configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). +~> Note: Registering a remote as an integration requires the use of [Packer's plugin docs configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). -`packer init` allows users to require and install remote Packer plugins, those not bundled with Packer core, that have been published to GitHub automatically. -To help with the discovery of remote Packer plugins on GitHub, plugins maintainers can choose to register plugin documentation for each component directly on the [Packer Documentation Page](/packer/docs). +`packer init` allows users to require and install remote Packer plugins published to GitHub. +To help with the discovery of remote Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](/packer/integrations). -The registration process requires the creation of a `docs.zip` file archive containing the `.mdx` files for each of the plugin components in the remote plugin's repository. A working example can be seen at the [packer-plugin-docker repository](https://github.com/hashicorp/packer-plugin-docker/releases/latest). +The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) be added to your plugin repository for configuring the Packer integration pipeline and plugin documentation for rendering +on the [Packer Integrations](/packer/integrations) portal. -Once in place the remote plugin can be added to Packer's website builds by opening a pull-request against [hashicorp/packer](https://github.com/hashicorp/packer), with the needed configuration for pulling in the remote documentation. +The following steps are required for registering your plugin as an integration -Remote plugins will have their components listed under the respected types (i.e builders, provisioners, etc) using the names specified in the remote block configuration, and labeled with their respective [tier and namespace](/packer/docs/plugins#tiers-and-namespaces). +1. Update your plugin documentation structure according to the templated defined with the [Packer Plugin Scaffolding template]. +New plugins generated from this template may have the necessary structure in place. If so you can jump to step 4. +1. For the integrations library, only one top-level README per integration is supported. Any top-level index.mdx files that exist +within a plugins existing documentation will need to migrate to a top-level README. +1. Update your top-level integration README to include a description, plugin installation steps, available components section, and, any, additional sections +needed to inform users on how to work with your integration. Refer to [Packer scaffolding plugin](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/docs/README.md) for an example. +1. Update the top-level README for each of the components within your integration to follow the structure defined in the scaffolding template. +1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new/choose). +Please provide all the requested information to help expedite the integration request. -To register a plugin follow one of the following setups +By opening an integration request, you are asking a member of the to Packer team to review your plugin integration configuration, plugin documentation, +and, finally, to open an internal pull-request to finalize the integration setup. - - - -Documentation for a plugin is maintained within the `docs` directory and served on GitHub. - -To include plugin docs on the website, a global pre-hook has been added to the main scaffolding [.goreleaser.yml](https://github.com/hashicorp/packer-plugin-scaffolding/blob/42e5b0b1e575879b0477cb6d4291e027f4d92f85/.goreleaser.yml#L10) file, that if uncommented will generate and include a docs.zip file as part of the plugin release. - -The `docs.zip` file contains all of the `.mdx` files under the plugins root `docs/` directory that the website can consume remotely. - - - - - -You can generate the required documentation structure manually by creating a zip file called `docs.zip` of the docs directory and including that zip file in the plugin release. - -```/bin/bash -[[ -d docs/ ]] && zip -r docs.zip docs/ -``` - - - - -Once the first `docs.zip` file has been included into a release you will need to open a one time pull-request against [hashicorp/packer](https://github.com/hashicorp/packer) to register the plugin docs. - -This is done by adding the block below for the respective plugin to the file [website/data/plugins-manifest.json](https://github.com/hashicorp/packer/blob/main/website/data/plugins-manifest.json). - -```json -{ - "title": "Scaffolding", - "path": "scaffolding", - "repo": "hashicorp/packer-plugin-scaffolding", - "version": "latest", - "sourceBranch": "main" -} -``` - -If a plugin maintainer wishes to only include a specific version of released docs, then the `"version"` key in the above configuration should be set to a released version of the plugin. Otherwise it should be set to `"latest"`. - -The `"sourceBranch"` key in the above configuration ensures potential contributors can link back to source files in the plugin repository from the Packer docs site. If a `"sourceBranch"` value is not present, it will default to `"main"`. - -### Testing Plugin Documentation - -Before publishing the `docs.zip` file, you might want to preview your documentation changes. -We provide a mechanism that allows to preview how the docs will look like within -the Packer documentation. - -Follow the next steps to get the Packer website running and preview the documentation changes: - -- Get the [Packer source code](https://github.com/hashicorp/packer). Our website code is under the [website folder](https://github.com/hashicorp/packer/tree/main/website). -- Generate the `docs.zip` file. You can find above the steps to do so. -- Add the `zipFile` attribute to the plugin entry in `plugins-manifest.json`. The value should be the full path of the `docs.zip` generated. For example: - -```json -{ - "title": "Scaffolding", - "path": "scaffolding", - "repo": "hashicorp/packer-plugin-scaffolding", - "version": "latest", - "sourceBranch": "main", - "zipFile": "/Users/myuser/Packer/plugins/packer-plugin-scaffolding/docs.zip" -} -``` +Plugin integrations will be listed as a [Packer Integration](/packer/integrations), with details on how to install and use your the plugin. -- Go to the [website folder](https://github.com/hashicorp/packer/tree/main/website). - In the website README, follow the steps to [run the website with node](https://github.com/hashicorp/packer/tree/main/website#with-node). -- Once the website is up and running, the plugin documentation should be available in `http://localhost:3000/docs`. +Plugin integrations, once deployed, can be updated manually, or automatically upon a new release, by the plugin authors. Changes to the defined documentation structure +or parent repository should be communicated to the Packer team to ensure a working integration pipeline. ## Plugin Development Tips and FAQs From 97624ce26eaa4cdc303939a87ddf80a534459445 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Thu, 30 Nov 2023 17:13:55 -0500 Subject: [PATCH 2/6] Replace relative links --- website/content/docs/plugins/creation/index.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 3ba19fee600..55425367b7b 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -9,7 +9,7 @@ page_title: Extending # Developing Plugins Packer is extensible and supports plugins that let you -create and use custom builders, provisioners, post-processors, and data sources. This page explains how to develop Packer plugins. Before you begin, we recommend reviewing the Packer documentation and the instructions for [installing external plugins](/packer/docs/plugins/install-plugins). +create and use custom builders, provisioners, post-processors, and data sources. This page explains how to develop Packer plugins. Before you begin, we recommend reviewing the Packer documentation and the instructions for [installing external plugins](https://developer.hashicorp.com/packer/docs/plugins/install-plugins). ~> **Warning** This is an advanced topic. You should have strong knowledge of Packer before you start writing plugins. @@ -146,7 +146,7 @@ because the `packer init` command only supports multi-component plugins. Next, build your plugin as you would any other Go application. The resulting binary is the plugin that can be installed using -[standard installation procedures](/packer/docs/plugins#installing-plugins). +[standard installation procedures](https://developer.hashicorp.com/packer/docs/plugins#installing-plugins). This documentation explains how to implement each type of plugin interface: builders, data sources, provisioners, and post-processors. @@ -212,17 +212,17 @@ Here's what you need to create releases using GitHub Actions: releaser is working. The tag must be a valid [Semantic Version](https://semver.org/) preceded with a `v`. Once the tag is pushed, the github actions you just configured will automatically build release binaries that Packer can download using `packer init`. For more details on how to install a plugin using `packer init`, see the - [init docs](/packer/docs/commands/init). + [init docs](https://developer.hashicorp.com/packer/docs/commands/init). ## Registering Plugins ~> Note: Registering a remote as an integration requires the use of [Packer's plugin docs configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). `packer init` allows users to require and install remote Packer plugins published to GitHub. -To help with the discovery of remote Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](/packer/integrations). +To help with the discovery of remote Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations). The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) be added to your plugin repository for configuring the Packer integration pipeline and plugin documentation for rendering -on the [Packer Integrations](/packer/integrations) portal. +on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. The following steps are required for registering your plugin as an integration @@ -239,7 +239,7 @@ Please provide all the requested information to help expedite the integration re By opening an integration request, you are asking a member of the to Packer team to review your plugin integration configuration, plugin documentation, and, finally, to open an internal pull-request to finalize the integration setup. -Plugin integrations will be listed as a [Packer Integration](/packer/integrations), with details on how to install and use your the plugin. +Plugin integrations will be listed as a [Packer Integration](https://developer.hashicorp.com/packer/integrations), with details on how to install and use your the plugin. Plugin integrations, once deployed, can be updated manually, or automatically upon a new release, by the plugin authors. Changes to the defined documentation structure or parent repository should be communicated to the Packer team to ensure a working integration pipeline. From 72385d6b8a61a0e4a737f9f52c155a3472b185c6 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Fri, 1 Dec 2023 08:00:41 -0500 Subject: [PATCH 3/6] Address review feedback --- website/content/docs/plugins/creation/index.mdx | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 55425367b7b..5cb3d8d3c00 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -216,23 +216,23 @@ Here's what you need to create releases using GitHub Actions: ## Registering Plugins -~> Note: Registering a remote as an integration requires the use of [Packer's plugin docs configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). +~> Note: Registering a plugin as an integration requires the use of [Packer's Incofiguration configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). -`packer init` allows users to require and install remote Packer plugins published to GitHub. -To help with the discovery of remote Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations). +To help with the discovery of Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations). -The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) be added to your plugin repository for configuring the Packer integration pipeline and plugin documentation for rendering -on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. +The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) be added to your plugin repository for configuring the Packer integration pipeline and +a specific directory structure for plugin documentation to be render on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. -The following steps are required for registering your plugin as an integration +You can execute the following steps to register your plugin as an integration: -1. Update your plugin documentation structure according to the templated defined with the [Packer Plugin Scaffolding template]. -New plugins generated from this template may have the necessary structure in place. If so you can jump to step 4. +1. Update your plugin documentation structure according to the template defined with the [Packer Plugin Scaffolding template]. +New plugins generated from this template may have the necessary structure in place. If so you can jump to step 3. 1. For the integrations library, only one top-level README per integration is supported. Any top-level index.mdx files that exist within a plugins existing documentation will need to migrate to a top-level README. 1. Update your top-level integration README to include a description, plugin installation steps, available components section, and, any, additional sections needed to inform users on how to work with your integration. Refer to [Packer scaffolding plugin](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/docs/README.md) for an example. 1. Update the top-level README for each of the components within your integration to follow the structure defined in the scaffolding template. +1. Add the integration configuration file [metadata.hcl](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.web-docs/metadata.hcl) to the plugins `.web-docs` directory. 1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new/choose). Please provide all the requested information to help expedite the integration request. From 5a96009ba192adc292e754b5f4a24db2e4cf17dd Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Fri, 1 Dec 2023 08:45:44 -0500 Subject: [PATCH 4/6] Add example shell commands for copying integration files into plugin repository --- .../content/docs/plugins/creation/index.mdx | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 5cb3d8d3c00..273144bfbcd 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -236,6 +236,33 @@ needed to inform users on how to work with your integration. Refer to [Packer sc 1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new/choose). Please provide all the requested information to help expedite the integration request. +#### [Example] Add integration files to existing plugin repository + +```shell +## Update Plugin repository with integration config, workflows, and scripts +cd packer-plugin-name +mkdir -p .web-docs/scripts + +# Download packer-plugin-scaffolding repo copy files +curl -L -O https://github.com/hashicorp/packer-plugin-scaffolding/archive/refs/heads/main.zip +unzip main.zip +cp packer-plugin-scaffolding-main/.web-docs/metadata.hcl .web-docs/ +cp -r packer-plugin-scaffolding-main/.web-docs/scripts/ .web-docs/scripts/ +cp .github/workflows/notify-integration-release-via-* .github/workflows/ + +# Remove downloaded scaffolding project +rm main.zip +rm -rf packer-plugin-scaffolding-main + +# Add the following commands to your plugin GNUmakefile +generate: install-packer-sdc + @go generate ./... + @rm -rf .docs + @packer-sdc renderdocs -src docs -partials docs-partials/ -dst .docs/ + @./.web-docs/scripts/compile-to-webdocs.sh "." ".docs" ".web-docs" "" + @rm -r ".docs" +``` + By opening an integration request, you are asking a member of the to Packer team to review your plugin integration configuration, plugin documentation, and, finally, to open an internal pull-request to finalize the integration setup. From ea92d8b1f38d1debce15a4d7685e38d715b49cd5 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Fri, 1 Dec 2023 09:59:50 -0500 Subject: [PATCH 5/6] Remove reference to deprecated single components --- .../content/docs/plugins/creation/index.mdx | 47 +------------------ 1 file changed, 1 insertion(+), 46 deletions(-) diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 273144bfbcd..7d7939f4683 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -49,15 +49,7 @@ danger of colliding dependencies. - [`github.com/hashicorp/packer-plugin-sdk/plugin`](https://pkg.go.dev/github.com/hashicorp/packer-plugin-sdk/plugin) - Contains the code to serve the plugin. This handles all the inter-process communication. -Basic examples of serving your component are shown below. Note that if you -define a multi-component plugin, you can (but do not need to) add more than one -component per plugin binary. The multi-component plugin is also compatible with -download and installation via `packer init`, whereas the single-component plugin -is not. - - - - +Basic examples of serving your component are shown below. ```go // main.go @@ -107,43 +99,6 @@ the following components available: - the `my-foo` post-processor - the `my-bar` provisioner - - - - -```go -// main.go - -import ( - "github.com/hashicorp/packer-plugin-sdk/plugin" -) - -// Assume this implements the packer.Builder interface -type Builder struct{} - -func main() { - server, err := plugin.Server() - if err != nil { - panic(err) - } - server.RegisterBuilder(new(Builder)) - server.Serve() -} -``` - -This `server.Serve()` invocation handles all the details of communicating with -Packer core and serving your component over RPC. As long as your struct being -registered implements one of the component interfaces, Packer will now be able -to launch your plugin and use it. - -Please note that single-component plugins exist for backwards-compatability. We -would rather you register your component using the multi-component method shown -in the other tab, even if you only have one component in your binary. This is -because the `packer init` command only supports multi-component plugins. - - - - Next, build your plugin as you would any other Go application. The resulting binary is the plugin that can be installed using [standard installation procedures](https://developer.hashicorp.com/packer/docs/plugins#installing-plugins). From cae2c5eb621ed341e93b8672b6455daa3b43d239 Mon Sep 17 00:00:00 2001 From: Wilken Rivera Date: Tue, 5 Dec 2023 15:31:03 -0500 Subject: [PATCH 6/6] Apply suggestions from code review Co-authored-by: Lucas Bajolet <105649352+lbajolet-hashicorp@users.noreply.github.com> Co-authored-by: Sylvia Moss --- .../content/docs/plugins/creation/index.mdx | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/content/docs/plugins/creation/index.mdx b/website/content/docs/plugins/creation/index.mdx index 7d7939f4683..cb6d7d70682 100644 --- a/website/content/docs/plugins/creation/index.mdx +++ b/website/content/docs/plugins/creation/index.mdx @@ -171,25 +171,25 @@ Here's what you need to create releases using GitHub Actions: ## Registering Plugins -~> Note: Registering a plugin as an integration requires the use of [Packer's Incofiguration configuration](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/docs). +~> Note: Registering a plugin as an integration requires the documentation to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs). To help with the discovery of Packer plugins, plugins maintainers can choose to register their plugin as a [Packer Integration](https://developer.hashicorp.com/packer/integrations). -The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) be added to your plugin repository for configuring the Packer integration pipeline and -a specific directory structure for plugin documentation to be render on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. +The registration process requires [metadata configuration](https://github.com/hashicorp/integration-template#metadata-configuration) to be added to your plugin repository for configuring the Packer integration pipeline and +a specific directory structure for plugin documentation to be rendered on the [Packer Integrations](https://developer.hashicorp.com/packer/integrations) portal. You can execute the following steps to register your plugin as an integration: -1. Update your plugin documentation structure according to the template defined with the [Packer Plugin Scaffolding template]. -New plugins generated from this template may have the necessary structure in place. If so you can jump to step 3. +1. Update your plugin documentation structure to match the [Scaffolding example layout](https://github.com/hashicorp/packer-plugin-scaffolding/tree/main/.web-docs). +New plugins generated from this template have the necessary structure in place. If so you can jump to step 3. 1. For the integrations library, only one top-level README per integration is supported. Any top-level index.mdx files that exist -within a plugins existing documentation will need to migrate to a top-level README. +within a plugin's existing documentation will need to migrate to a top-level README. 1. Update your top-level integration README to include a description, plugin installation steps, available components section, and, any, additional sections needed to inform users on how to work with your integration. Refer to [Packer scaffolding plugin](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/docs/README.md) for an example. 1. Update the top-level README for each of the components within your integration to follow the structure defined in the scaffolding template. 1. Add the integration configuration file [metadata.hcl](https://github.com/hashicorp/packer-plugin-scaffolding/blob/main/.web-docs/metadata.hcl) to the plugins `.web-docs` directory. -1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new/choose). -Please provide all the requested information to help expedite the integration request. +1. Open a request for integration issue with the Packer team - [Open Request](https://github.com/hashicorp/packer/issues/new?labels=new-plugin-contribution&template=plugin_integration.md). +Provide all the requested information to help expedite the integration request. #### [Example] Add integration files to existing plugin repository @@ -199,7 +199,7 @@ cd packer-plugin-name mkdir -p .web-docs/scripts # Download packer-plugin-scaffolding repo copy files -curl -L -O https://github.com/hashicorp/packer-plugin-scaffolding/archive/refs/heads/main.zip +wget https://github.com/hashicorp/packer-plugin-scaffolding/archive/refs/heads/main.zip unzip main.zip cp packer-plugin-scaffolding-main/.web-docs/metadata.hcl .web-docs/ cp -r packer-plugin-scaffolding-main/.web-docs/scripts/ .web-docs/scripts/