Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add instance_id to state so ID is usable in provisioners #84

Closed
wants to merge 3 commits into from
Closed

Add instance_id to state so ID is usable in provisioners #84

wants to merge 3 commits into from

Conversation

vheon
Copy link

@vheon vheon commented Nov 6, 2023

Hi,

The change I'm proposing should be pretty straightforward but I'm having difficulty testing it as packer is not picking up my locally built plugin and keep using the "official" one. Also the content of the linked CONTRIBUTING.md file seems sort of not updated.

https://github.com/hashicorp/packer-plugin-lxd/blob/main/.github/CONTRIBUTING.md#opening-an-pull-request

Any pointers would be welcome on this 🙏

@vheon vheon requested a review from a team as a code owner November 6, 2023 14:37
@hashicorp-cla
Copy link

hashicorp-cla commented Nov 6, 2023

CLA assistant check

Thank you for your submission! We require that all contributors sign our Contributor License Agreement ("CLA") before we can accept the contribution. Read and sign the agreement

Learn more about why HashiCorp requires a CLA and what the CLA includes

Have you signed the CLA already but the status is still pending? Recheck it.

@lbajolet-hashicorp
Copy link
Contributor

Hi @vheon,

From a quick glance at the code, this looks OK to me!

For testing, when you build the plugin, it will output a binary, which can then be loaded by Packer according to the rules specfied in this documentation section.

By any chance, could you share your template so we can help?

Also FYI, before we can merge this PR, we'll need you to sign our CLA first. This is not a pressing matter as we still need to ensure this works as expected, but please be aware this is required by our legal department for external contributions.

Thanks, and please let us know if we can help further!

@vheon
Copy link
Author

vheon commented Nov 7, 2023

Looking at the documentation I thought that I could simply build the plugin and copy it in the directory where I ran packer build. As of right now I have this simple test case:

~/code/test-lxd-packer
❯ tree
 .
├──  main.pkr.hcl
├──  packer-plugin-lxd
└──  playbooks
   └──  main.yml

main.pkr.hcl

packer {
  required_plugins {
    lxd = {
      version = ">= 1.0.1"
      source  = "github.com/hashicorp/lxd"
    }
    ansible = {
      source  = "github.com/hashicorp/ansible"
      version = "~> 1"
    }
  }
}

source "lxd" "centos7" {
  image = "images:centos/7"
}

build {
  name = "test"

  source "lxd.centos7" {
    output_image = "test-image"
    publish_properties = {
      description = "Make with Packer"
    }
  }

  provisioner "ansible" {
    playbook_file = "./playbooks/main.yml"
    user = "root"
    extra_arguments = [ "--extra-vars", "ansible_host=${build.ID} ansible_connection=${source.type}" ]
    ansible_env_vars = ["ANSIBLE_FORCE_COLOR=1"]
  }
}

and playbook/main.yml

---
- name: Test play
  hosts: all
  gather_facts: false
  tasks:
    - name: Test
      ansible.builtin.ping:

the output of PACKER_LOG=1 packer build main.pkr.hcl is the following

2023/11/07 15:27:36 [INFO] Packer version: 1.9.4 [go1.21.0 linux amd64]
2023/11/07 15:27:36 [TRACE] discovering plugins in
2023/11/07 15:27:36 [TRACE] discovering plugins in /home/linuxbrew/.linuxbrew/bin
2023/11/07 15:27:36 [INFO] Discovered potential plugin: ansible = /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64
2023/11/07 15:27:36 [INFO] Discovered potential plugin: docker = /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/docker/packer-plugin-docker_v1.0.8_x5.0_linux_amd64
2023/11/07 15:27:36 [INFO] Discovered potential plugin: lxd = /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64
2023/11/07 15:27:36 found external [-packer-default-plugin-name- local] provisioner from ansible plugin
2023/11/07 15:27:36 [INFO] found external [-packer-default-plugin-name-] builders from docker plugin
2023/11/07 15:27:36 [INFO] found external [import push save tag] post-processors from docker plugin
2023/11/07 15:27:36 [INFO] found external [-packer-default-plugin-name-] builders from lxd plugin
2023/11/07 15:27:36 [INFO] Discovered potential plugin: lxd = packer-plugin-lxd
2023/11/07 15:27:36 [INFO] found external [-packer-default-plugin-name-] builders from lxd plugin
2023/11/07 15:27:36 [INFO] PACKER_CONFIG env var not set; checking the default config file path
2023/11/07 15:27:36 [INFO] PACKER_CONFIG env var set; attempting to open config file: /home/vheon/.packerconfig
2023/11/07 15:27:36 [WARN] Config file doesn't exist: /home/vheon/.packerconfig
2023/11/07 15:27:36 Old default config directory found: /home/vheon/.packer.d
2023/11/07 15:27:36 [INFO] Setting cache directory: /home/vheon/.cache/packer
2023/11/07 15:27:36 Old default config directory found: /home/vheon/.packer.d
2023/11/07 15:27:36 [TRACE] listing potential installations for "github.com/hashicorp/lxd" that match ">= 1.0.1". plugingetter.ListInstallationsOptions{FromFolders:[]string{"", "/home/linuxbrew/.linuxbrew/bin"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00020a080)}}}}
2023/11/07 15:27:36 [TRACE] Found the following "github.com/hashicorp/lxd" installations: [{/home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 v1.0.1}]
2023/11/07 15:27:36 [INFO] found external [-packer-default-plugin-name-] builders from lxd plugin
2023/11/07 15:27:36 [TRACE] listing potential installations for "github.com/hashicorp/ansible" that match "~> 1". plugingetter.ListInstallationsOptions{FromFolders:[]string{"", "/home/linuxbrew/.linuxbrew/bin"}, BinaryInstallationOptions:plugingetter.BinaryInstallationOptions{APIVersionMajor:"5", APIVersionMinor:"0", OS:"linux", ARCH:"amd64", Ext:"", Checksummers:[]plugingetter.Checksummer{plugingetter.Checksummer{Type:"sha256", Hash:(*sha256.digest)(0xc00020a080)}}}}
2023/11/07 15:27:36 [TRACE] Found the following "github.com/hashicorp/ansible" installations: [{/home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 v1.1.0}]
2023/11/07 15:27:36 found external [-packer-default-plugin-name- local] provisioner from ansible plugin
2023/11/07 15:27:36 [INFO] Starting external plugin /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 start builder -packer-default-plugin-name-
2023/11/07 15:27:36 Starting plugin: /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 []string{"/home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64", "start", "builder", "-packer-default-plugin-name-"}
2023/11/07 15:27:36 Waiting for RPC address for: /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64
2023/11/07 15:27:36 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Plugin address: unix /tmp/packer-plugin3035673860
2023/11/07 15:27:36 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Waiting for connection...
2023/11/07 15:27:36 Received unix RPC address for /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64: addr is /tmp/packer-plugin3035673860
2023/11/07 15:27:36 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Serving a plugin connection...
2023/11/07 15:27:36 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 [TRACE] starting builder -packer-default-plugin-name-
2023/11/07 15:27:36 [INFO] Starting external plugin /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 start provisioner -packer-default-plugin-name-
2023/11/07 15:27:36 Starting plugin: /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 []string{"/home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64", "start", "provisioner", "-packer-default-plugin-name-"}
2023/11/07 15:27:36 Waiting for RPC address for: /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64
2023/11/07 15:27:36 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Plugin address: unix /tmp/packer-plugin1935208451
2023/11/07 15:27:36 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Waiting for connection...
2023/11/07 15:27:36 Received unix RPC address for /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64: addr is /tmp/packer-plugin1935208451
2023/11/07 15:27:36 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 Serving a plugin connection...
2023/11/07 15:27:36 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:36 [TRACE] starting provisioner -packer-default-plugin-name-
2023/11/07 15:27:37 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:37 ansible-playbook version: 2.14.1
2023/11/07 15:27:37 Build debug mode: false
2023/11/07 15:27:37 Force build: false
2023/11/07 15:27:37 On error:
2023/11/07 15:27:37 Waiting on builds to complete...
2023/11/07 15:27:37 Starting build run: test.lxd.centos7
test.lxd.centos7: output will be in this color.
2023/11/07 15:27:37 Running builder:
2023/11/07 15:27:37 [INFO] (telemetry) Starting builder lxd.centos7

==> test.lxd.centos7: Creating container...
2023/11/07 15:27:37 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:37 Executing lxc command: []string{"launch", "--ephemeral=false", "--vm=false", "--profile=default", "images:centos/7", "packer-centos7"}
2023/11/07 15:27:39 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:39 stdout: Creating packer-centos7
2023/11/07 15:27:39 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: Starting packer-centos7
Remapping container filesystemgin-lxd_v1.0.1_x5.0_linux_amd64 plugin:
2023/11/07 15:27:39 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:39 stderr:
2023/11/07 15:27:42 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:42 Sleeping for 3 seconds...
2023/11/07 15:27:42 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:42 Unable to load communicator config from state to populate provisionHookData
2023/11/07 15:27:42 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:42 Running the provision hook
2023/11/07 15:27:42 [INFO] (telemetry) Starting provisioner ansible
2023/11/07 15:27:43 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:43 ansible-playbook version: 2.14.1
==> test.lxd.centos7: Provisioning with Ansible...
    test.lxd.centos7: Setting up proxy adapter for Ansible....
2023/11/07 15:27:43 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:43 Creating inventory file for Ansible run...
2023/11/07 15:27:43 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:43 SSH proxy: serving on 127.0.0.1:40321
==> test.lxd.centos7: Executing Ansible: ansible-playbook -e packer_build_name="centos7" -e packer_builder_type=lxd --extra-vars ansible_host=ERR_ID_NOT_IMPLEMENTED_BY_BUILDER ansible_connection=lxd -e ansible_ssh_private_key_file=/tmp/ansible-key1668697098 -i /tmp/packer-provisioner-ansible4222930341 /home/vheon/code/test-lxd-packer/playbooks/main.yml
    test.lxd.centos7:
    test.lxd.centos7: PLAY [Test play] ***************************************************************
    test.lxd.centos7:
    test.lxd.centos7: TASK [Test] ********************************************************************
    test.lxd.centos7: fatal: [default]: UNREACHABLE! => {"changed": false, "msg": "Failed to create temporary directory. In some cases, you may have been able to authenticate and did not have permissions on the target directory. Consider changing the remote tmp path in ansible.cfg to a path rooted in \"/tmp\", for more error information use -vvv. Failed command was: ( umask 77 && mkdir -p \"` echo ~/.ansible/tmp `\"&& mkdir \"` echo ~/.ansible/tmp/ansible-tmp-1699367264.574535-3383261-71828469674855 `\" && echo ansible-tmp-1699367264.574535-3383261-71828469674855=\"` echo ~/.ansible/tmp/ansible-tmp-1699367264.574535-3383261-71828469674855 `\" ), exited with result 1", "unreachable": true}
    test.lxd.centos7:
    test.lxd.centos7: PLAY RECAP *********************************************************************
    test.lxd.centos7: default                    : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
    test.lxd.centos7:
2023/11/07 15:27:44 packer-plugin-ansible_v1.1.0_x5.0_linux_amd64 plugin: 2023/11/07 15:27:44 shutting down the SSH proxy
2023/11/07 15:27:44 [INFO] (telemetry) ending ansible
2023/11/07 15:27:44 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:44 Executing lxc command: []string{"delete", "--force", "packer-centos7"}
==> test.lxd.centos7: Unregistering and deleting deleting container...
2023/11/07 15:27:46 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:46 stdout:
2023/11/07 15:27:46 packer-plugin-lxd_v1.0.1_x5.0_linux_amd64 plugin: 2023/11/07 15:27:46 stderr:
2023/11/07 15:27:46 [INFO] (telemetry) ending lxd.centos7
==> Wait completed after 8 seconds 652 milliseconds
2023/11/07 15:27:46 machine readable: error-count []string{"1"}
==> Some builds didn't complete successfully and had errors:
2023/11/07 15:27:46 machine readable: test.lxd.centos7,error []string{"Error executing Ansible: Non-zero exit status: exit status 4"}
==> Builds finished but no artifacts were created.
2023/11/07 15:27:46 [INFO] (telemetry) Finalizing.
Build 'test.lxd.centos7' errored after 8 seconds 652 milliseconds: Error executing Ansible: Non-zero exit status: exit status 4

==> Wait completed after 8 seconds 652 milliseconds

==> Some builds didn't complete successfully and had errors:
--> test.lxd.centos7: Error executing Ansible: Non-zero exit status: exit status 4

==> Builds finished but no artifacts were created.
2023/11/07 15:27:46 waiting for all plugin processes to complete...
2023/11/07 15:27:46 /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/ansible/packer-plugin-ansible_v1.1.0_x5.0_linux_amd64: plugin process exited
2023/11/07 15:27:46 /home/linuxbrew/.linuxbrew/bin/github.com/hashicorp/lxd/packer-plugin-lxd_v1.0.1_x5.0_linux_amd64: plugin process exited

Looking at the log it doesn't seems to pick up the plugin in the current directory. Any help would be appreciated.

@lbajolet-hashicorp
Copy link
Contributor

Hey @vheon,

Sounds on point, you're running into this behaviour: "Where applicable, some installation processes such as packer init may override the plugin loading process."

To be fair, we think this is confusing, and are re-thinking our process for installing/loading plugins from Packer, please refer to this issue on Packer for information. If you have an opinion on this subject, please don't hesitate to share it in this issue!

For your case, if you want to be able to load the plugin, there's essentially two choices at the moment:

  1. Remove the required_plugins block for the plugin you're trying to load out-of-band
  2. Replicate the installation process from packer init for your manually compiled plugin, with a final version number for Packer to consider it. Alternatively, you may try with Packer compiled from this branch, and then you may install the plugin using packer plugins install --path <path_to_plugin> github.com/hashicorp/lxd 1.0.2. The convention may change by the time we finalize this, but you will be able to automatically install it.

Hope that helps!

@nywilken
Copy link
Contributor

nywilken commented Dec 5, 2023

@vheon the changes look good but we need you to sign the CLA before we can approve and merge.

@nywilken
Copy link
Contributor

Hi there 👋, I'm going to close this pull-request since there hasn't been any updates to it since the last request for the CLA. But if you are still interested in seeing this change please feel free to leave a comment after the signing the CLA we will gladly reopen.

@nywilken nywilken closed this Feb 26, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants