Add support for Windows 2019/2022 to OCI capi provider image builder #1051

joekr · 2023-01-11T02:42:11Z

What this PR does / why we need it:
Adds support to build windows images to using with CAPOCI provider using cluster-api.

Which issue(s) this PR fixes (optional, in fixes #(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged): No open issues

Additional context
the oci_iad_windows_bm.json file contents

{
  "availability_domain": "<myAD>",
  "build_name": "windows",
  "base_image_ocid": "ocid1.image.oc1.iad.aaaaaaaazt5nv5koc2sxjagzfvdv7gnza2te7gheiscw2ukdrwcdk4mwhyxq",
  "compartment_ocid": "<myCompartment",
  "kubernetes_deb_version": "1.24.9-00",
  "kubernetes_rpm_version": "1.24.9-0",
  "kubernetes_semver": "v1.24.9",
  "kubernetes_series": "v1.24",
  "ocpus": "128",
  "region": "us-ashburn-1",
  "shape": "BM.Standard.E4.128",
  "subnet_ocid": "<mySubnet>"
}

Build logs

$ OPC_USER_PASSWORD="<mypassword>"  no_proxy=* PACKER_VAR_FILES=oci_iad_windows_bm.json make build-oci-windows-2019

...
...
...

==> oracle-oci: Provisioning with powershell script: /var/folders/yp/cx82s01j4m1d34krx_q97xs80000gn/T/powershell-provisioner2441768962
    oracle-oci: Using cloudbase-init unattend file for sysprep: C:\Program Files\Cloudbase Solutions\Cloudbase-Init\conf\Unattend.xml
    oracle-oci: IMAGE_STATE_COMPLETE
    oracle-oci: IMAGE_STATE_UNDEPLOYABLE
    oracle-oci: IMAGE_STATE_GENERALIZE_RESEAL_TO_OOBE
    oracle-oci: >>> Sysprep complete ...
==> oracle-oci: Creating image from instance...
==> oracle-oci: Updating image schema...
==> oracle-oci: Created image (ocid1.image.oc1.iad.XXXX).
==> oracle-oci: Terminating instance (ocid1.instance.oc1.iad.XXXX)...
==> oracle-oci: Terminated instance.
==> oracle-oci: Running post-processor: manifest
Build 'oracle-oci' finished after 33 minutes 47 seconds.

==> Wait completed after 33 minutes 47 seconds

==> Builds finished. The artifacts of successful builds are:
--> oracle-oci: An image was created: 'cluster-api-windows-v1.24.9-1673378936' (OCID: ocid1.image.oc1.iad.XXXX) in region 'us-ashburn-1'
--> oracle-oci: An image was created: 'cluster-api-windows-v1.24.9-1673378936' (OCID: ocid1.image.oc1.iad.XXXX) in region 'us-ashburn-1'

k8s-ci-robot · 2023-01-11T02:42:13Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

images/capi/packer/oci/scripts/enable_second_nic.ps1

jsturtevant · 2023-01-12T03:39:44Z

/assign

jsturtevant · 2023-01-13T17:19:01Z

images/capi/ansible/windows/roles/cloudbase-init/tasks/main.yml

+- name: Set up cloudbase-init-oci configuration 
+  win_template:
+    src: templates/cloudbase-init-oci.conf
+    dest: '{{ programfiles.stdout | trim }}\Cloudbase Solutions\Cloudbase-Init\conf\cloudbase-init.conf'


what is the difference between these? most configuration should be configurable through cloudbase_metadata_services and cloudbase_plugins I think?

I saw I could modify the cloudbase_plugins but it appeared to only be addition, I couldn't figure out how to remove plugins OCI doesn't support https://github.com/kubernetes-sigs/image-builder/blob/master/images/capi/ansible/windows/roles/cloudbase-init/templates/cloudbase-init.conf#L35.

The OCI template removes

cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin,

I think it might be better to move these out of the base template to avoid duplicating this file as much as we can. what do you think?

Makes sense to me.

I've moved all the plugins to their respective packer-windows.json files and reduced the amount of default cloudbase_plugins in the cloudbase-init.conf file.

images/capi/packer/oci/packer-windows.json

jsturtevant · 2023-01-17T17:03:05Z

/retest

jsturtevant · 2023-01-18T18:29:06Z

centos failed when updating with yum. I don't know much about yum and centos, would a retry to this command help flakes?

 [Errno -1] Metadata file does not match checksum\nTrying other mirror.\n\n\n One of the configured repositories failed (CentOS-7 - Updates (OpenLogic)),\n and yum doesn't have enough cached data to continue. At this point the only\n safe thing yum can do is fail. There are a few ways to work \"fix\" this:\n\n     1. Contact the upstream for the repository and get them to fix the problem.\n\n     2. Reconfigure the baseurl/etc. for the repository, to point to a working\n        upstream. This is most often useful if you are using a newer\n        distribution release than is supported by the repository (and the\n        packages for the previous distribution release still work).\n\n     3. Run the command with the repository temporarily disabled\n            yum --disablerepo=updates-openlogic ...\n\n     4. Disable the repository permanently, so yum won't use it by default. Yum\n        will then just ignore the repository until you permanently enable it\n        again or use --enablerepo for temporary usage:\n\n            yum-config-manager --disable updates-openlogic\n        or\n            subscription-manager repos --disable=updates-openlogic\n\n     5. Configure the failing repository to be skipped, if it is unavailable.\n        Note that yum will try to contact the repo. when it runs most commands,\n        so will have to try and fail each time (and thus. yum will be be much\n        slower). If it is a very temporary problem though, this is often a nice\n        compromise:\n\n            yum-config-manager --save --setopt=updates-openlogic.skip_if_unavailable=true\n\nfailure: repodata/filelists.sqlite.bz2 from updates-openlogic: [Errno 256] No more mirrors to try.\nhttp://olcentgbl.trafficmanager.net/centos/7/updates/x86_64/repodata/filelists.sqlite.bz2: [Errno -1] Metadata file does not match checksum\n", "obsoletes":

/retest

joekr · 2023-01-19T14:17:01Z

I had to push a change. After removing "wins_url" the image would build, but failed to join the cluster. Using "wins_url": "", fixes the joining issue.

joekr · 2023-01-19T14:44:43Z

/retest

jsturtevant · 2023-01-19T16:08:50Z

images/capi/packer/oci/scripts/set_bootstrap.sh

+
+echo "Changing Password in winrm_bootstrap.txt"
+
+sed "s/(\[adsi\].*/([adsi](\"WinNT:\/\/\"+\$opcUser.caption).replace(\"\\\\\",\"\/\")).SetPassword(\"$OPC_USER_PASSWORD\")/g" packer/oci/scripts/winrm_bootstrap.txt | tee packer/oci/scripts/winrm_bootstrap.txt >/dev/null


Is this going to leave the password in a file as plain text?

It will modify the password set in the winrm_bootstrap.txt to be set a launch. From my understanding I need to set a password at launch as our image process sets the password to automatically reset on launch and it won't be known.

Here is an old example related issue: hashicorp/packer#7033 (comment)

I'm not sure how to use the ENV to set the value in winrm_bootstrap.txt without saving it to the file for launch.

so this password is only used during bootstrapping but the password when a new image is created will be generated? If that is the case can we make a comment so that we know we are not leaking passwords in plain text here?

I have updated how this works now. When the image is built we temporarily set the password to the user provided password, when the image create the file containing the provided password is deleted. I've also updated the book to call that out.

joekr · 2023-01-23T22:52:02Z

/retest

jsturtevant · 2023-01-30T16:35:54Z

/lgtm
/approve

k8s-ci-robot · 2023-01-30T16:36:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: joekr, jsturtevant

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~docs/OWNERS~~ [jsturtevant]
~~images/capi/OWNERS~~ [jsturtevant]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jan 11, 2023

k8s-ci-robot requested review from jsturtevant and mboersma January 11, 2023 02:42

joekr commented Jan 11, 2023

View reviewed changes

images/capi/packer/oci/scripts/enable_second_nic.ps1 Show resolved Hide resolved

joekr force-pushed the CAPOCI_windows_support branch from e138a87 to 1150d54 Compare January 11, 2023 17:23

joekr marked this pull request as ready for review January 11, 2023 17:24

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 11, 2023

k8s-ci-robot requested a review from AverageMarcus January 11, 2023 17:24

joekr force-pushed the CAPOCI_windows_support branch from 1150d54 to 3914053 Compare January 11, 2023 17:28

k8s-ci-robot assigned jsturtevant Jan 12, 2023

jsturtevant reviewed Jan 13, 2023

View reviewed changes

joekr force-pushed the CAPOCI_windows_support branch from 3914053 to e445cba Compare January 16, 2023 22:35

joekr force-pushed the CAPOCI_windows_support branch from e445cba to 02d6674 Compare January 18, 2023 13:47

joekr force-pushed the CAPOCI_windows_support branch from 02d6674 to 325c8fb Compare January 19, 2023 14:15

jsturtevant reviewed Jan 19, 2023

View reviewed changes

joekr mentioned this pull request Jan 24, 2023

Add windows documentation oracle/cluster-api-provider-oci#206

Closed

Add support for Windows 2019/2022 to OCI capi provider image builder

25eba9c

joekr force-pushed the CAPOCI_windows_support branch from 325c8fb to 25eba9c Compare January 26, 2023 16:31

joekr requested review from jsturtevant and removed request for mboersma and AverageMarcus January 27, 2023 18:53

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 30, 2023

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 30, 2023

k8s-ci-robot merged commit 80fe0cf into kubernetes-sigs:master Jan 30, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for Windows 2019/2022 to OCI capi provider image builder #1051

Add support for Windows 2019/2022 to OCI capi provider image builder #1051

joekr commented Jan 11, 2023 •

edited

Loading

k8s-ci-robot commented Jan 11, 2023

jsturtevant commented Jan 12, 2023

jsturtevant Jan 13, 2023

joekr Jan 16, 2023 •

edited

Loading

jsturtevant Jan 17, 2023

joekr Jan 17, 2023

joekr Jan 18, 2023

jsturtevant commented Jan 17, 2023

jsturtevant commented Jan 18, 2023

joekr commented Jan 19, 2023 •

edited

Loading

joekr commented Jan 19, 2023

jsturtevant Jan 19, 2023

joekr Jan 19, 2023

jsturtevant Jan 24, 2023

joekr Jan 26, 2023

joekr commented Jan 23, 2023

jsturtevant commented Jan 30, 2023

k8s-ci-robot commented Jan 30, 2023


		echo "Changing Password in winrm_bootstrap.txt"

		sed "s/(\[adsi\].*/([adsi](\"WinNT:\/\/\"+\$opcUser.caption).replace(\"\\\\\",\"\/\")).SetPassword(\"$OPC_USER_PASSWORD\")/g" packer/oci/scripts/winrm_bootstrap.txt \| tee packer/oci/scripts/winrm_bootstrap.txt >/dev/null

Add support for Windows 2019/2022 to OCI capi provider image builder #1051

Add support for Windows 2019/2022 to OCI capi provider image builder #1051

Conversation

joekr commented Jan 11, 2023 • edited Loading

k8s-ci-robot commented Jan 11, 2023

jsturtevant commented Jan 12, 2023

Choose a reason for hiding this comment

joekr Jan 16, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jsturtevant commented Jan 17, 2023

jsturtevant commented Jan 18, 2023

joekr commented Jan 19, 2023 • edited Loading

joekr commented Jan 19, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joekr commented Jan 23, 2023

jsturtevant commented Jan 30, 2023

k8s-ci-robot commented Jan 30, 2023

joekr commented Jan 11, 2023 •

edited

Loading

joekr Jan 16, 2023 •

edited

Loading

joekr commented Jan 19, 2023 •

edited

Loading