Skip to content

Commit

Permalink
Add support for Windows 2019/2022 to OCI capi provider image builder
Browse files Browse the repository at this point in the history
  • Loading branch information
joekr committed Jan 26, 2023
1 parent 0b8a9e2 commit 25eba9c
Show file tree
Hide file tree
Showing 17 changed files with 439 additions and 12 deletions.
39 changes: 38 additions & 1 deletion docs/book/src/capi/providers/oci.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ the different operating systems.
| `ubuntu-1804.json` | The settings for the Ubuntu 18.04 image |
| `ubuntu-2004.json` | The settings for the Ubuntu 20.04 image |
| `ubuntu-2204.json` | The settings for the Ubuntu 22.04 image |

| `windows-2019.json` | The settings for the Windows Server 2019 image |
| `windows-2022.json` | The settings for the Windows Server 2022 image |

#### Common options

Expand All @@ -42,6 +43,7 @@ list, and greater explanation can be found in the

| Variable | Description | Default | Mandatory |
|----------|-------------|---------|---------|
| `base_image_ocid` | The OCID of an existing image to build upon. | | No |
| `compartment_ocid` | The OCID of the compartment that the instance will run in. | | Yes |
| `subnet_ocid` | The OCID of the subnet within which a new instance is launched and provisioned. | | Yes |
| `availability_domain` | The name of the Availability Domain within which a new instance is launched and provisioned. The names of the Availability Domains have a prefix that is specific to your tenancy. | | Yes |
Expand Down Expand Up @@ -83,4 +85,39 @@ is defined in images/capi/packer/config/containerd.json.
"kubernetes_rpm_repo": "https://packages.cloud.google.com/yum/repos/kubernetes-el7-aarch64",
"containerd_sha256": "9ac616b5f23c1d10353bd45b26cb736efa75dfef31a2113baff2435dbc7becb8"
}
```

### Building a Windows image

> NOTE: In order to use Windows with CAPI a Baremetal instance is required. This means a Baremetal instance is required for building the image as well. The OCIDs for the 2019
and 2022 Datacenter edition of Windows can be found in their respective documentation:
>
> - [Windows server 2019](https://docs.oracle.com/en-us/iaas/images/windows-server-2019-bm/)
> - [Windows server 2022](https://docs.oracle.com/en-us/iaas/images/windows-server-2022-bm/)
#### Windows environment variables

| Variable | Description | Default | Mandatory |
|----------|-------------|---------|---------|
| `OPC_USER_PASSWORD` | The password to set the OPC user to when creating the image. This will be used for accessing instances using this image. | | Yes |

> NOTE: The `OPC_USER_PASSWORD` will be set in the `winrm_bootstrap.txt` file temporarily, while building the image.
This is required in order for winrm to access the instance building the image. Once the build process is complete
the password will be deleted along with the fil so the password isn't stored long term in a cleartext file.

#### Build a Windows based image

The following example json would use the [Windows Server 2019 Datacenter Edition BM E4 image in the us-ashburn-1 region](https://docs.oracle.com/en-us/iaas/images/image/4d56c93a-2165-49b0-9c6e-f9e9a9b05011/).

```json
{
"build_name": "windows",
"base_image_ocid": "<image_OCID>",
"ocpus": "128",
"shape": "BM.Standard.E4.128",
"region": "us-ashburn-1",
"compartment_ocid": "Fill compartment OCID here",
"subnet_ocid": "Fill Subnet OCID here",
"availability_domain": "Fill Availability Domain here"
}
```
10 changes: 8 additions & 2 deletions images/capi/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,7 @@ AZURE_BUILD_VHD_NAMES ?= $(addprefix azure-vhd-,$(VHD_TARGETS))
AZURE_BUILD_SIG_NAMES ?= $(addprefix azure-sig-,$(SIG_TARGETS))
AZURE_BUILD_SIG_GEN2_NAMES ?= $(addsuffix -gen2,$(addprefix azure-sig-,$(SIG_GEN2_TARGETS)))

OCI_BUILD_NAMES ?= oci-ubuntu-1804 oci-ubuntu-2004 oci-ubuntu-2204 oci-oracle-linux-8 oci-oracle-linux-9
OCI_BUILD_NAMES ?= oci-ubuntu-1804 oci-ubuntu-2004 oci-ubuntu-2204 oci-oracle-linux-8 oci-oracle-linux-9 oci-windows-2019 oci-windows-2022

DO_BUILD_NAMES ?= do-centos-7 do-ubuntu-1804 do-ubuntu-2004

Expand Down Expand Up @@ -485,7 +485,9 @@ $(RAW_VALIDATE_TARGETS): deps-raw

.PHONY: $(OCI_BUILD_TARGETS)
$(OCI_BUILD_TARGETS): deps-oci
packer build $(PACKER_NODE_FLAGS) -var-file="$(abspath packer/oci/$(subst build-oci-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/oci/packer.json
$(if $(findstring windows,$@),./packer/oci/scripts/set_bootstrap.sh,)
packer build $(if $(findstring windows,$@),$(PACKER_WINDOWS_NODE_FLAGS),$(PACKER_NODE_FLAGS)) -var-file="$(abspath packer/oci/$(subst build-oci-,,$@).json)" $(ABSOLUTE_PACKER_VAR_FILES) packer/oci/packer$(findstring -windows,$@).json
$(if $(findstring windows,$@),./packer/oci/scripts/unset_bootstrap.sh,)

.PHONY: $(OCI_VALIDATE_TARGETS)
$(OCI_VALIDATE_TARGETS): deps-oci
Expand Down Expand Up @@ -678,6 +680,8 @@ build-oci-ubuntu-2004: ## Builds the OCI ubuntu-2004 image
build-oci-ubuntu-2204: ## Builds the OCI ubuntu-2204 image
build-oci-oracle-linux-8: ## Builds the OCI Oracle Linux 8.x image
build-oci-oracle-linux-9: ## Builds the OCI Oracle Linux 9.x image
build-oci-windows-2019: ## Builds the OCI Windows Server 2019 image
build-oci-windows-2022: ## Builds the OCI Windows Server 2022 image
build-oci-all: $(OCI_BUILD_TARGETS) ## Builds all OCI image

build-osc-ubuntu-2004: ## Builds Ubuntu 20.04 Outscale Snapshot
Expand Down Expand Up @@ -790,6 +794,8 @@ validate-oci-ubuntu-2004: ## Validates the OCI ubuntu-2004 image packer config
validate-oci-ubuntu-2204: ## Validates the OCI ubuntu-2204 image packer config
validate-oci-oracle-linux-8: ## Validates the OCI Oracle Linux 8.x image packer config
validate-oci-oracle-linux-9: ## Validates the OCI Oracle Linux 9.x image packer config
validate-oci-windows-2019: ## Validates the OCI Windows 2019 image packer config
validate-oci-windows-2022: ## Validates the OCI Windows 2022 image packer config
validate-oci-all: $(OCI_VALIDATE_TARGETS) ## Validates all OCI image packer config

validate-osc-ubuntu-2004: ## Validates Ubuntu 20.04 Outscale Snapshot Packer config
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -32,8 +32,5 @@ ephemeral_disk_volume_label="Temporary Storage"
netbios_host_name_compatibility={{ netbios_host_name_compatibility }}

metadata_services={{ cloudbase_metadata_services }}
plugins=cloudbaseinit.plugins.windows.createuser.CreateUserPlugin,
cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin,
cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin,
cloudbaseinit.plugins.common.userdata.UserDataPlugin,
plugins=cloudbaseinit.plugins.common.userdata.UserDataPlugin,
{{ cloudbase_plugins }}
2 changes: 1 addition & 1 deletion images/capi/packer/ami/packer-windows.json
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@
"cloudbase_init_url": "https://github.com/cloudbase/cloudbase-init/releases/download/{{user `cloudbase_init_version`}}/CloudbaseInitSetup_{{user `cloudbase_init_version` | replace_all `.` `_` }}_x64.msi",
"cloudbase_metadata_services": "cloudbaseinit.metadata.services.ec2service.EC2Service",
"cloudbase_metadata_services_unattend": "cloudbaseinit.metadata.services.base.EmptyMetadataService",
"cloudbase_plugins": "cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin, cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin",
"cloudbase_plugins": "cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin, cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin, cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin",
"cloudbase_plugins_unattend": "cloudbaseinit.plugins.common.mtu.MTUPlugin",
"containerd_sha256": null,
"containerd_url": "",
Expand Down
2 changes: 1 addition & 1 deletion images/capi/packer/azure/packer-windows.json
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@
"cloudbase_logging_serial_port": "COM2,115200,N,8",
"cloudbase_metadata_services": "cloudbaseinit.metadata.services.azureservice.AzureService",
"cloudbase_metadata_services_unattend": "cloudbaseinit.metadata.services.base.EmptyMetadataService",
"cloudbase_plugins": "cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.windows.azureguestagent.AzureGuestAgentPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin",
"cloudbase_plugins": "cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin, cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.windows.azureguestagent.AzureGuestAgentPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin",
"cloudbase_plugins_unattend": "cloudbaseinit.plugins.common.mtu.MTUPlugin",
"community_gallery_image_id": "",
"containerd_url": "",
Expand Down
2 changes: 1 addition & 1 deletion images/capi/packer/nutanix/packer-windows.json
Original file line number Diff line number Diff line change
Expand Up @@ -120,7 +120,7 @@
"cloudbase_init_url": "https://github.com/cloudbase/cloudbase-init/releases/download/{{user `cloudbase_init_version`}}/CloudbaseInitSetup_{{user `cloudbase_init_version` | replace_all `.` `_` }}_x64.msi",
"cloudbase_metadata_services": "cloudbaseinit.metadata.services.configdrive.ConfigDriveService",
"cloudbase_metadata_services_unattend": "cloudbaseinit.metadata.services.base.EmptyMetadataService",
"cloudbase_plugins": "cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin, cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin, cloudbaseinit.plugins.common.userdata.UserDataPlugin, cloudbaseinit.plugins.common.localscripts.LocalScriptsPlugin, cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin",
"cloudbase_plugins": "cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.common.setuserpassword.SetUserPasswordPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin, cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin, cloudbaseinit.plugins.common.sshpublickeys.SetUserSSHPublicKeysPlugin, cloudbaseinit.plugins.common.userdata.UserDataPlugin, cloudbaseinit.plugins.common.localscripts.LocalScriptsPlugin, cloudbaseinit.plugins.windows.createuser.CreateUserPlugin, cloudbaseinit.plugins.windows.extendvolumes.ExtendVolumesPlugin",
"cloudbase_plugins_unattend": "cloudbaseinit.plugins.common.mtu.MTUPlugin",
"containerd_sha256": null,
"containerd_url": "",
Expand Down
148 changes: 148 additions & 0 deletions images/capi/packer/oci/packer-windows.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
{
"builders": [
{
"availability_domain": "{{user `availability_domain`}}",
"base_image_filter": {
"operating_system": "{{user `operating_system`}}",
"operating_system_version": "{{user `operating_system_version`}}"
},
"base_image_ocid": "{{user `base_image_ocid`}}",
"communicator": "winrm",
"compartment_ocid": "{{user `compartment_ocid`}}",
"image_name": "cluster-api-{{user `build_name`}}-{{user `kubernetes_semver`}}-{{user `build_timestamp`}}",
"shape": "{{user `shape`}}",
"shape_config": {
"ocpus": "{{user `ocpus`}}"
},
"subnet_ocid": "{{user `subnet_ocid`}}",
"tenancy_ocid": "{{user `tenancy_ocid`}}",
"type": "oracle-oci",
"user_data_file": "packer/oci/scripts/winrm_bootstrap.txt",
"user_ocid": "{{user `user_ocid`}}",
"winrm_insecure": true,
"winrm_password": "{{user `opc_user_password`}}",
"winrm_port": 5986,
"winrm_timeout": "10m",
"winrm_use_ntlm": true,
"winrm_use_ssl": true,
"winrm_username": "opc"
}
],
"post-processors": [
{
"custom_data": {
"build_date": "{{isotime}}",
"build_name": "{{user `build_name`}}",
"build_timestamp": "{{user `build_timestamp`}}",
"build_type": "node",
"containerd_version": "{{user `containerd_version`}}",
"kubernetes_cni_semver": "{{user `kubernetes_cni_semver`}}",
"kubernetes_semver": "{{user `kubernetes_semver`}}",
"kubernetes_source_type": "{{user `kubernetes_source_type`}}",
"os_name": "{{user `distro_name`}}",
"resource_group_name": "{{user `resource_group_name`}}",
"storage_account_name": "{{user `storage_account_name`}}"
},
"output": "{{user `manifest_output`}}",
"strip_path": true,
"type": "manifest"
}
],
"provisioners": [
{
"elevated_password": "{{.WinRMPassword}}",
"elevated_user": "opc",
"script": "ansible/windows/ansible_winrm.ps1",
"type": "powershell"
},
{
"extra_arguments": [
"-e",
"ansible_winrm_server_cert_validation=ignore ansible_winrm_operation_timeout_sec=120 ansible_winrm_read_timeout_sec=150",
"--extra-vars",
"{{user `ansible_common_vars`}}",
"--extra-vars",
"{{user `azure_extra_vars`}}",
"--extra-vars",
"{{user `ansible_extra_vars`}}",
"--extra-vars",
"{{user `ansible_user_vars`}}"
],
"max_retries": 5,
"pause_before": "15s",
"playbook_file": "ansible/windows/node_windows.yml",
"type": "ansible",
"use_proxy": false,
"user": "opc"
},
{
"restart_timeout": "10m",
"type": "windows-restart"
},
{
"destination": "C:\\Users\\opc\\",
"source": "./packer/oci/scripts/sysprep.ps1",
"type": "file"
},
{
"destination": "C:\\Users\\opc\\",
"source": "./packer/oci/scripts/attach_secondary_vnic.ps1",
"type": "file"
},
{
"destination": "C:\\Windows\\Setup\\Scripts\\",
"source": "./packer/oci/scripts/enable_second_nic.ps1",
"type": "file"
},
{
"inline": [
"rm -Force -Recurse C:\\var\\log\\kubelet\\*"
],
"type": "powershell"
},
{
"elevated_password": "{{.WinRMPassword}}",
"elevated_user": "opc",
"inline": [
"C:\\Users\\opc\\sysprep.ps1"
],
"type": "powershell"
}
],
"variables": {
"additional_debug_files": null,
"ansible_common_vars": "",
"ansible_extra_vars": "",
"ansible_user_vars": "",
"azure_extra_vars": "wire_server_users={{user `wire_server_users`}}",
"build_name": null,
"build_timestamp": "{{timestamp}}",
"cloudbase_init_url": "https://github.com/cloudbase/cloudbase-init/releases/download/{{user `cloudbase_init_version`}}/CloudbaseInitSetup_{{user `cloudbase_init_version` | replace_all `.` `_` }}_x64.msi",
"cloudbase_logging_serial_port": "COM1,9600,N,8",
"cloudbase_metadata_services": "cloudbaseinit.metadata.services.httpservice.HttpService",
"cloudbase_metadata_services_unattend": "cloudbaseinit.metadata.services.httpservice.HttpService",
"cloudbase_plugins": "cloudbaseinit.plugins.common.ephemeraldisk.EphemeralDiskPlugin, cloudbaseinit.plugins.common.mtu.MTUPlugin, cloudbaseinit.plugins.common.sethostname.SetHostNamePlugin",
"cloudbase_plugins_unattend": "cloudbaseinit.plugins.common.mtu.MTUPlugin",
"containerd_url": "",
"containerd_version": null,
"ib_version": "{{env `IB_VERSION`}}",
"image_version": "latest",
"kubernetes_base_url": "https://kubernetesreleases.blob.core.windows.net/kubernetes/{{user `kubernetes_semver`}}/binaries/node/windows/{{user `kubernetes_goarch`}}",
"manifest_output": "manifest.json",
"nssm_url": null,
"ocpus": "2",
"opc_user_password": "{{env `OPC_USER_PASSWORD`}}",
"prepull": null,
"private_virtual_network_with_public_ip": "",
"runtime": "containerd",
"shape": "VM.Standard.E4.Flex",
"virtual_network_name": "",
"virtual_network_resource_group_name": "",
"virtual_network_subnet_name": "",
"vm_size": "",
"windows_service_manager": null,
"windows_updates_kbs": null,
"wins_url": "",
"wire_server_users": ""
}
}
39 changes: 39 additions & 0 deletions images/capi/packer/oci/scripts/attach_secondary_vnic.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
function Get-Second-Vnic-Ocid() {
$ocid = ""
$vnics = Invoke-RestMethod -Uri "http://169.254.169.254/opc/v1/vnics/"
if ($vnics.Count -eq 2) {
$ocid = $vnics[1].vnicId
} else {
Write-Host "vnics count not equal 2"
}
return $ocid
}

$vnicId = Get-Second-Vnic-Ocid
Write-Host "found vnic id: ${vnicId}"


$retryDelaySeconds = 30
# We should continue to retry indefinitely until the vnic is
# detected by IMDS
# https://docs.oracle.com/en-us/iaas/Content/Compute/Tasks/gettingmetadata.htm
while($vnicId -eq "") {
$vnicId = Get-Second-Vnic-Ocid
Write-Host("Getting second vnic failed. Waiting " + $retryDelaySeconds + " seconds before next attempt.")
Start-Sleep -Seconds $retryDelaySeconds
}

if ($vnicId -ne "") {
Write-Host "Pulling down the secondary_vnic_windows_configure.ps1"
Invoke-WebRequest -Uri "https://docs.oracle.com/en-us/iaas/Content/Resources/Assets/secondary_vnic_windows_configure.ps1" -OutFile "C:\Users\opc\secondary_vnic_windows_configure.ps1"

Write-Host "calling script using ${vnicId}"

, 'Y', 'A' | powershell "C:\Users\opc\secondary_vnic_windows_configure.ps1 ${vnicId}"
Write-Error "secondary_vnic_windows_configure.ps1 - done"

$ipconfig = ipconfig
Write-Error "${ipconfig}"
}else{
Write-Error "VNIC OCID is empty. Can't configure."
}
47 changes: 47 additions & 0 deletions images/capi/packer/oci/scripts/enable_second_nic.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@

$newNetAdapterName = "Ethernet 2"

# check for two nics
$netAdapters = Get-NetAdapter
if ($netAdapters.Length -le 1) {
Write-Output "Could not find multiple Network Adapters."
Exit 1
}

$secondNic = $netAdapters[1]

# make sure the network adapter is known
if ($secondNic.Name -ne "") {
Write-Output "Changing ${secondNic.Name} to ${newNetAdapterName} ..."
try
{
Rename-NetAdapter -Name $secondNic.Name -NewName "${newNetAdapterName}"
$secondNic.Name = $newNetAdapterName
}
catch
{
Write-Output "Could not rename net adapter"
Write-Output $_
}
} else {
Write-Output "Can not change network adapter named: ${secondNic.Name}"
}

# check that second is disabled
if ($secondNic.Status -ne "up") {

try
{
Enable-NetAdapter -Name $secondNic.Name
Write-Output "${secondNic.Name} enabled ..."
}
catch
{
Write-Output "Could not enable net adapter"
Write-Output $_
}
} else {
Write-Output "${secondNic.Name} already enabled ..."
}

Remove-Item -Path .\enable_second_nic.ps1
Loading

0 comments on commit 25eba9c

Please sign in to comment.