Handle latest iso's not being immutable #1334

AverageMarcus · 2023-10-27T13:11:26Z

Is your feature request related to a problem? Please describe.

For our providers that build based on top of an iso image we've seen a problem several times now with Ubuntu (maybe others?) where the latest release is only available as a non-specific URL that is updated as new releases are made.

For example, Ubuntu 23.04 is currently (at time of writing) only available at the following location:

https://releases.ubuntu.com/23.04/ubuntu-23.04-live-server-amd64.iso

When a new release of Ubuntu 23.04 is made, this image is replaced but the URL remains the same. There isn't a location to get the specific patch version of the release that we've been able to find.

This causes the following problems:

checksums become out of date so we need to keep updating them even though the URL isn't changing
breaking changes could possibly be introduced into a new release of Ubuntu that could break for users of image-builder without there being any changes made to image-builder or the provided variables. This means image-builder can't guarentee reproducible builds.

Describe the solution you'd like

Ideally there would be a location where we can get the patch versions of Ubuntu images but I'm not aware of such a place existing.

Describe alternatives you've considered

We could fetch the checksum on the fly from the https://releases.ubuntu.com/23.04/SHA256SUMS file but this doesn't solve the reproducibility issue.

Additional context

For old releases, they become available at a different URL (e.g. https://old-releases.ubuntu.com/releases/jammy/ubuntu-22.04.2-live-server-amd64.iso) but the same doesn't seem to be possible with the latest patch release.

/kind feature

The text was updated successfully, but these errors were encountered:

k8s-triage-robot · 2024-01-31T07:38:16Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

AverageMarcus · 2024-01-31T10:01:26Z

/remove-lifecycle stale

AverageMarcus · 2024-04-05T07:42:53Z

A little more investigation...

The current latest Ubuntu release is available at:

https://releases.ubuntu.com/22.04.4/ubuntu-22.04.4-live-server-amd64.iso

With curl:

curl https://releases.ubuntu.com/22.04.4/ubuntu-22.04.4-live-server-amd64.iso
HTTP/1.1 200 OK
Date: Fri, 05 Apr 2024 07:23:52 GMT
Server: Apache/2.4.29 (Ubuntu)
Last-Modified: Fri, 16 Feb 2024 23:52:36 GMT
ETag: "7d6eb800-61188703a4abd"
Accept-Ranges: bytes
Content-Length: 2104408064
Content-Type: application/x-iso9660-image

+-----------------------------------------+
| NOTE: binary data not shown in terminal |
+-----------------------------------------+

The previous Ubuntu release can be fetched using:

https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso

But this actually returns a HTTP 302 redirect to https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso

With curl:

curl https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso">here</a>.</p>
<hr>
<address>Apache/2.4.29 (Ubuntu) Server at releases.ubuntu.com Port 443</address>
</body></html>
HTTP/1.1 301 Moved Permanently
Date: Fri, 05 Apr 2024 07:46:28 GMT
Server: Apache/2.4.29 (Ubuntu)
Location: https://old-releases.ubuntu.com/releases/22.04.2/ubuntu-22.04.2-live-server-amd64.iso
Content-Length: 379
Content-Type: text/html; charset=iso-8859-1

I'm going to check if Packer correctly follows these redirects. If so, this should be resolved as long as we use the full version in the URL.

If that doesn't work, I have discovered that at least some Packer builders support a iso_urls property in place of iso_url which can be used to support multiple URLs. This will try the next in the list if one fails. We could use this to supply both the releases.ubuntu.com and the old-releases.ubuntu.com URLs so it should fallback when needed.

AverageMarcus · 2024-04-05T07:54:24Z

I just confirmed that this works by using the following vars with make build-qemu-ubuntu-2204:

{
  "boot_command_prefix": "c<wait>linux /casper/vmlinuz --- autoinstall ds='nocloud-net;s=http://{{ .HTTPIP }}:{{ .HTTPPort }}/22.04/'<enter><wait><wait><wait>initrd /casper/initrd<enter><wait><wait><wait>boot<enter>",
  "build_name": "ubuntu-2204",
  "distribution_version": "2204",
  "distro_name": "ubuntu",
  "guest_os_type": "ubuntu-64",
  "iso_checksum": "5e38b55d57d94ff029719342357325ed3bda38fa80054f9330dc789cd2d43931",
  "iso_checksum_type": "sha256",
  "iso_url": "https://releases.ubuntu.com/22.04.2/ubuntu-22.04.2-live-server-amd64.iso",
  "os_display_name": "Ubuntu 22.04",
  "shutdown_command": "shutdown -P now",
  "unmount_iso": "true"
}

So as long as we use the https://releases.ubuntu.com/XX.XX.X/ style URLs (needs the patch value) we should be ok.

AverageMarcus · 2024-04-05T08:01:04Z

It seems this is only true of the recent LTS releases and doesn't apply to 23.04 yet as it's still in beta.

AverageMarcus · 2024-06-17T06:50:11Z

😩

So, the patch version URLs are only available from the first patch version. The initial release of Ubuntu 24.04 (e.g. 24.04.0) doesn't have an immutable URL and is only available from the one that points to the latest for that version. 😞

k8s-triage-robot · 2024-09-15T07:08:35Z

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

After 90d of inactivity, lifecycle/stale is applied
After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

Mark this issue as fresh with /remove-lifecycle stale
Close this issue with /close
Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

AverageMarcus · 2024-09-15T07:34:21Z

/lifecycle frozen

k8s-ci-robot added the kind/feature Categorizes issue or PR as related to a new feature. label Oct 27, 2023

AverageMarcus mentioned this issue Oct 27, 2023

QEMU RockyLinux 9 and Ubuntu 23.04 #1332

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 31, 2024

k8s-ci-robot removed the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Jan 31, 2024

AverageMarcus mentioned this issue Apr 5, 2024

Update Ubuntu 22.04 ISO URLs to use latest stable URLs #1438

Merged

AverageMarcus mentioned this issue Jun 17, 2024

Add Ubuntu 24.04 for QEMU #1491

Merged

This was referenced Sep 5, 2024

Ubuntu 22.04 redirects to 22.04.1 - proxmox json needs update #1560

Closed

Update Ubuntu 24.04 to latest (pinned) ISO #1561

Merged

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Sep 15, 2024

k8s-ci-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Sep 15, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Handle latest iso's not being immutable #1334

Handle latest iso's not being immutable #1334

AverageMarcus commented Oct 27, 2023

k8s-triage-robot commented Jan 31, 2024

AverageMarcus commented Jan 31, 2024

AverageMarcus commented Apr 5, 2024 •

edited

Loading

AverageMarcus commented Apr 5, 2024

AverageMarcus commented Apr 5, 2024

AverageMarcus commented Jun 17, 2024

k8s-triage-robot commented Sep 15, 2024

AverageMarcus commented Sep 15, 2024

Handle latest iso's not being immutable #1334

Handle latest iso's not being immutable #1334

Comments

AverageMarcus commented Oct 27, 2023

k8s-triage-robot commented Jan 31, 2024

AverageMarcus commented Jan 31, 2024

AverageMarcus commented Apr 5, 2024 • edited Loading

AverageMarcus commented Apr 5, 2024

AverageMarcus commented Apr 5, 2024

AverageMarcus commented Jun 17, 2024

k8s-triage-robot commented Sep 15, 2024

AverageMarcus commented Sep 15, 2024

AverageMarcus commented Apr 5, 2024 •

edited

Loading