Migrate to Solaris CBE for updated packages, don't use OpenCSW packages

[bradh352]

This should resolve vmactions/solaris-vm#38 as well as vmactions/solaris-vm#39

I tried using the original ISO, but that ISO doesn't support virtio so discards/trim don't work and the image size ballooned.

I've now switched to the CBE image.

IMPORTANT: This ISO image needs to be copied into a release artifact within vmactions/solaris-builder and the link needs to be updated in conf/solaris-11.4-gcc.conf and conf/solaris-11.4.conf to point to the new location. My fork will be deleted once this gets merged and therefore so will the current download location.

Due to concerns with packages that are in OpenCSW but not in the official solaris repository, the OpenCSW package manager is still installed but it is not actually used, its meant for end users if they need it.

Changes:

• Use ubuntu-latest as the build image
• Use jlumbroso/free-disk-space@main to clean up some host VM space so we don't run out
• Switch to more modern Solaris CBE ISO
• Install packages from the Solaris repository instead of OpenCSW (uploaded now in 0.0.4 release assests)
• Use virtio for the VM disk bus, and pass the discard=unmap option
• Enable trim support in ZFS

Updates needed to https://github.com/vmactions/solaris-vm/:

• Update BUILDER_VERSION in conf/11.4-gcc.conf and conf/11.4.conf
• Add some sort of logic to potentially have to download multiple parts and combine them back together (the -cbe-gcc build is larger than 2G now, its split into .qcow2.xz.0 and .qcow2.xz.1)

[whoschek]

Does this preserve the ability to install zstd and mbuffer like so? https://github.com/whoschek/wbackup-zfs/blob/a34b5f470482303540ff1c301962b3705ce428da/.github/workflows/python-app.yml#L281?

In the current solaris VM image zstd and mbuffer are only available from OpenCSW. I don't care whether it comes from pkgutil or pkg as long as it can be easily installed from somewhere.

[bradh352]

I'd assume we could do both I guess since zstd nor mbuffer are in Solaris's package list. At this point I'm trying to get the image size down, this might all be moot if I can't.

[bradh352]

I don't think I can iterate on this any more. The image size for the gcc version is just slightly too large to fit within the 2G limit so I had to create a way to split the result.

Here's the result of the last run:
https://github.com/bradh352/vmactions-solaris-builder/actions/runs/10022448221

You can see:

-rw-r--r-- 1 runner docker 1.5G Jul 20 19:57 solaris-11.4.qcow2.xz

-rw-r--r-- 1 runner docker 2.0G Jul 20 20:58 solaris-11.4-gcc.qcow2.xz.0
-rw-r--r-- 1 runner docker  19M Jul 20 20:58 solaris-11.4-gcc.qcow2.xz.1

Maybe since a split is required and it would need to be handled in vmactions/solaris-vm anyhow, it should be viewed as an opportunity to install more packages on the -gcc build.
I'd suggest these packages which would help cover a lot more use-cases: developer/gcc/gcc-c++ gnu-make cmake ninja pkg-config git

[bradh352]

I decided to go ahead and update the package list for the 'gcc' build to include the other recommended packages. Having a .xz.1 as 19MB is a bit weird, so makes sense to just go ahead and let it be a couple hundred MB and include useful stuff.

[Neilpang]

don't change the build.yml here, it was generated from the template from base builder:
https://github.com/vmactions/base-builder/blob/main/.github/tpl/build.tpl.yml

Don't change the Readme.md, it's auto generated.

The build.sh is also generated from base builder:

https://github.com/vmactions/base-builder/blob/main/build.sh

And it's shared by all the other builders, so if you need to change the base, be careful.

The project is generated/synced from the base builder by a workflow:

https://github.com/vmactions/solaris-builder/blob/main/.github/workflows/generate.yml

[bradh352]

hrm, that definitely complicates things a bit then. I didn't realize there was so much cross-dependency stuff going on. I'm not sure how to handle changing the build.sh as there is a lot we need to do in solaris that might not be needed anywhere else. Maybe adding a new step of "hooks/sysUpdate.sh" that if it exists runs, then reboots before running "hooks/postBuild.sh" would meet the needs of this.

The build.yml changes however I think make sense to be merged into the base as those are pretty sane.

The readme actually got updated automatically when I created a tag in my own repo so I didn't directly do that.

I'll try to submit a PR for base-builder, and we'll go from there. Just will make it a multi-step process since i'll need that merged before I can update this PR properly.

[bradh352]

@Neilpang Does this looks reasonable? vmactions/base-builder#1

If so, it should be pretty easy to move stuff around for solaris into the sysUpdate.sh and postBuild.sh hooks which I think will be compliant with what you're looking for.

[bradh352]

https://www.oracle.com/solaris/solaris11/downloads/solaris-downloads.html#

[bradh352]

but like I said I have it saved here: https://github.com/bradh352/vmactions-solaris-builder/releases/download/2.0.0/solaris-11.4.42.111.0-x86-cbe.iso

[bradh352]

re-synced with latest changes ...

[bradh352]

NOTE: also needs changes in vmactions/base-builder#1

[bradh352]

I made the changes you requested.

I should note that I don't think there's any reason to keep the non-CBE-based builds at all. Not sure if anyone would find such an old build useful at all.

The other alternative would be for the non-CBE builds to actually be upgraded, which when I tested it was possible, but the qcow2 image sizes really got huge because the ide driver doesn't support discard/trim. I didn't investigate if it was possible to switch the disk type post-upgrade to virtio and then try to evict the data and compact the qcow2 image. (virtio was added in 11.4.33).

Finally, the non-CBE image probably shouldn't be used at all from a licensing perspective. The CBE image is meant to be free for developers, the other image you have I'm not sure what the licensing is.

[bradh352]

last build log from my fork with these changes: https://github.com/bradh352/vmactions-solaris-builder/actions/runs/10773345217

All appears successful.

[bradh352]

@Neilpang thanks for merging vmactions/base-builder#1 ... just need this merged now :)

[Neilpang]

don't change the build.yml here, it was generated from the template from base builder.

I think we can just replace the default image with CBE image.
So, there would be only "11.4" and "11.4-gcc", which both use the lastest CBE iso to build.

you don't need cbe-gcc any more.

[bradh352]

don't change the build.yml here, it was generated from the template from base builder.

I think we can just replace the default image with CBE image. So, there would be only "11.4" and "11.4-gcc", which both use the lastest CBE iso to build.

you don't need cbe-gcc any more.

yeah, i know build.yml is generated, however i have to have it committed to actually build and test in my fork which is why its committed. I'll make the change to only use CBE at least. it won't hurt to have the build.yml committed, however, since your build script will just regenerate it base-builder when its in the right place.

[bradh352]

changes committed, building now in my fork : https://github.com/bradh352/vmactions-solaris-builder/actions/runs/10792803832