-
Notifications
You must be signed in to change notification settings - Fork 31
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Offline image configuration #534
Comments
Besides figuring out how to pass the mount path:
|
I think resources that support I think we would want a different model that requires starting the image, configure, and shutting it down and not mix it with actual offline configuration. My initial thinking is that DSC would include a type: Microsoft.DSC/OfflineGroup
properties:
imagePath: <path to vhd/vhdx to mount>
resources: <array of resources that support `Offline` mode> This group resource would be responsible for:
For Windows, |
Thinking about this further, one of the problems is that resource authors won't think in terms of online/offline, but I suspect many resources can work against a file path. So a simpler proposal would be a type: Microsoft.Windows/DiskImage
properties:
action: <mount|unmount>
imagePath: <file path to vhd/vhdx/wim>
mountPath: <mounted path> So this resource simply takes care of mounting or unmounting an image (basically replicating the Then resources like The main downside to this approach is the configuration needs explicit unmount, but this approach would be easier to implement. |
My teammates and I talked yesterday---let me see if I can capture what we discussed & my thoughts. Our problemWe often debug kernel-level issues that prevent the system from booting, so we need to ensure that KD is configured before the device ever boots. We currently do that (in IXPTools - internal only) with a bevy of PowerShell scripts that mount our VHDs, mount their registries, change some files, and then boot them (and then run some subsequent offline scripts after unattend OOBE finishes). There are a few other benefits to offline configuration---it's static & easy to verify, it's much faster than waiting for OOBE to finish, it can access stuff on the host machine (especially resources that live on our VPN)---but the main requirement is to have tools like KD and SSH guaranteed running before the device boots. For physical devices, we then choose to boot from VHD. Also: bootstrappingOne other frustration is that Windows automation is not self-bootstrapping: there is no way to configure a Windows installation to be connectable (at least publicly) at first boot. With TShell, we can install a .cab offline, but there's no equivalent for SSH (even though I can deploy the certs offline, I can't For example, Docker setups rely on substantial amounts of scripting and Also: device-agnosticityWe'd also like any solution to be device-agnostic: we'd love to be able to run configuration on existing devices and offline ones. For example, we have many scripts that do something like this:
Since we haven't generalized the offline step, if someone wants to run that script on an existing machine, our answer is basically, "too bad. Create a new VM." Which sucks. A new option for offline provisioningYou proposed these options in previous comments:
We want to propose a 3rd option:
This keeps the YAML files device-agnostic: you can run the same provisioning on an online or offline device. This is in keeping with the overall DSC v3 philosophy: nominally a Exploring offline mode (
|
@citelao some good thinking there. I like the As you mention, I don't think we want to turn The only other major point we should consider before closing towards a design (even if we don't implement it now) is for non-Windows. For example, would it be useful to use |
I'm glad you resonate with the
The only reason I'd push back on this is bootstrapping: if Compare this to Ansible: yes, I need to set up SSH on my remote machines first, but I can run everything else through Ansible itself. Versus the current approach of
Agreed. Offline provisioning seems like an obvious benefit to all platforms. But it strikes me that I couldn't find any real competitive art here. Is it because Linux boots so quickly that online provisioning (boot VM, make changes, shut down) is fast enough? TBH, I'm not sure how Also, I'm sure you knew this, but I looked into the servicing |
@citelao - just a quick note on terminology we often use that may not be clear, but when we're referencing "higher level solutions" or "higher order tools," we're talking about things like Ansible (and Machine Configuration, Puppet, Chef, Salt, etc) - DSCv3 is designed, like PSDSC before it, to provide a highly effective way to design, implement, and call resources that those tools can readily integrate with. There's an exponential complexity problem when designing configuration management tooling, and that problem grows large enough that there are multiple companies with hundreds of people trying to solve it. That's not to say that DSC shouldn't be bootstrap-aware, or that remote execution is strictly out-of-bounds, but that DSC isn't trying to replicate Machine Configuration or the enterprise offerings of other configuration management tools. But there are some domains where investing in them in DSC is more costly for less benefit than ensuring higher order tools can integrate and use their own models for calling DSC itself or resources that adhere to the DSC contract and publish a manifest. |
@citelao was talking to another internal partner regarding the need for init, so I created this issue to cover that topic with regards to bootstrapping #557 I wonder for the actual mount/unmounting of various types of files: vhd, vhdx, wim, cab, etc... it might be better for DSC to just provide an extension mechanism that a different tool just handles the mounting/unmounting and simply returns the path to the mounted image. For example, we would have an extension manifest that is similar to a resource manifest that advertises the mount/unmount capability with a well defined JSON output format (initially just need the path I think) to be returned via stdout. The manifest would declare what file formats the tool supports. In the case of VHD, it could literally just be a .ps1 script that calls |
@michaeltlombardi Absoutely---I have no intent to (a) step on anyone's toes or (b) reimplement something that another tool does better. But my understanding of the state of things is that there does not exist a good solution for pre-provisioning VHDs (etc) in situations like this that can also be run on live machines. In other words, I can either:
But not both---which seems like something that would be useful. "Get this machine into this state," regardless if online or offline. Perhaps our scenario is uncommon, but our (internal) customers keep asking if our provisioning scripts can run on already-running devices. We've had to tell them "no" each time because they're designed to run on offline VMs only. That said---I'm beginning to wonder if I misunderstand our requirements. Some example scenarios we'd want to run online:
I wonder if our aspirational tool must be dual-mode "offline, mounted" or "online, over SSH", since our provisioning steps largely require info from a host machine.
I'm not certain I understand issue 557---is it about downloading artifacts? We obtain artifacts many ways, plus we cache them and a bunch of other complicated stuff. I'd rather let our tool obtain the files, then use Or is 557 more about creating a "nupkg" equivalent file?
Yes, @jazzdelightsme was the initial person to point out the complexity here. His proposal was to have |
@citelao, I'm not sure if this helps, but back when I was doing infrastructure engineering, we used a combination of packer (which Azure VM Image Builder is built on) and our configuration management tool - at that time, Chef - to pre-configure everything we could do without the machine being joined and networked up in it's eventual spot, then finishing the configuration at deployment time. We were migrating into blue/green deployment, but when I left that role we had something close to 80/20 for pre-deploy configuration and post-deploy, with continuous enforcement of post-deploy. I don't remember finding any cases that we couldn't solve that way, and it helped us move the vast majority of slow/intensive processing into image preparation. We kept a small library of versioned images and a much more heterogenous and sprawling set of deployment configs for our various app teams. One thing to highlight is that our post-deployment configurations were a super-set of pre-deployment, we just didn't do upgrades to installed software (because we were migrating apps to blue/green as we onboarded them to this model). With DSC that would look something like having a pre-deploy configuration document and using the |
No, #463 is the "nupkg" type solution 557 is a potential alternative additional approach. For this one, imagine you want to install software locally and don't or can't (due to disk space) have it as a zip pkg. I would agree 463 would be a more common scenario than 557 and for me would take priority.
That would certainly simplify things and make it generally useful without tying it to any specific tools/technology. I think my preference right now is this approach although I'd call it |
adding to this
No it's not, but like any configuration process you need to equate for handoffs between each step of the process especially between teams & you only make it more complex in trying to have an all in one solution for both the online & offline scenario, but it do able. This is why you can build in your end to end process to do checks at each stage and depending which stage it is you either skip a series of steps or do those steps. I used Composite Resources and Script Resources for these along with Wait For Resources which replication and expansion of these are I believe a part of the plans for v3 but I've been out the loop somewhat with the development and plans for v3. My experience comes from the use of Lability for use not only in Lab environments where I could build and deploy them whilst I have been totally disconnected from the internet or any other network but also similarly to packer, build and package an image/image layer, that could be used elsewhere as needed (much like how containers are built today) I even said to Jeffery Snover that would be how I would configure containers by starting with base os, dsc config for the next layer & so on and so forth until the full layering is completed and a device/complex group of devices is in the overall desired state. However I also had a need for not only Lab deployments to local machines, or other machines in my control in the local network, I had a need to do so with Azure VMs and utilised Azure Lab Services and before it Azure DevTest Labs when spinning up repeatable labs when I've delivered training in this area. @rikhepworth wrote this great article - Define Once, Deploy Everywhere (Sort of...) about how he's used bits of this & may have some thoughts on this thread.
2nd is doable if you also have access to jump up from the VM to it's host via the network layer, like via WinRM, or SSH,
In the 3rd - Nothing stopping an encoded string that represents the installer file and setting that in the mounted disk, I have done this using segment based configuration to deploy parts of a file and then stitch them together to resemble the end file inside the VM's Disk for deploying a VHDX into a nested VM where I could not give the VM any networking access - that's not a great process but doable. I personally would rather see dsc not add a 1..10 | foreach { dsc config set --path Myconfig.yml --offline-settings Device$_.yml --background} That then would allow you to build as many devices as you need with the same config that would look like this # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
resources:
- name: Offline
type: Microsoft.DSC/OfflineConfiguration
- name: Current user registry example
type: DSCCommunity.NetworkingDSC/HostsFile
properties:
HostName: google.com
ipaddress: 127.0.0.1
ensure: present
dependsOn:
- "[resourceId('Microsoft.DSC/OfflineConfiguration', 'Offline')" with this path.yml containing any details for the offline resource and how to interact with it # example.offline.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/offline.json
OfflineConfig:
- type: VHD
Path: C:\myvhd.vhdx
MountTo : Any DriveLetter or # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/offline.json
OfflineConfig:
- type: Device
DeviceID: <As Returned by Get-PNPDeviceID>
Mount: Locally
MountTo : Any DriveLetter or # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/offline.json
OfflineConfig:
- type: Device
DeviceID: <As Returned by Get-PNPDeviceID>
Mount: VM
MountTo : <VMID> 1..10 | foreach { dsc config set --path Myconfig.yml --off-device-settings Device$_.yml --background} I know this is beyond "Offline" but this would also be worthwhile for considering the wider "configuring a device that is not where I am running dsc from" # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
resources:
- name: OffDevice
type: Microsoft.DSC/OffDevice
- name: Current user registry example
type: DSCCommunity.NetworkingDSC/HostsFile
properties:
HostName: google.com
ipaddress: 127.0.0.1
ensure: present
dependsOn:
- "[resourceId('Microsoft.DSC/OffDevice', 'OffDevice')" # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
resources:
- name: AWSDisk
type: Microsoft.DSC/OffDevice
provider: AWS
path: <AWS Disk accessible URI> with this you pass the connection element off to the AWS provider and that deals with Auth and securing the connection to that disk and making any required edits to it as needed. This could also optionally mount it as if it's local, using SMB over Quic, but that's an implementation detail that is outscope on this converation # example.dsc.config.yaml
$schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
resources:
- name: AWSDisk
type: Microsoft.DSC/OffDevice
provider: AWS
path: <AWS Disk accessible URI> I say this as especially as there would be a need for this to mount or interact with other disk types that may include filesystems that isn't usually visible to the host OS or even connected to the host but can be coerced to be interacted with via either attaching the device to a VM instead or may require other installed software inc drivers to be able to be interacted with in the host OS or can be remotely interacted with via other methods. Examples of that include filesystems used on devices like Games Consoles, phones etc. I'd prefer this than explicit passing of a mounted path due to flexibility and reducing any potential in leaking out a sensitive item like the ImageName DiskName RemoteDrive in any logging on the device where the dsc executable runs. It would also enable the ability to invoke parallel configurations to similar types of devices across different environments think Azure, AWS, GCP and Private Clouds and even potentially to customer environments where dsc is available too. |
@kilasuit With $schema: https://raw.githubusercontent.com/PowerShell/DSC/main/schemas/2024/04/config/document.json
metadata:
Microsoft.DSC:
deploymentSupport:
- online
- offline
resources:
- name: windows product name
type: Microsoft.Windows/Registry
properties:
hivePath: "[concat(getMountedPath(), '\\Windows\\System32\\Config\\SOFTWARE')]"
keyPath: HKLM\Software\Microsoft\Windows NT\CurrentVersion
valueName: ProductName The |
Summary of the new feature / enhancement
Support offline image configuration by having a group resource handle the mounting/unmounting of an image and passing the mounted path to resources that can support it.
Proposed technical implementation details (optional)
Introduce a tag
offline
that indicates resources support being given a mounted path to work against instead of the current live system. This would include theregistry
resource and others likefile
,windowsservice
, etc...How to pass the mounted path to resources? Some options:
_mountedPath
mountedPath
The text was updated successfully, but these errors were encountered: