Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content promotion workflow for WSUS and Foreman/Satellite #57

Open
corporate-gadfly opened this issue Apr 30, 2020 · 2 comments
Open

Content promotion workflow for WSUS and Foreman/Satellite #57

corporate-gadfly opened this issue Apr 30, 2020 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@corporate-gadfly
Copy link
Contributor

This is mostly off-topic, so feel free to label this issue as question.

What strategies are available for patching systems in waves where QA lags production by, let's say, 2 weeks? So, you watch your QA systems for issues for 2 weeks and then roll out the patches to production.

Is it along the lines of yum repos (for RHEL) which are maintained with a 2 week lag in updates?

@nmaludy nmaludy self-assigned this Apr 30, 2020
@nmaludy nmaludy added the enhancement New feature or request label Apr 30, 2020
@nmaludy
Copy link
Member

nmaludy commented Apr 30, 2020

@corporate-gadfly We handle this at a different level than Bolt, although there are talks internally about using Bolt (and this module) to aide us in the process.

We use two tools:

  • Red Hat Satellite (open source = Foreman): In this tool they have "content views". It ha features of different "environments' and we can promote "content" aka packages up through the different environments based on our patching schedule.
  • WSUS : Here we create a list of patches we approve in WSUS. Approve them in our DEV WSUS, then go to our PROD WSUS and approve the same list based on our patching schedule.

The idea is to create some plans / tasks to talk to Satellite/Foreman and WSUS to handle these content promotion workflows.

@nmaludy nmaludy reopened this Apr 30, 2020
@nmaludy nmaludy changed the title Patching strategy where QA machines lag production machines Content promotion workflow for WSUS and Foreman/Satellite Apr 30, 2020
@yachub
Copy link

yachub commented May 1, 2020

As far as the Satellite/Foreman side goes, for our RHEL servers we use the content views and lifecycle environments feature to stage package updates to a Development environment throughout a week timeframe, then the next week those exact same packages are promoted and pushed out to the Production environment. (This has actually saved us a few times when we found an issue with a package during the development patching cycle before it made it to production!)

We handle the actual auto-installation and auto-rebooting of all hosts using the "remote execution" feature in Satellite/Foreman to create a "Job" (similarly could use bolt, but I like the nice GUI interface that presents stats, outputs, and results of the job). Our hosts are organized into different "Host Collections" to identify their patching window/timeframe. When a job is created the remote execution command targets all hosts in a host collection and executes yum-cron (However, the yum-cron service itself always remains stopped and disabled), which give you the advantage of staggering updates/reboots throughout a specified timeframe rather than all at once, plus I like using the email feature of yum-cron.

I wrote scripts that automate the promotion of content through lifecycle environments and create the patching/reboot jobs, which are created using the Hammer CLI tool.

I have a full Foreman/Katello demo environment that demonstrates all of this and can be experimented with in a vagrant environment.

The repo contains all of the automated scripts mentioned above as well as a couple Puppet profiles to setup yum-cron and the katello agent.

The workflow has been working pretty well for us over the past 2 years. And I would happy to have further discussion around it!

On related note, we also did something similar with Chocolatey packages for Windows, where we manage 2 separate internal Chocolatey server lifecycle environments, and use choco upgrade to automatically update packages during patching windows. Packages are built, pushed, and staged through environments using GitLab-CI. I don't have anything official written up on it, but would be happy to share more if anyone is interested.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

3 participants