Releases: vmware-tanzu/sonobuoy
v0.15.1
Sonobuoy v0.15.1
What's new
- New command
sonobuoy resultsto access results from any plugin.- See the docs for more details on this new command. TL;DR; it is now possible to read results from any plugin without opening up the tarball.
- New flag
--wait-output- The flag defaults to 'silent' (existing behavior) but you can also adjust it to 'spinner' which will ensure that CI systems (and users) don't think that the command has hung.
- New plugin definition option
SkipCleanup- This feature increases your ability to run tests and experiments which may need more manual debugging after the test run. Previously, Sonobuoy would automatically delete all the plugin pods once they were completed preventing manual inspection.
What's better
- Kubernetes version support
- If the Kubernetes version of the cluster is greater than the maximum version supported by Sonobuoy, a warning is displayed instead of an error. This should make it easier for users who are trying to test on new clusters when Kubernetes versions get released.
- The plugin definitions are written into the 'plugins/' directory for higher visibility.
- Multiple commands now rely on labels instead of exact pod names when targeting the Sonobuoy aggregator pod. This helps facilitate situations where Sonobuoy is being deployed as a Job, Deployment, or ReplicaSet.
- Extra query options
- Numerous options were added to facilitate getting pod logs to be more in line with upstream options.
- Return exit code 1 if sonobuoy can't pull all images required for the e2e tests
Bug fixes
- Fixed a bug where the UUID may not be set on a config in certain code paths.
Changelog
4acae8c Bump version to 0.15.1 and add versioned docs
b89b8bb Add some documentation for new results command (#825)
7713c02 Allow multiple result files to be post-processed (#824)
e39cde3 Default to ResultFormat=raw if unset (#822)
d91bc40 Save results in a canonical format via postprocessing (#813)
a6dc3a3 Update some vars and literals to use consts (#816)
18d5aea Change max supported version error to warning (#815)
d739225 Merge pull request #622 from ekj1711/wait-emit-output
fe23f98 Write plugin definitions in their respective folders (#812)
f0ffecc Remove use of global sonobuoy pod name (#811)
020a7e4 Remove plugin.Definition (#810)
5519e20 Wait should emit some output
6240935 Preserve single file result names and keep job/ds depth the same (#785)
c942509 Ensure only failed tests are run with --rerun-failed (#801)
e3e7514 Pin images used in kind config and reduce number of workers (#803)
160b58a Upgrade kind to v0.4.0 (#804)
4a5c1fd Create a UUID for loaded configs if necessary (#807)
0152d01 Add Docker to requirements for sonobuoy images (#805)
700ac79 Merge pull request #800 from MengmengZHANG/fieldSelector
8380685 add fieldSelectors & labelSelectors for getPodLogs (#778)
5009540 Generate Plugins ConfigMap if no plugins specified (#799)
443f20a Fix a broken link on "Conformance Testing" page (#798)
90cd0d1 Allow cleanup to be skipped on a per plugin basis (#796)
ad75ce4 Merge pull request #795 from jonasrosland/redirect
4e5187e Small CTA fix
456ec57 Add redirect for docs
afcd78a Fix links to doc pages on new site (#794)
c029d9f Fix preflight test error message (#791)
dca7c17 Fix retrieve command when using dynamic pod name (#790)
bc9e84b Look up sonobuoy master pod name by label (#787)
f298701 issue 777 add more query options for getPodLogs (#788)
c3b5afa Return exit code 1 if sonobuoy can't pull all images required for the e2e tests (#786)
fd7722d Add e2e-repo-config flag to images pull command (#781)
c7c4345 Add new blog post for dry-run and --plugin-env flag (#774)
2439cd4 Fix logo reference on main README (#773)
20e0df5 Update README to point to docs site (#771)
6811a38 Add v0.15.0 doc snapshot and automate process (#770)
v0.15.0
Sonobuoy v0.15.0
What's new:
Versioned Documentation
The Sonobuoy website now has documentation available for each release of Sonobuoy going back to v0.13.0. As new versions are released, documentation specific to that version will be available.
The docs for this release are available at sonobuoy.io/docs/v0.15.0.
What's better:
This release takes place very close to our previous release, v0.14.3, and focuses more on bug fixes and improving the contributor experience.
Bug fixes
- Mitigated a bug where there would be a failure due to the container being terminated even though results had been sent. Sonobuoy will now wait longer for results after they have been sent and the container terminated before deeming the run to be a failure.
- Fixed a bug where a panic would happen if a
nilconfiguration parameter was passed to most of the functions in the Sonobuoy client library.
Contributor experience improvements
- Improved the debugging experience in the CI builds. More logging has been added to help track down and debug issues when builds fail.
- Added a check to ensure that a developer commits any changes resulting from
depchanges. - Improved tests which previously checked that the build version was correctly included when generating a Sonobuoy config. These tests needed to be updated on every release but no longer require this as they instead check a static version.
- Improved the documentation for the Sonobuoy release process to include more details on steps required for minor version increases which correspond to new Kubernetes releases.
- Added a
nativeMakefile target so that Sonobuoy can be built locally using the same flags as in CI without using Docker.
Changelog
7ad3675 Prepare v0.15.0 release (#769)
c34d3ee Update release process documentation (#768)
41ce5d4 Tweak the timing of considering pod as failing to be more permissive (#764)
ee916a1 Add basic validation of configs (#763)
f1fbf8a Add versioned docs to Sonobuoy site (#756)
7e5103d Fixup debug line to target regular files and print file name (#762)
e257b50 Apply changes from running gofmt (#758)
daa7ab3 Add Bridget to the team on the website (#760)
6c522c8 Add native Makefile target (#757)
8ac5ca2 Ensure dep has been run in CI (#755)
985c071 Update golden file tests to use static versions (#751)
2630934 Fixup to travis debug lines (#753)
4901e55 Add vscode to gitignore (#749)
7d7bd81 Set GitSHA when using goreleaser (#750)
v0.14.3
Sonobuoy v0.14.3
What's New:
A new Sonobuoy website: sonobuoy.io
The website is generated from documentation in the repo itself and uses Netlify to generate a site preview when we change those documents.
Setting arbitrary env vars on plugins
A new flag --plugin-env can be used to set any env var on any plugin. This will prevent you from having to (1) add a flag to Sonobuoy to edit your env vars (2) save and edit the YAML for the plugin in order to modify the env var.
You can set env vars on any plugin (whether or not it is a custom or built-in plugin) by referring to the plugin by name:
sonobuoy run --plugin-env e2e.E2E_DRYRUN=true --plugin ./myplugin.yaml --plugin-env myPlugin.FOO=BAR
Custom annotations on Pods
In some situations you may need to have the Sonobuoy pods all lauched with custom annotations, for example, to use KIAM permissions. A new field, CustomAnnotations, in the config file format supports this.
For example, to add the annotation foo:bar to your pods:
sonobuoy gen config | jq '.CustomAnnotations={"foo":"bar"}' > myConfig.json
sonobuoy run --config myConfig.json
What's Better:
Improved the feedback loop when a plugin failed to run
Up to this point, if a plugin failed to run, the logic in the server overreacted and would exit entirely. Now, it reports the error for that plugin and continues the run as normal.
Improved the feedback loop when a plugin exited without returning results.
In this case, the Sonobuoy sidecar would continue to wait for the results from the main container (your plugin) despite it already having exited. Now, the aggregator will watch for this condition and report it as an error running the plugin. This avoids the aggregator server hanging until the timeout waiting for results.
Improved stability in situations where network failures are high.
The Sonobuoy sidecar on the plugins automatically was retrying to send plugin results if the connection failed for some reason. However, the aggregator server was not accepting retries appropriately. This led to situations where your plugin results would be missing or truncated, despite the pod logs of the plugin showing more full results.
Other bug fixes:
-
Fixed a bug which could cause the Sonobuoy status to never report completed if no plugins were run.
-
Fixed a bug which prevented the user from providing an empty array of resources to gather (indicating to gather them all).
-
Fixed a bug which caused numerous resource directories to be created in the results tarball, even if no resources of that type existed.
-
Fixed a deadlock issue which could occur if you tried to run
sonobuoy logswhen pods were not yet ready. -
Fixed a bug which would have caused a manually specified conformance image to be improperly manipulated causing a failed image lookup.
-
Documentation improvements:
- Clarified the support matrix (3 minor versions total)
- Added documentation describing how to use a Sonobuoy image which is in a private registry.
- Fixed a broken link in the conformance docs
-
Code cleanup:
- Combined two code flows into one for reuse when handling plugin results.
- Made GetStatus library command have a signature consistent with other methods
- Code cleanup including removing redundant and confusing fields from
GenConfig
Changelog
6d02613 Fix build and cleanup test to ensure clean state for release (#747)
439de5f Bump version to v0.14.3 (#743)
6466ac3 Fix excerpt separator being shown in blog posts (#742)
3324bc4 Add checks for if a container has terminated (#734)
a7bba9b Add daemonSet monitoring tests (#740)
61ac7be Add tests for job.Monitor logic (#739)
edb4bf5 Adjust README description of support matrix (#737)
4d4ff7a Merge pull request #730 from johnSchnake/retryResults
f51d88d Reprocess results if there was a processing error
98d7d32 Unify http/non-http result handling (#729)
cfd9cfc Use a struct for GetStatus to be consistent (#731)
a07aed2 Add ability to specify custom annotations (#723)
f77a792 Documentation usability and readability update (#727)
d5c624c Fix/simplify the config loading logic enabling empty resources (#725)
f2c885d Add initial annotation to the pod when starting (#724)
d34e9cf Don't create resource directories if not querying for resources (#726)
e9aba66 Properly report error when a plugin fails to run (#720)
90b3df8 Ensure plugin selection list and resources can be empty (#722)
0fa9c4f Merge pull request #712 from smalltalk-ai/master
34a4918 Update version number
dad5f21 Update README | Remove enhancements
875aae4 Update docs from master branch not gh-pages
702c22a Updates to homepage links
68df46a Adding in Jekyll site
9f0c2b0 Set any plugin env vars from the CLI (#709)
9a19511 Fix race/deadlock issues with logs when a pod is not ready (#719)
affd4fb Ensure conformance image version is prefixed with 'v' (#717)
770b85b Fix broken link in conformance doc (#718)
c76f2f5 Use go modules when getting KinD (#714)
a4e9a32 Merge pull request #705 from johnSchnake/pullSecretsDoc
65e2319 Merge pull request #704 from padlar/padlar/issue-691
108d168 remove redundant fields from GenConfig
9070e29 Add documentation about how to pull private Sonobuoy image
3bef2a4 Add example for command runner plugin
v0.14.2
What's New in v0.14.2
Improved Plugin Support
The major improvement on this release are the --plugin flag and the sonobuoy gen plugin command.
Use sonobuoy gen plugin to generate a custom plugin definition and then use --plugin with either sonobuoy gen or sonobuoy run to select which plugins to run. For example, to run a custom plugin and the e2e tests together, run the command :
$ sonobuoy gen plugin --name myPlugin --image myImage > customPlugin.yaml
$ sonobuoy run --plugin customPlugin.yaml --e2e
See more details in a this new example.
In addition, the aggregation server will, by default, run all the plugins it can find when it starts. This differs from previous releases where it would only run plugins specified in the Sonobuoy configuration's Plugins field. Now, by default that field is null and it will run everything. If you want, you can still specify that field manually like you did before to require a specific list of plugins to run. If you want to skip all plugins you can provide the empty array ([]).
Dynamic Client for Queries
The query logic (which gathers information about your server after the plugins run) used to have a discrete list of resources it could query. We've updated this so that it dynamically crawls the API to enable it to gather more information. If you want to limit the values it gathers, just set the Resources field in the Sonobuoy configuration file to list the API resource names you want. By default it still explicitly lists the resources it intends to gather, but if you remove the Resources field you can see it gathers everything it can (with the exception of secrets). This can be really useful if you want to see things like CRDs.
The logs also now show explicitly which items the query logic is skipping; that way you can be confident about what you are querying for and what items you could add.
Sonobuoy Configuration File Generation
We added a new command: sonobuoy gen config which will spit out the default Sonobuoy configuration. This has two purposes:
- Helps you understand what the default values are
- Makes it easy to take the default and modify a single value or two which are of interest. Now you don't have to wonder what the fields are named or what the defaults are if you just want to slightly tweak something.
ImagePullSecrets for Sonobuoy Image
If you are hosting the Sonobuoy image in a private Docker registry which required a password, in previous releases it was not possible to get Sonobuoy to work without a custom build because the templates and data types didn't "know" the imagePullSecrets field.
Support for that has been added. It requires setting the ImagePullSecrets field in the Sonobuoy configuration and then also manually adding the YAML for your secret in the output from sonobuoy gen and running it all manually via kubectl apply. Future releases may consider how to better handle creating additional resources like this.
Server Timeout Flag
One of the most commonly edited fields in the Sonobuoy config file is the timeout for the aggregation server (how long it will wait for plugins to complete). We added a new --timeout flag to sonobuoy gen and sonobuoy run to make this easier.
Context Flag for KUBECONFIG
A new --context flag has been added everywhere --kubeconfig was supported in order to automatically handle multi-context config files.
Other changes
Preflight checks will now error if you try to run Sonobuoy in an existing namespace. This is to help prevent the situation where you create Sonobuoy in a namespace which you don't want to later destroy (meaning you've got to clean things up manually). If you really want to create Sonobuoy in an existing namespace, you still can: just provide the --skip-preflight flag.
Changelog
ebc7916 Add coverage profile to gitignore to ensure clean state after tests
a38e39e Bump version to v0.14.2
d5927ae Fix servergroups query
ee6ecc8 Update queryies to use the dynamic client
a85b38f Add ability to set args when defining a plugin
ec8b6c6 Properly set default for plugin job driver
bf155c8 Support imagePullSecrets for sonobuoy workers
aaa5e13 Add CLI flag for aggregation timeout
1b56c31 Add plugin flag for custom, local plugins
1a72321 Add sonobuoy gen config file and documentation
012648c Tighten preflight check to err on existing namespace
a5d8610 Update client go and other deps
7af1d5a Run all loaded plugins by default
89bb5b4 Modify the json serialization of config object
f488867 Add test that will run through command creation
b420055 Fix flaky unit test
8f123df Change logic in template to use a list of plugin manifests
0f8ab46 Add gen plugin command to generate boilerplate for plugins
ee335d2 Centralize the process of going from kubeconfig to sonobuoy client
d54a882 Defer assignment of Docker gcr.io secret
566e80a Add context flag whenever kubeconfig flag is added
8eb9fb5 Add CodeCov reports
v0.14.1
Changelog
797da23 Bump version for 0.14.1
fc4acbb Use heptio kube-conformance image for any cluster < v1.14.1
33f61c2 Fix CRD query
c3a2050 Error e2e command if no e2e results
18a07b9 Add validation to focus/skip regexp flags
755fafb Update release documentation
a027811 Add additional documentation about testing in airgap
v0.14.0
Changelog
94af630 Clean up a few artifacts for clean state
01ab41b Add a short version option for scripting
5138a19 Bump versions for v0.14
68f15a2 Add image list for v1.14 Kubernetes
5d4aaac Use workers default image and pull IfNotPresent
c27ab1a Push images in Jenkins
ccc8d5a Remove image pushing from Travis
94cceb3 Add image download to tar, image pull to docker client, image tag/push to registry
0c41438 Replace alpine with debian-slim
365de15 Show list of images used for upstream Kubernetes E2E tests
e22d4d6 Add missing registry entry for airgap example
3b9c7aa Remove debug lines
ffd4016 Default to 'auto' for gen command
de60f31 Update README
6ac2f2d Use upstream conformance image when >= v1.13
0e520eb Add support for KUBE_TEST_REPO_LIST and custom registries
60e9447 Update to go1.12
c134ed8 Restructured cmd package
bea7092 Run E2E tests inside a kind cluster
69d20b0 List files created by retrieve command
3ab6710 Update logo
152531c Fix go vet error in daemonset.go
fd5adba Add wait flags for delete and run
a152311 Prioritize flags over config values when running gen
47bb372 Fix problem for building locally on non-Linux OS
0042c40 Provide more clear output from sonobuoy e2e
fd6bd41 Update version command to print GitSHA, Min/Max K8S Versions & current cluster API version
eccb315 Acknowledge state between plugins complete and done
081c43f Project Sonobuoy renaming (#577)
af37d18 Added checks to release document (#566)