Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use DaemonSet for node scanning #1134

Merged
merged 27 commits into from
Jun 21, 2024
Merged

use DaemonSet for node scanning #1134

merged 27 commits into from
Jun 21, 2024

Conversation

slntopp
Copy link
Member

@slntopp slntopp commented Jun 19, 2024

No description provided.

Copy link

github-actions bot commented Jun 19, 2024

Test Results

  5 files  ±0   50 suites  ±0   2h 36m 54s ⏱️ + 12m 19s
301 tests  - 5  301 ✅  - 5  0 💤 ±0  0 ❌ ±0 
370 runs   - 5  370 ✅  - 5  0 💤 ±0  0 ❌ ±0 

Results for commit 9dc1997. ± Comparison against base commit 59392e5.

This pull request removes 11 and adds 6 tests. Note that renamed tests count towards both.
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestConfigMapName
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestConfigMapName/should_be_prefix+base+hash_when_longer_than_52_chars
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestConfigMapName/should_be_prefix+base+suffix_when_shorter_than_52_chars
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_CleanDeploymentsForDeletedNodes
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_CreateDeployments
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_CreateDeployments_Switch
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_Deployment_NodeScanningStatus
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_UpdateDeployments
go.mondoo.com/mondoo-operator/tests/integration ‑ TestAuditConfigCustomNamespaceSuite/TestReconcile_Nodes_Deployments
go.mondoo.com/mondoo-operator/tests/integration ‑ TestAuditConfigOOMSuite/TestOOMNodeScan_Deployment
…
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_CreateDaemonSets
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_CreateDaemonSets_Switch
go.mondoo.com/mondoo-operator/controllers/nodes ‑ TestDeploymentHandlerSuite/TestReconcile_UpdateDaemonSets
go.mondoo.com/mondoo-operator/tests/integration ‑ TestAuditConfigCustomNamespaceSuite/TestReconcile_Nodes_DaemonSet
go.mondoo.com/mondoo-operator/tests/integration ‑ TestAuditConfigOOMSuite/TestOOMNodeScan_DaemonSet
go.mondoo.com/mondoo-operator/tests/integration ‑ TestAuditConfigSuite/TestReconcile_Nodes_DaemonSet

♻️ This comment has been updated with latest results.

@slntopp slntopp marked this pull request as ready for review June 20, 2024 16:01
@slntopp slntopp requested review from imilchev and czunker June 20, 2024 16:01
controllers/nodes/deployment_handler.go Show resolved Hide resolved
controllers/nodes/deployment_handler_test.go Outdated Show resolved Hide resolved
controllers/nodes/deployment_handler_test.go Outdated Show resolved Hide resolved
controllers/nodes/deployment_handler_test.go Show resolved Hide resolved
controllers/nodes/deployment_handler_test.go Outdated Show resolved Hide resolved
controllers/nodes/resources.go Show resolved Hide resolved
tests/integration/audit_config_base_suite.go Show resolved Hide resolved
tests/integration/audit_config_oom_test.go Outdated Show resolved Hide resolved
tests/integration/audit_config_oom_test.go Outdated Show resolved Hide resolved
tests/integration/audit_config_oom_test.go Show resolved Hide resolved
}
dep.Spec.Template.Spec.Tolerations = k8s.TaintsToTolerations(node.Spec.Taints)
ds.Spec.Template.Annotations[ignoreQueryAnnotationPrefix+"mondoo-kubernetes-security-pod-runasnonroot"] = ignoreAnnotationValue
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independent of this PR:
Looks like this no longer works and we can remove it:
image

@imilchev What do you think.

@czunker
Copy link
Contributor

czunker commented Jun 21, 2024

Testing:

  • minikube with 2 nodes
  • I installed the current operator with Deployment node scanning.
  • I uploaded an operator build from this PR into minikube
  • I switched the image
  • I manually changed the cluster role

And I see a DaemonSet and Pods for the nodes.

🎉 Thanks @slntopp

@czunker
Copy link
Contributor

czunker commented Jun 21, 2024

Nodes are also regularly reported to the Platform. 👍

PlatformIDs also looking good:

[
  "//platformid.api.mondoo.app/runtime/k8s/uid/e6276680-c3d3-43ff-9977-7fd9ecf72117/node/minikube-m02",
  "//platformid.api.mondoo.app/runtime/ssh/hostkey/SHA256-AnkEEFa/ZAaXxqgvGwKJDykIFH4E3WTRvfsY7U7YQcg",
  "//platformid.api.mondoo.app/runtime/ssh/hostkey/SHA256-HJpfzDi5PymOmQxFBE4fk5Wc78OMoT1kd5+J2EOZwSo",
  "//platformid.api.mondoo.app/runtime/ssh/hostkey/SHA256-vB+V+ZPY8oumOHLlDKOlS+yf6osiT1ssEEWmNxF0QMU",
  "minikube-m02"
]

@czunker
Copy link
Contributor

czunker commented Jun 21, 2024

I think, I found a bug.
I installed the operator with deployments for the node scanning. With the new operator that switched to a DaemonSet. ✔️

But then, I switched to CronJobs for the node scanning. The CronJob got created, but the DaemonSet is still present:

k -n mondoo-operator get all                                                                                                                                                                                                                   
NAME                                                      READY   STATUS      RESTARTS      AGE
pod/mondoo-client-k8s-scan-28649450-ntb75                 0/1     Completed   0             24m
pod/mondoo-client-node-rz7sl                              1/1     Running     0             4h9m
pod/mondoo-client-node-xq5mn                              1/1     Running     0             4h9m
pod/mondoo-client-scan-api-68b4cc949f-4swkd               1/1     Running     0             6h23m
pod/mondoo-operator-controller-manager-57b4db899b-zcb7k   1/1     Running     2 (24m ago)   6h18m

NAME                                                         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/mondoo-client-scan-api                               ClusterIP   10.105.211.169   <none>        8080/TCP   6h42m
service/mondoo-operator-controller-manager-metrics-service   ClusterIP   10.105.73.28     <none>        8080/TCP   6h42m

NAME                                DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/mondoo-client-node   2         2         2       2            2           <none>          6h17m

NAME                                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/mondoo-client-scan-api               1/1     1            1           6h42m
deployment.apps/mondoo-operator-controller-manager   1/1     1            1           6h42m

NAME                                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/mondoo-client-scan-api-68b4cc949f               1         1         1       6h23m
replicaset.apps/mondoo-client-scan-api-fbccdfc94                0         0         0       6h42m
replicaset.apps/mondoo-operator-controller-manager-5569d69fcb   0         0         0       6h42m
replicaset.apps/mondoo-operator-controller-manager-57b4db899b   1         1         1       6h23m

NAME                                            SCHEDULE        TIMEZONE   SUSPEND   ACTIVE   LAST SCHEDULE   AGE
cronjob.batch/mondoo-client-k8s-scan            50 * * * *      <none>     False     0        42m             6h42m
cronjob.batch/mondoo-client-node-gc             31 */12 * * *   <none>     False     0        <none>          6h42m
cronjob.batch/mondoo-client-node-minikube       50 * * * *      <none>     False     0        <none>          108s
cronjob.batch/mondoo-client-node-minikube-m02   50 * * * *      <none>     False     0        <none>          108s

NAME                                        STATUS     COMPLETIONS   DURATION   AGE
job.batch/mondoo-client-k8s-scan-28649450   Complete   1/1           84s        24m

@slntopp
Copy link
Member Author

slntopp commented Jun 21, 2024

Good catch!
Just updated this one

@czunker
Copy link
Contributor

czunker commented Jun 21, 2024

Good catch! Just updated this one

Fixed! Thanks 🎉

Copy link
Contributor

@czunker czunker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @slntopp

Copy link
Member

@imilchev imilchev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Let's ship it!

@slntopp slntopp merged commit 0cb897f into main Jun 21, 2024
21 checks passed
@slntopp slntopp deleted the mik/node-scanning-daemonset branch June 21, 2024 13:56
@github-actions github-actions bot locked and limited conversation to collaborators Jun 21, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants