Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⭐️ delayed asset discovery #3496

Merged
merged 8 commits into from
Mar 7, 2024
Merged

⭐️ delayed asset discovery #3496

merged 8 commits into from
Mar 7, 2024

Conversation

imilchev
Copy link
Member

@imilchev imilchev commented Mar 5, 2024

needs #3492

Copy link
Contributor

github-actions bot commented Mar 5, 2024

Test Results

2 854 tests  +5   2 853 ✅ +5   1m 22s ⏱️ -8s
  186 suites ±0       1 💤 ±0 
    5 files   ±0       0 ❌ ±0 

Results for commit 023880f. ± Comparison against base commit eed1b07.

This pull request removes 3 and adds 8 tests. Note that renamed tests count towards both.
go.mondoo.com/cnquery/v10/llx ‑ TestSuccess/2024-03-06_10:34:20.294474983_+0000_UTC_m=+0.008575732
go.mondoo.com/cnquery/v10/llx ‑ TestTruthy/2024-03-06_10:34:20.294474983_+0000_UTC_m=+0.008575732
go.mondoo.com/cnquery/v10/llx ‑ TestTruthy/2024-03-06_10:34:20.294474983_+0000_UTC_m=+0.008575732#01
go.mondoo.com/cnquery/v10/llx ‑ TestSuccess/2024-03-06_13:44:55.107182654_+0000_UTC_m=+0.006560970
go.mondoo.com/cnquery/v10/llx ‑ TestTruthy/2024-03-06_13:44:55.107182654_+0000_UTC_m=+0.006560970
go.mondoo.com/cnquery/v10/llx ‑ TestTruthy/2024-03-06_13:44:55.107182654_+0000_UTC_m=+0.006560970#01
go.mondoo.com/cnquery/v10/providers-sdk/v1/plugin ‑ TestDeprecatedAddRuntime_DisableDelayedDiscovery
go.mondoo.com/cnquery/v10/providers/os/connection/container ‑ TestNewImageConnection_DelayDiscovery
go.mondoo.com/cnquery/v10/providers/os/connection/container ‑ TestNewImageConnection_DisableDelayDiscovery
go.mondoo.com/cnquery/v10/providers/os/connection/docker ‑ TestAssetNameForRemoteImages_DisableDelayedDiscovery
go.mondoo.com/cnquery/v10/providers/os/provider ‑ TestLocalConnectionIdDetectors_DelayedDiscovery

♻️ This comment has been updated with latest results.

Base automatically changed from ivan/global-connection-ids to main March 6, 2024 10:31
imilchev added 3 commits March 6, 2024 11:33
Signed-off-by: Ivan Milchev <[email protected]>
Signed-off-by: Ivan Milchev <[email protected]>
@imilchev imilchev force-pushed the ivan/delayed-discovery branch from a70a04f to e31f61c Compare March 6, 2024 10:34
imilchev added 2 commits March 6, 2024 11:39
Signed-off-by: Ivan Milchev <[email protected]>
@imilchev imilchev force-pushed the ivan/delayed-discovery branch 2 times, most recently from b4cd275 to a3ae12c Compare March 6, 2024 14:01
@imilchev imilchev force-pushed the ivan/delayed-discovery branch 2 times, most recently from 1ce3a8d to 0d012bd Compare March 6, 2024 15:29
@imilchev imilchev marked this pull request as ready for review March 6, 2024 15:32
@imilchev imilchev force-pushed the ivan/delayed-discovery branch from 0d012bd to abbb13e Compare March 6, 2024 15:45
Copy link
Contributor

@czunker czunker left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why or in which cases should this be used?

Is it intended, that this is not exposed as a flag?

explorer/scan/discovery.go Outdated Show resolved Hide resolved
providers/os/connection/container/image_connection.go Outdated Show resolved Hide resolved
inv := &inventory.Config{Options: map[string]string{}}
_, err = container.NewImageConnection(1, inv, &inventory.Asset{}, img)
require.NoError(t, err)
assert.True(t, inv.DelayDiscovery)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we add a check for the platform here and in the other test?

Here, we shouldn't have any platform information. In the other test we should know it is an alpine image.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

discovery is handled in the provider, not in the connection. At this point, we never have a platform regardless of the flag. There are tests that verify the provider sets the platform based on the flag in the provider tests

explorer/scan/local_scanner.go Outdated Show resolved Hide resolved
providers-sdk/v1/inventory/inventory.proto Outdated Show resolved Hide resolved
@imilchev
Copy link
Member Author

imilchev commented Mar 7, 2024

Why or in which cases should this be used?

Is it intended, that this is not exposed as a flag?

I am not sure I understand the question. This is an inventory setting. There is nothing for the user to specify here. The goal is that every provider knows which discovery takes a lot memory, disk space or IO. It is up to the provider to decide whether delayed discovery should be used for an asset or not. If delayed discovery is used, then just 1 operation at a time will be active during a scan. That's not something the user should decide or care about

Signed-off-by: Ivan Milchev <[email protected]>
@imilchev imilchev merged commit 1b98d7f into main Mar 7, 2024
14 checks passed
@imilchev imilchev deleted the ivan/delayed-discovery branch March 7, 2024 14:52
@github-actions github-actions bot locked and limited conversation to collaborators Mar 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants