[Docs]: docs should more clearly describe the cve scanning feature and how to enable it

[mikemccracken]

zot version

v1.4.3

Describe the bug

looking for details about how to config and use the cve scanning feature, I only see references to the zli command and the search extension at https://zotregistry.io/v1.4.3/admin-guide/admin-configuration/?h=cve#enhanced-searching-and-querying-images

We should have a separate section for this, and clearly explain what it scans and when, how to enable it, and what tools it uses under the hood.

also, does it require the search extension?

I assume it scans each tag by extracting it then scanning the resulting rootfs, probably on push, but I think we should be explicit in the docs.

To reproduce

n/a

Expected behavior

No response

Screenshots

No response

Additional context

No response

[andaaron]

Some clarifications in general:

1. The search extension is (for now) mandatory. The search extension is (still) the only way to access scan results.

2. There is a background task for downloading the latest Trivy DB available (by default from the trivy project URL on GH, but it can be configured with an alternate source). The interval at which this background task is run can be changed by the user.

3. Regarding when the scan runs, it is not on push:

• Originally the scan was done on every search request, later we started caching the results (a previous 2.0.0 RC), and more recently (in the most recent 2.0.0 RC) we started scanning images in advance.
• We have the scheduler running background tasks periodically (as soon as zot starts and the DB is downloaded) to scan images.
• If a user requests results for an image which does not have results in the cache, it is scanned at that time (the server does not reply until the results are available).
• Cache results are stored using the manifest digests as keys, so if a manifest is stored in a different repo, or with a different tag name, the results are reused. If someone pushes a new manifest and reuses a tag, that new manifest will be picked up on the next time the schedule starts running trivy scan tasks.
• The cache is purged entirely after a new version of the CVE DB is downloaded from upstream.

4. We are using the trivy code as a Go library, we don't require the user to download any trivy tooling in advance before the zot server is started.

5. There is a requirement for the host to have sufficient space on the temporary partition, as the trivy library stores files under /tmp without us being able to control this behavior.

• the DBs are downloaded under /tmp, and then moved to the final destination under the zot root dir. This is something out of our control and out of trivy developers control, this is a limitation of the download library they use to copy oras artifacts. After the operation is finished the temporary folders created should remain empty.
• trivy will unpack large layers under /tmp, in order to run the CVE scanner on the content. So if you use large layers, you need to have sufficient space under /tmp to unpack them. Smaller layers will be unpacked in-memory.

6. DBs (not sure if this needs to be documented as part of this task)

• The trivy DB should initially have under 100 MB, and will grow while storing new scan results.
• The trivy Java DB is quite larger, a few hundred MB, and theoretically should not grow (this needs to be verified).

7. The scan itself is similar to what the trivy image command runs, with options https://github.com/project-zot/zot/blob/main/pkg/extensions/search/cve/trivy/scanner.go#L38
   This is the equivalent of trivy image --scanners vuln --vuln-type os,library. The rest of the options in the code are related to how zot handles trivy DB download, input images, and processing the results.

Not sure how much of this needs to get into the end-user documentation

[mikemccracken]

Thanks for all the details @andaaron - I think most of this should be in the docs, although number 3 can probably be summarized a bit. If we got all this info into the docs, it'd resolve all my questions in this issue.

in particulara the details in 5 and 6 are very relevant for someone wanting to deploy this, so they can size things.

maybe point 7 can be left out, but even that might be useful. and if there isn't an easy way to tell which version of the trivy go library was used in a given zot build, we should consider adding that IMO.