New Parser: Kiuwan SCA Scan #10064

mwager · 2024-04-30T05:29:15Z

As discussed in slack #defectdojo-dev:

Checklist

This checklist is for your information.

Make sure to rebase your PR against the very latest dev.
Features/Changes should be submitted against the dev.
Give a meaningful name to your PR, as it may end up being used in the release notes.
Your code is flake8 compliant.
Your code is python 3.11 compliant.
If this is a new feature and not a bug fix, you've included the proper documentation in the docs at https://github.com/DefectDojo/django-DefectDojo/tree/dev/docs as part of this PR.
Add applicable tests to the unit tests.

(cc @flmarkus)

…he UI display

dryrunsecurity · 2024-04-30T05:29:56Z

Hi there 👋, @DryRunSecurity here, below is a summary of our analysis and findings.

DryRun Security	Status	Findings
Configured Codepaths Analyzer	✅	0 findings
IDOR Analyzer	✅	0 findings
Sensitive Files Analyzer	✅	0 findings
Server-Side Request Forgery Analyzer	✅	0 findings
SQL Injection Analyzer	✅	0 findings
Authn/Authz Analyzer	✅	0 findings
Secrets Analyzer	✅	0 findings

Note

🟢 Risk threshold not exceeded.

Change Summary (click to expand)

The following is a summary of changes in this pull request made by me, your security buddy 🤖. Note that this summary is auto-generated and not meant to be a definitive list of security issues but rather a helpful summary from a security perspective.

Summary:

This pull request introduces several changes related to the integration of the Kiuwan Scanner tool with the DefectDojo application security management platform. The changes include updates to the documentation, the addition of a new parser for Kiuwan Software Composition Analysis (SCA) scan results, and the inclusion of sample Kiuwan SCA scan data for testing purposes.

The key highlights of these changes are:

Documentation Updates: The changes improve the documentation for the Kiuwan Scanner integration, including the addition of information about the Kiuwan REST API for exporting scan results and the classification of Kiuwan as a Static Application Security Testing (SAST) tool.
Kiuwan SCA Parser Implementation: A new parser has been added to the DefectDojo application to handle the import of Kiuwan SCA scan results. The parser is designed to accurately map the Kiuwan scan data to the DefectDojo finding format, including details such as CVEs, CWEs, CVSS scores, and EPSS data.
Deduplication and Muting of Findings: The parser includes logic to deduplicate findings and handle muted findings from the Kiuwan scans, which is an important security feature to ensure that the DefectDojo application only reports on active, unresolved vulnerabilities.
Sample Kiuwan SCA Scan Data: The pull request includes sample Kiuwan SCA scan data in JSON format, which can be used for unit testing and integration testing purposes. This data includes both a scan with no vulnerabilities and a scan with multiple high-severity vulnerabilities affecting various software components.

Overall, these changes demonstrate a focused effort to improve the integration and management of Kiuwan security scan data within the DefectDojo application. From an application security perspective, these changes are positive and should help organizations better identify, prioritize, and address security vulnerabilities in their applications.

Files Changed:

docs/content/en/integrations/parsers/file/kiuwan.md: Documentation updates for the Kiuwan Scanner integration.
docs/content/en/integrations/parsers/file/kiuwan-sca.md: New documentation for the Kiuwan SCA (Software Composition Analysis) integration.
dojo/tools/kiuwan_sca/parser.py: Implementation of the Kiuwan SCA parser.
unittests/scans/kiuwan-sca/kiuwan_sca_no_vuln.json: Sample Kiuwan SCA scan data with no vulnerabilities.
dojo/settings/settings.dist.py: Addition of the Kiuwan SCA parser to the DefectDojo application.
unittests/scans/kiuwan-sca/kiuwan_sca_many_vuln.json: Sample Kiuwan SCA scan data with multiple high-severity vulnerabilities.
unittests/tools/test_kiuwan_sca_parser.py: Unit tests for the Kiuwan SCA parser.

Powered by DryRun Security

… kiuwan-sca * 'kiuwan-sca' of github.com:mwager/django-DefectDojo: Update versions in application files Product Metrics: Performance Enhancements (DefectDojo#10059) String Based Filtering: Follow on for DefectDojo#10038 (DefectDojo#10050) update semgrep tests (DefectDojo#10058) Jira Webhook: Reorg logging and responses (DefectDojo#10049) Similar Findings: Create Toggle (DefectDojo#10047) Bump social-auth-app-django from 5.4.0 to 5.4.1 (DefectDojo#10026) Update versions in application files Update versions in application files Updated DryRun Security config (DefectDojo#10037) Filtering Performance: Add opt-in setting for converting to string ba… (DefectDojo#10038) Updates to semgrep parser (DefectDojo#10033) Update versions in application files

manuel-sommer · 2024-04-30T06:45:40Z

dojo/tools/kiuwan_sca/parser.py

+            finding.mitigation = "See Kiuwan Web UI"
+            finding.static_finding = True
+
+            key = hashlib.sha256(


Hi @mwager, the PR looks quite good.

I don't know if you need the key at all if you choose DEDUPE_ALGO_HASH_CODE.

Thx! As I understood, this is only for deduping per import? i.e. removing dupes in one import file, therefore preventing further deduplication later? Maybe I got this wrong then?

But I can remove it if this is not needed. We tested it with lots of prod data and it does what we expect :)

Let's wait for the reviews of the maintainers. Ah, good point, maybe you are right, but why would you have duplicates in one file?

Good question :) TBH I just followed the Kiuwan SAST importer logic: https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/tools/kiuwan/parser.py#L118

which is using the default legacy behavior based on:

static scanner: ['title', 'cwe', 'line', 'file_path', 'description']

So probably I got this wrong, let's wait for other reviewers and I am happy to clean this up if necessary.

mwager · 2024-05-07T06:15:01Z

@mtesauro Any more work needed from my side? I don't understand the error from the "Detect Merge conflicts" action:

See here: https://github.com/DefectDojo/django-DefectDojo/actions/runs/8889852139/job/24429470629?pr=10064

Or is it just that more reviewers need to approve?

mtesauro · 2024-06-10T13:59:44Z

@mwager

Any more work needed from my side? I don't understand the error from the "Detect Merge conflicts" action:

That "Detect Merge conflicts" is just GH having issues running GH Actions and nothing you need to do. You're good for now as there's an issue with the REST tests we're trying to figure out so hold tight for a bit and we can start the approvals once we sort out whatever is going on with the REST tests.

github-actions · 2024-06-21T21:59:26Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-06-24T07:41:41Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

github-actions · 2024-07-03T20:19:27Z

This pull request has conflicts, please resolve those before we can evaluate the pull request.

github-actions · 2024-07-04T05:41:22Z

Conflicts have been resolved. A maintainer will review the pull request shortly.

mwager · 2024-07-04T05:58:36Z

@mtesauro We really need this in master because via Kubernetes we would need to build and host our own images to use it. Without this Parser we cannot import our supply chain scans which is quite an issue. I think the Rest test Issues are fixed now?

mtesauro · 2024-07-04T16:51:36Z

@mwager

This PR is targeting the dev branch so, once it's got the needed approvals, it will be merged into main/master on the first Monday in August and be part of version 2.37.0

I just approved the tests to run, assuming those are good, we can start getting the needed approvals. Thanks for you patience - summer is 'interesting' with people going on holiday and such.

mtesauro · 2024-07-05T21:09:39Z

@mwager
Hate to suggest this but it may make sense to close this PR and open a new one with the same changes. It's likely easier that trying to get this one green on the GH Action tests. We recently did several updates to brittle/flaky tests and I suspect this PR has those old tests associated with it since I just re-kicked off the tests and the same ones failed again.

mwager · 2024-07-06T09:47:10Z

Re-opened here, hope this is good to go now :)

mwager added 17 commits April 25, 2024 10:50

Update kiuwan docs

0ac5a69

Add Kiuwan SCA test files

718d8b1

Add Kiuwan SCA Parser Unittest

c44d701

Add Kiuwan SCA parser implementation

54e50d9

Add more fields to the Kiuwan SCA parser

9b4e9ef

Update parser docs

1d8146d

Add test case for "muted" findings

a137892

Update finding title

ae533cd

Minor cleanupo

4374447

Update hashing logic

ab6ad9f

Remove print statement

9660329

Add optional epss support

d07b9d5

Add epss unit test

cd37fe9

Add custom deduplication logic as default static is not enough for SCA

274fb16

Remove cve as it is not allowed for deduplication

9345115

Set finding title to component name as this makes more sense within t…

fb052cc

…he UI display

Merge branch 'DefectDojo:master' into kiuwan-sca

84ec7f7

github-actions bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR docs unittests parser labels Apr 30, 2024

mwager added 3 commits April 30, 2024 07:35

Fix lint warnings

690e6cd

Fix another lint warning

13a6bf5

manuel-sommer reviewed Apr 30, 2024

View reviewed changes

github-actions bot added the conflicts-detected label Jun 21, 2024

Merge branch 'dev' into kiuwan-sca

fc44d4f

github-actions bot removed the conflicts-detected label Jun 24, 2024

mwager added 4 commits June 24, 2024 07:46

Fix lint error

eb1711d

fix lint errors

dbb0ed9

fix lint errors

3bbf308

fix lint errors

70bdd19

github-actions bot added the conflicts-detected label Jul 3, 2024

Merge branch 'dev' into kiuwan-sca

a137ba3

github-actions bot removed the conflicts-detected label Jul 4, 2024

mwager added 2 commits July 4, 2024 07:47

Fix lint

589b249

Fix lint

5c8a080

mwager closed this Jul 6, 2024

mwager mentioned this pull request Jul 6, 2024

New Parser: Kiuwan SCA #10522

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

New Parser: Kiuwan SCA Scan #10064

New Parser: Kiuwan SCA Scan #10064

mwager commented Apr 30, 2024 •

edited

Loading

dryrunsecurity bot commented Apr 30, 2024 •

edited

Loading

manuel-sommer Apr 30, 2024

mwager Apr 30, 2024

manuel-sommer Apr 30, 2024

mwager Apr 30, 2024

mwager commented May 7, 2024 •

edited

Loading

mtesauro commented Jun 10, 2024

github-actions bot commented Jun 21, 2024

github-actions bot commented Jun 24, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 4, 2024

mwager commented Jul 4, 2024

mtesauro commented Jul 4, 2024

mtesauro commented Jul 5, 2024

mwager commented Jul 6, 2024

New Parser: Kiuwan SCA Scan #10064

New Parser: Kiuwan SCA Scan #10064

Conversation

mwager commented Apr 30, 2024 • edited Loading

dryrunsecurity bot commented Apr 30, 2024 • edited Loading

manuel-sommer Apr 30, 2024

Choose a reason for hiding this comment

mwager Apr 30, 2024

Choose a reason for hiding this comment

manuel-sommer Apr 30, 2024

Choose a reason for hiding this comment

mwager Apr 30, 2024

Choose a reason for hiding this comment

mwager commented May 7, 2024 • edited Loading

mtesauro commented Jun 10, 2024

github-actions bot commented Jun 21, 2024

github-actions bot commented Jun 24, 2024

github-actions bot commented Jul 3, 2024

github-actions bot commented Jul 4, 2024

mwager commented Jul 4, 2024

mtesauro commented Jul 4, 2024

mtesauro commented Jul 5, 2024

mwager commented Jul 6, 2024

mwager commented Apr 30, 2024 •

edited

Loading

dryrunsecurity bot commented Apr 30, 2024 •

edited

Loading

mwager commented May 7, 2024 •

edited

Loading