Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New Parser: Kiuwan SCA Scan #10064

Closed
wants to merge 28 commits into from
Closed
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
0ac5a69
Update kiuwan docs
mwager Apr 25, 2024
718d8b1
Add Kiuwan SCA test files
mwager Apr 25, 2024
c44d701
Add Kiuwan SCA Parser Unittest
mwager Apr 25, 2024
54e50d9
Add Kiuwan SCA parser implementation
mwager Apr 25, 2024
9b4e9ef
Add more fields to the Kiuwan SCA parser
mwager Apr 25, 2024
1d8146d
Update parser docs
mwager Apr 25, 2024
a137892
Add test case for "muted" findings
mwager Apr 25, 2024
ae533cd
Update finding title
mwager Apr 25, 2024
4374447
Minor cleanupo
mwager Apr 25, 2024
ab6ad9f
Update hashing logic
mwager Apr 25, 2024
9660329
Remove print statement
mwager Apr 25, 2024
d07b9d5
Add optional epss support
mwager Apr 26, 2024
cd37fe9
Add epss unit test
mwager Apr 26, 2024
274fb16
Add custom deduplication logic as default static is not enough for SCA
mwager Apr 26, 2024
9345115
Remove cve as it is not allowed for deduplication
mwager Apr 26, 2024
fb052cc
Set finding title to component name as this makes more sense within t…
mwager Apr 26, 2024
84ec7f7
Merge branch 'DefectDojo:master' into kiuwan-sca
mwager Apr 30, 2024
690e6cd
Fix lint warnings
mwager Apr 30, 2024
596760a
Merge branch 'kiuwan-sca' of github.com:mwager/django-DefectDojo into…
mwager Apr 30, 2024
13a6bf5
Fix another lint warning
mwager Apr 30, 2024
fc44d4f
Merge branch 'dev' into kiuwan-sca
mwager Jun 24, 2024
eb1711d
Fix lint error
mwager Jun 24, 2024
dbb0ed9
fix lint errors
mwager Jun 24, 2024
3bbf308
fix lint errors
mwager Jun 24, 2024
70bdd19
fix lint errors
mwager Jun 24, 2024
a137ba3
Merge branch 'dev' into kiuwan-sca
mwager Jul 4, 2024
589b249
Fix lint
mwager Jul 4, 2024
5c8a080
Fix lint
mwager Jul 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions docs/content/en/integrations/parsers/file/kiuwan-sca.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
title: "Kiuwan Scanner (SCA i.e. \"Insights\")"
toc_hide: true
---
Import Kiuwan Insights Scan in JSON format. Export via API endpoint `insights/analysis/security` as json and create a file for importing to DefectDojo.

### Example Code

Data can be fetched from the [Kiuwan REST API](https://static.kiuwan.com/rest-api/kiuwan-rest-api.html) like this:

```
import requests, json
headers = {'Authorization': 'Basic $KIUWAN_TOKEN', 'Accept' : 'application/json'}

appName = "Test"
analysisCode = "A-111-1111111111"

URL = "https://api.kiuwan.com/insights/analysis/security?analysisCode=" + analysisCode + "&application=" + appName
response = requests.get(url = URL, headers = headers)
jsonResponse = r.json()
data = jsonResponse["data"]
saveFile("result.json", json.dumps(data, indent=2))
```

### Sample Scan Data
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan-sca).
6 changes: 3 additions & 3 deletions docs/content/en/integrations/parsers/file/kiuwan.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
title: "Kiuwan Scanner"
title: "Kiuwan Scanner (SAST)"
toc_hide: true
---
Import Kiuwan Scan in CSV format. Export as CSV Results on Kiuwan.
Import Kiuwan SAST Scan in CSV format. Export as CSV Results on Kiuwan, or via the [Kiuwan REST API](https://static.kiuwan.com/rest-api/kiuwan-rest-api.html) endpoint `vulnerabilities/export` (type=csv).

### Sample Scan Data
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan).
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan).
4 changes: 3 additions & 1 deletion dojo/settings/settings.dist.py
Original file line number Diff line number Diff line change
Expand Up @@ -1250,7 +1250,8 @@ def saml2_attrib_map_format(dict):
'Snyk Code Scan': ['vuln_id_from_tool', 'file_path'],
'Bearer CLI': ['title', 'severity'],
'Nancy Scan': ['title', 'vuln_id_from_tool'],
'Wiz Scan': ['title', 'description', 'severity']
'Wiz Scan': ['title', 'description', 'severity'],
'Kiuwan SCA Scan': ['description', 'severity', 'component_name', 'component_version', 'cwe']
}

# Override the hardcoded settings here via the env var
Expand Down Expand Up @@ -1467,6 +1468,7 @@ def saml2_attrib_map_format(dict):
'Nosey Parker Scan': DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
'Bearer CLI': DEDUPE_ALGO_HASH_CODE,
'Wiz Scan': DEDUPE_ALGO_HASH_CODE,
'Kiuwan SCA Scan': DEDUPE_ALGO_HASH_CODE,
}

# Override the hardcoded settings here via the env var
Expand Down
Empty file.
89 changes: 89 additions & 0 deletions dojo/tools/kiuwan_sca/parser.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
import json
import hashlib

from dojo.models import Finding

__author__ = "mwager"


class KiuwanSCAParser:
SEVERITY = {
"-": "Low",
"LOW": "Low",
"MEDIUM": "Medium",
"HIGH": "High",
"CRITICAL": "Critical",
"Low": "Low",
"Medium": "Medium",
"High": "High",
"Critical": "Critical"
}

def get_scan_types(self):
return ["Kiuwan SCA Scan"]

def get_label_for_scan_types(self, scan_type):
return scan_type

def get_description_for_scan_types(self, scan_type):
return "Import Kiuwan Insights Scan in JSON format. Export as JSON using Kiuwan REST API."

def get_findings(self, filename, test):
data = json.load(filename)
dupes = dict()

for row in data:
# if a finding was "muted" in the Kiuwan UI, we ignore it (e.g. marked as false positive)
if row["muted"] is True:
continue

finding = Finding(test=test)
finding.unique_id_from_tool = row["id"]
finding.cve = row["cve"]
finding.description = row["description"]
finding.severity = self.SEVERITY[row["securityRisk"]]

if "components" in row and len(row["components"]) > 0:
finding.component_name = row["components"][0]["artifact"]
finding.component_version = row["components"][0]["version"]
finding.title = finding.component_name + " v" + str(finding.component_version)

if not finding.title:
finding.title = row["cve"]

if "cwe" in row:
try:
finding.cwe = int(row["cwe"].replace("CWE-", ""))
except Exception:
pass

if "epss_score" in row:
finding.epss_score = row["epss_score"]
if "epss_percentile" in row:
finding.epss_percentile = row["epss_percentile"]

if "cVSSv3BaseScore" in row:
finding.cvssv3_score = float(row["cVSSv3BaseScore"])

finding.references = "See Kiuwan Web UI"
finding.mitigation = "See Kiuwan Web UI"
finding.static_finding = True

key = hashlib.sha256(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mwager, the PR looks quite good.

I don't know if you need the key at all if you choose DEDUPE_ALGO_HASH_CODE.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thx! As I understood, this is only for deduping per import? i.e. removing dupes in one import file, therefore preventing further deduplication later? Maybe I got this wrong then?

grafik

But I can remove it if this is not needed. We tested it with lots of prod data and it does what we expect :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's wait for the reviews of the maintainers. Ah, good point, maybe you are right, but why would you have duplicates in one file?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good question :) TBH I just followed the Kiuwan SAST importer logic: https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/tools/kiuwan/parser.py#L118

which is using the default legacy behavior based on:

static scanner:  ['title', 'cwe', 'line', 'file_path', 'description']

So probably I got this wrong, let's wait for other reviewers and I am happy to clean this up if necessary.

(
finding.description
+ "|"
+ finding.severity
+ "|"
+ finding.component_name
+ "|"
+ finding.component_version
+ "|"
+ str(finding.cwe)
).encode("utf-8")
).hexdigest()

if key not in dupes:
dupes[key] = finding

return list(dupes.values())
Loading
Loading