-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
New Parser: Kiuwan SCA Scan #10064
Closed
Closed
New Parser: Kiuwan SCA Scan #10064
Changes from 20 commits
Commits
Show all changes
28 commits
Select commit
Hold shift + click to select a range
0ac5a69
Update kiuwan docs
mwager 718d8b1
Add Kiuwan SCA test files
mwager c44d701
Add Kiuwan SCA Parser Unittest
mwager 54e50d9
Add Kiuwan SCA parser implementation
mwager 9b4e9ef
Add more fields to the Kiuwan SCA parser
mwager 1d8146d
Update parser docs
mwager a137892
Add test case for "muted" findings
mwager ae533cd
Update finding title
mwager 4374447
Minor cleanupo
mwager ab6ad9f
Update hashing logic
mwager 9660329
Remove print statement
mwager d07b9d5
Add optional epss support
mwager cd37fe9
Add epss unit test
mwager 274fb16
Add custom deduplication logic as default static is not enough for SCA
mwager 9345115
Remove cve as it is not allowed for deduplication
mwager fb052cc
Set finding title to component name as this makes more sense within t…
mwager 84ec7f7
Merge branch 'DefectDojo:master' into kiuwan-sca
mwager 690e6cd
Fix lint warnings
mwager 596760a
Merge branch 'kiuwan-sca' of github.com:mwager/django-DefectDojo into…
mwager 13a6bf5
Fix another lint warning
mwager fc44d4f
Merge branch 'dev' into kiuwan-sca
mwager eb1711d
Fix lint error
mwager dbb0ed9
fix lint errors
mwager 3bbf308
fix lint errors
mwager 70bdd19
fix lint errors
mwager a137ba3
Merge branch 'dev' into kiuwan-sca
mwager 589b249
Fix lint
mwager 5c8a080
Fix lint
mwager File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- | ||
title: "Kiuwan Scanner (SCA i.e. \"Insights\")" | ||
toc_hide: true | ||
--- | ||
Import Kiuwan Insights Scan in JSON format. Export via API endpoint `insights/analysis/security` as json and create a file for importing to DefectDojo. | ||
|
||
### Example Code | ||
|
||
Data can be fetched from the [Kiuwan REST API](https://static.kiuwan.com/rest-api/kiuwan-rest-api.html) like this: | ||
|
||
``` | ||
import requests, json | ||
headers = {'Authorization': 'Basic $KIUWAN_TOKEN', 'Accept' : 'application/json'} | ||
|
||
appName = "Test" | ||
analysisCode = "A-111-1111111111" | ||
|
||
URL = "https://api.kiuwan.com/insights/analysis/security?analysisCode=" + analysisCode + "&application=" + appName | ||
response = requests.get(url = URL, headers = headers) | ||
jsonResponse = r.json() | ||
data = jsonResponse["data"] | ||
saveFile("result.json", json.dumps(data, indent=2)) | ||
``` | ||
|
||
### Sample Scan Data | ||
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan-sca). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,8 +1,8 @@ | ||
--- | ||
title: "Kiuwan Scanner" | ||
title: "Kiuwan Scanner (SAST)" | ||
toc_hide: true | ||
--- | ||
Import Kiuwan Scan in CSV format. Export as CSV Results on Kiuwan. | ||
Import Kiuwan SAST Scan in CSV format. Export as CSV Results on Kiuwan, or via the [Kiuwan REST API](https://static.kiuwan.com/rest-api/kiuwan-rest-api.html) endpoint `vulnerabilities/export` (type=csv). | ||
|
||
### Sample Scan Data | ||
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan). | ||
Sample Kiuwan Scanner scans can be found [here](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/kiuwan). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,89 @@ | ||
import json | ||
import hashlib | ||
|
||
from dojo.models import Finding | ||
|
||
__author__ = "mwager" | ||
|
||
|
||
class KiuwanSCAParser: | ||
SEVERITY = { | ||
"-": "Low", | ||
"LOW": "Low", | ||
"MEDIUM": "Medium", | ||
"HIGH": "High", | ||
"CRITICAL": "Critical", | ||
"Low": "Low", | ||
"Medium": "Medium", | ||
"High": "High", | ||
"Critical": "Critical" | ||
} | ||
|
||
def get_scan_types(self): | ||
return ["Kiuwan SCA Scan"] | ||
|
||
def get_label_for_scan_types(self, scan_type): | ||
return scan_type | ||
|
||
def get_description_for_scan_types(self, scan_type): | ||
return "Import Kiuwan Insights Scan in JSON format. Export as JSON using Kiuwan REST API." | ||
|
||
def get_findings(self, filename, test): | ||
data = json.load(filename) | ||
dupes = dict() | ||
|
||
for row in data: | ||
# if a finding was "muted" in the Kiuwan UI, we ignore it (e.g. marked as false positive) | ||
if row["muted"] is True: | ||
continue | ||
|
||
finding = Finding(test=test) | ||
finding.unique_id_from_tool = row["id"] | ||
finding.cve = row["cve"] | ||
finding.description = row["description"] | ||
finding.severity = self.SEVERITY[row["securityRisk"]] | ||
|
||
if "components" in row and len(row["components"]) > 0: | ||
finding.component_name = row["components"][0]["artifact"] | ||
finding.component_version = row["components"][0]["version"] | ||
finding.title = finding.component_name + " v" + str(finding.component_version) | ||
|
||
if not finding.title: | ||
finding.title = row["cve"] | ||
|
||
if "cwe" in row: | ||
try: | ||
finding.cwe = int(row["cwe"].replace("CWE-", "")) | ||
except Exception: | ||
pass | ||
|
||
if "epss_score" in row: | ||
finding.epss_score = row["epss_score"] | ||
if "epss_percentile" in row: | ||
finding.epss_percentile = row["epss_percentile"] | ||
|
||
if "cVSSv3BaseScore" in row: | ||
finding.cvssv3_score = float(row["cVSSv3BaseScore"]) | ||
|
||
finding.references = "See Kiuwan Web UI" | ||
finding.mitigation = "See Kiuwan Web UI" | ||
finding.static_finding = True | ||
|
||
key = hashlib.sha256( | ||
( | ||
finding.description | ||
+ "|" | ||
+ finding.severity | ||
+ "|" | ||
+ finding.component_name | ||
+ "|" | ||
+ finding.component_version | ||
+ "|" | ||
+ str(finding.cwe) | ||
).encode("utf-8") | ||
).hexdigest() | ||
|
||
if key not in dupes: | ||
dupes[key] = finding | ||
|
||
return list(dupes.values()) |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hi @mwager, the PR looks quite good.
I don't know if you need the key at all if you choose DEDUPE_ALGO_HASH_CODE.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thx! As I understood, this is only for deduping per import? i.e. removing dupes in one import file, therefore preventing further deduplication later? Maybe I got this wrong then?
But I can remove it if this is not needed. We tested it with lots of prod data and it does what we expect :)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's wait for the reviews of the maintainers. Ah, good point, maybe you are right, but why would you have duplicates in one file?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good question :) TBH I just followed the Kiuwan SAST importer logic: https://github.com/DefectDojo/django-DefectDojo/blob/master/dojo/tools/kiuwan/parser.py#L118
which is using the default legacy behavior based on:
So probably I got this wrong, let's wait for other reviewers and I am happy to clean this up if necessary.