Burp Enterprise: Support newer format

[Maffooch]

• Simplify/Solidify BurpE parser to work with newer formats
• Add support for CAPEC plus multiple CWE via vulnerability IDs
• Clean up finding format

[sc-8300]

[dryrunsecurity]

DryRun Security Summary

The provided code changes focus on enhancing the integration and parsing of Burp Enterprise scan reports in the OWASP Dependency-Track application, including updates to configuration files, vulnerability URL generation, and the BurpEnterpriseParser class, as well as the introduction of new unit tests to ensure the continued integrity and security of the application.

Expand for full summary

Summary:

The provided code changes cover various updates and improvements to the OWASP Dependency-Track application, with a focus on enhancing the integration and parsing of Burp Enterprise scan reports. The changes include updates to configuration files, vulnerability URL generation, and the BurpEnterpriseParser class.

From an application security perspective, the key changes are:

1. Updating Vulnerability Source URLs: The changes to the dojo/settings/settings.dist.py file add new vulnerability source URLs for CAPEC and CWE definitions, which can provide more context and information for users when reviewing security findings.

2. Improving Vulnerability URL Generation: The changes to the display_tags.py file enhance the vulnerability_url function to handle special cases for vulnerability ID prefixes, ensuring that the generated URLs are correct and accessible.

3. Enhancing Burp Enterprise Report Parsing: The changes to the dojo/tools/burp_enterprise/parser.py file significantly improve the parsing of Burp Enterprise HTML reports, including more robust HTML parsing, structured extraction of finding details, and better handling of vulnerability IDs and CWE information.

4. Expanding Unit Tests: The changes to the unittests/tools/test_burp_enterprise_parser.py file introduce new unit tests that cover various formats of the Burp Enterprise scan report, including the identification of security issues such as CORS policy vulnerabilities, WAF detection, HSTS enforcement, and Content Security Policy (CSP) issues.

Overall, these changes focus on improving the security and reliability of the OWASP Dependency-Track application, particularly in the area of integrating and analyzing Burp Enterprise scan reports. The updates to the vulnerability source URLs, URL generation, and the BurpEnterpriseParser class are positive security enhancements, while the expanded unit tests help ensure the continued integrity and security of the application.

Files Changed:

1. dojo/settings/.settings.dist.py.sha256sum: The SHA-256 hash value of the dojo/settings/.settings.dist.py file has been updated, indicating a change to the configuration file.
2. dojo/settings/settings.dist.py: The changes add new vulnerability source URLs for CAPEC and CWE definitions, and expand the list of acceptable file types that can be uploaded to DefectDojo objects.
3. dojo/templatetags/display_tags.py: The changes improve the vulnerability_url function to handle special cases for vulnerability ID prefixes, making the URL generation more robust and reliable.
4. dojo/tools/burp_enterprise/parser.py: The changes significantly enhance the BurpEnterpriseParser class, improving the parsing of Burp Enterprise HTML reports, extracting structured finding details, and handling vulnerability IDs and CWE information.
5. unittests/tools/test_burp_enterprise_parser.py: The changes introduce new unit tests that cover various formats of the Burp Enterprise scan report, including the identification of security issues such as CORS policy vulnerabilities, WAF detection, HSTS enforcement, and CSP issues.

Code Analysis

We ran 9 analyzers against 6 files and 0 analyzers had findings. 9 analyzers had no findings.

Riskiness

🟢 Risk threshold not exceeded.

View PR in the DryRun Dashboard.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[mtesauro]

Approved

[fopinappb]

Hi @Maffooch I see in this refactor the commonly "dedupe within parser" was left out.

Is this the new approach, leave it to the post-save deduplication?

That would make sense to me to reduce complexity and chance of inconsistency

I see hashcode fields were not changed and continue to NOT include description which is the only place with the full path for the finding. This will make issues of same category on different paths/parameters to be considered duplicates

Should it be like that?

[Maffooch]

Hi @fopinappb thanks for the questions! The decision to dedupe within the parser or not is often a nuanced internal discussion. I have found that it is entirely dependent on how the report is structured. With Burp Enterprise specifically, the reports are structured with the title of a finding, and then all endpoints underneath it to reflect the many instances of a given vulnerability.

The changes made in this PR still do the dedupe within the parser, but it is not done in the same way as many other parsers. This is due to the table of contents at the beginning of a burp enterprise report. This table of contents is the best place to determine what endpoints are set on a give vulnerability. The parser works by first scraping the table of contents, and aggregating results there based on finding title. Once that is complete, more details about the vulnerability itself are added to the aggregated value later on in the create_findings function

I see hashcode fields were not changed and continue to NOT include description which is the only place with the full path for the finding. This will make issues of same category on different paths/parameters to be considered duplicates

Do you have an example of a report with query parameters that can be used to determine how to proceed? I would imagine the table of contents on the report would list paths with different query parameters as unique items, but that may not be the case

[fopina]

Thanks for taking the time to comment @Maffooch

I think my main question/misunderstanding is probably addressed in the other issue I opened for burp parser too, so I'll continue there for focusing!