Fix nuclei deduplication

[fopina]

Description

Many nuclei templates have several matchers, such as tech-detect
If there are several matchers, each is reported as a separate finding.

The current parser puts matcher under component_name yet the hash fields do not include component_name resulting in findings being closed as duplicates when they're not (such as multiple technologies for the mentioned template)

The deduplication done within the nuclei parser itself considers matcher as key already which was not aligned with hash fields, as those didn't consider component_name.

alternative

Another option could be appending matcher to title instead of changing the hash code fields. This would allow seeing the matcher in the findings list (that does not show components) but we would lose the ability to filter by component.

[dryrunsecurity]

DryRun Security Summary

The pull request focuses on updates related to the Nuclei scanner integration within the DefectDojo application, including updating test files, configuration files, and the NucleiParser class to improve the handling and deduplication of Nuclei scan results.

Expand for full summary

Summary:

The code changes in this pull request primarily focus on updates related to the Nuclei scanner integration within the DefectDojo application. The changes include:

1. Updating the unittests/scans/nuclei/multiple_matches.json file with new entries from running the "Wappalyzer Technology Detection" template on a test website. This includes detecting the "Genexus" technology and the presence of a Google reCAPTCHA, which should be reviewed to ensure the reCAPTCHA implementation is secure.

2. Updating the SHA-256 checksum file dojo/settings/.settings.dist.py.sha256sum to reflect changes made to the settings.dist.py configuration file. This is a routine update to maintain the integrity of the application's configuration files.

3. Enhancing the NucleiParser class in the unittests/tools/test_nuclei_parser.py file to improve the handling of Nuclei scan results with multiple matches for a single template. This helps ensure that the security findings reported by the Nuclei scanner are accurately reflected in the DefectDojo application.

4. Modifying the HASHCODE_FIELDS_PER_SCANNER and DEDUPLICATION_ALGORITHM_PER_PARSER settings in the dojo/settings/settings.dist.py file to improve the deduplication of findings from the "Nuclei Scan" parser. This change can help enhance the accuracy and reliability of the vulnerability management process in DefectDojo.

Overall, these changes appear to be focused on improving the integration and handling of Nuclei scanner results within the DefectDojo application, which is a valuable contribution to the application's security capabilities.

Files Changed:

1. unittests/scans/nuclei/multiple_matches.json: This file was updated with new entries from running the "Wappalyzer Technology Detection" template on a test website, including the detection of the "Genexus" technology and a Google reCAPTCHA.

2. dojo/settings/.settings.dist.py.sha256sum: The SHA-256 checksum for the settings.dist.py configuration file was updated, likely due to changes made to the underlying configuration file.

3. unittests/tools/test_nuclei_parser.py: A new test case called test_parse_same_template_multiple_matches was added to the NucleiParser class to ensure proper handling of Nuclei scan results with multiple matches for a single template.

4. dojo/settings/settings.dist.py: The HASHCODE_FIELDS_PER_SCANNER and DEDUPLICATION_ALGORITHM_PER_PARSER settings were updated to improve the deduplication of findings from the "Nuclei Scan" parser.

Code Analysis

We ran 9 analyzers against 4 files and 0 analyzers had findings. 9 analyzers had no findings.

Riskiness

🟢 Risk threshold not exceeded.

View PR in the DryRun Dashboard.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[Maffooch]

@fopina this PR says there are conflicts, but idk what they are though... Pretty strange

[fopina]

@Maffooch I received the notification yet it's still missing approvals and I was considering just waiting for all approvals before resolving them

Yet, as I came here to comment, I also resolved them 👍 (it was the settings shasum)

[github-actions]

Conflicts have been resolved. A maintainer will review the pull request shortly.

[mtesauro]

Approved