summarizing multiple similar findings into problems

[LeoOMaia]

Description

• Feature:
  We created a "Problems" tab that disambiguates similar findings based on the script_id that detected them. This allows us to consolidate most findings into a single problem, enabling the vulnerability analyst to more accurately identify all types of issues without duplication.
• Bug fix implemented by this PR:
  A bug in the OpenVAS XML parser has been fixed, where it was not correctly identifying the script_id and was always returning None.

Test results

We tested the creation and association of findings to a specific problem according to the JSON file we provided, which already identifies similar findings by grouping them based on their script_id. We also tested that after creating the findings and problems, deleting a finding would update the problem by reducing the number of associated findings, and if all findings related to a problem were deleted, the problem would be automatically removed. Additionally, we verified the logic where if all findings become inactive, the problem status changes from open to closed, and if at least one finding remains active, the problem stays open.

[dryrunsecurity]

DryRun Security Summary

The pull request enhances DefectDojo's security management capabilities by introducing a new Problem model, integrating security scanning tools like Nuclei, and improving problem tracking and user interface functionality.

Expand for full summary

Summary:

The code changes in this pull request focus on enhancing the functionality and security capabilities of the DefectDojo application, a popular open-source application security management platform. The key changes include:

1. Sensitive Code Path Tracking: The changes update the .dryrunsecurity.yaml file to include a new sensitive code path, 'dojo/problem/*.py', indicating that the "problem" module or functionality within the application may handle security-critical data.
2. Problem Management Integration: The code introduces a new Problem model and associated functionality to allow the application to better organize and manage security findings by associating them with specific problems or vulnerabilities.
3. Asynchronous Task Handling: The changes include the addition of a new asynchronous task, daily_cache_update, which is responsible for updating a cache related to the problem management functionality.
4. Improved User Interface: The code updates the application's user interface, including the addition of new views and templates for listing and managing problems and their associated findings.
5. Security Scanning Integration: The changes integrate the Nuclei security scanning tool with the DefectDojo application, allowing the platform to ingest and manage the findings from Nuclei scans.

From an application security perspective, these changes demonstrate a strong focus on improving the overall security posture of the DefectDojo application. The introduction of the Problem model, the tracking of sensitive code paths, and the integration of security scanning tools all contribute to a more comprehensive and effective security management solution.

Files Changed:

1. .dryrunsecurity.yaml: This file is used for security scanning and code auditing purposes. The changes add a new sensitive code path, 'dojo/problem/*.py', indicating that the "problem" module may handle security-critical data.
2. dojo/models.py: This file introduces a new Problem model, which can be used to associate security findings with specific problems or vulnerabilities.
3. dojo/db_migrations/0219_problem_finding_problem.py: This database migration file adds the Problem model and a problem field to the existing Finding model, allowing the association of findings with problems.
4. dojo/problem/helper.py: This file contains utility functions for managing the mapping between findings and problems, including caching and validation mechanisms.
5. dojo/problem/update_mappings.py: This file adds a new asynchronous task, daily_cache_update, responsible for updating the cache related to the problem management functionality.
6. dojo/problem/views.py: This file introduces new views for listing problems, including open, closed, and all problems, as well as a view for displaying the findings associated with a specific problem.
7. dojo/templates/dojo/problem_findings.html: This new template file is used to display the findings associated with a specific problem.
8. dojo/templatetags/problem_tags.py: This file contains custom Django template tags related to the problem management functionality.
9. dojo/templates/dojo/problems_list_snippet.html: This template file is used to display the list of problems.
10. dojo/tools/nmap/parser.py: This file has been updated to associate the findings from Nmap scans with the corresponding problems in the application.
11. dojo/tools/openvas/xml_parser.py: Similar to the Nmap parser, this file has been updated to associate the findings from OpenVAS scans with the corresponding problems.
12. dojo/tools/nuclei/parser.py: This new file introduces the integration of the Nuclei security scanning tool with the DefectDojo application, allowing the platform to ingest and manage the findings from Nuclei scans.

Code Analysis

We ran 9 analyzers against 21 files and 2 analyzers had findings. 7 analyzers had no findings.

Analyzer	Findings
Configured Codepaths Analyzer	17 findings
Sensitive Files Analyzer	4 findings

Overall Riskiness

🔴 Risk threshold exceeded.

We've notified @mtesauro, @grendel513.

View PR in the DryRun Dashboard.

[cunha]

The most important comments are changing some of the algorithms to have better asymptotic performance.

[cunha]

Some additional suggestions in the comments.