Malware protection 6: Malware integration #3604

mallexxx · 2024-11-27T13:43:17Z

Task/Issue URL: https://app.asana.com/0/481882893211075/1208033567421351/f
BSK PR: duckduckgo/BrowserServicesKit#1099

Description:

add .malware threat and special page types

Optional E2E tests:

Run PIR E2E tests
Check this to run the Personal Information Removal end to end tests. If updating CCF, or any PIR related code, tick this.

Steps to test this PR:
0.1. Check out https://github.com/duckduckgo/content-scope-scripts/tree/pr-releases/pr-1268 and npm install; npm run build; add reference to the project
0.2. Check out https://github.com/duckduckgo/privacy-dashboard/tree/pr-releases/pr-250 and npm install; npm run build; add reference to the project

or use the Review build

Validate http://privacy-test-pages.site/security/badware/malware.html detects malware
Validate http://privacy-test-pages.site/security/badware/phishing.html detects phishing
Validate Privacy Dashboard displaying status on overridden navigation
Validate Help/Report links (not ready yet)

Definition of Done:

Does this PR satisfy our Definition of Done?

Internal references:

Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation

… and cleanup

…ting in code

Task/Issue URL: https://app.asana.com/0/481882893211075/1208033567421351/f BSK PR: duckduckgo/BrowserServicesKit#1092 **Description**: - Refactor APIClient to use Networking.APIRequestV2; implement generic request/response (BSK) **Optional E2E tests**: - [ ] Run PIR E2E tests Check this to run the Personal Information Removal end to end tests. If updating CCF, or any PIR related code, tick this. **Steps to test this PR**: 1. Validate Phishing Protection change sets and Match API calls work as before  **Definition of Done**: * [ ] Does this PR satisfy our [Definition of Done](https://app.asana.com/0/1202500774821704/1207634633537039/f)? --- ###### Internal references: [Pull Request Review Checklist](https://app.asana.com/0/1202500774821704/1203764234894239/f) [Software Engineering Expectations](https://app.asana.com/0/59792373528535/199064865822552) [Technical Design Template](https://app.asana.com/0/59792373528535/184709971311943) [Pull Request Documentation](https://app.asana.com/0/1202500774821704/1204012835277482/f)

Task/Issue URL: https://app.asana.com/0/1202406491309510/1208033567421351/f Tech design: https://app.asana.com/0/481882893211075/1208736595187321/f BSK PR: duckduckgo/BrowserServicesKit#1093 **Description**: - Update BSK ref with Malicious Site Protection data store refactoring - Update malicious site configuration file storage path to match the Configuration storage **Optional E2E tests**: - [ ] Run PIR E2E tests Check this to run the Personal Information Removal end to end tests. If updating CCF, or any PIR related code, tick this. **Steps to test this PR**: 1. Validate app config is loaded correctly (`ConfigurationStore.swift`) 2. Activate Feature Flag override for malicious site protections in Debug -> Feature Flag overrides 3. Visit https://privacy-test-pages.site/security/badware/phishing.html, validate phishing detection works without changes 4. Validate tests pass 5. Validate update manager updates the phishing data 6. validate `./scripts/update_phishing_detection_data.sh` updates malicious site data  **Definition of Done**: * [ ] Does this PR satisfy our [Definition of Done](https://app.asana.com/0/1202500774821704/1207634633537039/f)? --- ###### Internal references: [Pull Request Review Checklist](https://app.asana.com/0/1202500774821704/1203764234894239/f) [Software Engineering Expectations](https://app.asana.com/0/59792373528535/199064865822552) [Technical Design Template](https://app.asana.com/0/59792373528535/184709971311943) [Pull Request Documentation](https://app.asana.com/0/1202500774821704/1204012835277482/f) --------- Co-authored-by: Anh Do <[email protected]> Co-authored-by: Dominik Kapusta <[email protected]> Co-authored-by: Alessandro Boron <[email protected]> Co-authored-by: Michal Smaga <[email protected]> Co-authored-by: Anka <[email protected]> Co-authored-by: Sabrina Tardio <[email protected]> Co-authored-by: kshann <[email protected]> Co-authored-by: Anka <[email protected]> Co-authored-by: Anka <[email protected]> Co-authored-by: Dax Mobile <[email protected]> Co-authored-by: Anka <[email protected]> Co-authored-by: Sam Symons <[email protected]> Co-authored-by: Shilpa Modi <[email protected]> Co-authored-by: Anka <[email protected]> Co-authored-by: Maxim Tsoy <[email protected]> Co-authored-by: Fernando Bunn <[email protected]> Co-authored-by: Diego Rey Mendez <[email protected]> Co-authored-by: Anka <[email protected]>

…on-1

…formative logging. update malicious site data

SabrinaTardio

I left a couple of comments and questions!

SabrinaTardio · 2024-12-05T10:54:23Z

DuckDuckGo/Localizable.xcstrings

@@ -35845,6 +35845,18 @@
        }
      }
    },
+    "malware.error.page.tab.title" : {


we need translations

yes, as soon as we get our copy approved

SabrinaTardio · 2024-12-05T10:55:15Z

DuckDuckGo/MaliciousSiteProtection/MaliciousSiteProtectionManager.swift

@@ -46,11 +46,12 @@ extension MaliciousSiteProtectionManager {
    struct EmbeddedDataProvider: MaliciousSiteProtection.EmbeddedDataProviding {

        private enum Constants {
-            static let embeddedDataRevision = 1694418
-            static let phishingEmbeddedHashPrefixDataSHA = "d9eccd24d05ce16d4ab877574df728f69be6c7840aea00e1be11aeafffb0b1dc"
+            // TODO: Rollback the revision and filter (with -f) set when malware is available on the server


Can we remove it?

yes, this is to be removed before merging. for now it‘s there to test malware.html handling using local filters as remote doesn‘t catch it

SabrinaTardio · 2024-12-05T10:59:12Z

DuckDuckGo/PinnedTabs/View/PinnedTabView.swift

@@ -269,9 +270,18 @@ struct PinnedTabInnerView: View {
        }
    }

+    private var faviconImage: NSImage? {
+        if let error = model.error, (error as NSError as? URLError)?.code == .serverCertificateUntrusted || (error as NSError as? MaliciousSiteError != nil) {


I am not sure if we wanted this for the SSL. Can we double check with product?

SabrinaTardio · 2024-12-05T11:05:21Z

DuckDuckGo/Tab/TabExtensions/SpecialErrorPageTabExtension.swift

-            webView?.close()
-            return
+        Task { @MainActor in
+            await self.webView?.openNewTabFromErrorPage()


I don’t think SSL navigation was causing problems…
do we want to generalise this behaviour for all?

SabrinaTardio · 2024-12-05T11:08:31Z

UnitTests/TabExtensionsTests/ErrorPageTabExtensionTest.swift

@@ -195,7 +200,8 @@ final class ErrorPageTabExtensionTest: XCTestCase {
        errorPageExtention.leaveSiteAction()

        // THEN
-        XCTAssertTrue(mockWebView.goBackCalled)
+        XCTAssertTrue(closeTabCalled)


These tests are Failing… also probably we should update the titles

SabrinaTardio

Assuming that we want to have the same behaviour for SSL this looks good to me

…on-6

Task/Issue URL: https://app.asana.com/0/481882893211075/1208033567421351/f iOS PR: duckduckgo/iOS#3690 macOS PR: duckduckgo/macos-browser#3604

Task/Issue URL:https://app.asana.com/0/481882893211075/1208033567421351/f BSK PR: duckduckgo/BrowserServicesKit#1099 macOS PR: duckduckgo/macos-browser#3604

…on-6

Task/Issue URL: https://app.asana.com/0/481882893211075/1208033567421351/f iOS PR: macOS PR: duckduckgo/macos-browser#3604

…on-6

mallexxx added 12 commits November 22, 2024 21:24

Rename PhishingDetection to MaliciousSiteProtection; code adjustments…

afd0746

… and cleanup

fix linter issues

45c982d

Update UserText key names; Bump BSK; disable TODO warnings

f041a1a

refactor Malicious Site Protection APIClient

3cdf77d

fix test

aca479c

Update Malicious site data storing

712f7d6

fix BSK ref

21f4585

fix BSK ref

d135c21

Refactor malicious site data update manager

5687819

Refactor Special Error Types

35cf020

Package.resolved

8cb5739

Integrate Malware protection

6f29625

mallexxx added the includes BSK changes label Nov 27, 2024

mallexxx requested a review from SabrinaTardio November 27, 2024 13:43

mallexxx and others added 16 commits November 28, 2024 14:41

Update protections update script and data

87fd08c

validate revision returned by api in update script; fix revision upda…

9ef094d

…ting in code

temporarily include malware test page to the filters

be5bde1

fix sh linter issue

bafd937

Package.resolved

fcc6627

bump BSK

eb8be0b

Merge remote-tracking branch 'origin/main' into alex/malware-protecti…

4d40826

…on-1

bump BSK

87f5f47

Merge branch 'alex/malware-protection-1' into alex/malware-protection-5

fc9db2d

Merge branch 'alex/malware-protection-5' into alex/malware-protection-6

e380729

fix linter warning

5b82f9b

fix update_phishing_detection_data.sh; update malicious site data

30460c2

Merge branch 'alex/malware-protection-5' into alex/malware-protection-6

94f1595

fix updating hashes by update_phishing_detection_data.sh, add more in…

961e60f

…formative logging. update malicious site data

ship review fixes

61c94a0

SabrinaTardio reviewed Dec 5, 2024

View reviewed changes

fix tests

88dbfb1

SabrinaTardio approved these changes Dec 5, 2024

View reviewed changes

mallexxx added 2 commits December 5, 2024 20:04

add Warning: to Malware title

be2f2ee

go back for Ssl error page on Leave site action

941332a

mallexxx mentioned this pull request Dec 6, 2024

Malware protection (BSK macOS) duckduckgo/iOS#3690

Merged

14 tasks

mallexxx and others added 8 commits December 6, 2024 17:34

Merge remote-tracking branch 'origin/main' into alex/malware-protecti…

3ee8985

…on-6

update copy xcstrings

8893893

update data

d672847

fix page reloading on "Accept risk"

c92cb45

remove TODO-s

4564fcf

fix tests

01c9932

update BSK dependency

f39d20f

update copy

3b80c27

mallexxx mentioned this pull request Dec 10, 2024

Malware protection 6: Malware integration duckduckgo/BrowserServicesKit#1099

Merged

6 tasks

fix privacy dashboard button not updated

6307071

mallexxx added a commit to duckduckgo/iOS that referenced this pull request Dec 10, 2024

Malware protection (BSK macOS) (#3690)

1dfa23b

Task/Issue URL:https://app.asana.com/0/481882893211075/1208033567421351/f BSK PR: duckduckgo/BrowserServicesKit#1099 macOS PR: duckduckgo/macos-browser#3604

mallexxx added 4 commits December 10, 2024 16:36

BSK -> 221.0.0

dfb58cb

rm privacy-dashboard

d53d4f6

Merge remote-tracking branch 'origin/main' into alex/malware-protecti…

28f752e

…on-6

cleanup

890480a

mallexxx mentioned this pull request Dec 10, 2024

remove MaliciousSiteProtectionSubfeature duckduckgo/BrowserServicesKit#1131

Merged

6 tasks

maliciousSiteProtectionErrorPage feature flag -> maliciousSiteProtection

d4c60ee

mallexxx added a commit to duckduckgo/BrowserServicesKit that referenced this pull request Dec 10, 2024

remove MaliciousSiteProtectionSubfeature (#1131)

b7f7ba9

Task/Issue URL: https://app.asana.com/0/481882893211075/1208033567421351/f iOS PR: macOS PR: duckduckgo/macos-browser#3604

mallexxx added 2 commits December 10, 2024 21:06

BSK -> 221.0.1

244467d

Merge remote-tracking branch 'origin/main' into alex/malware-protecti…

e14814f

…on-6

mallexxx merged commit 708655c into main Dec 10, 2024
20 checks passed

mallexxx deleted the alex/malware-protection-6 branch December 10, 2024 15:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Malware protection 6: Malware integration #3604

Malware protection 6: Malware integration #3604

mallexxx commented Nov 27, 2024 •

edited

Loading

SabrinaTardio left a comment •

edited

Loading

SabrinaTardio Dec 5, 2024

mallexxx Dec 5, 2024

SabrinaTardio Dec 5, 2024

mallexxx Dec 5, 2024 •

edited

Loading

SabrinaTardio Dec 5, 2024

SabrinaTardio Dec 5, 2024

SabrinaTardio Dec 5, 2024

SabrinaTardio left a comment

Malware protection 6: Malware integration #3604

Malware protection 6: Malware integration #3604

Conversation

mallexxx commented Nov 27, 2024 • edited Loading

Internal references:

SabrinaTardio left a comment • edited Loading

Choose a reason for hiding this comment

SabrinaTardio Dec 5, 2024

Choose a reason for hiding this comment

mallexxx Dec 5, 2024

Choose a reason for hiding this comment

SabrinaTardio Dec 5, 2024

Choose a reason for hiding this comment

mallexxx Dec 5, 2024 • edited Loading

Choose a reason for hiding this comment

SabrinaTardio Dec 5, 2024

Choose a reason for hiding this comment

SabrinaTardio Dec 5, 2024

Choose a reason for hiding this comment

SabrinaTardio Dec 5, 2024

Choose a reason for hiding this comment

SabrinaTardio left a comment

Choose a reason for hiding this comment

mallexxx commented Nov 27, 2024 •

edited

Loading

SabrinaTardio left a comment •

edited

Loading

mallexxx Dec 5, 2024 •

edited

Loading