update ClickToLoad to use shared C-S-S (#2162)

Task/Issue URL:
https://app.asana.com/0/72649045549333/1205105078450227/f
Tech Design URL:
https://app.asana.com/0/1201720254973470/1206422390411022/f
CC:

**Description**:
This requires the corresponding BSK PR to test:
duckduckgo/BrowserServicesKit#760
Note that his PR also relies on a custom config URL for testing
purposes, to enable this new codepath. Once we merge the [corresponding
privacy config
PR](https://royal-thankful-weight.glitch.me/macos-config.json), the URL
will be reverted to the production version prior to final review &
merge.

**Steps to test this PR**:
Verify blocking
1. Navigate to
https://privacy-test-pages.site/privacy-protections/click-to-load/
2. Ensure request are blocked ("Facebook Resources Loads: NONE")
3. No social buttons should be shown ("All the social buttons from the
SDK" & "All the social buttons in iFrames")
4. Login button should be replaced with a DDG overlay and hover
5. All posts should be replaced with DDG overlays

Verify clicking through overlays
1. Click on several of the CTL overlays. They should disappear and be
replaced with the underlying FB content (note that this test page is
rather old so some posts may take several seconds to load, or even time
out)

Verify protections toggling
1. Turn protections off
4. FB requests should be allowed ("Facebook Resources Loads: DETECTED")
5. Both sets of social buttons should show up 
6. Login button should by the stock FB one (no DDG hover tip)
7. All FB content should be shown without any CTL overlays

Test login with FB
1. Navigate to
https://www.eventbrite.com/signin/?iss=https%3A%2F%2Fid.auth.eventbrite.com%2F
and/or https://www.grubhub.com/login
2. For eventbrite, a small round FB icon should be present
3. For grubhub, "Continue with Facebook" should be present
4. Click on the login button
5. You should see a DDG dialog warning the user about continuing
("Logging in with Facebook lets them track you")
6. Click on "Log In"
7. A new window should open with an FB login

At this point that means that CTL blocking rules have been removed and
the real FB sdk injected in place of the surrogate (otherwise the FB
window content would not appear)
Optionally you can complete the login process, but its not strictly
necessary.

<!--
Tagging instructions
If this PR isn't ready to be merged for whatever reason it should be
marked with the `DO NOT MERGE` label (particularly if it's a draft)
If it's pending Product Review/PFR, please add the `Pending Product
Review` label.

If at any point it isn't actively being worked on/ready for
review/otherwise moving forward (besides the above PR/PFR exception)
strongly consider closing it (or not opening it in the first place). If
you decide not to close it, make sure it's labelled to make it clear the
PRs state and comment with more information.
-->

---
###### Internal references:
[Pull Request Review
Checklist](https://app.asana.com/0/1202500774821704/1203764234894239/f)
[Software Engineering
Expectations](https://app.asana.com/0/59792373528535/199064865822552)
[Technical Design
Template](https://app.asana.com/0/59792373528535/184709971311943)
[Pull Request
Documentation](https://app.asana.com/0/1202500774821704/1204012835277482/f)

---------

Co-authored-by: Jacek Łyp <[email protected]>
Co-authored-by: Bartek Waresiak <[email protected]>

DuckDuckGo.xcodeproj/project.xcworkspace/xcshareddata/swiftpm/Package.resolved

@@ -32,8 +32,8 @@
       "kind" : "remoteSourceControl",
       "location" : "https://github.com/duckduckgo/BrowserServicesKit",
       "state" : {
-        "revision" : "ddb0eb213c081a5d6d3f78f4df5dff11bc974218",
-        "version" : "155.0.0"
+        "revision" : "e9e239fe5dfeab87dcacbf5a31c8f555623a4ce2",
+        "version" : "156.0.0"
       }
     },
     {
@@ -176,8 +176,8 @@
       "kind" : "remoteSourceControl",
       "location" : "https://github.com/duckduckgo/TrackerRadarKit",
       "state" : {
-        "revision" : "c01e6a59d000356b58ec77053e0a99d538be56a5",
-        "version" : "2.1.1"
+        "revision" : "1403e17eeeb8493b92fb9d11eb8c846bb9776581",
+        "version" : "2.1.2"
       }
     },
     {

DuckDuckGo/Application/AppConfigurationURLProvider.swift

@@ -56,7 +56,7 @@ struct AppConfigurationURLProvider: ConfigurationURLProviding {
         case .bloomFilterExcludedDomains: return URL(string: "https://staticcdn.duckduckgo.com/https/https-mobile-v2-false-positives.json")!
         case .privacyConfiguration: return customPrivacyConfigurationUrl ?? URL(string: "https://staticcdn.duckduckgo.com/trackerblocking/config/v4/macos-config.json")!
         case .surrogates: return URL(string: "https://staticcdn.duckduckgo.com/surrogates.txt")!
-        case .trackerDataSet: return URL(string: "https://staticcdn.duckduckgo.com/trackerblocking/v5/current/macos-tds.json")!
+        case .trackerDataSet: return URL(string: "https://staticcdn.duckduckgo.com/trackerblocking/v6/current/macos-tds.json")!
         // In archived repo, to be refactored shortly (https://staticcdn.duckduckgo.com/useragents/social_ctp_configuration.json)
         case .FBConfig: return URL(string: "https://staticcdn.duckduckgo.com/useragents/")!
         }

DuckDuckGo/ContentBlocker/AppPrivacyConfigurationDataProvider.swift

@@ -22,8 +22,8 @@ import BrowserServicesKit
 final class AppPrivacyConfigurationDataProvider: EmbeddedDataProvider {
 
     public struct Constants {
-        public static let embeddedDataETag = "\"c46f529760ac695b143fb664350ad760\""
-        public static let embeddedDataSHA = "00383f1d5e5e8b5da067c1527ed88027093422c85f362fcfa95cda12c5870987"
+        public static let embeddedDataETag = "\"d05f1997ae293869ab8c448e4cbc3e39\""
+        public static let embeddedDataSHA = "0ea4b7874c38d81d5e24505585d8b941069e7e7893a05a7cea9f1745adc4fa52"
     }
 
     var embeddedDataEtag: String {

DuckDuckGo/ContentBlocker/AppTrackerDataSetProvider.swift

@@ -22,8 +22,8 @@ import BrowserServicesKit
 final class AppTrackerDataSetProvider: EmbeddedDataProvider {
 
     public struct Constants {
-        public static let embeddedDataETag = "\"ea184137cdaa19ca5de76352215a9e0e\""
-        public static let embeddedDataSHA = "faa4dfbef4903710374b153c9a87e09b713fc19d64fa0bcfd1fd392fff93af21"
+        public static let embeddedDataETag = "\"6fa664462bd29b66c1b13eed51072afa\""
+        public static let embeddedDataSHA = "52d29aa8942e518db24a5adbcca63148aac9c01a8a28c3f2c4bc517cb80b18ff"
     }
 
     var embeddedDataEtag: String {

DuckDuckGo/ContentBlocker/ClickToLoadUserScript.swift

@@ -17,77 +17,88 @@
 //
 
 import WebKit
+import Common
 import UserScript
-import BrowserServicesKit
 
 protocol ClickToLoadUserScriptDelegate: AnyObject {
 
-    func clickToLoadUserScriptAllowFB(_ script: UserScript, replyHandler: @escaping (Bool) -> Void)
+    func clickToLoadUserScriptAllowFB() -> Bool
 }
 
-final class ClickToLoadUserScript: NSObject, UserScript, WKScriptMessageHandlerWithReply {
+final class ClickToLoadUserScript: NSObject, WKNavigationDelegate, Subfeature {
+    weak var broker: UserScriptMessageBroker?
+    weak var webView: WKWebView?
 
-    var injectionTime: WKUserScriptInjectionTime { .atDocumentEnd }
-    var forMainFrameOnly: Bool { false }
-    var messageNames: [String] { ["getImage", "enableFacebook", "initClickToLoad" ] }
-    let source: String
+    weak var delegate: ClickToLoadUserScriptDelegate?
+
+#if DEBUG
+    var devMode: Bool = true
+#else
+    var devMode: Bool = false
+#endif
+
+    // this isn't an issue to be set to 'all' because the page
+    public let messageOriginPolicy: MessageOriginPolicy = .all
+    public let featureName: String = "clickToLoad"
 
-    init(scriptSourceProvider: ScriptSourceProviding) {
-        self.source = scriptSourceProvider.clickToLoadSource
+    // MARK: - Subfeature
+
+    public func with(broker: UserScriptMessageBroker) {
+        self.broker = broker
     }
 
-    weak var delegate: ClickToLoadUserScriptDelegate?
+    // MARK: - MessageNames
 
-    @MainActor
-    func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage,
-                               replyHandler: @escaping (Any?, String?) -> Void) {
-        if message.name == "initClickToLoad" {
-            let host = message.body as? String
-            let controller = userContentController as? UserContentController
-            let privacyConfigurationManager = controller!.privacyConfigurationManager
-            let privacyConfiguration = privacyConfigurationManager.privacyConfig
-
-            let locallyProtected = privacyConfiguration.isProtected(domain: host)
-            let featureEnabled = privacyConfiguration.isFeature(.clickToPlay, enabledForDomain: host)
-            if locallyProtected && featureEnabled {
-                replyHandler(true, nil)
-            } else {
-                replyHandler(false, nil)
-            }
-            return
-        }
-        if message.name == "enableFacebook" {
-            guard let delegate = delegate else { return }
-            delegate.clickToLoadUserScriptAllowFB(self) { _ in
-                guard let isLogin = message.body as? Bool else {
-                    replyHandler(nil, nil)
-                    return
-                }
-
-                replyHandler(isLogin, nil)
-            }
-
-            return
+    enum MessageNames: String, CaseIterable {
+        case getClickToLoadState
+        case unblockClickToLoadContent
+        case updateFacebookCTLBreakageFlags
+        case addDebugFlag
+    }
+
+    public func handler(forMethodNamed methodName: String) -> Handler? {
+        switch MessageNames(rawValue: methodName) {
+        case .getClickToLoadState:
+            return handleGetClickToLoadState
+        case .unblockClickToLoadContent:
+            return handleUnblockClickToLoadContent
+        case .updateFacebookCTLBreakageFlags:
+            return handleDebugFlagsMock
+        case .addDebugFlag:
+            return handleDebugFlagsMock
+        default:
+            assertionFailure("ClickToLoadUserScript: Failed to parse User Script message: \(methodName)")
+            return nil
         }
+    }
 
-        var image: String
+    @MainActor
+    private func handleGetClickToLoadState(params: Any, message: UserScriptMessage) -> Encodable? {
+        webView = message.messageWebView
+        return [
+            "devMode": devMode,
+            "youtubePreviewsEnabled": false
+        ]
+    }
 
-        guard let arg = message.body as? String else {
-            replyHandler(nil, nil)
-            return
-        }
-        if message.name == "getImage" {
-            image = ClickToLoadModel.getImage[arg]!
-        } else {
-            assertionFailure("Uknown message type")
-            replyHandler(nil, nil)
-            return
-        }
-        replyHandler(image, nil)
+    @MainActor
+    private func handleUnblockClickToLoadContent(params: Any, message: UserScriptMessage) -> Encodable? {
+        guard let delegate = delegate else { return false }
+
+        // only worry about CTL FB for now
+        return delegate.clickToLoadUserScriptAllowFB()
     }
 
-    func userContentController(_ userContentController: WKUserContentController, didReceive message: WKScriptMessage) {
-        assertionFailure("SHOULDN'T BE HERE!")
+    @MainActor
+    private func handleDebugFlagsMock(params: Any, message: UserScriptMessage) -> Encodable? {
+        // breakage flags not supported on Mac yet
+        return nil
     }
 
+    @MainActor
+    public func displayClickToLoadPlaceholders() {
+        guard let webView else { return }
+
+        broker?.push(method: "displayClickToLoadPlaceholders", params: ["ruleAction": ["block"]], for: self, into: webView)
+    }
 }

DuckDuckGo/ContentBlocker/ContentBlockerRulesLists.swift

@@ -23,35 +23,6 @@ import CryptoKit
 
 final class ContentBlockerRulesLists: DefaultContentBlockerRulesListsSource {
 
-    enum Constants {
-        static let clickToLoadRulesListName = "ClickToLoad"
-    }
-
-    static var fbTrackerDataFile: Data = {
-        do {
-            let url = Bundle.main.url(forResource: "fb-tds", withExtension: "json")!
-            return try Data(contentsOf: url)
-        } catch {
-            fatalError("Failed to load FB-TDS")
-        }
-    }()
-
-    static var fbTrackerDataSet: TrackerRadarKit.TrackerData = {
-        do {
-            return try JSONDecoder().decode(TrackerData.self, from: fbTrackerDataFile)
-        } catch {
-            fatalError("Failed to JSON decode FB-TDS")
-        }
-    }()
-
-    func MD5(data: Data) -> String {
-        let digest = Insecure.MD5.hash(data: data)
-
-        return digest.map {
-            String(format: "%02hhx", $0)
-        }.joined()
-    }
-
     private let adClickAttribution: AdClickAttributing
 
     init(trackerDataManager: TrackerDataManager, adClickAttribution: AdClickAttributing) {
@@ -66,21 +37,24 @@ final class ContentBlockerRulesLists: DefaultContentBlockerRulesListsSource {
            let tdsRulesIndex = result.firstIndex(where: { $0.name == DefaultContentBlockerRulesListsSource.Constants.trackerDataSetRulesListName }) {
             let tdsRules = result[tdsRulesIndex]
             let allowlistedTrackerNames = adClickAttribution.allowlist.map { $0.entity }
-            let splitter = AdClickAttributionRulesSplitter(rulesList: tdsRules, allowlistedTrackerNames: allowlistedTrackerNames)
-            if let splitRules = splitter.split() {
+            let adSplitter = AdClickAttributionRulesSplitter(rulesList: tdsRules, allowlistedTrackerNames: allowlistedTrackerNames)
+            if let splitRules = adSplitter.split() {
                 result.remove(at: tdsRulesIndex)
                 result.append(splitRules.0)
                 result.append(splitRules.1)
             }
         }
 
-        // Add new ones
-        let etag = MD5(data: Self.fbTrackerDataFile)
-        let dataSet: TrackerDataManager.DataSet = TrackerDataManager.DataSet(Self.fbTrackerDataSet, etag)
-        let CTLRulesList = ContentBlockerRulesList(name: Constants.clickToLoadRulesListName,
-                                                   trackerData: nil,
-                                                   fallbackTrackerData: dataSet)
-            result.append(CTLRulesList)
+        // split CTL rules so they can be managed separately from the main list when user clicks through a CTL dialog
+        if let tdsRulesIndex = result.firstIndex(where: { $0.name == DefaultContentBlockerRulesListsSource.Constants.trackerDataSetRulesListName }) {
+            let tdsRules = result[tdsRulesIndex]
+            let ctlSplitter = ClickToLoadRulesSplitter(rulesList: tdsRules)
+            if let splitRules = ctlSplitter.split() {
+                result.remove(at: tdsRulesIndex)
+                result.append(splitRules.withoutBlockCTL)
+                result.append(splitRules.withBlockCTL)
+            }
+        }
 
         return result
     }

DuckDuckGo/ContentBlocker/ContentBlocking.swift

@@ -137,7 +137,7 @@ final class AppContentBlocking {
             switch listName {
             case defaultTDSListName:
                 listType = .tds
-            case ContentBlockerRulesLists.Constants.clickToLoadRulesListName:
+            case DefaultContentBlockerRulesListsSource.Constants.clickToLoadRulesListName:
                 listType = .clickToLoad
             case AdClickAttributionRulesSplitter.blockingAttributionRuleListName(forListNamed: defaultTDSListName):
                 listType = .blockingAttribution