update ClickToLoad to use shared C-S-S

[ladamski]

Task/Issue URL: https://app.asana.com/0/72649045549333/1205105078450227/f
Tech Design URL: https://app.asana.com/0/1201720254973470/1206422390411022/f
CC:

Description:
This requires the corresponding BSK PR to test: duckduckgo/BrowserServicesKit#760
Note that his PR also relies on a custom config URL for testing purposes, to enable this new codepath. Once we merge the corresponding privacy config PR, the URL will be reverted to the production version prior to final review & merge.

Steps to test this PR:
Verify blocking

1. Navigate to https://privacy-test-pages.site/privacy-protections/click-to-load/
2. Ensure request are blocked ("Facebook Resources Loads: NONE")
3. No social buttons should be shown ("All the social buttons from the SDK" & "All the social buttons in iFrames")
4. Login button should be replaced with a DDG overlay and hover
5. All posts should be replaced with DDG overlays

Verify clicking through overlays

1. Click on several of the CTL overlays. They should disappear and be replaced with the underlying FB content (note that this test page is rather old so some posts may take several seconds to load, or even time out)

Verify protections toggling

1. Turn protections off
2. FB requests should be allowed ("Facebook Resources Loads: DETECTED")
3. Both sets of social buttons should show up
4. Login button should by the stock FB one (no DDG hover tip)
5. All FB content should be shown without any CTL overlays

Test login with FB

1. Navigate to https://www.eventbrite.com/signin/?iss=https%3A%2F%2Fid.auth.eventbrite.com%2F and/or https://www.grubhub.com/login
2. For eventbrite, a small round FB icon should be present
3. For grubhub, "Continue with Facebook" should be present
4. Click on the login button
5. You should see a DDG dialog warning the user about continuing ("Logging in with Facebook lets them track you")
6. Click on "Log In"
7. A new window should open with an FB login

At this point that means that CTL blocking rules have been removed and the real FB sdk injected in place of the surrogate (otherwise the FB window content would not appear)
Optionally you can complete the login process, but its not strictly necessary.

---

Internal references:

Pull Request Review Checklist
Software Engineering Expectations
Technical Design Template
Pull Request Documentation

[github-actions]

	Fails
🚫	Content Tracker URL mismatch. Please check DuckDuckGo/AppDelegate/AppConfigurationURLProvider.swift and scripts/update_embedded.sh
🚫	Privacy Config URL mismatch. Please check DuckDuckGo/AppDelegate/AppConfigurationURLProvider.swift and scripts/update_embedded.sh

	Warnings
⚠️	PR has more than 500 lines of code changing. Consider splitting into smaller PRs if possible.

Generated by 🚫 dangerJS against 5e5feb1

[ladamski]

@bwaresiak I moved ClickToLoadRulesSplitter to BSK, and updated the tests accordingly. Note a few things:

• I renamed CTLReferenceTests -> ClickToLoadBlockingTests to avoid confusion as there are actually separate CTL reference tests (but we can't currently support them due to their use of random custom action names)
  -- I still need to clean up the fallback return login per your comments
• Need to flesh out more test cases
• Address a few remaining syntax comments

Hope to have something for "final" review by EOD Mon

[ladamski]

@bwaresiak I fixed the ClickToLoadBlockingTests and added some test cases. All are now passing but please let me know if you think there are other test cases that we should add. There were two issues with the test:

• self.tds was undefined, causing it to fail some test cases
• more tricky, the CBR lists were all hardcoded to use test as name, so we weren't really testing with two different lists. I fixed that and now the tests seem to behave correctly

I'm still working through some issues with SurrogateUserScriptTests, not sure what exactly it is yet since debugging surrogates.js inside of integration tests is a real pain.. please let me know if you have any thoughts on that front? At least some way to get debug info out of it, if not set breakpoints.

Might be worth another code review pass overall, since I have fixed the other code comments as well (except the one for embedded TDS, etc but there's a reason I will note there)

[bwaresiak]

@ladamski let's sync on the debugging, I might have some tricks to share. :) I'll have a look at tests and changes Today.

[ladamski]

@bwaresiak ultimately I didn't have much luck with the Safari debugging path (by the time the content script showed up in the debugger it was already past my breakpoints..), so I just added some debug messaging to monitor execution. That demonstrated that the control flow was correct and the CTL tracker load was being detected & surrogate injected.

Turns out the issue is related to the fact that these surrogate tests assume essentially sync execution, and CTL involves a bit of async messages to check current CTL enabled state, so the tests are evaluated before that RT is complete. Adding some delays to test evaluation results in the expected results. I pushed an example here: duckduckgo/BrowserServicesKit@la/clicktoload-redux...la/surrogate-tests

I know adding delays is an ugly "fix" so I'm definitely open to a better way to coordinate test completion with these async operations. Also one more note, for some reason testWhenSiteIsNotInExceptionListThenSurrogatesAreInjected is still failing though so not sure what the issue there is, I don't think its related to my changes offhand but haven't had time to dig into that one in detail yet.

[ladamski]

Note that I merged the changes from the test branch in the prior comment back into the working BSK PR.

[bwaresiak]

@ayoy I'm a little confused by Danger complaining about config & TDS urls here - I think URLs are actually matching.

issues have been resolved

[ayoy]

@ayoy I'm a little confused by Danger complaining about config & TDS urls here - I think URLs are actually matching.

@bwaresiak yep, we occasionally see this check reporting a false negative. Sorry about that. We should revisit the logic here. Thanks for double checking!