Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Privacy Stats module for collecting stats about blocked trackers #1097

Merged
merged 52 commits into from
Nov 29, 2024
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
f0fc267
Add PrivacyStats module
ayoy Nov 24, 2024
7514bf2
Implement storing privacy stats
ayoy Nov 24, 2024
c429cb4
Add API to calculate 7 day privacy stats
ayoy Nov 24, 2024
8646940
Add deleting outdated entries
ayoy Nov 24, 2024
92204a7
Use context.perform where possible
ayoy Nov 24, 2024
9d96447
Rewrite PrivacyStats using structured concurrency
ayoy Nov 25, 2024
b3267f8
Add API to delete all privacy stats entities
ayoy Nov 26, 2024
dc6857f
Nullify current stats object when clearing data
ayoy Nov 26, 2024
a36718b
Reload empty stats after clearing data
ayoy Nov 26, 2024
14e7955
Code renaming and refactoring
ayoy Nov 27, 2024
f5cb69f
Further refactoring
ayoy Nov 27, 2024
a8be3d4
Switch to storing blocked tracker stats by company
ayoy Nov 27, 2024
7a7cd5b
Remove in-memory cache
ayoy Nov 27, 2024
8e4d160
Clean up the code
ayoy Nov 27, 2024
0006a08
Rename
ayoy Nov 27, 2024
58932bb
More renaming
ayoy Nov 27, 2024
c48e48b
Fix Swiftlint violations
ayoy Nov 27, 2024
213e873
Move code around
ayoy Nov 27, 2024
3c501dc
Mark commitChangesSubject/Publisher nonisolated
ayoy Nov 27, 2024
9e32661
Add PrivacyStatsTests module
ayoy Nov 27, 2024
98a7511
Update C-S-S to 6.40.0
ayoy Nov 27, 2024
e1d6796
Move topCompanies to the client, make stats initializer sync and add …
ayoy Nov 27, 2024
3ac66a5
Add unit tests for PrivacyStats
ayoy Nov 28, 2024
09dd899
Switch to daily packs, not hourly
ayoy Nov 28, 2024
c6cd844
Add placeholder for PrivacyStatsUtilsTests
ayoy Nov 28, 2024
68d34b5
Add EventMapping to PrivacyStats
ayoy Nov 28, 2024
d1549cd
Add tests for PrivacyStatsUtils
ayoy Nov 28, 2024
113d575
Add tests for CurrentPack
ayoy Nov 28, 2024
e21233a
Add unit tests for deleting outdated objects
ayoy Nov 28, 2024
02e4e2e
Add more docs and fix SwiftLint violations
ayoy Nov 28, 2024
f1e8467
Merge branch 'main' into dominik/privacy-stats
ayoy Nov 28, 2024
a10132a
Fix flaky test
ayoy Nov 28, 2024
c19d9d4
Cancel commit task on CurrentPack deinit
ayoy Nov 28, 2024
f1cd8fc
Another fix
ayoy Nov 28, 2024
0ab9617
Don't notify about change when saving current day pack
ayoy Nov 28, 2024
25bc3e1
Remove unneeded code from CurrentPackTests
ayoy Nov 28, 2024
a455e65
Merge branch 'main' into dominik/privacy-stats
ayoy Nov 28, 2024
b51633f
Update C-S-S to 6.41.0
ayoy Nov 29, 2024
3e39c62
Merge branch 'main' into dominik/privacy-stats
ayoy Nov 29, 2024
d50db7e
Update Sources/Common/Extensions/DateExtension.swift
ayoy Nov 29, 2024
9041b8a
Remove Date.isSameHour
ayoy Nov 29, 2024
46897e6
Extract entity name into a constant
ayoy Nov 29, 2024
184595e
Remove unnecessary return statements
ayoy Nov 29, 2024
6f605ec
Use NSBatchDeleteRequest for deleting objects
ayoy Nov 29, 2024
661849c
Replace handling app termination with a dedicated function to be used…
ayoy Nov 29, 2024
f291fab
Update Sources/PrivacyStats/PrivacyStats.swift
ayoy Nov 29, 2024
b85e699
Reset context after executing NSBatchDeleteRequest
ayoy Nov 29, 2024
e0d3d33
Make currentPack non-optional lazy var
ayoy Nov 29, 2024
3dd3b56
Send a stats update event after resetting data
ayoy Nov 29, 2024
82fa9e6
Merge branch 'main' into dominik/privacy-stats
ayoy Nov 29, 2024
c0b708c
Increase timeout in testThatMultipleCallsToRecordBlockedTrackerOnlyTr…
ayoy Nov 29, 2024
e7db292
Make initializeCurrentPack a static function and currenPack a let
ayoy Nov 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -539,6 +539,20 @@
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
<BuildActionEntry
buildForTesting = "YES"
buildForRunning = "YES"
buildForProfiling = "YES"
buildForArchiving = "YES"
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "PrivacyStats"
BuildableName = "PrivacyStats"
BlueprintName = "PrivacyStats"
ReferencedContainer = "container:">
</BuildableReference>
</BuildActionEntry>
</BuildActionEntries>
</BuildAction>
<TestAction
Expand Down Expand Up @@ -818,6 +832,16 @@
ReferencedContainer = "container:">
</BuildableReference>
</TestableReference>
<TestableReference
skipped = "NO">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "PrivacyStatsTests"
BuildableName = "PrivacyStatsTests"
BlueprintName = "PrivacyStatsTests"
ReferencedContainer = "container:">
</BuildableReference>
</TestableReference>
</Testables>
</TestAction>
<LaunchAction
Expand Down
4 changes: 2 additions & 2 deletions Package.resolved
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@
"kind" : "remoteSourceControl",
"location" : "https://github.com/duckduckgo/content-scope-scripts",
"state" : {
"revision" : "dfef00ef77f5181d1d8a4f7cc88f7b7c0514dd34",
"version" : "6.39.0"
"revision" : "c4bb146afdf0c7a93fb9a7d95b1cb255708a470d",
"version" : "6.41.0"
}
},
{
Expand Down
24 changes: 23 additions & 1 deletion Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,15 @@ let package = Package(
.library(name: "Onboarding", targets: ["Onboarding"]),
.library(name: "BrokenSitePrompt", targets: ["BrokenSitePrompt"]),
.library(name: "PageRefreshMonitor", targets: ["PageRefreshMonitor"]),
.library(name: "PrivacyStats", targets: ["PrivacyStats"]),
],
dependencies: [
.package(url: "https://github.com/duckduckgo/duckduckgo-autofill.git", exact: "15.1.0"),
.package(url: "https://github.com/duckduckgo/GRDB.swift.git", exact: "2.4.2"),
.package(url: "https://github.com/duckduckgo/TrackerRadarKit", exact: "3.0.0"),
.package(url: "https://github.com/duckduckgo/sync_crypto", exact: "0.3.0"),
.package(url: "https://github.com/gumob/PunycodeSwift.git", exact: "3.0.0"),
.package(url: "https://github.com/duckduckgo/content-scope-scripts", exact: "6.39.0"),
.package(url: "https://github.com/duckduckgo/content-scope-scripts", exact: "6.41.0"),
.package(url: "https://github.com/duckduckgo/privacy-dashboard", exact: "7.2.1"),
.package(url: "https://github.com/httpswift/swifter.git", exact: "1.5.0"),
.package(url: "https://github.com/duckduckgo/bloom_cpp.git", exact: "3.0.0"),
Expand Down Expand Up @@ -445,6 +446,20 @@ let package = Package(
.define("DEBUG", .when(configuration: .debug))
]
),
.target(
name: "PrivacyStats",
dependencies: [
"Common",
"Persistence",
"TrackerRadarKit"
],
resources: [
.process("PrivacyStats.xcdatamodeld")
],
swiftSettings: [
.define("DEBUG", .when(configuration: .debug))
]
),

// MARK: - Test Targets
.testTarget(
Expand Down Expand Up @@ -679,6 +694,13 @@ let package = Package(
"PageRefreshMonitor"
]
),
.testTarget(
name: "PrivacyStatsTests",
dependencies: [
"PrivacyStats",
"TestUtils",
]
),
],
cxxLanguageStandard: .cxx11
)
10 changes: 9 additions & 1 deletion Sources/Common/Extensions/DateExtension.swift
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,10 @@ public extension Date {
return Calendar.current.isDate(date1, inSameDayAs: date2)
}

static func isSameHour(_ date1: Date, _ date2: Date) -> Bool {
return Calendar.current.isDate(date1, equalTo: date2, toGranularity: .hour)
ayoy marked this conversation as resolved.
Show resolved Hide resolved
}

static var startOfDayTomorrow: Date {
let tomorrow = Calendar.current.date(byAdding: .day, value: 1, to: Date())!
return Calendar.current.startOfDay(for: tomorrow)
Expand All @@ -64,7 +68,11 @@ public extension Date {
}

var startOfDay: Date {
return Calendar.current.startOfDay(for: self)
return Calendar.current.startOfDay(for: self)
}

func daysAgo(_ days: Int) -> Date {
return Calendar.current.date(byAdding: .day, value: -days, to: self)!
ayoy marked this conversation as resolved.
Show resolved Hide resolved
}

static var startOfMinuteNow: Date {
Expand Down
24 changes: 24 additions & 0 deletions Sources/PrivacyStats/Logger+PrivacyStats.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
//
// Logger+PrivacyStats.swift
//
// Copyright © 2024 DuckDuckGo. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

import Foundation
import os.log

public extension Logger {
static var privacyStats = { Logger(subsystem: "Privacy Stats", category: "") }()
}
Loading
Loading