Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

experiment default metrics pixels #1107

Merged
merged 57 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
216a00e
implement experiment manager
SabrinaTardio Nov 8, 2024
d8b385c
Merge branch 'main' into sabrina/add-experiment-manager
SabrinaTardio Nov 8, 2024
914a5ed
fix lint issues
SabrinaTardio Nov 8, 2024
93247a3
fix linting issue
SabrinaTardio Nov 8, 2024
5d530f2
wrap UserDefaults
SabrinaTardio Nov 8, 2024
2dcef1e
fix linting
SabrinaTardio Nov 8, 2024
5aec88d
refactor
SabrinaTardio Nov 8, 2024
b188d67
fix linting
SabrinaTardio Nov 8, 2024
1054923
address some comments
SabrinaTardio Nov 8, 2024
c05aa96
minor refactor
SabrinaTardio Nov 8, 2024
8387472
use Constants enum
SabrinaTardio Nov 8, 2024
e862ad8
add Targets and configurations
SabrinaTardio Nov 11, 2024
6090a15
fix linting issues
SabrinaTardio Nov 11, 2024
14f92e9
initial implementation
SabrinaTardio Nov 12, 2024
8bda08c
add tests
SabrinaTardio Nov 14, 2024
9a19db2
Merge branch 'main' into sabrina/add-experiment-logic
SabrinaTardio Nov 14, 2024
56c54a7
refactor
SabrinaTardio Nov 15, 2024
86b7c97
getAllActiveExperiments tests
SabrinaTardio Nov 15, 2024
2595fcf
sort linting issues
SabrinaTardio Nov 15, 2024
86d6f7b
fix linting
SabrinaTardio Nov 15, 2024
95c4470
don't break the API
SabrinaTardio Nov 15, 2024
c30f0b4
Merge branch 'main' into sabrina/add-experiment-logic
SabrinaTardio Nov 15, 2024
727e813
implement PixelExperimentKit
SabrinaTardio Nov 21, 2024
69dc2fd
implement framework
SabrinaTardio Nov 27, 2024
cfe3b13
clean up
SabrinaTardio Nov 27, 2024
1e426e2
Merge branch 'main' into sabrina/experiment-framework
SabrinaTardio Nov 27, 2024
26fa2ed
fix linting
SabrinaTardio Nov 28, 2024
ee2f891
have only one interface method
SabrinaTardio Nov 28, 2024
b3224fe
fix lint
SabrinaTardio Nov 28, 2024
b057431
address comments
SabrinaTardio Nov 28, 2024
e92cbb5
Merge branch 'sabrina/experiment-framework' into sabrina/experiment-p…
SabrinaTardio Nov 28, 2024
4756b60
update after merge
SabrinaTardio Nov 29, 2024
4f2d28b
Merge branch 'main' into sabrina/experiment-pixels
SabrinaTardio Nov 29, 2024
d202980
cleanup
SabrinaTardio Nov 29, 2024
bfb7a5e
fix lint errors
SabrinaTardio Nov 29, 2024
a98e9c4
add pixel unique parameter test
SabrinaTardio Nov 29, 2024
7747bdc
remove unwanted code
SabrinaTardio Nov 29, 2024
92bb24f
refactor
SabrinaTardio Nov 29, 2024
e7801f9
refactor and add more tests
SabrinaTardio Dec 2, 2024
fc1c31e
add tests
SabrinaTardio Dec 2, 2024
f03d0e3
Merge branch 'main' into sabrina/experiment-pixels
SabrinaTardio Dec 2, 2024
f24740a
fix linting
SabrinaTardio Dec 2, 2024
aef82dd
linting
SabrinaTardio Dec 2, 2024
9b65cc7
linting
SabrinaTardio Dec 2, 2024
4992efe
implement firing enrolment pixel
SabrinaTardio Dec 3, 2024
4412bd4
Merge branch 'main' into sabrina/experiment-pixels
SabrinaTardio Dec 3, 2024
b202b44
fix tests
SabrinaTardio Dec 3, 2024
c62fac4
generalise headers and prefix name
SabrinaTardio Dec 3, 2024
cc6c1b6
address comments
SabrinaTardio Dec 4, 2024
b567a22
add isDry for new unique params
SabrinaTardio Dec 4, 2024
2b0f434
fix linting
SabrinaTardio Dec 4, 2024
15eed7b
fix tests
SabrinaTardio Dec 4, 2024
e0e45ca
fix iOS pixel naming
SabrinaTardio Dec 4, 2024
cd6d89a
address comments and remove m_ from experiments
SabrinaTardio Dec 5, 2024
9568123
remove unwanted change
SabrinaTardio Dec 5, 2024
8db91da
fix mac pixel names
SabrinaTardio Dec 9, 2024
8c2a9c6
fix test
SabrinaTardio Dec 4, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -547,6 +547,9 @@
buildForAnalyzing = "YES">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "PixelExperimentKit"
BuildableName = "PixelExperimentKit"
BlueprintName = "PixelExperimentKit"
BlueprintIdentifier = "PrivacyStats"
BuildableName = "PrivacyStats"
BlueprintName = "PrivacyStats"
Expand Down Expand Up @@ -836,6 +839,9 @@
skipped = "NO">
<BuildableReference
BuildableIdentifier = "primary"
BlueprintIdentifier = "PixelExperimentKitTests"
BuildableName = "PixelExperimentKitTests"
BlueprintName = "PixelExperimentKitTests"
BlueprintIdentifier = "PrivacyStatsTests"
BuildableName = "PrivacyStatsTests"
BlueprintName = "PrivacyStatsTests"
Expand Down
20 changes: 20 additions & 0 deletions Package.swift
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ let package = Package(
.library(name: "DuckPlayer", targets: ["DuckPlayer"]),
.library(name: "PhishingDetection", targets: ["PhishingDetection"]),
.library(name: "Onboarding", targets: ["Onboarding"]),
.library(name: "PixelExperimentKit", targets: ["PixelExperimentKit"]),
.library(name: "BrokenSitePrompt", targets: ["BrokenSitePrompt"]),
.library(name: "PageRefreshMonitor", targets: ["PageRefreshMonitor"]),
.library(name: "PrivacyStats", targets: ["PrivacyStats"]),
Expand Down Expand Up @@ -428,6 +429,19 @@ let package = Package(
.define("DEBUG", .when(configuration: .debug))
]
),
.target(
name: "PixelExperimentKit",
dependencies: [
"PixelKit",
"BrowserServicesKit"
],
resources: [
.process("Resources")
],
swiftSettings: [
.define("DEBUG", .when(configuration: .debug))
]
),
.target(
name: "BrokenSitePrompt",
dependencies: [
Expand Down Expand Up @@ -676,6 +690,12 @@ let package = Package(
"Onboarding"
]
),
.testTarget(
name: "PixelExperimentKitTests",
dependencies: [
"PixelExperimentKit"
]
),
.testTarget(
name: "SpecialErrorPagesTests",
dependencies: [
Expand Down
63 changes: 63 additions & 0 deletions Sources/PixelExperimentKit/ExperimentActionPixelManager.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
//
// ExperimentActionPixelManager.swift
//
// Copyright © 2024 DuckDuckGo. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

import Foundation

public protocol ExperimentActionPixelStore {
func removeObject(forKey defaultName: String)
func integer(forKey defaultName: String) -> Int
func set(_ value: Int, forKey defaultName: String)
}

public protocol ExperimentActionPixelManaging {
func incrementAndCheckThreshold(forKey key: String, threshold: Int, isInWindow: Bool) -> Bool
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
}

public struct ExperimentActionPixelManager: ExperimentActionPixelManaging {
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
let store: ExperimentActionPixelStore
private let syncQueue = DispatchQueue(label: "com.pixelkit.experimentActionSyncQueue")

public init(store: ExperimentActionPixelStore = UserDefaults.standard) {
self.store = store
}

public func incrementAndCheckThreshold(forKey key: String, threshold: Int, isInWindow: Bool) -> Bool {
syncQueue.sync {
// Remove the key if is not in window
guard isInWindow else {
store.removeObject(forKey: key)
return false
}

// Increment the current count
let currentCount = store.integer(forKey: key)
let newCount = currentCount + 1
store.set(newCount, forKey: key)

// Check if the threshold is exceeded
if newCount >= threshold {
store.removeObject(forKey: key)
return true
}
return false
}
}

}

extension UserDefaults: ExperimentActionPixelStore {}
233 changes: 233 additions & 0 deletions Sources/PixelExperimentKit/PixelExperimentKit.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,233 @@
//
// PixelExperimentKit.swift
//
// Copyright © 2024 DuckDuckGo. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

import PixelKit
import BrowserServicesKit
import Foundation

struct ExperimentEvent: PixelKitEvent {
var name: String
var parameters: [String: String]?
}

extension PixelKit {

struct Constants {
static let enrollmentEventPrefix = "experiment_enroll"
static let metricsEventPrefix = "experiment_metrics"
static let metricKey = "metric"
static let conversionWindowDaysKey = "conversionWindowDays"
static let valueKey = "value"
static let enrollmentDateKey = "enrollmentDate"
static let searchMetricValue = "search"
static let appUseMetricValue = "app_use"
}

// Static property to hold shared dependencies
struct ExperimentConfig {
static var featureFlagger: FeatureFlagger?
static var store: ExperimentActionPixelManaging = ExperimentActionPixelManager()
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
static var fireFunction: (PixelKitEvent, PixelKit.Frequency, Bool) -> Void = { event, frequency, includeAppVersion in
fire(event, frequency: frequency, includeAppVersionParameter: includeAppVersion)
}
}

// Setup method to initialize dependencies
public static func configureExperimentKit(
featureFlagger: FeatureFlagger,
store: ExperimentActionPixelManaging = ExperimentActionPixelManager(),
fire: @escaping (PixelKitEvent, PixelKit.Frequency, Bool) -> Void = { event, frequency, includeAppVersion in
fire(event, frequency: frequency, includeAppVersionParameter: includeAppVersion)
}
) {
ExperimentConfig.featureFlagger = featureFlagger
ExperimentConfig.store = store
ExperimentConfig.fireFunction = fire
}

/// Fires a pixel indicating the user's enrollment in an experiment.
/// - Parameters:
/// - subfeatureID: Identifier for the subfeature associated with the experiment.
/// - experiment: Data about the experiment like cohort and enrollment date
public static func fireExperimentEnrollmentPixel(subfeatureID: SubfeatureID, experiment: ExperimentData) {
let eventName = "\(Self.Constants.enrollmentEventPrefix)_\(subfeatureID)_\(experiment.cohortID)"
let event = ExperimentEvent(name: eventName, parameters: [Self.Constants.enrollmentDateKey: experiment.enrollmentDate.toYYYYMMDDInET()])
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
ExperimentConfig.fireFunction(event, .uniqueByNameAndParameters, false)
}

/// Fires a pixel for a specific action in an experiment, based on conversion window and value thresholds (if value is a number).
/// - Parameters:
/// - subfeatureID: Identifier for the subfeature associated with the experiment.
/// - metric: The name of the metric being tracked (e.g., "searches").
/// - conversionWindowDays: The range of days after enrollment during which the action is valid.
/// - value: A specific value associated to the action. It could be the target number of actions required to fire the pixel.
///
/// This function:
/// 1. Validates if the experiment is active.
/// 2. Ensures the user is within the specified conversion window.
/// 3. Tracks actions performed and sends the pixel once the target value is reached (if applicable).
public static func fireExperimentPixel(for subfeatureID: SubfeatureID, metric: String, conversionWindowDays: ClosedRange<Int>, value: String) {
// Check is active experiment for user
guard let featureFlagger = ExperimentConfig.featureFlagger else {
assertionFailure("PixelKit is not configured for experiments")
return
}
guard let experimentData = featureFlagger.getAllActiveExperiments()[subfeatureID] else { return }

Self.fireExperimentPixelForActiveExperiment(subfeatureID, experimentData: experimentData, metric: metric, conversionWindowDays: conversionWindowDays, value: value)
}

/// Fires search-related experiment pixels for all active experiments.
///
/// This function iterates through all active experiments and triggers
/// pixel firing based on predefined search-related value and conversion window mappings.
/// - The value and conversion windows define when and how many search actions
/// must occur before the pixel is fired.
public static func fireSearchExperimentPixels() {
let valueConversionDictionary: [Int: [ClosedRange<Int>]] = [
1: [0...0, 1...1, 2...2, 3...3, 4...4, 5...5, 6...6, 7...7, 5...7],
4: [5...7, 8...15],
6: [5...7, 8...15],
11: [5...7, 8...15],
21: [5...7, 8...15],
30: [5...7, 8...15]
]
guard let featureFlagger = ExperimentConfig.featureFlagger else {
assertionFailure("PixelKit is not configured for experiments")
return
}
featureFlagger.getAllActiveExperiments().forEach { experiment in
fireExperimentPixelsFor(
experiment.key,
experimentData: experiment.value,
metric: Self.Constants.searchMetricValue,
valueConversionDictionary: valueConversionDictionary
)
}
}

/// Fires app retention-related experiment pixels for all active experiments.
///
/// This function iterates through all active experiments and triggers
/// pixel firing based on predefined app retention value and conversion window mappings.
/// - The value and conversion windows define when and how many app usage actions
/// must occur before the pixel is fired.
public static func fireAppRetentionExperimentPixels() {
let valueConversionDictionary: [Int: [ClosedRange<Int>]] = [
1: [1...1, 2...2, 3...3, 4...4, 5...5, 6...6, 7...7, 5...7],
4: [5...7, 8...15],
6: [5...7, 8...15],
11: [5...7, 8...15],
21: [5...7, 8...15],
30: [5...7, 8...15]
]
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
guard let featureFlagger = ExperimentConfig.featureFlagger else {
assertionFailure("PixelKit is not configured for experiments")
return
}
featureFlagger.getAllActiveExperiments().forEach { experiment in
fireExperimentPixelsFor(
experiment.key,
experimentData: experiment.value,
metric: Self.Constants.appUseMetricValue,
valueConversionDictionary: valueConversionDictionary
)
}
}

private static func fireExperimentPixelsFor(
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
_ experiment: SubfeatureID,
experimentData: ExperimentData,
metric: String,
valueConversionDictionary: [Int: [ClosedRange<Int>]]
) {
valueConversionDictionary.forEach { value, ranges in
ranges.forEach { range in
fireExperimentPixelForActiveExperiment(
experiment,
experimentData: experimentData,
metric: metric,
conversionWindowDays: range,
value: "\(value)"
)
}
}
}

private static func fireExperimentPixelForActiveExperiment(_ subfeatureID: SubfeatureID, experimentData: ExperimentData, metric: String, conversionWindowDays: ClosedRange<Int>, value: String) {
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
// Set parameters, event name, store key
let eventName = "\(Self.Constants.metricsEventPrefix)_\(subfeatureID)_\(experimentData.cohortID)"
let parameters: [String: String] = [
Self.Constants.metricKey: metric,
Self.Constants.conversionWindowDaysKey: "\(conversionWindowDays.lowerBound.description)-\(conversionWindowDays.upperBound.description)",
Self.Constants.valueKey: value,
Self.Constants.enrollmentDateKey: experimentData.enrollmentDate.toYYYYMMDDInET()
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
]
let event = ExperimentEvent(name: eventName, parameters: parameters)
let eventStoreKey = "\(eventName)_\(parameters.toString())"

// Determine if the user is within the conversion window
let isInWindow = isUserInConversionWindow(conversionWindowDays, enrollmentDate: experimentData.enrollmentDate)

// Check if value is a number
if let numberOfAction = Int(value), numberOfAction > 1 {
// Increment or remove based on conversion window status
let shouldSendPixel = ExperimentConfig.store.incrementAndCheckThreshold(
forKey: eventStoreKey,
threshold: numberOfAction,
isInWindow: isInWindow
)

// Send the pixel only if conditions are met
if shouldSendPixel {
ExperimentConfig.fireFunction(event, .uniqueByNameAndParameters, false)
}
} else if isInWindow {
// If value is not a number, send the pixel only if within the window
ExperimentConfig.fireFunction(event, .uniqueByNameAndParameters, false)
}
}

private static func isUserInConversionWindow(
_ conversionWindowRange: ClosedRange<Int>,
enrollmentDate: Date
) -> Bool {
let calendar = Calendar.current
guard let startOfWindow = enrollmentDate.addDays(conversionWindowRange.lowerBound),
let endOfWindow = enrollmentDate.addDays(conversionWindowRange.upperBound) else {
return false
}

let currentDate = calendar.startOfDay(for: Date())
return currentDate >= calendar.startOfDay(for: startOfWindow) &&
currentDate <= calendar.startOfDay(for: endOfWindow)
}
}

extension Date {
public func toYYYYMMDDInET() -> String {
let formatter = DateFormatter()
formatter.dateFormat = "yyyy-MM-dd"
formatter.timeZone = TimeZone(identifier: "America/New_York")
return formatter.string(from: self)
}

func addDays(_ days: Int) -> Date? {
return Calendar.current.date(byAdding: .day, value: days, to: self)
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
}
}
25 changes: 25 additions & 0 deletions Sources/PixelKit/Extensions/Disctionary+PixelKit.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
//
// Disctionary+PixelKit.swift
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
//
// Copyright © 2024 DuckDuckGo. All rights reserved.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//

public extension Dictionary where Key: Comparable {
func toString(pairSeparator: String = ":", entrySeparator: String = ",") -> String {
return self.sorted(by: { $0.key < $1.key })
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
SabrinaTardio marked this conversation as resolved.
Show resolved Hide resolved
.map { "\($0.key)\(pairSeparator)\($0.value)" }
.joined(separator: entrySeparator)
}
}
Loading
Loading