Closes #2989: Add deletion-request ping support for legacy_client_id #2992

psymoon · 2020-06-24T16:14:57Z

This is following the guideline from https://mozilla.github.io/glean/book/user/pings/deletion_request.html#the-deletion-request-ping.

mdboom · 2020-06-24T16:25:03Z

app/build.gradle

+// Generate markdown docs for the collected metrics.
+ext.gleanGenerateMarkdownDocs = true
+ext.gleanDocsDirectory = "$rootDir/docs/glean"
+apply from: 'https://github.com/mozilla-mobile/android-components/raw/v' + rootProject.ext.moz_components_version + '/components/service/glean/scripts/sdk_generator.gradle'


This is no longer the supported way to use the Glean code generation plugin. See here:

https://mozilla.github.io/glean/book/user/adding-glean-to-your-project.html#adding-new-metrics

@mdboom this is using older a-c/glean version (24.0.0), I get the following compilation error:
Plugin with id 'org.mozilla.telemetry.glean-gradle-plugin' not found.

Ok. Be aware that the sdk_generator.gradle script has no way to ensure that it's running the correct version of the Glean generator, so this problem of generating the wrong code could re-occur in the future. If there's any way you could upgrade to a-c 27.0.0 or later, you would have access to the Glean Gradle plugin, which resolves that issue.

@mdboom I think we will address this in #2990.
AFAIK, we are currently lacking QA support for big a-c upgrade. I think it's safer to upgrade a-c version to 24 for deletion-request ping support.

@mcomella what do you think about this ^?

Since we're only expecting to build one build from this in the next few weeks, I'd think this is probably fine to move forward with this.

@mdboom Do you agree?

@psymoon If mdboom agrees, can you file an issue about "upgrade to a-c 27 and use glean plugin" and talk to athomasmoz to ensure it's at the top of the backlog? It should be the first thing we do before adding more code (or at least validate that it won't break). Ideally, we'll upgrade to the latest version, but if not, we should do that issue.

That makes sense. Thanks for checking.

Filed #2993.

psymoon · 2020-06-24T19:38:34Z

Request for data collection review form

What questions will you answer with this data?

This data will be used to delete legacy telemetry data associated with the id when they opt out of telemetry.

Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?

This allows us to comply with user privacy regulations.

What alternative methods did you consider to answer these questions? Why were they not sufficient?

Existing legacy telemetry does not currently have the ability to send a deletion request ping and this was the easiest way to get that information

Can current instrumentation answer these questions?

N/A

List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories on the Mozilla wiki.

Measurement Description	Data Collection Category	Tracking Bug #
Legacy telemetry client id	Category 1	Issue 2989

How long will this data be collected?

For the lifetime of the application, owned by the amazon-dev team.

What populations will you measure?

All users with telemetry enabled, on release channel.

If this data collection is default on, what is the opt-out mechanism for users?

There is an option in settings to stop sending usage data that will be used as the opt-out mechanism.

Please provide a general description of how you will analyze this data.

This will be analyzed with the normal Glean methods such as GUD and GLAM.

Where do you intend to share the results of your analysis?

Internally with Mozilla, through normal Glean reporting methods.

mcomella

I'm not going to be able to finish the review today but here are a few nits I've found.

app/build.gradle

app/src/main/java/org/mozilla/tv/firefox/pocket/PocketVideoFetchScheduler.kt

app/src/main/java/org/mozilla/tv/firefox/utils/ServiceLocator.kt

liuche

Data Review Form (to be filled by Data Stewards)

Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, this is documented in metrics.md, as well as in the Glean telemetry docs: https://mozilla.github.io/glean/book/user/pings/deletion_request.html

Is there a control mechanism that allows the user to turn the data collection on and off?

This ping will be sent when the user turns off telemetry so that all their telemetry data can be deleted by the server. Any additional telemetry will not be collected after turning data collection off, so yes, data collection can be turned off by the in-app telemetry toggle.

If the request is for permanent data collection, is there someone who will monitor the data over time?

This ping is sent only by the client to request deleting data, so we will not have data from this ping.

Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Type 2

Is the data collection request for default-on or default-off?

Default on for telemetry

Does the instrumentation include the addition of any new identifiers (whether anonymous or otherwise; e.g., username, random IDs, etc. See the appendix for more details)?

Sends client id, which is necessary to delete the telemetry data associated with this client

Is the data collection covered by the existing Firefox privacy notice?

Yes

Does there need to be a check-in in the future to determine whether to renew the data? (Yes/No) (If yes, set a todo reminder or file a bug if appropriate)**

No, data deletion request

Does the data collection use a third-party collection tool? If yes, escalate to legal.

No

mcomella

lgtm. I verified the fix by launching the app in an emulator, disabling telemetry upload, and verifying the breakpoint I set that enqueues the deletion ping in the Glean library was set. This is the breakpoint I checked:

I didn't know how to verify the legacy client ID was set so I didn't verify that – I expect QA will have more expertise to do that by validating what hits the server. However, the code looks straightforward so I'm not too concerned.

Thanks psymoon!

mcomella · 2020-06-25T22:12:21Z

app/build.gradle

@@ -4,6 +4,9 @@

 import org.mozilla.gradle.tasks.ValidateAndroidAppReleaseConfiguration

+plugins {
+    id "com.jetbrains.python.envs" version "0.0.26"


Why is this needed? Is this used by the glean script?

Yep. Related docs, FYI: https://mozilla.github.io/glean/book/user/adding-glean-to-your-project.html#adding-new-metrics

mcomella · 2020-06-25T22:14:22Z

app/build.gradle

@@ -216,6 +224,10 @@ dependencies {
    androidTestImplementation 'com.squareup.okhttp3:mockwebserver:3.10.0'
    testImplementation 'com.squareup.okhttp3:mockwebserver:3.10.0'

+    // For the initial release of Glean 19, we require consumer applications to
+    // depend on a separate library for unit tests. This will be removed in future releases.
+    testImplementation "org.mozilla.telemetry:glean-forUnitTests:21.3.0"


Should this be 24.0.0? Same as ext.moz_components_version?

No, this is correct. This is the version of Glean that ships with a-c 2.4.0.0. This is the gotcha I was talking about that the Glean Gradle plugin solves (handling this mapping automatically). Unfortunately, the Glean Gradle plugin isn't available prior to a-c 27.0.0.

This is the minimum version we would need to update to get the deletion request ping.

…egration

mcomella · 2020-07-14T00:17:44Z

I didn't know how to verify the legacy client ID was set so I didn't verify that

I reverified using the glean debug ping viewer and saw a legacy client ID was set. However, I didn't verify it was the same one as on the device – I'm going to assume that is correct.

psymoon requested a review from mcomella June 24, 2020 16:14

mdboom suggested changes Jun 24, 2020

View reviewed changes

psymoon force-pushed the ccpa branch 2 times, most recently from df32e3a to 9f14808 Compare June 24, 2020 19:32

psymoon closed this Jun 24, 2020

psymoon reopened this Jun 24, 2020

psymoon force-pushed the ccpa branch from 9f14808 to fcee127 Compare June 24, 2020 20:00

mcomella reviewed Jun 25, 2020

View reviewed changes

psymoon mentioned this pull request Jun 25, 2020

Upgrade to a-c 27 and use glean plugin #2993

Open

psymoon requested a review from mdboom June 25, 2020 14:29

liuche approved these changes Jun 25, 2020

View reviewed changes

mcomella approved these changes Jun 25, 2020

View reviewed changes

mdboom approved these changes Jun 26, 2020

View reviewed changes

psymoon mentioned this pull request Jun 26, 2020

build-pr CI reporting failure with BUILD SUCCESSFUL #2995

Open

psymoon added 5 commits June 26, 2020 16:25

Issue mozilla-mobile#2989: Add deletion-request ping

f3cae3b

Issue mozilla-mobile#2989: Add glean metrics.md for deletion-request

6fe1af7

Issue mozilla-mobile#2989: Add glean markdown and metrics support

ce27ec9

Issue mozilla-mobile#2989: Upgrade a-c version to 24

a897844

This is the minimum version we would need to update to get the deletion request ping.

Issue mozilla-mobile#2989: Set legacy clientID UUID from TelemetryInt…

857ceaf

…egration

psymoon force-pushed the ccpa branch from fcee127 to a08f843 Compare June 26, 2020 20:26

Closes mozilla-mobile#2989: Add glean unit test dependency

5a54864

psymoon force-pushed the ccpa branch from a08f843 to 5a54864 Compare June 26, 2020 20:31

psymoon merged commit f2e6895 into mozilla-mobile:master Jun 26, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Closes #2989: Add deletion-request ping support for legacy_client_id #2992

Closes #2989: Add deletion-request ping support for legacy_client_id #2992

psymoon commented Jun 24, 2020 •

edited

Loading

mdboom Jun 24, 2020

psymoon Jun 24, 2020 •

edited

Loading

mdboom Jun 24, 2020

psymoon Jun 24, 2020

psymoon Jun 24, 2020

mcomella Jun 25, 2020

mdboom Jun 25, 2020

psymoon Jun 25, 2020

psymoon commented Jun 24, 2020

mcomella left a comment

liuche left a comment

mcomella left a comment

mcomella Jun 25, 2020

mdboom Jun 26, 2020

mcomella Jun 25, 2020

mdboom Jun 26, 2020

mcomella commented Jul 14, 2020

Closes #2989: Add deletion-request ping support for legacy_client_id #2992

Closes #2989: Add deletion-request ping support for legacy_client_id #2992

Conversation

psymoon commented Jun 24, 2020 • edited Loading

Choose a reason for hiding this comment

psymoon Jun 24, 2020 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

psymoon commented Jun 24, 2020

Request for data collection review form

mcomella left a comment

Choose a reason for hiding this comment

liuche left a comment

Choose a reason for hiding this comment

Data Review Form (to be filled by Data Stewards)

mcomella left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mcomella commented Jul 14, 2020

psymoon commented Jun 24, 2020 •

edited

Loading

psymoon Jun 24, 2020 •

edited

Loading