feat(): Add usage_reporting template to glean_usage sql_generator #6581
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kik-kik
force-pushed
the
feat/add_usage_reporting-to-glean_usage
branch
from
0b452ba to
26dcc56
Compare
kik-kik
commented
Nov 29, 2024
| require_partition_filter: true | ||
| clustering: | ||
| fields: | ||
| - normalized_channel |
There was a problem hiding this comment.
@irrationalagent So far it appears most of the values in the new ping for normalized_channel are null, is this expected?
There was a problem hiding this comment.
We plan to fix this.
kik-kik
commented
Nov 29, 2024
| -- client_info.app_build, | ||
| normalized_os, | ||
| normalized_os_version, | ||
| -- client_info.locale, |
There was a problem hiding this comment.
@irrationalagent all of the client_info.* fields are missing in the new ping. Are we planning on adding those or should I just delete these from the code?
There was a problem hiding this comment.
Some of these will be added back, not under client_info but likely as "normal" metrics e.g. string metrics. Specifically, we can expect these fields to be added as metrics:
App_version
App_build
first_run_date
reason
OS
Duration
Is default browser
Distribution_id
Channel
I will follow up about where exactly in the ping we will expect to see those.
kik-kik
commented
Nov 29, 2024
| normalized_os, | ||
| normalized_os_version, | ||
| -- client_info.locale, | ||
| normalized_country_code, |
There was a problem hiding this comment.
@irrationalagent Are we ok to include the country code instead of locale (no longer available due to it coming from the client_info)?
There was a problem hiding this comment.
country code should stay, this is not reported in the ping itself but assigned on the backend
kik-kik
commented
Nov 29, 2024
| CAST(TRUE AS INT64) & CAST(is_active AS INT64) AS days_active_bits, | ||
| udf.days_since_created_profile_as_28_bits( | ||
| DATE_DIFF(submission_date, first_run_date, DAY) | ||
| ) AS days_created_profile_bits, |
There was a problem hiding this comment.
@irrationalagent Is the days_created_profile_bits field important? Right now it seems first_run_date is not available in the new ping. Is this intended?
kik-kik
force-pushed
the
feat/add_usage_reporting-to-glean_usage
branch
from
8281806 to
cc26ab9
Compare
This comment has been minimized.
This comment has been minimized.
kik-kik
changed the title
kik-kik
force-pushed
the
feat/add_usage_reporting-to-glean_usage
branch
from
cc26ab9 to
0a92d91
Compare
This comment has been minimized.
This comment has been minimized.
kik-kik
force-pushed
the
feat/add_usage_reporting-to-glean_usage
branch
from
eecc977 to
ace64de
Compare
|
Waiting for the missing fields to be added to the usage_reporting ping. |
…ed in the usage_reporting derived tables
kik-kik
force-pushed
the
feat/add_usage_reporting-to-glean_usage
branch
from
ace64de to
303a4bf
Compare
Integration report for "feat: remove dau_reporting ping as we decided it should not be included in the usage_reporting derived tables"
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
feat(): Add dau_reporting template to glean_usage sql_generator
Description
This is a replacement to to PR##6399
The main difference being a clean git history and all resource being renamed from
dau_reportin_*tousage_reporting_*.Also, we still need to account for data sent via the old
dau_reportingping. This is handled here via theusage_reporting_clients_daily_v1template.Original description:
** Note: modified to use the updated naming**
Adding templates to set up ELT for the usage_reporting ping. This is a new minimal ping which will be used specifically for reporting the number of users of our applications. The templates mimic our existing ETL setup for the baseline ping and will result in the following artifacts being generated in BigQuery for each GLEAN app sending us the
usage_reportingping:Tables:
[app_name]_derived.usage_reporting_clients_daily_v1- Keeping a record of each client that sent us the usage_reporting ping and limited selected numbers of associated with them.[app_name]_derived.usage_reporting_clients_first_seen_v1- Letting us know when we observed a specific client for the first time in our warehouse.[app_name]_derived.usage_reporting_clients_last_seen_v1- Keeping track of when is the last time we saw each client, and when is the last time they were active, and how long it's been since the profile has been created.Views:
[app_name].usage_reporting_clients_daily[app_name].usage_reporting_clients_first_seen[app_name].usage_reporting_clients_last_seen