Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DENG-6887] - Build necessary aggregate tables for FX dashboards related to desktop events #6733

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

alekhyamoz
Copy link
Contributor

@alekhyamoz alekhyamoz commented Dec 23, 2024

Related to the data retention project - https://mozilla-hub.atlassian.net/browse/DENG-6887.
This PR builds necessary aggregate tables pulling from telemetry.events

Note - The PR will be merged after the holiday season.

┆Issue is synchronized with this Jira Task

@alekhyamoz alekhyamoz force-pushed the DENG-6887_desktop_events branch 3 times, most recently from 5a626ab to fba6145 Compare December 23, 2024 18:20
@alekhyamoz alekhyamoz force-pushed the DENG-6887_desktop_events branch from fba6145 to bd24cb0 Compare December 23, 2024 18:25
@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot

This comment has been minimized.

@dataops-ci-bot
Copy link

Integration report for "Fix view definition"

sql.diff

Click to expand!
Only in /tmp/workspace/generated-sql/dags/: bqetl_fx_cert_error_privacy_dashboard.py
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_fx_cert_error_privacy_dashboard.py /tmp/workspace/generated-sql/dags/bqetl_fx_cert_error_privacy_dashboard.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_fx_cert_error_privacy_dashboard.py	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_fx_cert_error_privacy_dashboard.py	2024-12-26 17:46:49.000000000 +0000
@@ -0,0 +1,131 @@
+# Generated via https://github.com/mozilla/bigquery-etl/blob/main/bigquery_etl/query_scheduling/generate_airflow_dags.py
+
+from airflow import DAG
+from airflow.sensors.external_task import ExternalTaskMarker
+from airflow.sensors.external_task import ExternalTaskSensor
+from airflow.utils.task_group import TaskGroup
+import datetime
+from operators.gcp_container_operator import GKEPodOperator
+from utils.constants import ALLOWED_STATES, FAILED_STATES
+from utils.gcp import bigquery_etl_query, bigquery_dq_check, bigquery_bigeye_check
+
+docs = """
+### bqetl_fx_cert_error_privacy_dashboard
+
+Built from bigquery-etl repo, [`dags/bqetl_fx_cert_error_privacy_dashboard.py`](https://github.com/mozilla/bigquery-etl/blob/generated-sql/dags/bqetl_fx_cert_error_privacy_dashboard.py)
+
+#### Description
+
+This DAG builds the desktop event_aggregates table for unique users count
+
+#### Owner
+
+[email protected]
+
+#### Tags
+
+* impact/tier_3
+* repo/bigquery-etl
+"""
+
+
+default_args = {
+    "owner": "[email protected]",
+    "start_date": datetime.datetime(2024, 12, 19, 0, 0),
+    "end_date": None,
+    "email": ["[email protected]", "[email protected]"],
+    "depends_on_past": False,
+    "retry_delay": datetime.timedelta(seconds=300),
+    "email_on_failure": True,
+    "email_on_retry": False,
+    "retries": 2,
+}
+
+tags = ["impact/tier_3", "repo/bigquery-etl"]
+
+with DAG(
+    "bqetl_fx_cert_error_privacy_dashboard",
+    default_args=default_args,
+    schedule_interval="40 16 * * *",
+    doc_md=docs,
+    tags=tags,
+) as dag:
+
+    wait_for_bq_main_events = ExternalTaskSensor(
+        task_id="wait_for_bq_main_events",
+        external_dag_id="copy_deduplicate",
+        external_task_id="bq_main_events",
+        execution_delta=datetime.timedelta(seconds=56400),
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    wait_for_event_events = ExternalTaskSensor(
+        task_id="wait_for_event_events",
+        external_dag_id="copy_deduplicate",
+        external_task_id="event_events",
+        execution_delta=datetime.timedelta(seconds=56400),
+        check_existence=True,
+        mode="reschedule",
+        poke_interval=datetime.timedelta(minutes=5),
+        allowed_states=ALLOWED_STATES,
+        failed_states=FAILED_STATES,
+        pool="DATA_ENG_EXTERNALTASKSENSOR",
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_normalized_channel__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__fx_cert_error_unique_users_normalized_channel__v1",
+        destination_table="fx_cert_error_unique_users_normalized_channel_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=["[email protected]", "[email protected]"],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_os__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__fx_cert_error_unique_users_os__v1",
+        destination_table="fx_cert_error_unique_users_os_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=["[email protected]", "[email protected]"],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
+    telemetry_derived__fx_privacy_dau_agg__v1 = bigquery_etl_query(
+        task_id="telemetry_derived__fx_privacy_dau_agg__v1",
+        destination_table="fx_privacy_dau_agg_v1",
+        dataset_id="telemetry_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=["[email protected]", "[email protected]"],
+        date_partition_parameter="submission_date",
+        depends_on_past=False,
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_normalized_channel__v1.set_upstream(
+        wait_for_bq_main_events
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_normalized_channel__v1.set_upstream(
+        wait_for_event_events
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_os__v1.set_upstream(
+        wait_for_bq_main_events
+    )
+
+    telemetry_derived__fx_cert_error_unique_users_os__v1.set_upstream(
+        wait_for_event_events
+    )
+
+    telemetry_derived__fx_privacy_dau_agg__v1.set_upstream(wait_for_bq_main_events)
+
+    telemetry_derived__fx_privacy_dau_agg__v1.set_upstream(wait_for_event_events)
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: fx_cert_error_unique_users_normalized_channel
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: fx_cert_error_unique_users_os
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry: fx_privacy_dau_agg
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: fx_cert_error_unique_users_normalized_channel_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: fx_cert_error_unique_users_os_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived: fx_privacy_dau_agg_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/apple_ads_external/ios_app_campaign_stats_v1/bigconfig.yml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/apple_ads_external/ios_app_campaign_stats_v1/bigconfig.yml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/apple_ads_external/ios_app_campaign_stats_v1/bigconfig.yml	2024-12-26 17:39:24.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/apple_ads_external/ios_app_campaign_stats_v1/bigconfig.yml	2024-12-26 17:43:17.000000000 +0000
@@ -1,7 +1,6 @@
 type: BIGCONFIG_FILE
-
 tag_deployments:
-  - collection:
+- collection:
       name: Operational Checks
       notification_channels:
         - slack: '#de-bigeye-triage'
@@ -24,10 +23,10 @@
         metrics:
           - saved_metric_id: is_not_null
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
       - column_selectors:
@@ -35,10 +34,10 @@
         metrics:
           - saved_metric_id: is_2_char_len
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
       - column_selectors:
@@ -46,17 +45,17 @@
         metrics:
           - saved_metric_id: volume
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
           - saved_metric_id: freshness
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_ads_derived/android_app_campaign_stats_v1/bigconfig.yml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_ads_derived/android_app_campaign_stats_v1/bigconfig.yml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/google_ads_derived/android_app_campaign_stats_v1/bigconfig.yml	2024-12-26 17:39:24.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/google_ads_derived/android_app_campaign_stats_v1/bigconfig.yml	2024-12-26 17:43:16.000000000 +0000
@@ -1,7 +1,6 @@
 type: BIGCONFIG_FILE
-
 tag_deployments:
-  - collection:
+- collection:
       name: Operational Checks
       notification_channels:
         - slack: '#de-bigeye-triage'
@@ -23,10 +22,10 @@
         metrics:
           - saved_metric_id: is_not_null
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
       - column_selectors:
@@ -34,10 +33,10 @@
         metrics:
           - saved_metric_id: is_2_char_len
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
       - column_selectors:
@@ -45,17 +44,17 @@
         metrics:
           - saved_metric_id: volume
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
           - saved_metric_id: freshness
             lookback:
-              lookback_type: DATA_TIME
               lookback_window:
                 interval_type: DAYS
                 interval_value: 28
+        lookback_type: DATA_TIME
             rct_overrides:
               - date
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/metadata.yaml	2024-12-26 17:41:52.000000000 +0000
@@ -0,0 +1,14 @@
+friendly_name: Fx Cert Error Unique Users Normalized Channel
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/taar
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.fx_cert_error_unique_users_normalized_channel_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_normalized_channel/view.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.fx_cert_error_unique_users_normalized_channel`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.fx_cert_error_unique_users_normalized_channel_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/metadata.yaml	2024-12-26 17:41:52.000000000 +0000
@@ -0,0 +1,14 @@
+friendly_name: Fx Cert Error Unique Users Os
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/taar
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.fx_cert_error_unique_users_os_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_cert_error_unique_users_os/view.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.fx_cert_error_unique_users_os`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.fx_cert_error_unique_users_os_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/metadata.yaml	2024-12-26 17:41:52.000000000 +0000
@@ -0,0 +1,14 @@
+friendly_name: Fx Privacy Dau Agg
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:dataops-managed/taar
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.telemetry_derived.fx_privacy_dau_agg_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry/fx_privacy_dau_agg/view.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,7 @@
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.telemetry.fx_privacy_dau_agg`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.telemetry_derived.fx_privacy_dau_agg_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/metadata.yaml	2024-12-26 17:41:50.000000000 +0000
@@ -0,0 +1,26 @@
+friendly_name: Firefox Cert Error Dashboard - Unique users by normalized channel
+description: |-
+  Unique users experiencing cert error page per day per channel
+owners:
+- [email protected]
+labels:
+  incremental: true
+  owner1: akommasani
+  table_type: aggregate
+  shredder_mitigation: true
+  dag: bqetl_fx_cert_error_privacy_dashboard
+scheduling:
+  dag_name: bqetl_fx_cert_error_privacy_dashboard
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: false
+    expiration_days: null
+  range_partitioning: null
+  clustering: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references: {}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/query.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,14 @@
+SELECT
+  submission_date,
+  normalized_channel,
+  COUNT(DISTINCT client_id) AS nbr_unique_users
+FROM
+  `mozdata.telemetry.events`
+WHERE
+  event_category = 'security.ui.certerror'
+  AND event_object = 'aboutcerterror'
+  AND event_method = 'load'
+  AND submission_date = @submission_date
+GROUP BY
+  submission_date,
+  normalized_channel
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_normalized_channel_v1/schema.yaml	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,13 @@
+fields:
+- mode: NULLABLE
+  name: submission_date
+  type: DATE
+  description: Submission Date
+- mode: NULLABLE
+  name: normalized_channel
+  type: STRING
+  description: Normalized Channel
+- name: nbr_unique_users
+  type: INTEGER
+  mode: NULLABLE
+  description: Number of Unique users (distinct client ids)
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/metadata.yaml	2024-12-26 17:41:50.000000000 +0000
@@ -0,0 +1,26 @@
+friendly_name: Firefox Cert Error Dashboard - Unique users by os
+description: |-
+  Unique users experiencing cert error page per day per os
+owners:
+- [email protected]
+labels:
+  incremental: true
+  owner1: akommasani
+  table_type: aggregate
+  shredder_mitigation: true
+  dag: bqetl_fx_cert_error_privacy_dashboard
+scheduling:
+  dag_name: bqetl_fx_cert_error_privacy_dashboard
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: false
+    expiration_days: null
+  range_partitioning: null
+  clustering: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references: {}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/query.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,14 @@
+SELECT
+  submission_date,
+  os,
+  COUNT(DISTINCT client_id) AS nbr_unique_users
+FROM
+  `mozdata.telemetry.events`
+WHERE
+  event_category = 'security.ui.certerror'
+  AND event_object = 'aboutcerterror'
+  AND event_method = 'load'
+  AND submission_date = @submission_date
+GROUP BY
+  submission_date,
+  os
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_cert_error_unique_users_os_v1/schema.yaml	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,13 @@
+fields:
+- mode: NULLABLE
+  name: submission_date
+  type: DATE
+  description: Submission Date
+- mode: NULLABLE
+  name: os
+  type: STRING
+  description: OS
+- name: nbr_unique_users
+  type: INTEGER
+  mode: NULLABLE
+  description: Number of Unique users (distinct client ids)
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/metadata.yaml	2024-12-26 17:41:50.000000000 +0000
@@ -0,0 +1,26 @@
+friendly_name: Firefox Privacy Dashboard - DAU interracting with the privacy panel
+description: |-
+  DAU interracting with the privacy panel
+owners:
+- [email protected]
+labels:
+  incremental: true
+  owner1: akommasani
+  table_type: aggregate
+  shredder_mitigation: true
+  dag: bqetl_fx_cert_error_privacy_dashboard
+scheduling:
+  dag_name: bqetl_fx_cert_error_privacy_dashboard
+bigquery:
+  time_partitioning:
+    type: day
+    field: submission_date
+    require_partition_filter: false
+    expiration_days: null
+  range_partitioning: null
+  clustering: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references: {}
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/query.sql	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,15 @@
+SELECT
+  event_object,
+  submission_date,
+  COUNT(*) AS nbr_events,
+  COUNT(DISTINCT client_id) AS nbr_distinct_users,
+FROM
+  `moz-fx-data-shared-prod.telemetry.events`
+WHERE
+  event_category = 'security.ui.protectionspopup'
+  AND submission_date = @submission_date
+GROUP BY
+  event_object,
+  submission_date
+ORDER BY
+  submission_date
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/telemetry_derived/fx_privacy_dau_agg_v1/schema.yaml	2024-12-26 17:39:27.000000000 +0000
@@ -0,0 +1,17 @@
+fields:
+- mode: NULLABLE
+  name: submission_date
+  type: DATE
+  description: Submission Date
+- mode: NULLABLE
+  name: event_object
+  type: STRING
+  description: Submission Date
+- mode: NULLABLE
+  name: nbr_events
+  type: INTEGER
+  description: Number of events
+- name: nbr_distinct_users
+  type: INTEGER
+  mode: NULLABLE
+  description: Number of distinct users

Link to full diff

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants