From e619ffa000b727d7315d4db1730475c778194391 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Wed, 6 Dec 2023 16:51:07 -0800 Subject: [PATCH 01/48] Update sl-jdbc.md We're making some changes to our API (not released yet), so I'm preemptively adding info. --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index aba309566f8..b250c9a7818 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -254,11 +254,14 @@ Where filters in API allow for a filter list or string. We recommend using the f Where Filters have a few objects that you can use: -- `Dimension()` - Used for any categorical or time dimensions. If used for a time dimension, granularity is required - `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` +- `Dimension()` - Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` - `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'MONTH')`. + +Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'month')`. + +For both `TimeDimension()` and `Dimension()` objects, the grain is only required if the aggregation time dimension for the measures / metrics associated with the where filter is not the same for all measures. - Use the following example to query using a `where` filter with the string format: @@ -315,7 +318,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], order_by=[-'order_gross_profit'] }} ``` -If you are ordering by an object that's been operated on (e.g., change granularity), or you are using the full object notation, descending order must look like: +If you are ordering by an object that's been operated on (e.g., you changed the the granularity of the time dimension), or you are using the full object notation, descending order must look like: ```bash select * from {{ From dc49f197ec31317f2b069c64c413eb9f4804274a Mon Sep 17 00:00:00 2001 From: rpourzand Date: Thu, 7 Dec 2023 17:55:44 -0800 Subject: [PATCH 02/48] Update sl-jdbc.md Adding an example from Paul --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 60 ++++++++++++++++++++- 1 file changed, 58 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index b250c9a7818..2a9c5e95199 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -259,9 +259,65 @@ Where Filters have a few objects that you can use: - `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimesion` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'month')`. +Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimension` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'month')`. + +For both `TimeDimension()` and `Dimension()` objects, the grain is only required if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. + +For example, consider this Semantic Model and Metric Config which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. + +```yaml +--- +semantic_model: + name: my_model_source + +defaults: + agg_time_dimension: created_month + + measures: + - name: measure_0 + agg: sum + - name: measure_1 + agg: sum + agg_time_dimension: order_year + + dimensions: + - name: created_month + type: time + type_params: + time_granularity: month + - name: order_year + type: time + time_granularity: year +... + + +metrics: + name: metric_0 + description: A metric with a month grain. + type: simple + type_params: + measure: measure_0 + + name: metric_1 + description: A metric with a year grain + type: simple + type_params: + measure: measure_1 + +``` + +Assuming the user is querying metric_0 and metric_1 together, a valid filter would be: + + * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` + +Invalid Filters would be: + + `* "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query + are defined based on measures with different grains. + + * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - + metric_1 is not available at a month grain. -For both `TimeDimension()` and `Dimension()` objects, the grain is only required if the aggregation time dimension for the measures / metrics associated with the where filter is not the same for all measures. - Use the following example to query using a `where` filter with the string format: From be5c52c0a68fd2f5e2a9958661e52eefb7640511 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Thu, 7 Dec 2023 17:57:48 -0800 Subject: [PATCH 03/48] Update sl-graphql.md copying the same over for jdbc -- maybe we can figure out a way to put this somewhere so we don't have to duplicate --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 63 ++++++++++++++++++- 1 file changed, 60 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index b7d13d0d453..0fd55397bc9 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -432,18 +432,18 @@ mutation { The `where` filter takes a list argument (or a string for a single input). Depending on the object you are filtering, there are a couple of parameters: - - `Dimension()` — Used for any categorical or time dimensions. If used for a time dimension, granularity is required. For example, `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`. + - `Dimension()` — Used for any categorical or time dimensions. For example, `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`. - `Entity()` — Used for entities like primary and foreign keys, such as `Entity('order_id')`. -Note: If you prefer a more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension name and also requires granularity. For example, `TimeDimension('metric_time', 'MONTH')`. +Note: If you prefer a more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension and optionally the granularity level. `TimeDimension('metric_time', 'month')`. ```graphql mutation { createQuery( environmentId: BigInt! metrics:[{name: "order_total"}] - groupBy:[{name: "customer__customer_type"}, {name: "metric_time", grain: MONTH}] + groupBy:[{name: "customer__customer_type"}, {name: "metric_time", grain: month}] where:[{sql: "{{ Dimension('customer__customer_type') }} = 'new'"}, {sql:"{{ Dimension('metric_time').grain('month') }} > '2022-10-01'"}] ) { queryId @@ -451,6 +451,63 @@ mutation { } ``` +For both `TimeDimension()` and `Dimension()` objects, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. + +For example, consider this Semantic Model and Metric Configuration which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. + +```yaml +--- +semantic_model: + name: my_model_source + +defaults: + agg_time_dimension: created_month + + measures: + - name: measure_0 + agg: sum + - name: measure_1 + agg: sum + agg_time_dimension: order_year + + dimensions: + - name: created_month + type: time + type_params: + time_granularity: month + - name: order_year + type: time + time_granularity: year +... + + +metrics: + name: metric_0 + description: A metric with a month grain. + type: simple + type_params: + measure: measure_0 + + name: metric_1 + description: A metric with a year grain + type: simple + type_params: + measure: measure_1 + +``` + +Assuming the user is querying metric_0 and metric_1 together, a valid filter would be: + + * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` + +Invalid Filters would be: + + `* "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query + are defined based on measures with different grains. + + * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - + metric_1 is not available at a month grain. + **Query with Order** ```graphql From 8c510b7c3cf7ca717c2d7a1fcc1337f10ad78726 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Thu, 7 Dec 2023 17:58:45 -0800 Subject: [PATCH 04/48] Update sl-jdbc.md timedimension is no longer optional --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 2a9c5e95199..3c771cc182c 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -259,7 +259,7 @@ Where Filters have a few objects that you can use: - `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimension` input takes the time dimension name and also requires granularity, like this: `TimeDimension('metric_time', 'month')`. +Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimension` input takes the time dimension name optionally requires granularity, like this: `TimeDimension('metric_time', 'month')`. For both `TimeDimension()` and `Dimension()` objects, the grain is only required if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. From 436577f237fedc60038fb38e7d05ede0d97695d8 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Thu, 7 Dec 2023 18:00:09 -0800 Subject: [PATCH 05/48] Update sl-jdbc.md fixing some updates --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 3c771cc182c..220d4fb2cd4 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -261,7 +261,7 @@ Where Filters have a few objects that you can use: Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimension` input takes the time dimension name optionally requires granularity, like this: `TimeDimension('metric_time', 'month')`. -For both `TimeDimension()` and `Dimension()` objects, the grain is only required if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. +For both `TimeDimension()` and `Dimension()` objects, the grain is only required if in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. For example, consider this Semantic Model and Metric Config which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. @@ -287,10 +287,10 @@ defaults: time_granularity: month - name: order_year type: time - time_granularity: year + type_params: + time_granularity: year ... - metrics: name: metric_0 description: A metric with a month grain. @@ -306,7 +306,7 @@ metrics: ``` -Assuming the user is querying metric_0 and metric_1 together, a valid filter would be: +Assuming the user is querying metric_0 and metric_1 together in a single request, a valid WHERE filter would be: * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` From c250bb6490a1d4646d53907378cc789b9a283567 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Thu, 7 Dec 2023 18:00:59 -0800 Subject: [PATCH 06/48] Update sl-graphql.md fixing typo --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 0fd55397bc9..163380bb647 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -477,7 +477,8 @@ defaults: time_granularity: month - name: order_year type: time - time_granularity: year + type_params: + time_granularity: year ... From 61af03e70e666c1e702dca66428bed253b092f5a Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 8 Dec 2023 14:52:59 -0800 Subject: [PATCH 07/48] Update sl-graphql.md making paul's reco's --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 163380bb647..7b4eeddfee6 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -436,7 +436,7 @@ The `where` filter takes a list argument (or a string for a single input). Depen - `Entity()` — Used for entities like primary and foreign keys, such as `Entity('order_id')`. -Note: If you prefer a more strongly typed `where` clause, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension and optionally the granularity level. `TimeDimension('metric_time', 'month')`. +Note: If you prefer a `where` clause with a more explicit path, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension and optionally the granularity level. `TimeDimension('metric_time', 'month')`. ```graphql mutation { @@ -502,8 +502,8 @@ Assuming the user is querying metric_0 and metric_1 together, a valid filter wou * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` Invalid Filters would be: - - `* "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query + + * ` "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query are defined based on measures with different grains. * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - From c5afac9f933dc2bba6aa7f28c5f782afd2ba1448 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 8 Dec 2023 14:53:53 -0800 Subject: [PATCH 08/48] Update sl-jdbc.md paul's recos --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 220d4fb2cd4..b1d90721d24 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -312,7 +312,7 @@ Assuming the user is querying metric_0 and metric_1 together in a single request Invalid Filters would be: - `* "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query + * `"{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query are defined based on measures with different grains. * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - From 384eba8856f59a341af767b744dc866673f9d66a Mon Sep 17 00:00:00 2001 From: rpourzand Date: Tue, 19 Dec 2023 10:59:29 -0800 Subject: [PATCH 09/48] Update sl-graphql.md also adding a dimension only example to it --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 3591c36cbd3..774b0c53d87 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -204,6 +204,8 @@ DimensionType = [CATEGORICAL, TIME] ### Querying +When querying for data, _either_ a `groupBy` _or_ a `metrics` selection is required. + **Create Dimension Values query** ```graphql @@ -428,6 +430,19 @@ mutation { } ``` +**Query a categorical dimension on its own** + +```graphql +mutation { + createQuery( + environmentId: 123456 + groupBy: [{name: "customer__customer_type"}] + ) { + queryId + } +} +``` + **Query with a where filter** The `where` filter takes a list argument (or a string for a single input). Depending on the object you are filtering, there are a couple of parameters: From bc2239ac113dcbeff4e97830768475ada91b4c9b Mon Sep 17 00:00:00 2001 From: rpourzand Date: Wed, 20 Dec 2023 17:07:12 -0800 Subject: [PATCH 10/48] Update sl-jdbc.md updating per jordan's input --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 69428878f80..a1f79208e6d 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -256,12 +256,12 @@ Where Filters have a few objects that you can use: - `Dimension()` - Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` -- `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` +- `TimeDimension()` - Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` +- `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` -Note: If you prefer a more explicit path to create the `where` clause, you can optionally use the `TimeDimension` feature. This helps separate out categorical dimensions from time-related ones. The `TimeDimension` input takes the time dimension name optionally requires granularity, like this: `TimeDimension('metric_time', 'month')`. -For both `TimeDimension()` and `Dimension()` objects, the grain is only required if in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. +For `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. For example, consider this Semantic Model and Metric Config which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. From a73e79398bb432a46133c729594eafe8ba3a4743 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:23:45 -0500 Subject: [PATCH 11/48] Update sl-graphql.md format yaml --- .../docs/docs/dbt-cloud-apis/sl-graphql.md | 41 ++++++++----------- 1 file changed, 16 insertions(+), 25 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 774b0c53d87..28ac38936ca 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -471,20 +471,17 @@ For both `TimeDimension()` and `Dimension()` objects, the grain is only required For example, consider this Semantic Model and Metric Configuration which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. ```yaml ---- semantic_model: name: my_model_source - -defaults: - agg_time_dimension: created_month +defaults: + agg_time_dimension: created_month measures: - name: measure_0 agg: sum - name: measure_1 agg: sum agg_time_dimension: order_year - dimensions: - name: created_month type: time @@ -494,35 +491,29 @@ defaults: type: time type_params: time_granularity: year -... - metrics: - name: metric_0 - description: A metric with a month grain. - type: simple - type_params: - measure: measure_0 - - name: metric_1 - description: A metric with a year grain - type: simple - type_params: - measure: measure_1 - + - name: metric_0 + description: A metric with a month grain. + type: simple + type_params: + measure: measure_0 + - name: metric_1 + description: A metric with a year grain. + type: simple + type_params: + measure: measure_1 ``` -Assuming the user is querying metric_0 and metric_1 together, a valid filter would be: +Assuming the user is querying `metric_0` and `metric_1` together, a valid filter would be: * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` -Invalid Filters would be: +Invalid filters would be: - * ` "{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query - are defined based on measures with different grains. + * ` "{{ TimeDimension('metric_time') }} > '2020-01-01'"` — metrics in the query b are defined based on measures with different grains. - * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - - metric_1 is not available at a month grain. + * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` — `metric_1` is not available at a month grain. **Query with Order** From ceea7dc2bc35825d1dfd8ed26c2dbf9e616e7bef Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:25:26 -0500 Subject: [PATCH 12/48] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 28ac38936ca..77d50d1d293 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -468,7 +468,7 @@ mutation { For both `TimeDimension()` and `Dimension()` objects, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. -For example, consider this Semantic Model and Metric Configuration which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. +For example, consider this Semantic model and Metric configuration, which contains two metrics that are aggregated across different time grains. This example shows a single semantic model, but the same goes for metrics across more than one semantic model. ```yaml semantic_model: From 5f832e9d6ecb9468aeb615ef66f227203e7360f5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:25:50 -0500 Subject: [PATCH 13/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index a1f79208e6d..09fb73c1c9f 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -254,7 +254,7 @@ Where filters in API allow for a filter list or string. We recommend using the f Where Filters have a few objects that you can use: -- `Dimension()` - Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` +- `Dimension()` — Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` - `TimeDimension()` - Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` From 1b774190a0e67f9c7e5641eddb687f16bd025e70 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:26:04 -0500 Subject: [PATCH 14/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 09fb73c1c9f..32e57208126 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -256,7 +256,7 @@ Where Filters have a few objects that you can use: - `Dimension()` — Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` -- `TimeDimension()` - Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` +- `TimeDimension()` — Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` - `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` From e06152133bcd3184181cfc371ba05152b45ae626 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:26:29 -0500 Subject: [PATCH 15/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 32e57208126..98fe99fc713 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -258,7 +258,7 @@ Where Filters have a few objects that you can use: - `TimeDimension()` — Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` -- `Entity()` - Used for entities like primary and foreign keys - `Entity('order_id')` +- `Entity()` — Used for entities like primary and foreign keys - `Entity('order_id')` For `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. From 7aa722d1e834b64b74451d79637adeafae1a137d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:28:37 -0500 Subject: [PATCH 16/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 98fe99fc713..60fd45d2424 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -261,7 +261,7 @@ Where Filters have a few objects that you can use: - `Entity()` — Used for entities like primary and foreign keys - `Entity('order_id')` -For `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. +For `TimeDimension()`, the grain is only required in the `WHERE` filter if the aggregation time dimensions for the measures and metrics associated with the where filter have different grains. For example, consider this Semantic Model and Metric Config which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. From 7dc125e79f33ec8dba66c98534a581eca4791b20 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:34:29 -0500 Subject: [PATCH 17/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 60fd45d2424..b6eb03affde 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -263,7 +263,7 @@ Where Filters have a few objects that you can use: For `TimeDimension()`, the grain is only required in the `WHERE` filter if the aggregation time dimensions for the measures and metrics associated with the where filter have different grains. -For example, consider this Semantic Model and Metric Config which contains two metrics that are aggregated across different time grains. This example shows a single Semantic Model, but the same goes for metrics across more than one Semantic Model. +For example, consider this Semantic model and Metric config, which contains two metrics that are aggregated across different time grains. This example shows a single semantic model, but the same goes for metrics across more than one semantic model. ```yaml --- From ece5d3f662660a66ab6d971272d2cc22c12ca966 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 21 Dec 2023 09:39:20 -0500 Subject: [PATCH 18/48] Update sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 70 +++++++++++---------- 1 file changed, 36 insertions(+), 34 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index b6eb03affde..41d57994cc7 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -177,8 +177,6 @@ To query metric values, here are the following parameters that are available: |`order` | Order the data returned by a particular field | `order_by=['order_gross_profit']`, use `-` for descending, or full object notation if the object is operated on: `order_by=[Metric('order_gross_profit').descending(True)`] | Optional | | `compile` | If true, returns generated SQL for the data platform but does not execute | `compile=True` | Optional | - - ## Note on time dimensions and `metric_time` You will notice that in the list of dimensions for all metrics, there is a dimension called `metric_time`. `Metric_time` is a reserved keyword for the measure-specific aggregation time dimensions. For any time-series metric, the `metric_time` keyword should always be available for use in queries. This is a common dimension across *all* metrics in a semantic graph. @@ -266,20 +264,17 @@ For `TimeDimension()`, the grain is only required in the `WHERE` filter if the a For example, consider this Semantic model and Metric config, which contains two metrics that are aggregated across different time grains. This example shows a single semantic model, but the same goes for metrics across more than one semantic model. ```yaml ---- semantic_model: name: my_model_source - -defaults: - agg_time_dimension: created_month +defaults: + agg_time_dimension: created_month measures: - name: measure_0 agg: sum - name: measure_1 agg: sum agg_time_dimension: order_year - dimensions: - name: created_month type: time @@ -288,36 +283,31 @@ defaults: - name: order_year type: time type_params: - time_granularity: year -... + time_granularity: year metrics: - name: metric_0 - description: A metric with a month grain. - type: simple - type_params: - measure: measure_0 - - name: metric_1 - description: A metric with a year grain - type: simple - type_params: - measure: measure_1 + - name: metric_0 + description: A metric with a month grain. + type: simple + type_params: + measure: measure_0 + - name: metric_1 + description: A metric with a year grain. + type: simple + type_params: + measure: measure_1 ``` -Assuming the user is querying metric_0 and metric_1 together in a single request, a valid WHERE filter would be: +Assuming the user is querying `metric_0` and `metric_1` together in a single request, a valid `WHERE` filter would be: * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` Invalid Filters would be: - * `"{{ TimeDimension('metric_time') }} > '2020-01-01'"` - metrics in the query - are defined based on measures with different grains. - - * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` - - metric_1 is not available at a month grain. + * `"{{ TimeDimension('metric_time') }} > '2020-01-01'"` — metrics in the query are defined based on measures with different grains. + * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` — `metric_1` is not available at a month grain. - Use the following example to query using a `where` filter with the string format: @@ -350,7 +340,8 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], group_by=[Dimension('metric_time')], limit=10) }} -``` +``` + ### Query with Order By Examples Order By can take a basic string that's a Dimension, Metric, or Entity and this will default to ascending order @@ -373,7 +364,8 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], limit=10, order_by=[-'order_gross_profit'] }} -``` +``` + If you are ordering by an object that's been operated on (e.g., you changed the the granularity of the time dimension), or you are using the full object notation, descending order must look like: ```bash @@ -413,14 +405,24 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], -- **Why do some dimensions use different syntax, like `metric_time` versus `[Dimension('metric_time')`?**
- When you select a dimension on its own, such as `metric_time` you can use the shorthand method which doesn't need the “Dimension” syntax. However, when you perform operations on the dimension, such as adding granularity, the object syntax `[Dimension('metric_time')` is required. + + +When you select a dimension on its own, such as `metric_time` you can use the shorthand method which doesn't need the “Dimension” syntax. However, when you perform operations on the dimension, such as adding granularity, the object syntax `[Dimension('metric_time')` is required. + + + + + +The double underscore `"__"` syntax indicates a mapping from an entity to a dimension, as well as where the dimension is located. For example, `user__country` means someone is looking at the `country` dimension from the `user` table. + + + -- **What does the double underscore `"__"` syntax in dimensions mean?**
- The double underscore `"__"` syntax indicates a mapping from an entity to a dimension, as well as where the dimension is located. For example, `user__country` means someone is looking at the `country` dimension from the `user` table. + + +The default output follows the format `{time_dimension_name}__{granularity_level}`. So for example, if the time dimension name is `ds` and the granularity level is yearly, the output is `ds__year`. -- **What is the default output when adding granularity?**
- The default output follows the format `{time_dimension_name}__{granularity_level}`. So for example, if the time dimension name is `ds` and the granularity level is yearly, the output is `ds__year`. + ## Related docs From b7cd78a646175341f0632b7a89027f4d89e34491 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 21 Dec 2023 10:05:47 -0500 Subject: [PATCH 19/48] add toggles --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 41d57994cc7..b3d9b961957 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -403,24 +403,24 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], ## FAQs - +
- - -When you select a dimension on its own, such as `metric_time` you can use the shorthand method which doesn't need the “Dimension” syntax. However, when you perform operations on the dimension, such as adding granularity, the object syntax `[Dimension('metric_time')` is required. + +When you select a dimension on its own, such as `metric_time` you can use the shorthand method which doesn't need the “Dimension” syntax. +However, when you perform operations on the dimension, such as adding granularity, the object syntax `[Dimension('metric_time')` is required. - + The double underscore `"__"` syntax indicates a mapping from an entity to a dimension, as well as where the dimension is located. For example, `user__country` means someone is looking at the `country` dimension from the `user` table. - - - -The default output follows the format `{time_dimension_name}__{granularity_level}`. So for example, if the time dimension name is `ds` and the granularity level is yearly, the output is `ds__year`. + +The default output follows the format `{{time_dimension_name}__{granularity_level}}`. + +So for example, if the `time_dimension_name` is `ds` and the granularity level is yearly, the output is `ds__year`. From 7b8cc9ad8da5b62d89a03e1bddb40173fd473d27 Mon Sep 17 00:00:00 2001 From: Matt Shaver <60105315+matthewshaver@users.noreply.github.com> Date: Fri, 22 Dec 2023 14:24:36 -0500 Subject: [PATCH 20/48] Apply suggestions from code review --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index b3d9b961957..6c57cdec161 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -252,11 +252,11 @@ Where filters in API allow for a filter list or string. We recommend using the f Where Filters have a few objects that you can use: -- `Dimension()` — Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')` +- `Dimension()` — Used for any categorical or time dimensions. `Dimension('metric_time').grain('week')` or `Dimension('customer__country')`. -- `TimeDimension()` — Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')` +- `TimeDimension()` — Used as a more explicit definition for time dimensions, optionally takes in a granularity `TimeDimension('metric_time', 'month')`. -- `Entity()` — Used for entities like primary and foreign keys - `Entity('order_id')` +- `Entity()` — Used for entities like primary and foreign keys - `Entity('order_id')`. For `TimeDimension()`, the grain is only required in the `WHERE` filter if the aggregation time dimensions for the measures and metrics associated with the where filter have different grains. From 162bf7af95244af5a49fd144aa01faa4d031d18d Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 17 Jan 2024 11:48:52 +0000 Subject: [PATCH 21/48] add fill nulls width --- website/docs/docs/build/conversion-metrics.md | 2 + website/docs/docs/build/cumulative-metrics.md | 34 ++++++++++++ website/docs/docs/build/derived-metrics.md | 34 ++++++++++++ website/docs/docs/build/ratio-metrics.md | 53 +++++++++++++++---- website/docs/docs/build/simple.md | 37 ++++++++++++- 5 files changed, 148 insertions(+), 12 deletions(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index 2238655fbe0..3f24b7749b9 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -33,6 +33,8 @@ The specification for conversion metrics is as follows: | `base_property` | The property from the base semantic model that you want to hold constant. | Entity or Dimension | Optional | | `conversion_property` | The property from the conversion semantic model that you want to hold constant. | Entity or Dimension | Optional | +Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. + The following code example displays the complete specification for conversion metrics and details how they're applied: ```yaml diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index 94ed762d1f0..09d04c87f2c 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -21,6 +21,8 @@ This metric is common for calculating things like weekly active users, or month- | `window` | The accumulation window, such as 1 month, 7 days, 1 year. This can't be used with `grain_to_date`. | Optional | | `grain_to_date` | Sets the accumulation grain, such as month will accumulate data for one month. Then restart at the beginning of the next. This can't be used with `window`. | Optional | +Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. + The following displays the complete specification for cumulative metrics, along with an example: ```yaml @@ -244,3 +246,35 @@ group by metric_time limit 100 ``` + +### Additional settings + +Use the following additional settings to customize your conversion metrics: + +- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + + +To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: + +```yaml +- name: vist_to_buy_conversion_rate_7_day_window + description: "Conversion rate from viewing a page to making a purchase" + type: conversion + label: Visit to Seller Conversion Rate (7 day window) + type_params: + conversion_type_params: + calculation: conversions + base_measure: visits + conversion_measure: + name: buys + fill_nulls_with: 0 + entity: user + window: 7 days + +``` + +This will return the following results: + + + diff --git a/website/docs/docs/build/derived-metrics.md b/website/docs/docs/build/derived-metrics.md index fc7961bbe7f..172cdbfbe08 100644 --- a/website/docs/docs/build/derived-metrics.md +++ b/website/docs/docs/build/derived-metrics.md @@ -23,6 +23,8 @@ In MetricFlow, derived metrics are metrics created by defining an expression usi | `filter` | Optional filter to apply to the metric. | Optional | | `offset_window` | Set the period for the offset window, such as 1 month. This will return the value of the metric one month from the metric time. | Required | +Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. + The following displays the complete specification for derived metrics, along with an example. ```yaml @@ -157,3 +159,35 @@ bookings - bookings_7_days_ago would be compile as 7438 - 7252 = 186. | d7_booking_change | metric_time__month | | ----------------- | ------------------ | | 186 | 2017-07-01 | + + +### Additional settings + +Use the following additional settings to customize your conversion metrics: + +- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + + +To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: + +```yaml +- name: vist_to_buy_conversion_rate_7_day_window + description: "Conversion rate from viewing a page to making a purchase" + type: conversion + label: Visit to Seller Conversion Rate (7 day window) + type_params: + conversion_type_params: + calculation: conversions + base_measure: visits + conversion_measure: + name: buys + fill_nulls_with: 0 + entity: user + window: 7 days + +``` + +This will return the following results: + + diff --git a/website/docs/docs/build/ratio-metrics.md b/website/docs/docs/build/ratio-metrics.md index 97efe0f55bf..fe6a9fb160a 100644 --- a/website/docs/docs/build/ratio-metrics.md +++ b/website/docs/docs/build/ratio-metrics.md @@ -22,6 +22,8 @@ Ratio allows you to create a ratio between two metrics. You simply specify a num | `filter` | Optional filter for the numerator or denominator. | Optional | | `alias` | Optional alias for the numerator or denominator. | Optional | +Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. + The following displays the complete specification for ratio metrics, along with an example. ```yaml @@ -61,16 +63,16 @@ The system will simplify and turn the numerator and denominator in a ratio metri ```sql select - subq_15577.metric_time as metric_time - , cast(subq_15577.mql_queries_created_test as double) / cast(nullif(subq_15582.distinct_query_users, 0) as double) as mql_queries_per_active_user + subq_15577.metric_time as metric_time, + cast(subq_15577.mql_queries_created_test as double) / cast(nullif(subq_15582.distinct_query_users, 0) as double) as mql_queries_per_active_user from ( select - metric_time - , sum(mql_queries_created_test) as mql_queries_created_test + metric_time, + sum(mql_queries_created_test) as mql_queries_created_test from ( select - cast(query_created_at as date) as metric_time - , case when query_status in ('PENDING','MODE') then 1 else 0 end as mql_queries_created_test + cast(query_created_at as date) as metric_time, + case when query_status in ('PENDING','MODE') then 1 else 0 end as mql_queries_created_test from prod_dbt.mql_query_base mql_queries_test_src_2552 ) subq_15576 group by @@ -78,12 +80,12 @@ from ( ) subq_15577 inner join ( select - metric_time - , count(distinct distinct_query_users) as distinct_query_users + metric_time, + count(distinct distinct_query_users) as distinct_query_users from ( select - cast(query_created_at as date) as metric_time - , case when query_status in ('MODE','PENDING') then email else null end as distinct_query_users + cast(query_created_at as date) as metric_time, + case when query_status in ('MODE','PENDING') then email else null end as distinct_query_users from prod_dbt.mql_query_base mql_queries_src_2585 ) subq_15581 group by @@ -125,3 +127,34 @@ metrics: ``` Note the `filter` and `alias` parameters for the metric referenced in the numerator. Use the `filter` parameter to apply a filter to the metric it's attached to. The `alias` parameter is used to avoid naming conflicts in the rendered SQL queries when the same metric is used with different filters. If there are no naming conflicts, the `alias` parameter can be left out. + +### Additional settings + +Use the following additional settings to customize your conversion metrics: + +- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + + +To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: + +```yaml +- name: vist_to_buy_conversion_rate_7_day_window + description: "Conversion rate from viewing a page to making a purchase" + type: conversion + label: Visit to Seller Conversion Rate (7 day window) + type_params: + conversion_type_params: + calculation: conversions + base_measure: visits + conversion_measure: + name: buys + fill_nulls_with: 0 + entity: user + window: 7 days + +``` + +This will return the following results: + + diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md index 1803e952a69..7d857c16a47 100644 --- a/website/docs/docs/build/simple.md +++ b/website/docs/docs/build/simple.md @@ -20,6 +20,8 @@ Simple metrics are metrics that directly reference a single measure, without any | `type_params` | The type parameters of the metric. | Required | | `measure` | The measure you're referencing. | Required | +Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. + The following displays the complete specification for simple metrics, along with an example. @@ -28,7 +30,7 @@ metrics: - name: The metric name # Required description: the metric description # Optional type: simple # Required - label: The value that will be displayed in downstream tools #Required + label: The value that will be displayed in downstream tools # Required type_params: # Required measure: The measure you're referencing # Required @@ -50,7 +52,7 @@ If you've already defined the measure using the `create_metric: true` parameter, type: simple # Pointers to a measure you created in a semantic model label: Count of customers type_params: - measure: customers # The measure youre creating a proxy of. + measure: customers # The measure you're creating a proxy of. - name: large_orders description: "Order with order values over 20." type: SIMPLE @@ -60,3 +62,34 @@ If you've already defined the measure using the `create_metric: true` parameter, filter: | # For any metric you can optionally include a filter on dimension values {{Dimension('customer__order_total_dim')}} >= 20 ``` + +### Additional settings + +Use the following additional settings to customize your conversion metrics: + +- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + + +To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: + +```yaml +- name: vist_to_buy_conversion_rate_7_day_window + description: "Conversion rate from viewing a page to making a purchase" + type: conversion + label: Visit to Seller Conversion Rate (7 day window) + type_params: + conversion_type_params: + calculation: conversions + base_measure: visits + conversion_measure: + name: buys + fill_nulls_with: 0 + entity: user + window: 7 days + +``` + +This will return the following results: + + From 92cce08882cb9d81022c106d898af02c0510c41b Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Wed, 17 Jan 2024 11:57:03 +0000 Subject: [PATCH 22/48] tweak --- website/docs/docs/build/ratio-metrics.md | 4 ++-- website/docs/docs/build/simple.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/docs/build/ratio-metrics.md b/website/docs/docs/build/ratio-metrics.md index fe6a9fb160a..7dad50f3581 100644 --- a/website/docs/docs/build/ratio-metrics.md +++ b/website/docs/docs/build/ratio-metrics.md @@ -54,8 +54,8 @@ metrics: type_params: numerator: food_orders denominator: orders - ``` + ## Ratio metrics using different semantic models The system will simplify and turn the numerator and denominator in a ratio metric from different semantic models by computing their values in sub-queries. It will then join the result set based on common dimensions to calculate the final ratio. Here's an example of the SQL generated for such a ratio metric. @@ -133,6 +133,7 @@ Note the `filter` and `alias` parameters for the metric referenced in the numera Use the following additional settings to customize your conversion metrics: - **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + @@ -152,7 +153,6 @@ To return zero in the final data set, you can set the value of a null conversion fill_nulls_with: 0 entity: user window: 7 days - ``` This will return the following results: diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md index 7d857c16a47..6249769989b 100644 --- a/website/docs/docs/build/simple.md +++ b/website/docs/docs/build/simple.md @@ -68,6 +68,7 @@ If you've already defined the measure using the `create_metric: true` parameter, Use the following additional settings to customize your conversion metrics: - **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. + @@ -87,7 +88,6 @@ To return zero in the final data set, you can set the value of a null conversion fill_nulls_with: 0 entity: user window: 7 days - ``` This will return the following results: From aacb466410f9b4cc2dcb6991302233b676eb08b8 Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Thu, 18 Jan 2024 12:23:30 +0000 Subject: [PATCH 23/48] add fills_null_width --- website/docs/docs/build/conversion-metrics.md | 5 +- website/docs/docs/build/cumulative-metrics.md | 59 +++++-------------- website/docs/docs/build/derived-metrics.md | 50 ++++------------ website/docs/docs/build/metrics-overview.md | 23 ++++---- website/docs/docs/build/ratio-metrics.md | 42 +++---------- website/docs/docs/build/simple.md | 36 ++--------- 6 files changed, 54 insertions(+), 161 deletions(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index 3f24b7749b9..1b4ccc6884e 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -32,6 +32,7 @@ The specification for conversion metrics is as follows: | `constant_properties` | List of constant properties. | List | Optional | | `base_property` | The property from the base semantic model that you want to hold constant. | Entity or Dimension | Optional | | `conversion_property` | The property from the conversion semantic model that you want to hold constant. | Entity or Dimension | Optional | +| `fill_nulls_with` | Set the value of a null conversion event to zero instead of null. | String | Optional | Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. @@ -40,10 +41,11 @@ The following code example displays the complete specification for conversion me ```yaml metrics: - name: The metric name # Required - description: the metric description # Optional + description: The metric description # Optional type: conversion # Required label: # Required type_params: # Required + fills_nulls_with: Set value to zero instead of null # Optional conversion_type_params: # Required entity: ENTITY # Required calculation: CALCULATION_TYPE # Optional. default: conversion_rate. options: conversions(buys) or conversion_rate (buys/visits), and more to come. @@ -91,6 +93,7 @@ Next, define a conversion metric as follows: type: conversion label: Visit to Buy Conversion Rate (7-day window) type_params: + fills_nulls_with: 0 conversion_type_params: base_measure: visits conversion_measure: sellers diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index 09d04c87f2c..85e6bf4f085 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -20,8 +20,7 @@ This metric is common for calculating things like weekly active users, or month- | `measure` | The measure you are referencing. | Required | | `window` | The accumulation window, such as 1 month, 7 days, 1 year. This can't be used with `grain_to_date`. | Optional | | `grain_to_date` | Sets the accumulation grain, such as month will accumulate data for one month. Then restart at the beginning of the next. This can't be used with `window`. | Optional | - -Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. +| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | The following displays the complete specification for cumulative metrics, along with an example: @@ -32,6 +31,7 @@ metrics: type: cumulative # Required label: The value that will be displayed in downstream tools # Required type_params: # Required + fill_nulls_with: Set value to zero instead of null # Optional measure: The measure you are referencing # Required window: The accumulation window, such as 1 month, 7 days, 1 year. # Optional. Cannot be used with grain_to_date grain_to_date: Sets the accumulation grain, such as month will accumulate data for one month, then restart at the beginning of the next. # Optional. Cannot be used with window @@ -39,6 +39,7 @@ metrics: ``` ## Limitations + Cumulative metrics are currently under active development and have the following limitations: - You are required to use [`metric_time` dimension](/docs/build/dimensions#time) when querying cumulative metrics. If you don't use `metric_time` in the query, the cumulative metric will return incorrect results because it won't perform the time spine join. This means you cannot reference time dimensions other than the `metric_time` in the query. @@ -61,12 +62,14 @@ metrics: description: The cumulative value of all orders type: cumulative type_params: + fill_nulls_with: 0 measure: order_total - name: cumulative_order_total_l1m label: Cumulative Order total (L1M) description: Trailing 1 month cumulative order amount type: cumulative type_params: + fills_nulls_with: 0 measure: order_total window: 1 month - name: cumulative_order_total_mtd @@ -74,6 +77,7 @@ metrics: description: The month to date value of all orders type: cumulative type_params: + fills_nulls_with: 0 measure: order_total grain_to_date: month ``` @@ -203,16 +207,16 @@ The current method connects the metric table to a timespine table using the prim ``` sql select - count(distinct distinct_users) as weekly_active_users - , metric_time + count(distinct distinct_users) as weekly_active_users, + metric_time from ( select - subq_3.distinct_users as distinct_users - , subq_3.metric_time as metric_time + subq_3.distinct_users as distinct_users, + subq_3.metric_time as metric_time from ( select - subq_2.distinct_users as distinct_users - , subq_1.metric_time as metric_time + subq_2.distinct_users as distinct_users, + subq_1.metric_time as metric_time from ( select metric_time @@ -225,8 +229,8 @@ from ( ) subq_1 inner join ( select - distinct_users as distinct_users - , date_trunc('day', ds) as metric_time + distinct_users as distinct_users, + date_trunc('day', ds) as metric_time from demo_schema.transactions transactions_src_426 where ( (date_trunc('day', ds)) >= cast('1999-12-26' as timestamp) @@ -243,38 +247,7 @@ from ( ) subq_3 ) group by - metric_time -limit 100 -``` - -### Additional settings - -Use the following additional settings to customize your conversion metrics: - -- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. - - -To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: - -```yaml -- name: vist_to_buy_conversion_rate_7_day_window - description: "Conversion rate from viewing a page to making a purchase" - type: conversion - label: Visit to Seller Conversion Rate (7 day window) - type_params: - conversion_type_params: - calculation: conversions - base_measure: visits - conversion_measure: - name: buys - fill_nulls_with: 0 - entity: user - window: 7 days + metric_time, +limit 100; ``` - -This will return the following results: - - - diff --git a/website/docs/docs/build/derived-metrics.md b/website/docs/docs/build/derived-metrics.md index 172cdbfbe08..802b26f598b 100644 --- a/website/docs/docs/build/derived-metrics.md +++ b/website/docs/docs/build/derived-metrics.md @@ -22,8 +22,7 @@ In MetricFlow, derived metrics are metrics created by defining an expression usi | `alias` | Optional alias for the metric that you can use in the expr. | Optional | | `filter` | Optional filter to apply to the metric. | Optional | | `offset_window` | Set the period for the offset window, such as 1 month. This will return the value of the metric one month from the metric time. | Required | - -Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. +| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | The following displays the complete specification for derived metrics, along with an example. @@ -34,6 +33,7 @@ metrics: type: derived # Required label: The value that will be displayed in downstream tools #Required type_params: # Required + fill_nulls_with: Set value to zero instead of null # Optional expr: the derived expression # Required metrics: # The list of metrics used in the derived metrics # Required - name: the name of the metrics. must reference a metric you have already defined # Required @@ -51,6 +51,7 @@ metrics: type: derived label: Order Gross Profit type_params: + fill_nulls_with: 0 expr: revenue - cost metrics: - name: order_total @@ -62,6 +63,7 @@ metrics: description: "The gross profit for each food order." type: derived type_params: + fill_nulls_with: 0 expr: revenue - cost metrics: - name: order_total @@ -98,6 +100,7 @@ The following example displays how you can calculate monthly revenue growth usin description: Percentage of customers that are active now and those active 1 month ago label: customer_retention type_params: + fill_nulls_with: 0 expr: (active_customers/ active_customers_prev_month) metrics: - name: active_customers @@ -117,6 +120,7 @@ You can query any granularity and offset window combination. The following examp type: derived label: d7 Bookings Change type_params: + fill_nulls_with: 0 expr: bookings - bookings_7_days_ago metrics: - name: bookings @@ -128,10 +132,10 @@ You can query any granularity and offset window combination. The following examp When you run the query `dbt sl query --metrics d7_booking_change --group-by metric_time__month` for the metric, here's how it's calculated. For dbt Core, you can use the `mf query` prefix. -1. We retrieve the raw, unaggregated dataset with the specified measures and dimensions at the smallest level of detail, which is currently 'day'. -2. Then, we perform an offset join on the daily dataset, followed by performing a date trunc and aggregation to the requested granularity. +1. Retrieve the raw, unaggregated dataset with the specified measures and dimensions at the smallest level of detail, which is currently 'day'. +2. Then, perform an offset join on the daily dataset, followed by performing a date trunc and aggregation to the requested granularity. For example, to calculate `d7_booking_change` for July 2017: - - First, we sum up all the booking values for each day in July to calculate the bookings metric. + - First, sum up all the booking values for each day in July to calculate the bookings metric. - The following table displays the range of days that make up this monthly aggregation. | | Orders | Metric_time | @@ -141,7 +145,7 @@ When you run the query `dbt sl query --metrics d7_booking_change --group-by met | | 78 | 2017-07-01 | | Total | 7438 | 2017-07-01 | -3. Next, we calculate July's bookings with a 7-day offset. The following table displays the range of days that make up this monthly aggregation. Note that the month begins 7 days later (offset by 7 days) on 2017-07-24. +3. Calculate July's bookings with a 7-day offset. The following table displays the range of days that make up this monthly aggregation. Note that the month begins 7 days later (offset by 7 days) on 2017-07-24. | | Orders | Metric_time | | - | ---- | -------- | @@ -150,7 +154,7 @@ When you run the query `dbt sl query --metrics d7_booking_change --group-by met | | 83 | 2017-06-24 | | Total | 7252 | 2017-07-01 | -4. Lastly, we calculate the derived metric and return the final result set: +4. Lastly, calculate the derived metric and return the final result set: ```bash bookings - bookings_7_days_ago would be compile as 7438 - 7252 = 186. @@ -159,35 +163,3 @@ bookings - bookings_7_days_ago would be compile as 7438 - 7252 = 186. | d7_booking_change | metric_time__month | | ----------------- | ------------------ | | 186 | 2017-07-01 | - - -### Additional settings - -Use the following additional settings to customize your conversion metrics: - -- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. - - -To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: - -```yaml -- name: vist_to_buy_conversion_rate_7_day_window - description: "Conversion rate from viewing a page to making a purchase" - type: conversion - label: Visit to Seller Conversion Rate (7 day window) - type_params: - conversion_type_params: - calculation: conversions - base_measure: visits - conversion_measure: - name: buys - fill_nulls_with: 0 - entity: user - window: 7 days - -``` - -This will return the following results: - - diff --git a/website/docs/docs/build/metrics-overview.md b/website/docs/docs/build/metrics-overview.md index ea602d0953f..60b103419e3 100644 --- a/website/docs/docs/build/metrics-overview.md +++ b/website/docs/docs/build/metrics-overview.md @@ -9,7 +9,7 @@ pagination_next: "docs/build/cumulative" Once you've created your semantic models, it's time to start adding metrics! Metrics can be defined in the same YAML files as your semantic models, or split into separate YAML files into any other subdirectories (provided that these subdirectories are also within the same dbt project repo) -The keys for metrics definitions are: +The keys for metrics definitions are: | Parameter | Description | Type | | --------- | ----------- | ---- | @@ -22,7 +22,6 @@ The keys for metrics definitions are: | `filter` | You can optionally add a filter string to any metric type, applying filters to dimensions, entities, or time dimensions during metric computation. Consider it as your WHERE clause. | Optional | | `meta` | Additional metadata you want to add to your metric. | Optional | - Here's a complete example of the metrics spec configuration: ```yaml @@ -39,14 +38,7 @@ metrics: null ``` -This page explains the different supported metric types you can add to your dbt project. - +This page explains the different supported metric types you can add to your dbt project. ### Conversion metrics @@ -55,10 +47,11 @@ This page explains the different supported metric types you can add to your dbt ```yaml metrics: - name: The metric name # Required - description: the metric description # Optional + description: The metric description # Optional type: conversion # Required label: # Required type_params: # Required + fills_nulls_with: Set value to zero instead of null # Optional conversion_type_params: # Required entity: ENTITY # Required calculation: CALCULATION_TYPE # Optional. default: conversion_rate. options: conversions(buys) or conversion_rate (buys/visits), and more to come. @@ -82,9 +75,10 @@ metrics: - support@getdbt.com type: cumulative type_params: + fills_nulls_with: 0 measures: - distinct_users - #Omitting window will accumulate the measure over all time + # Omitting window will accumulate the measure over all time window: 7 days ``` @@ -100,6 +94,7 @@ metrics: type: derived label: Order Gross Profit type_params: + fills_nulls_with: 0 expr: revenue - cost metrics: - name: order_total @@ -139,6 +134,7 @@ metrics: # Define the metrics from the semantic manifest as numerator or denominator type: ratio type_params: + fills_nulls_with: 0 numerator: cancellations denominator: transaction_amount filter: | # add optional constraint string. This applies to both the numerator and denominator @@ -157,6 +153,7 @@ metrics: filter: | # add optional constraint string. This applies to both the numerator and denominator {{ Dimension('customer__country') }} = 'MX' ``` + ### Simple metrics [Simple metrics](/docs/build/simple) point directly to a measure. You may think of it as a function that takes only one measure as the input. @@ -171,6 +168,7 @@ metrics: - name: cancellations type: simple type_params: + fills_nulls_with: 0 measure: cancellations_usd # Specify the measure you are creating a proxy for. filter: | {{ Dimension('order__value')}} > 100 and {{Dimension('user__acquisition')}} @@ -187,6 +185,7 @@ filter: | filter: | {{ TimeDimension('time_dimension', 'granularity') }} ``` + ### Further configuration You can set more metadata for your metrics, which can be used by other tools later on. The way this metadata is used will vary based on the specific integration partner diff --git a/website/docs/docs/build/ratio-metrics.md b/website/docs/docs/build/ratio-metrics.md index 7dad50f3581..f0db2963f4a 100644 --- a/website/docs/docs/build/ratio-metrics.md +++ b/website/docs/docs/build/ratio-metrics.md @@ -21,8 +21,7 @@ Ratio allows you to create a ratio between two metrics. You simply specify a num | `denominator` | The name of the metric used for the denominator, or structure of properties. | Required | | `filter` | Optional filter for the numerator or denominator. | Optional | | `alias` | Optional alias for the numerator or denominator. | Optional | - -Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. +| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | The following displays the complete specification for ratio metrics, along with an example. @@ -33,6 +32,7 @@ metrics: type: ratio # Required label: The value that will be displayed in downstream tools #Required type_params: # Required + fill_nulls_with: Value to zero instead of null # Optional numerator: The name of the metric used for the numerator, or structure of properties # Required name: Name of metric used for the numerator # Required filter: Filter for the numerator # Optional @@ -52,6 +52,7 @@ metrics: label: Food Order Ratio type: ratio type_params: + fill_nulls_with: 0 numerator: food_orders denominator: orders ``` @@ -117,6 +118,7 @@ metrics: - support@getdbt.com type: ratio type_params: + fill_nulls_with: 0 numerator: name: distinct_purchasers filter: | @@ -126,35 +128,7 @@ metrics: name: distinct_purchasers ``` -Note the `filter` and `alias` parameters for the metric referenced in the numerator. Use the `filter` parameter to apply a filter to the metric it's attached to. The `alias` parameter is used to avoid naming conflicts in the rendered SQL queries when the same metric is used with different filters. If there are no naming conflicts, the `alias` parameter can be left out. - -### Additional settings - -Use the following additional settings to customize your conversion metrics: - -- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. - - - -To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: - -```yaml -- name: vist_to_buy_conversion_rate_7_day_window - description: "Conversion rate from viewing a page to making a purchase" - type: conversion - label: Visit to Seller Conversion Rate (7 day window) - type_params: - conversion_type_params: - calculation: conversions - base_measure: visits - conversion_measure: - name: buys - fill_nulls_with: 0 - entity: user - window: 7 days -``` - -This will return the following results: - - +Note the `filter` and `alias` parameters for the metric referenced in the numerator. +- Use the `filter` parameter to apply a filter to the metric it's attached to. +- The `alias` parameter is used to avoid naming conflicts in the rendered SQL queries when the same metric is used with different filters. +- If there are no naming conflicts, the `alias` parameter can be left out. diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md index 6249769989b..b384c53489f 100644 --- a/website/docs/docs/build/simple.md +++ b/website/docs/docs/build/simple.md @@ -19,8 +19,7 @@ Simple metrics are metrics that directly reference a single measure, without any | `label` | The value that will be displayed in downstream tools. | Required | | `type_params` | The type parameters of the metric. | Required | | `measure` | The measure you're referencing. | Required | - -Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. +| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | The following displays the complete specification for simple metrics, along with an example. @@ -33,6 +32,7 @@ metrics: label: The value that will be displayed in downstream tools # Required type_params: # Required measure: The measure you're referencing # Required + fill_nulls_with: Value to zero instead of null # Optional ``` @@ -52,44 +52,16 @@ If you've already defined the measure using the `create_metric: true` parameter, type: simple # Pointers to a measure you created in a semantic model label: Count of customers type_params: + fills_nulls_with: 0 measure: customers # The measure you're creating a proxy of. - name: large_orders description: "Order with order values over 20." type: SIMPLE label: Large Orders type_params: + fill_nulls_with: 0 measure: orders filter: | # For any metric you can optionally include a filter on dimension values {{Dimension('customer__order_total_dim')}} >= 20 ``` -### Additional settings - -Use the following additional settings to customize your conversion metrics: - -- **Null conversion values:** Set null conversions to zero using `fill_nulls_with`. - - - -To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: - -```yaml -- name: vist_to_buy_conversion_rate_7_day_window - description: "Conversion rate from viewing a page to making a purchase" - type: conversion - label: Visit to Seller Conversion Rate (7 day window) - type_params: - conversion_type_params: - calculation: conversions - base_measure: visits - conversion_measure: - name: buys - fill_nulls_with: 0 - entity: user - window: 7 days -``` - -This will return the following results: - - From 76b60c38ef7d051a39b6cd8f34688089ba361772 Mon Sep 17 00:00:00 2001 From: Jordan Stein Date: Fri, 19 Jan 2024 13:00:59 -0800 Subject: [PATCH 24/48] fix typo and re-format sql --- website/docs/docs/build/conversion-metrics.md | 160 ++++++++---------- 1 file changed, 74 insertions(+), 86 deletions(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index 2238655fbe0..8d5f7eb24bf 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -105,19 +105,18 @@ This step joins the `BUYS` table to the `VISITS` table and gets all combinations The SQL generated in these steps looks like the following: ```sql -select - v.ds, - v.user_id, - v.referrer_id, - b.ds, - b.uuid, - 1 as buys -from visits v -inner join ( - select *, uuid_string() as uuid from buys -- Adds a uuid column to uniquely identify the different rows -) b -on -v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' +SELECT v.ds, + v.user_id, + v.referrer_id, + b.ds, + b.uuid, + 1 as buys +FROM visits v + INNER JOIN (SLECT *, uuid_string() as uuid + FROM buys -- Adds a uuid column to uniquely identify the different rows + ) b + ON + v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' ``` The dataset returns the following (note that there are two potential conversion events for the first visit): @@ -134,19 +133,17 @@ The dataset returns the following (note that there are two potential conversion Instead of returning the raw visit values, use window functions to link conversions to the closest base event. You can partition by the conversion source and get the `first_value` ordered by `visit ds`, descending to get the closest base event from the conversion event: ```sql -select - first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, - first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, - first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, - b.ds, - b.uuid, - 1 as buys -from visits v -inner join ( - select *, uuid_string() as uuid from buys -) b -on -v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' +SELECT first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, + first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, + first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, + b.ds, + b.uuid, + 1 as buys +FROM visits v + INNER JOIN (select *, uuid_string() as uuid + from buys) b + ON + v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' ``` @@ -168,19 +165,18 @@ To resolve this and eliminate duplicates, use a distinct select. The UUID also h Instead of regular select used in the [Step 2](#step-2-refine-with-window-function), use a distinct select to remove the duplicates: ```sql -select distinct - first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, - first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, - first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, - b.ds, - b.uuid, - 1 as buys -from visits v -inner join ( - select *, uuid_string() as uuid from buys -) b -on -v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day'; +SELECT DISTINCT first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, + first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, + first_value(v.referrer_id) + over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, + b.ds, + b.uuid, + 1 as buys +FROM visits v + INNER JOIN (select *, uuid_string() as uuid + from buys) b + ON + v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day'; ``` The dataset returns the following: @@ -201,38 +197,28 @@ You now have a dataset where every conversion is connected to a visit event. To Now that you’ve tied each conversion event to a visit, you can calculate the aggregated conversions and opportunities measures. Then, you can join them to calculate the actual conversion rate. The SQL to calculate the conversion rate is as follows: ```sql -select - coalesce(subq_3.metric_time__day, subq_13.metric_time__day) as metric_time__day, - cast(max(subq_13.buys) as double) / cast(nullif(max(subq_3.visits), 0) as double) as visit_to_buy_conversion_rate_7d -from ( -- base measure - select - metric_time__day, - sum(visits) as mqls - from ( - select - date_trunc('day', first_contact_date) as metric_time__day, - 1 as visits - from visits - ) subq_2 - group by - metric_time__day -) subq_3 -full outer join ( -- conversion measure - select - metric_time__day, - sum(buys) as sellers - from ( - -- ... - -- The output of this subquery is the table produced in Step 3. The SQL is hidden for legibility. - -- To see the full SQL output, add --explain to your conversion metric query. - ) subq_10 - group by - metric_time__day -) subq_13 -on - subq_3.metric_time__day = subq_13.metric_time__day -group by - metric_time__day +SELECT coalesce(subq_3.metric_time__day, subq_13.metric_time__day) as metric_time__day, + cast(max(subq_13.buys) as double) / + cast(nullif(max(subq_3.visits), 0) as double) as visit_to_buy_conversion_rate_7d +FROM ( -- base measure + SELECT metric_time__day, + sum(visits) as mqls + FROM (SELECT date_trunc('day', first_contact_date) as metric_time__day, + 1 as visits + FROM visits) subq_2 + GROUP BY metric_time__day) subq_3 + FULL OUTER JOIN ( -- conversion measure + SELECT metric_time__day, + sum(buys) as sellers + FROM ( + -- ... + -- The output of this subquery is the table produced in Step 3. The SQL is hidden for legibility. + -- To see the full SQL output, add --explain to your conversion metric query. + ) subq_10 + GROUP BY metric_time__day) subq_13 + ON + subq_3.metric_time__day = subq_13.metric_time__day +GROUP BY metric_time__day ``` ### Additional settings @@ -249,7 +235,7 @@ Use the following additional settings to customize your conversion metrics: To return zero in the final data set, you can set the value of a null conversion event to zero instead of null. You can add the `fill_nulls_with` parameter to your conversion metric definition like this: ```yaml -- name: vist_to_buy_conversion_rate_7_day_window +- name: visit_to_buy_conversion_rate_7_day_window description: "Conversion rate from viewing a page to making a purchase" type: conversion label: Visit to Seller Conversion Rate (7 day window) @@ -329,22 +315,24 @@ In this case, you want to set `product_id` as the constant property. You can spe You will add an additional condition to the join to make sure the constant property is the same across conversions. ```sql -select distinct - first_value(v.ds) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as ds, - first_value(v.user_id) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as user_id, - first_value(v.referrer_id) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as referrer_id, - buy_source.uuid, - 1 as buys -from {{ source_schema }}.fct_view_item_details v -inner join +SELECT DISTINCT first_value(v.ds) + over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as ds, + first_value(v.user_id) + over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as user_id, + first_value(v.referrer_id) + over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as referrer_id, + buy_source.uuid, + 1 as buys +FROM {{ source_schema }}.fct_view_item_details v +INNER JOIN ( - select *, {{ generate_random_uuid() }} as uuid from {{ source_schema }}.fct_purchases + SELECT *, {{ generate_random_uuid() }} as uuid FROM {{ source_schema }}.fct_purchases ) buy_source -on - v.user_id = buy_source.user_id - and v.ds <= buy_source.ds - and v.ds > buy_source.ds - interval '7 day' - and buy_source.product_id = v.product_id --Joining on the constant property product_id +ON + v.user_id = buy_source.user_id + AND v.ds <= buy_source.ds + AND v.ds > buy_source.ds - INTERVAL '7 day' + AND buy_source.product_id = v.product_id --Joining on the constant property product_id ``` From 4682138bba356a2297d5d120c38f6253caa1f371 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 19 Jan 2024 15:53:20 -0700 Subject: [PATCH 25/48] Document `DBT_PRINT` / `DBT_NO_PRINT` environment variables --- website/docs/reference/global-configs/print-output.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/website/docs/reference/global-configs/print-output.md b/website/docs/reference/global-configs/print-output.md index 112b92b546f..016ef46b6db 100644 --- a/website/docs/reference/global-configs/print-output.md +++ b/website/docs/reference/global-configs/print-output.md @@ -8,7 +8,7 @@ sidebar: "Print output" -By default, dbt includes `print()` messages in standard out (stdout). You can use the `NO_PRINT` config to prevent these messages from showing up in stdout. +By default, dbt includes `print()` messages in standard out (stdout). You can use the `DBT_NO_PRINT` environment variable to prevent these messages from showing up in stdout. @@ -23,7 +23,7 @@ config: -By default, dbt includes `print()` messages in standard out (stdout). You can use the `PRINT` config to prevent these messages from showing up in stdout. +By default, dbt includes `print()` messages in standard out (stdout). You can use the `DBT_PRINT` environment variable to prevent these messages from showing up in stdout. @@ -36,7 +36,7 @@ config: :::warning Syntax deprecation -The original `NO_PRINT` syntax has been deprecated, starting with dbt v1.5. Backward compatibility is supported but will be removed in an as-of-yet-undetermined future release. +The original `DBT_NO_PRINT` environment variable has been deprecated, starting with dbt v1.5. Backward compatibility is supported but will be removed in an as-of-yet-undetermined future release. ::: From 1fbb8df5d083f35b8b360c3362304d428f63a4e0 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 19 Jan 2024 16:05:55 -0700 Subject: [PATCH 26/48] Remove unnecessary ellipses --- website/docs/reference/global-configs/print-output.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/website/docs/reference/global-configs/print-output.md b/website/docs/reference/global-configs/print-output.md index 016ef46b6db..096ecf95d2d 100644 --- a/website/docs/reference/global-configs/print-output.md +++ b/website/docs/reference/global-configs/print-output.md @@ -46,8 +46,6 @@ Supply `--no-print` flag to `dbt run` to suppress `print()` messages from showin ```text dbt --no-print run -... - ``` ### Printer width From 930a26e2301011a92b410790670a640d210e4c8d Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 19 Jan 2024 16:07:45 -0700 Subject: [PATCH 27/48] Remove configuration examples that do not work --- .../reference/global-configs/print-output.md | 18 ------------------ 1 file changed, 18 deletions(-) diff --git a/website/docs/reference/global-configs/print-output.md b/website/docs/reference/global-configs/print-output.md index 096ecf95d2d..78de635f2dd 100644 --- a/website/docs/reference/global-configs/print-output.md +++ b/website/docs/reference/global-configs/print-output.md @@ -10,30 +10,12 @@ sidebar: "Print output" By default, dbt includes `print()` messages in standard out (stdout). You can use the `DBT_NO_PRINT` environment variable to prevent these messages from showing up in stdout. - - -```yaml -config: - no_print: true -``` - - - By default, dbt includes `print()` messages in standard out (stdout). You can use the `DBT_PRINT` environment variable to prevent these messages from showing up in stdout. - - -```yaml -config: - print: false -``` - - - :::warning Syntax deprecation The original `DBT_NO_PRINT` environment variable has been deprecated, starting with dbt v1.5. Backward compatibility is supported but will be removed in an as-of-yet-undetermined future release. From 6a32ba682ad541a89b71060a130ef1ea1d5f3979 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 19 Jan 2024 17:17:23 -0800 Subject: [PATCH 28/48] Update sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index fc095985160..9bef3c039f3 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -513,7 +513,7 @@ Assuming the user is querying `metric_0` and `metric_1` together, a valid filter Invalid filters would be: - * ` "{{ TimeDimension('metric_time') }} > '2020-01-01'"` — metrics in the query b are defined based on measures with different grains. + * ` "{{ TimeDimension('metric_time') }} > '2020-01-01'"` — metrics in the query are defined based on measures with different grains. * `"{{ TimeDimension('metric_time', 'month') }} > '2020-01-01'"` — `metric_1` is not available at a month grain. From 1d053634a3325ef1d6b1102e431de7f3bc01b240 Mon Sep 17 00:00:00 2001 From: rpourzand Date: Fri, 19 Jan 2024 17:18:33 -0800 Subject: [PATCH 29/48] Update sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 9bef3c039f3..230f84b0b98 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -468,7 +468,7 @@ mutation { } ``` -For both `TimeDimension()` and `Dimension()` objects, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. +For both `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. For example, consider this Semantic model and Metric configuration, which contains two metrics that are aggregated across different time grains. This example shows a single semantic model, but the same goes for metrics across more than one semantic model. From 357e5aa2e3489305f2f759638d9c18716ac5d50b Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:26:05 +0000 Subject: [PATCH 30/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 7296d1541ab..7b2b1438ae9 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -378,7 +378,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], }} ``` -If you are ordering by an object that's been operated on (e.g., you changed the the granularity of the time dimension), or you are using the full object notation, descending order must look like: +If you are ordering by an object that's been operated on (for example, you changed the the granularity of the time dimension), or you are using the full object notation, descending order must look like: ```bash select * from {{ From 192c2f01be98826adebacd7dd2627763796fc75a Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:27:47 +0000 Subject: [PATCH 31/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 7b2b1438ae9..6ff846fa7c7 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -315,7 +315,7 @@ Assuming the user is querying `metric_0` and `metric_1` together in a single req * `"{{ TimeDimension('metric_time', 'year') }} > '2020-01-01'"` -Invalid Filters would be: +Invalid filters would be: * `"{{ TimeDimension('metric_time') }} > '2020-01-01'"` — metrics in the query are defined based on measures with different grains. From 250b3ba78d8cb759089a0fa3bc045ae88dd8e587 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:28:47 +0000 Subject: [PATCH 32/48] Update website/docs/docs/dbt-cloud-apis/sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 6ff846fa7c7..6916f283898 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -415,7 +415,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], ## FAQs -
+ When you select a dimension on its own, such as `metric_time` you can use the shorthand method which doesn't need the “Dimension” syntax. From fb3b8f69b3f762e16736aed47bf1d5fb8111bf33 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:30:05 +0000 Subject: [PATCH 33/48] Update website/docs/docs/dbt-cloud-apis/sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 7a857f45087..177883ce19b 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -481,7 +481,7 @@ mutation { } ``` -For both `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures & metrics associated with the where filter have different grains. +For both `TimeDimension()`, the grain is only required in the WHERE filter if the aggregation time dimensions for the measures and metrics associated with the where filter have different grains. For example, consider this Semantic model and Metric configuration, which contains two metrics that are aggregated across different time grains. This example shows a single semantic model, but the same goes for metrics across more than one semantic model. From 610da3815e16fef976b47ff3d4b39b4b2980b154 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:31:04 +0000 Subject: [PATCH 34/48] Update sl-graphql.md --- website/docs/docs/dbt-cloud-apis/sl-graphql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-graphql.md b/website/docs/docs/dbt-cloud-apis/sl-graphql.md index 177883ce19b..f26a19a1930 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-graphql.md +++ b/website/docs/docs/dbt-cloud-apis/sl-graphql.md @@ -466,7 +466,7 @@ The `where` filter takes a list argument (or a string for a single input). Depen - `Entity()` — Used for entities like primary and foreign keys, such as `Entity('order_id')`. -Note: If you prefer a `where` clause with a more explicit path, you can optionally use `TimeDimension()` to separate out categorical dimensions from time ones. The `TimeDimension` input takes the time dimension and optionally the granularity level. `TimeDimension('metric_time', 'month')`. +Note: If you prefer a `where` clause with a more explicit path, you can optionally use `TimeDimension()` to separate categorical dimensions from time ones. The `TimeDimension` input takes the time dimension and optionally the granularity level. `TimeDimension('metric_time', 'month')`. ```graphql mutation { From 7f3cc44c400ac2b56c265e4fb7a4d4292d6efa08 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 11:33:20 +0000 Subject: [PATCH 35/48] Update sl-jdbc.md --- website/docs/docs/dbt-cloud-apis/sl-jdbc.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md index 6916f283898..5ef0c071c10 100644 --- a/website/docs/docs/dbt-cloud-apis/sl-jdbc.md +++ b/website/docs/docs/dbt-cloud-apis/sl-jdbc.md @@ -92,9 +92,9 @@ select * from {{ -Use this query to fetch dimension values for one or multiple metrics and single dimension. +Use this query to fetch dimension values for one or multiple metrics and a single dimension. -Note, `metrics` is a required argument that lists one or multiple metrics in it, and a single dimension. +Note, `metrics` is a required argument that lists one or multiple metrics, and a single dimension. ```bash select * from {{ @@ -105,9 +105,9 @@ semantic_layer.dimension_values(metrics=['food_order_amount'], group_by=['custom -Use this query to fetch queryable granularities for a list of metrics. This API request allows you to only show the time granularities that make sense for the primary time dimension of the metrics (such as `metric_time`), but if you want queryable granularities for other time dimensions, you can use the `dimensions()` call, and find the column queryable_granularities. +You can use this query to fetch queryable granularities for a list of metrics. This API request allows you to only show the time granularities that make sense for the primary time dimension of the metrics (such as `metric_time`), but if you want queryable granularities for other time dimensions, you can use the `dimensions()` call, and find the column queryable_granularities. -Note, `metrics` is a required argument that lists one or multiple metrics in it. +Note, `metrics` is a required argument that lists one or multiple metrics. ```bash select * from {{ @@ -124,7 +124,7 @@ select * from {{ Use this query to fetch available metrics given dimensions. This command is essentially the opposite of getting dimensions given a list of metrics. -Note, `group_by` is a required argument that lists one or multiple dimensions in it. +Note, `group_by` is a required argument that lists one or multiple dimensions. ```bash select * from {{ @@ -137,7 +137,7 @@ select * from {{ -Use this example query to fetch available granularities for all time dimesensions (the similar queryable granularities API call only returns granularities for the primary time dimensions for metrics). The following call is a derivative of the `dimensions()` call and specifically selects the granularities field. +You can use this example query to fetch available granularities for all time dimensions (the similar queryable granularities API call only returns granularities for the primary time dimensions for metrics). The following call is a derivative of the `dimensions()` call and specifically selects the granularity field. ```bash select NAME, QUERYABLE_GRANULARITIES from {{ @@ -344,7 +344,7 @@ where=["{{ Dimension('metric_time').grain('month') }} >= '2017-03-09'", "{{ Dime ### Query with a limit -Use the following example to query using a `limit` or `order_by` clauses: +Use the following example to query using a `limit` or `order_by` clause: ```bash select * from {{ @@ -356,7 +356,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], ### Query with Order By Examples -Order By can take a basic string that's a Dimension, Metric, or Entity and this will default to ascending order +Order By can take a basic string that's a Dimension, Metric, or Entity, and this will default to ascending order ```bash select * from {{ @@ -367,7 +367,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], }} ``` -For descending order, you can add a `-` sign in front of the object. However, you can only use this short hand notation if you aren't operating on the object or using the full object notation. +For descending order, you can add a `-` sign in front of the object. However, you can only use this short-hand notation if you aren't operating on the object or using the full object notation. ```bash select * from {{ @@ -378,7 +378,7 @@ semantic_layer.query(metrics=['food_order_amount', 'order_gross_profit'], }} ``` -If you are ordering by an object that's been operated on (for example, you changed the the granularity of the time dimension), or you are using the full object notation, descending order must look like: +If you are ordering by an object that's been operated on (for example, you changed the granularity of the time dimension), or you are using the full object notation, descending order must look like: ```bash select * from {{ From 7fef1110ba5a6681a46ea47b5ba5cc713cedbeec Mon Sep 17 00:00:00 2001 From: mirnawong1 Date: Tue, 23 Jan 2024 11:58:39 +0000 Subject: [PATCH 36/48] update sql --- website/docs/docs/build/conversion-metrics.md | 160 ++++++++++-------- 1 file changed, 85 insertions(+), 75 deletions(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index 8d5f7eb24bf..5b63a6bbbf1 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -105,18 +105,19 @@ This step joins the `BUYS` table to the `VISITS` table and gets all combinations The SQL generated in these steps looks like the following: ```sql -SELECT v.ds, - v.user_id, - v.referrer_id, - b.ds, - b.uuid, - 1 as buys -FROM visits v - INNER JOIN (SLECT *, uuid_string() as uuid - FROM buys -- Adds a uuid column to uniquely identify the different rows - ) b - ON - v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' +select + v.ds, + v.user_id, + v.referrer_id, + b.ds, + b.uuid, + 1 as buys +from visits v +inner join ( + select *, uuid_string() as uuid from buys -- Adds a uuid column to uniquely identify the different rows +) b +on +v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 days' ``` The dataset returns the following (note that there are two potential conversion events for the first visit): @@ -133,18 +134,19 @@ The dataset returns the following (note that there are two potential conversion Instead of returning the raw visit values, use window functions to link conversions to the closest base event. You can partition by the conversion source and get the `first_value` ordered by `visit ds`, descending to get the closest base event from the conversion event: ```sql -SELECT first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, - first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, - first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, - b.ds, - b.uuid, - 1 as buys -FROM visits v - INNER JOIN (select *, uuid_string() as uuid - from buys) b - ON - v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' - +select + first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, + first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, + first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, + b.ds, + b.uuid, + 1 as buys +from visits v +inner join ( + select *, uuid_string() as uuid from buys +) b +on +v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day' ``` The dataset returns the following: @@ -165,18 +167,19 @@ To resolve this and eliminate duplicates, use a distinct select. The UUID also h Instead of regular select used in the [Step 2](#step-2-refine-with-window-function), use a distinct select to remove the duplicates: ```sql -SELECT DISTINCT first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, - first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, - first_value(v.referrer_id) - over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, - b.ds, - b.uuid, - 1 as buys -FROM visits v - INNER JOIN (select *, uuid_string() as uuid - from buys) b - ON - v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day'; +select distinct + first_value(v.ds) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as v_ds, + first_value(v.user_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as user_id, + first_value(v.referrer_id) over (partition by b.ds, b.user_id, b.uuid order by v.ds desc) as referrer_id, + b.ds, + b.uuid, + 1 as buys +from visits v +inner join ( + select *, uuid_string() as uuid from buys +) b +on +v.user_id = b.user_id and v.ds <= b.ds and v.ds > b.ds - interval '7 day'; ``` The dataset returns the following: @@ -197,28 +200,38 @@ You now have a dataset where every conversion is connected to a visit event. To Now that you’ve tied each conversion event to a visit, you can calculate the aggregated conversions and opportunities measures. Then, you can join them to calculate the actual conversion rate. The SQL to calculate the conversion rate is as follows: ```sql -SELECT coalesce(subq_3.metric_time__day, subq_13.metric_time__day) as metric_time__day, - cast(max(subq_13.buys) as double) / - cast(nullif(max(subq_3.visits), 0) as double) as visit_to_buy_conversion_rate_7d -FROM ( -- base measure - SELECT metric_time__day, - sum(visits) as mqls - FROM (SELECT date_trunc('day', first_contact_date) as metric_time__day, - 1 as visits - FROM visits) subq_2 - GROUP BY metric_time__day) subq_3 - FULL OUTER JOIN ( -- conversion measure - SELECT metric_time__day, - sum(buys) as sellers - FROM ( - -- ... - -- The output of this subquery is the table produced in Step 3. The SQL is hidden for legibility. - -- To see the full SQL output, add --explain to your conversion metric query. - ) subq_10 - GROUP BY metric_time__day) subq_13 - ON - subq_3.metric_time__day = subq_13.metric_time__day -GROUP BY metric_time__day +select + coalesce(subq_3.metric_time__day, subq_13.metric_time__day) as metric_time__day, + cast(max(subq_13.buys) as double) / cast(nullif(max(subq_3.visits), 0) as double) as visit_to_buy_conversion_rate_7d +from ( -- base measure + select + metric_time__day, + sum(visits) as mqls + from ( + select + date_trunc('day', first_contact_date) as metric_time__day, + 1 as visits + from visits + ) subq_2 + group by + metric_time__day +) subq_3 +full outer join ( -- conversion measure + select + metric_time__day, + sum(buys) as sellers + from ( + -- ... + -- The output of this subquery is the table produced in Step 3. The SQL is hidden for legibility. + -- To see the full SQL output, add --explain to your conversion metric query. + ) subq_10 + group by + metric_time__day +) subq_13 +on + subq_3.metric_time__day = subq_13.metric_time__day +group by + metric_time__day ``` ### Additional settings @@ -315,25 +328,22 @@ In this case, you want to set `product_id` as the constant property. You can spe You will add an additional condition to the join to make sure the constant property is the same across conversions. ```sql -SELECT DISTINCT first_value(v.ds) - over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as ds, - first_value(v.user_id) - over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as user_id, - first_value(v.referrer_id) - over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as referrer_id, - buy_source.uuid, - 1 as buys -FROM {{ source_schema }}.fct_view_item_details v -INNER JOIN +select distinct + first_value(v.ds) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as ds, + first_value(v.user_id) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as user_id, + first_value(v.referrer_id) over (partition by buy_source.ds, buy_source.user_id, buy_source.session_id order by v.ds desc rows between unbounded preceding and unbounded following) as referrer_id, + buy_source.uuid, + 1 as buys +from {{ source_schema }}.fct_view_item_details v +inner join ( - SELECT *, {{ generate_random_uuid() }} as uuid FROM {{ source_schema }}.fct_purchases + select *, {{ generate_random_uuid() }} as uuid from {{ source_schema }}.fct_purchases ) buy_source -ON - v.user_id = buy_source.user_id - AND v.ds <= buy_source.ds - AND v.ds > buy_source.ds - INTERVAL '7 day' - AND buy_source.product_id = v.product_id --Joining on the constant property product_id - +on + v.user_id = buy_source.user_id + and v.ds <= buy_source.ds + and v.ds > buy_source.ds - interval '7 day' + and buy_source.product_id = v.product_id --Joining on the constant property product_id ``` From 6348e402b6dba43cd83033a06cec7bb1f127bf5a Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:01:50 +0000 Subject: [PATCH 37/48] Update website/docs/docs/build/conversion-metrics.md --- website/docs/docs/build/conversion-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index 48915c410f9..bd2a16c3146 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -32,7 +32,7 @@ The specification for conversion metrics is as follows: | `constant_properties` | List of constant properties. | List | Optional | | `base_property` | The property from the base semantic model that you want to hold constant. | Entity or Dimension | Optional | | `conversion_property` | The property from the conversion semantic model that you want to hold constant. | Entity or Dimension | Optional | -| `fill_nulls_with` | Set the value of a null conversion event to zero instead of null. | String | Optional | +| `fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | String | Optional | Refer to [additional settings](#additional-settings) to learn how to customize conversion metrics with settings for null values, calculation type, and constant properties. From 08652a4e806b60db12a86fc725050453ab042084 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:02:48 +0000 Subject: [PATCH 38/48] Update website/docs/docs/build/cumulative-metrics.md --- website/docs/docs/build/cumulative-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index 85e6bf4f085..c55f879a419 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -20,7 +20,7 @@ This metric is common for calculating things like weekly active users, or month- | `measure` | The measure you are referencing. | Required | | `window` | The accumulation window, such as 1 month, 7 days, 1 year. This can't be used with `grain_to_date`. | Optional | | `grain_to_date` | Sets the accumulation grain, such as month will accumulate data for one month. Then restart at the beginning of the next. This can't be used with `window`. | Optional | -| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | +| `fill_nulls_with` | Set the value in your metric definition instead of null (such as zero).| Optional | The following displays the complete specification for cumulative metrics, along with an example: From ed37744a422af36f5e08d35e0dc51613a84c9d12 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:03:14 +0000 Subject: [PATCH 39/48] Update website/docs/docs/build/derived-metrics.md --- website/docs/docs/build/derived-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/derived-metrics.md b/website/docs/docs/build/derived-metrics.md index fcc585500cb..f9ce13c0932 100644 --- a/website/docs/docs/build/derived-metrics.md +++ b/website/docs/docs/build/derived-metrics.md @@ -21,7 +21,7 @@ In MetricFlow, derived metrics are metrics created by defining an expression usi | `metrics` | The list of metrics used in the derived metrics. | Required | | `alias` | Optional alias for the metric that you can use in the expr. | Optional | | `filter` | Optional filter to apply to the metric. | Optional | -| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | +| `fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | | `offset_window` | Set the period for the offset window, such as 1 month. This will return the value of the metric one month from the metric time. | Optional | The following displays the complete specification for derived metrics, along with an example. From f4f319f04e4f95aee3bf83d6a3bf49af670cf673 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:03:41 +0000 Subject: [PATCH 40/48] Update website/docs/docs/build/derived-metrics.md --- website/docs/docs/build/derived-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/derived-metrics.md b/website/docs/docs/build/derived-metrics.md index f9ce13c0932..35adb12cb1a 100644 --- a/website/docs/docs/build/derived-metrics.md +++ b/website/docs/docs/build/derived-metrics.md @@ -33,7 +33,7 @@ metrics: type: derived # Required label: The value that will be displayed in downstream tools #Required type_params: # Required - fill_nulls_with: Set value to zero instead of null # Optional + fill_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional expr: the derived expression # Required metrics: # The list of metrics used in the derived metrics # Required - name: the name of the metrics. must reference a metric you have already defined # Required From fb5f85482011fb198fd4789ac0b401ea48c2fb40 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:05:13 +0000 Subject: [PATCH 41/48] Update website/docs/docs/build/simple.md --- website/docs/docs/build/simple.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md index b384c53489f..0d5704c5514 100644 --- a/website/docs/docs/build/simple.md +++ b/website/docs/docs/build/simple.md @@ -32,7 +32,7 @@ metrics: label: The value that will be displayed in downstream tools # Required type_params: # Required measure: The measure you're referencing # Required - fill_nulls_with: Value to zero instead of null # Optional + fill_nulls_with: Set value instead of null (such as zero) # Optional ``` From 7b3ffe1fa5e3a8284f05fb3e64e8d0eb315bb0e2 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:05:29 +0000 Subject: [PATCH 42/48] Update website/docs/docs/build/metrics-overview.md --- website/docs/docs/build/metrics-overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/metrics-overview.md b/website/docs/docs/build/metrics-overview.md index 60b103419e3..f6844c60498 100644 --- a/website/docs/docs/build/metrics-overview.md +++ b/website/docs/docs/build/metrics-overview.md @@ -51,7 +51,7 @@ metrics: type: conversion # Required label: # Required type_params: # Required - fills_nulls_with: Set value to zero instead of null # Optional + fills_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional conversion_type_params: # Required entity: ENTITY # Required calculation: CALCULATION_TYPE # Optional. default: conversion_rate. options: conversions(buys) or conversion_rate (buys/visits), and more to come. From 8384e7bd30dbdb75791ba32e52184e8fff85653c Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:05:36 +0000 Subject: [PATCH 43/48] Update website/docs/docs/build/ratio-metrics.md --- website/docs/docs/build/ratio-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/ratio-metrics.md b/website/docs/docs/build/ratio-metrics.md index f0db2963f4a..0a2ca985eea 100644 --- a/website/docs/docs/build/ratio-metrics.md +++ b/website/docs/docs/build/ratio-metrics.md @@ -21,7 +21,7 @@ Ratio allows you to create a ratio between two metrics. You simply specify a num | `denominator` | The name of the metric used for the denominator, or structure of properties. | Required | | `filter` | Optional filter for the numerator or denominator. | Optional | | `alias` | Optional alias for the numerator or denominator. | Optional | -| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | +| `fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | The following displays the complete specification for ratio metrics, along with an example. From 632d85b61ee479d64b4a050e1f4faa12d7b0a3ab Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:05:42 +0000 Subject: [PATCH 44/48] Update website/docs/docs/build/ratio-metrics.md --- website/docs/docs/build/ratio-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/ratio-metrics.md b/website/docs/docs/build/ratio-metrics.md index 0a2ca985eea..5de4128c1f5 100644 --- a/website/docs/docs/build/ratio-metrics.md +++ b/website/docs/docs/build/ratio-metrics.md @@ -32,7 +32,7 @@ metrics: type: ratio # Required label: The value that will be displayed in downstream tools #Required type_params: # Required - fill_nulls_with: Value to zero instead of null # Optional + fill_nulls_with: Set value instead of null (such as zero) # Optional numerator: The name of the metric used for the numerator, or structure of properties # Required name: Name of metric used for the numerator # Required filter: Filter for the numerator # Optional From 0ba2a0998ead320dee0db98d859347da524f053f Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:05:49 +0000 Subject: [PATCH 45/48] Update website/docs/docs/build/simple.md --- website/docs/docs/build/simple.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/simple.md b/website/docs/docs/build/simple.md index 0d5704c5514..fafb770dd04 100644 --- a/website/docs/docs/build/simple.md +++ b/website/docs/docs/build/simple.md @@ -19,7 +19,7 @@ Simple metrics are metrics that directly reference a single measure, without any | `label` | The value that will be displayed in downstream tools. | Required | | `type_params` | The type parameters of the metric. | Required | | `measure` | The measure you're referencing. | Required | -| `fill_nulls_with` | Set the value to zero instead of null in your metric definition. | Optional | +| `fill_nulls_with` | Set the value in your metric definition instead of null (such as zero). | Optional | The following displays the complete specification for simple metrics, along with an example. From a5b6462833e0401abd1fb78995a17fbd2450cd9d Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:06:19 +0000 Subject: [PATCH 46/48] Update website/docs/docs/build/cumulative-metrics.md --- website/docs/docs/build/cumulative-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index c55f879a419..ac3196dc874 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -31,7 +31,7 @@ metrics: type: cumulative # Required label: The value that will be displayed in downstream tools # Required type_params: # Required - fill_nulls_with: Set value to zero instead of null # Optional + fill_nulls_with: Set the value in your metric definition instead of null (such as zero)# Optional measure: The measure you are referencing # Required window: The accumulation window, such as 1 month, 7 days, 1 year. # Optional. Cannot be used with grain_to_date grain_to_date: Sets the accumulation grain, such as month will accumulate data for one month, then restart at the beginning of the next. # Optional. Cannot be used with window From 9defe732f6d9847c92bb1704bc31a8a2f9ee9637 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:06:42 +0000 Subject: [PATCH 47/48] Update website/docs/docs/build/conversion-metrics.md --- website/docs/docs/build/conversion-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/conversion-metrics.md b/website/docs/docs/build/conversion-metrics.md index bd2a16c3146..39b3d969b27 100644 --- a/website/docs/docs/build/conversion-metrics.md +++ b/website/docs/docs/build/conversion-metrics.md @@ -45,7 +45,7 @@ metrics: type: conversion # Required label: # Required type_params: # Required - fills_nulls_with: Set value to zero instead of null # Optional + fills_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional conversion_type_params: # Required entity: ENTITY # Required calculation: CALCULATION_TYPE # Optional. default: conversion_rate. options: conversions(buys) or conversion_rate (buys/visits), and more to come. From 5c274545fca62b5afd9e35456cb95fa035a67b34 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Tue, 23 Jan 2024 18:06:58 +0000 Subject: [PATCH 48/48] Update website/docs/docs/build/cumulative-metrics.md --- website/docs/docs/build/cumulative-metrics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/build/cumulative-metrics.md b/website/docs/docs/build/cumulative-metrics.md index ac3196dc874..ec962969c9e 100644 --- a/website/docs/docs/build/cumulative-metrics.md +++ b/website/docs/docs/build/cumulative-metrics.md @@ -31,7 +31,7 @@ metrics: type: cumulative # Required label: The value that will be displayed in downstream tools # Required type_params: # Required - fill_nulls_with: Set the value in your metric definition instead of null (such as zero)# Optional + fill_nulls_with: Set the value in your metric definition instead of null (such as zero) # Optional measure: The measure you are referencing # Required window: The accumulation window, such as 1 month, 7 days, 1 year. # Optional. Cannot be used with grain_to_date grain_to_date: Sets the accumulation grain, such as month will accumulate data for one month, then restart at the beginning of the next. # Optional. Cannot be used with window