GITBOOK-1581: No subject

rajagopr · Feb 9, 2023 · 376657e · 376657e
1 parent fadea31
commit 376657e
Show file tree

Hide file tree

Showing 18 changed files with 62 additions and 15 deletions.
diff --git a/.gitbook/assets/image (10) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png b/.gitbook/assets/image (10) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png
diff --git a/...sets/image (29) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png b/...sets/image (29) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png
diff --git a/...s/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png b/...s/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png
diff --git a/...s/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (2).png b/...s/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (2).png
diff --git a/...(2) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (2) (1) (1) (1).png b/...(2) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (2) (1) (1) (1).png
diff --git a/basics/components/deep-store.md b/basics/components/deep-store.md
@@ -26,13 +26,13 @@ The ingestion job then sends a notification about the new segment to the control
 
 For real-time tables, by default, a segment is first built-in memory by the server. It is then uploaded to the lead controller (as part of the Segment Completion Protocol sequence), which writes the segment into the deep store, as shown in the diagram below:
 
-![Server sends segment to Controller, which writes segments into the deep store](<../../.gitbook/assets/server-controller-deep-store (1).png>)
+![Server sends segment to Controller, which writes segments into the deep store](../../.gitbook/assets/server-controller-deep-store.png)
 
 Having all segments go through the controller can become a system bottleneck under heavy load, in which case you can use the peer download policy, as described in [Decoupling Controller from the Data Path](../../operators/operating-pinot/decoupling-controller-from-the-data-path.md).
 
 When using this configuration the server will directly write a completed segment to the deep store, as shown in the diagram below:
 
-![Server writing a segment into the deep store](<../../.gitbook/assets/server-deep-store (1).png>)
+![Server writing a segment into the deep store](../../.gitbook/assets/server-deep-store.png)
 
 ## Configuring the Deep Store
 

diff --git a/basics/components/exploring-pinot.md b/basics/components/exploring-pinot.md
@@ -20,7 +20,7 @@ If you want to view the contents of a server, click on its instance name. You'll
 
 To view the _baseballStats_ table, click on its name, which will show the following screen:
 
-![baseballStats Table](<../../.gitbook/assets/image (2) (1).png>)
+![baseballStats Table](<../../.gitbook/assets/image (2) (1) (1).png>)
 
 From this screen, we can edit or delete the table, edit or adjust its schema, as well as several other operations.
 

diff --git a/basics/getting-started/kubernetes-quickstart.md b/basics/getting-started/kubernetes-quickstart.md
@@ -321,7 +321,7 @@ kubectl port-forward service/my-trino 18080:8080 -n trino-quickstart
 
 **6.2.4 Query Pinot data using Trino CLI**
 
-![](<../../.gitbook/assets/image (43).png>)
+![](<../../.gitbook/assets/image (5) (1).png>)
 
 ### 6.3 Sample queries to execute
 

diff --git a/basics/getting-started/running-pinot-in-docker.md b/basics/getting-started/running-pinot-in-docker.md
@@ -8,7 +8,7 @@ In this guide we will learn about running Pinot in Docker.
 
 This guide assumes that you have installed [Docker](https://hub.docker.com/editions/community/docker-ce-desktop-mac) and have configured it with enough memory. A sample config is shown below:
 
-![Sample Docker resources](<../../.gitbook/assets/image (4) (2).png>)
+![Sample Docker resources](<../../.gitbook/assets/image (4) (1).png>)
 
 The latest Pinot Docker image is published at `apachepinot/pinot:latest` and you can see a list of [all published tags on Docker Hub](https://hub.docker.com/r/apachepinot/pinot/tags).
 

diff --git a/basics/getting-started/troubleshooting-pinot.md b/basics/getting-started/troubleshooting-pinot.md
@@ -6,7 +6,7 @@ Pinot offers various ways to assist with troubleshooting and debugging problems
 
 The table debug api can be invoked via the Swagger UI as follows:
 
-![Swagger - Table Debug Api](<../../.gitbook/assets/image (11) (2).png>)
+![Swagger - Table Debug Api](<../../.gitbook/assets/image (11) (1).png>)
 
 It can also be invoked directly by accessing the URL as follows. The api requires the `tableName`, and can optionally take `tableType (offline|realtime)` and `verbosity` level.
 

diff --git a/configuration-reference/cluster.md b/configuration-reference/cluster.md
@@ -58,4 +58,4 @@ JSON body contains the configs map for new/updated configs. E.g.
 
 Example:
 
-![](<../.gitbook/assets/image (9) (2).png>)
+![](<../.gitbook/assets/image (9) (1).png>)
diff --git a/developers/advanced/advanced-pinot-setup.md b/developers/advanced/advanced-pinot-setup.md
@@ -14,7 +14,7 @@ Setup Pinot by starting each component individually
 If running locally, please ensure your docker cluster has enough resources, below is a sample config.
 {% endhint %}
 
-![Sample docker resources](<../../.gitbook/assets/image (4) (1).png>)
+![Sample docker resources](<../../.gitbook/assets/image (4) (2).png>)
 
 **Pull docker image**
 

diff --git a/developers/advanced/v2-multi-stage-query-engine.md b/developers/advanced/v2-multi-stage-query-engine.md
@@ -10,7 +10,7 @@ The new Pinot query engine version 2 (a.k.a Multi-Stage V2 query engine) is desi
 
 It also resolves the bottleneck effect for the broker reduce stage where only a single machine is dedicated to perform heavy lifting such as high cardinality `GROUP BY` result merging; `ORDER BY` sorting, etc.
 
-## How to use the multi-stage query engine
+## How to enable the multi-stage query engine
 
 To enable the multi-stage engine,
 
@@ -31,12 +31,60 @@ To enable the multi-stage engine,
 
     <figure><img src="../../.gitbook/assets/image (51).png" alt=""><figcaption><p>Sample Query Screenshot</p></figcaption></figure>
 
+## How to programmatically access the multi-stage query engine
+
+There's 2 main way to make query against the multi-stage engine:
+
+### Via REST APIs
+
+Both the Controller query API and the Broker query API allows optional JSON payload for configuration. For example:
+
+* For Controller REST API
+
+```bash
+curl -X POST http://localhost:9000/sql -d 
+'
+{
+  "sql": "select * from baseballStats limit 10",
+  "trace": false,
+  "queryOptions": "useMultistageEngine=true"
+}
+'
+```
+
+* For Broker REST API
+
+```bash
+curl -X POST http://localhost:8000/query/sql -d '
+{
+  "sql": "select * from baseballStats limit 10",
+  "trace": false,
+  "queryOptions": "useMultistageEngine=true"
+}
+'
+```
+
+### Via Query Options
+
+When executing a query via a non-REST API route. we also support enabling the multi-stage engine via the query option. simply attach a query option at the top of your query will enable the multi-stage engine
+
+```sql
+SET useMultistageEngine=true; -- indicator to enable the multi-stage engine.
+SELECT * from baseballStats limit 10
+```
+
 ## Troubleshoot
 
 The V2 query engine is still in the beta phase, there might be various performance or feature gaps from the current query engine.
 
 Here are the general troubleshooting steps:
 
+### Cluster errors
+
+If you can't get the cluster to run and received an error indicating the multi-stage engine is not available:
+
+* Make sure your pinot-servers have access to the configured port: The default setting above assumes all pinot-servers are run on individual networks and all of them have access to the same port number available within that environment. If your cluster is not set up this way, you will have to manually configure them directly in your pinot-server config file.
+
 ### Semantic / Runtime errors
 
 * Try downloading the latest docker image or building from the latest master commit
@@ -71,7 +119,6 @@ We are continuously improving the multi-stage engine. However, since the multi-s
 * Currently, it doesn't incorporate table statistics into plan optimization.
 * Currently, it doesn't support complex aggregation functions, such as `COVAR_POP`.
 * Currently, it doens't support tenant isolation, only `DEFAULT_TENANT` is visible to the multi-stage engine.
-* Some functions that lack implementation with `@ScalarFunction` annotation will not be supported in intermediate stages.
 
 For more up-to-date tracking of feature and performance support please follow the Github tracking issues:
 

diff --git a/integrations/presto.md b/integrations/presto.md
@@ -68,7 +68,7 @@ Splits: 17 total, 17 done (100.00%)
 
 Meanwhile you can access [Presto Cluster UI](http://localhost:8080/ui/) to see query stats.
 
-![Presto Cluster UI](<../.gitbook/assets/image (16) (1).png>)
+![Presto Cluster UI](<../.gitbook/assets/image (16) (2).png>)
 {% endtab %}
 {% endtabs %}
 

diff --git a/integrations/superset.md b/integrations/superset.md
@@ -67,7 +67,7 @@ E.g.
 
 Below is an example for the QuickStart cluster, you can click `TEST CONNECTION` button to check if Pinot cluster is successfully connected.
 
-![Add Pinot cluster as a new Database](<../.gitbook/assets/image (18).png>)
+![Add Pinot cluster as a new Database](<../.gitbook/assets/image (37) (1).png>)
 
 ### Adding Pinot Table
 

diff --git a/operators/operating-pinot/segment-assignment.md b/operators/operating-pinot/segment-assignment.md
@@ -12,7 +12,7 @@ Segment assignment refers to the strategy of assigning each segment from a table
 
 Balanced Segment Assignment is the default assignment strategy, where each segment is assigned to the server with the least segments already assigned. With this strategy, each server will have balanced query load, and each query will be routed to all the servers. It requires minimum configuration, and works well for small use cases.
 
-![](../../.gitbook/assets/Balanced.png)
+![](../../.gitbook/assets/balanced.png)
 
 ## Replica-Group Segment Assignment
 

diff --git a/...w-to-connect-pinot-with-amazon-managed-streaming-for-apache-kafka-amazon-msk.md b/...w-to-connect-pinot-with-amazon-managed-streaming-for-apache-kafka-amazon-msk.md
@@ -54,7 +54,7 @@ This is configured through Amazon VPC Page.
 1. Record the Amazon MSK `SecurityGroup` from the Cluster page, in the above demo, it's `sg-01e7ab1320a77f1a9`.
 2. Open [Amazon VPC Page](https://us-west-2.console.aws.amazon.com/vpc/home), click on **`SecurityGroups`** on left bar. Find the EKS Security group: `eksctl-${PINOT_EKS_CLUSTER}-cluster/ClusterSharedNodeSecurityGroup.`
 
-![Amazon EKS ClusterSharedNodeSecurityGroup](<../../.gitbook/assets/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png>)
+![Amazon EKS ClusterSharedNodeSecurityGroup](<../../.gitbook/assets/image (9) (2) (2) (2) (2) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1) (1).png>)
 
 {% hint style="info" %}
 Please ensure you are picking **ClusterShardNodeSecurityGroup**

diff --git a/operators/tutorials/monitor-pinot-using-prometheus-and-grafana.md b/operators/tutorials/monitor-pinot-using-prometheus-and-grafana.md
@@ -139,7 +139,7 @@ Once open the dashboard, you can login with credential:
 
 `admin`/`[ PASSWORD GET FROM PREVIOUS STEP]`
 
-![Grafana Dashboard](<../../.gitbook/assets/image (47) (1).png>)
+![Grafana Dashboard](<../../.gitbook/assets/image (47) (1) (1).png>)
 
 * Add data source
Original file line number	Diff line number	Diff line change
Expand Up		@@ -58,4 +58,4 @@ JSON body contains the configs map for new/updated configs. E.g.

		Example:

		![](<../.gitbook/assets/image (9) (2).png>)
		![](<../.gitbook/assets/image (9) (1).png>)