Skip to content

Commit

Permalink
Update clickpipes object storage limitations (#2542)
Browse files Browse the repository at this point in the history
  • Loading branch information
kitop authored Jul 31, 2024
1 parent 4964d25 commit 5a21d82
Showing 1 changed file with 2 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Materialized views created while an Object Storage ClickPipe is running will not
For best results, we recommend stopping the pipe, creating the materialized views, and then restarting the pipe.

## Limitations

- Any changes to the destination table, its materialized views (including cascading materialized views), or the materialized view's target tables won't be picked up automatically by the pipe and can result in errors. For best results, we recommend stopping the pipe, making the necessary modifications, and then restarting the pipe.
- Role authentication is not available for S3 ClickPipes for ClickHouse Cloud instances deployed into GCP or Azure. It is only supported for AWS ClickHouse Cloud instances.
- ClickPipes will only attempt to ingest objects at 1GB or smaller in size. If a file is greater than 1 GB an error will be appended to the ClickPipes dedicated error table.
- S3 / GCS ClickPipes **does not** share a listing syntax with the [S3 Table Function](https://clickhouse.com/docs/en/sql-reference/table-functions/file#globs_in_path).
Expand Down Expand Up @@ -140,8 +140,4 @@ The Service Account permissions attached to the HMAC credentials should be `stor
## F.A.Q.
- **Does ClickPipes support GCS buckets prefixed with `gs://`?**

No. For interoprability reasons we ask you to replace your `gs://` bucket prefix with `https://storage.googleapis.com/`.

- **When will continuous ingestion be available?**

Continuous ingestion is being actively developed and is available to select number of customers for feedback. It will be available to all customers in due course.
No. For interoprability reasons we ask you to replace your `gs://` bucket prefix with `https://storage.googleapis.com/`.

0 comments on commit 5a21d82

Please sign in to comment.