cache_path format

[Arshil-Akkala]

In greptimedb-standalone, i am trying to set a cache_path and capacity. So, post setting this, I should be able to view the cache_path in the s3(Minio in my case) right?
The path which I have is "s3://bucket-name//relative-cache-path"
I am unable to view it in my s3. Where am I going wrong?

[killme2008]

The cache_path sets the local read cache directory for remote object storage. Since object storage systems like AWS S3 or Minio prioritize throughput over latency, they may not perform well for queries that scan multiple SST files.

The read cache stores recently accessed files in the local cache_path, improving the speed of subsequent queries that need to access them again. Therefore, it should be configured to point to a local machine rather than the object storage service.

Sorry for the misunderstanding and will enhance the documentation.

[killme2008]

@Arshil-Akkala My pleasure. It's recommended to enable the enable_experimental_write_cache too, it's good for performance, please read the performance tips doc:
https://docs.greptime.com/user-guide/administration/performance-tuning-tips#using-cache-for-object-stores

[Arshil-Akkala]

I had another question, what if the pod fails and get reallocated to another node.. what will happen in that case?

[zyy17]

@Arshil-Akkala

If you use our official helm-charts to deploy your DB, we use the following default configuration:

• Use K8s StatefulSets to manage Datanode or GreptimeDB Standalone instance;
• Use default storageclass to allocate the PVC, which is used for wal and cache data;

Suppose your default storageclass is distributed block storage, for example, AWS EBS(public cloud) or Ceph(self-host). In that case, the PVC remains even if your pod is terminated or relocated to another node. Whenever your pod recovers, it will attach to the original PVC and use the original data.

The storage will be bound to the specific node if your underly storageclass is local-storage(for example, local pv). Although your DB may work, you should still be careful with the binding. If you don't set the pod affinity(the pod must be scheduled to the specific node), your pod may not work well when it fails or is rescheduled(it probably will fail to be scheduled). We don't recommend using this deployment because it's hard to operate.

[Arshil-Akkala]

I meant in reference to caching, so is there something like distributed caching?

[killme2008]

I meant in reference to caching, so is there something like distributed caching?

No, we don't design the distributed caching in the system right now, but based on the PVC and statefulsets, we can archive almost the same mechanism just like it. We plan to design the in-house distributed caching.