Write partition data in separate data dir for iceberg sink

[ZENOTME]

Is your feature request related to a problem? Please describe.

Background

I have an iceberg table partitioned by the Hour of a timestamp, but all the data from RisingWave doesn't seem to respect the partition. It's just writing straight to the /data folder. In contrast, if I write to the table from Spark it will create a partitioned folder and write data to that partition, under /data/timestamp_hour=2024-03-18-19 for example.

Describe the solution you'd like

1. This feature depends on the iceberg sdk to provide a convenient to custom the data dir.
2. Provide the config in risingwave

Describe alternatives you've considered

No response

Additional context

No response

[chenzl25]

@ZENOTME Any updates?

[ZENOTME]

@ZENOTME Any updates?

No, I don't make progress for this now. If it's not a high priority features, we can postpone it to next version of iceberg sink.

[github-actions]

This issue has been open for 60 days with no activity.

If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.

You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄