You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, iceberg source does scan planning in the frontend node and sends the files needed to be scanned to compute nodes. Technically, we can use a file level scan API to scan those files. However, icelake lacks this API, so we need to reuse the table-level API and filter out the assigned files. I think we can add a file-level read API to icelake to avoid redundant scan planning.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered:
After apache/iceberg-rust#377, I think the iceberg-rust can distribute the file scan task to different compute nodes. And the iceberg-rust has better support for reading and is under active development. I find that we can add the new interface like load_table_v2 using iceberg-rust so that we can replace the icelake implementation about read with iceberg-rust.
This issue has been open for 60 days with no activity.
If you think it is still relevant today, and needs to be done in the near future, you can comment to update the status, or just manually remove the no-issue-activity label.
You can also confidently close this issue as not planned to keep our backlog clean.
Don't worry if you think the issue is still valuable to continue in the future.
It's searchable and can be reopened when it's time. 😄
Is your feature request related to a problem? Please describe.
Currently, iceberg source does scan planning in the frontend node and sends the files needed to be scanned to compute nodes. Technically, we can use a file level scan API to scan those files. However, icelake lacks this API, so we need to reuse the table-level API and filter out the assigned files. I think we can add a file-level read API to icelake to avoid redundant scan planning.
Describe the solution you'd like
No response
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: