Upgrade to Arrow 17

[galipremsagar]

Description

This PR upgrades to arrow-17 in cudf.

Checklist

• I am familiar with the Contributing Guidelines.
• New or existing tests cover these changes.
• The documentation is up to date with these changes.

[vyasr]

I don't think we want to upgrade this, certainly not so quickly. We know that moving our Arrow dependency too quickly can cause pain for some of our users (CC @beckernick), and also we're targeting removing the libarrow dependency in 24.10 anyway.

[bdice]

Thanks @vyasr - I had the same concern. Let's try to reduce our Arrow dependencies first, and we can upgrade this later in the 24.10 release cycle if still needed.

[galipremsagar]

I don't think we want to upgrade this, certainly not so quickly. We know that moving our Arrow dependency too quickly can cause pain for some of our users (CC @beckernick), and also we're targeting removing the libarrow dependency in 24.10 anyway.

Sounds good 👍

[vyasr]

@galipremsagar once #16640 merges, we should replace this PR with one that simply relaxes the pyarrow constraint. Would you be able to take point on testing the versions that we support? I expect that anything after pyarrow 13 should work for us (we need pyarrow 13 for apache/arrow#36162).

@seberg when we add the earliest dependency testing to cudf (the cudf version of rapidsai/rmm#1613) can we also ensure that the appropriate pyarrow (and pandas) versions are being tested there? Thank you!

[galipremsagar]

@galipremsagar once #16640 merges, we should replace this PR with one that simply relaxes the pyarrow constraint. Would you be able to take point on testing the versions that we support? I expect that anything after pyarrow 13 should work for us (we need pyarrow 13 for apache/arrow#36162).

@seberg when we add the earliest dependency testing to cudf (the cudf version of rapidsai/rmm#1613) can we also ensure that the appropriate pyarrow (and pandas) versions are being tested there? Thank you!

Sure 👍

[seberg]

hen we add the earliest dependency testing to cudf (the cudf version of rapidsai/rmm#1613) can we also ensure that the appropriate pyarrow (and pandas) versions are being tested there?

They already are part of gh-16570 (although you can double check the versions, and I guess this would modify them).

[vyasr]

Oh awesome I thought there was a cudf PR for this but couldn't find it. Thanks for pointing it out! Yes, we'd just want to update the versions there.

[jakirkham]

Right this is the cuDF PR: #16570

Think Matthew is going to work on adding some skips and xfails as appropriate: #16570 (comment)

Guessing once that is done it can be merged

[vyasr]

Replaced by #16681.