Can't write to a table with DECIMAL(38,0) field.

[denizsurmeli]

I have a column with type DECIMAL(38,0, but when I try to start ingestion I get the following error:

Ingest service exited with error DeltaWriteFailed { ending_offsets: "{\"0\":4999}", partition_counts: "{\"0\":5000}", source: Arrow { source: JsonError("Decimal(38, 0) type is not supported") } }

I believe this is a compatibility issue with the www.github.com/delta-io/delta-rs, a help would be appreciated. I am ready to provide more information.

Thanks in advance !

[houqp]

most likely a missing feature in arrow::ArrowWriter, we might just have to upgrade arrow.

[rtyler]

@denizsurmeli I am quite certain that this bug is fixed now with the current code in main. I will probably close this ticket when I write a regression test that covers this case

[rtyler]

@denizsurmeli 👋 hiya we just merged an upgrade to deltalake 0.16.5 and I'm extra certain that this bug should be fixed now, but haven't had the time to write a regression test for it. Would it be possible for you to test your workload again?