Support unsigned int types in features

[buhrmann]

Hi,
I'm trying to programmatically upload my first training dataset, but even though the docs say you support missing data in features, your validation in fact prevents missing data in integer columns:

client_python/arize/pandas/validation/validator.py

Line 1027 in 6f67886

allowed_datatypes = (

The list of allowed arrow types only contains non-nullable integer types. Is this an oversight or because you don't really support missing data in features?

Also, and perhaps alternatively, since you support manual upload of parquet and arrow files, do you plan to also support these via the Python SDK? My data is in Arrow to begin with, and so that would save me some manual work of converting to pandas, especially since it'll get converted back to Arrow anyway.

[buhrmann]

Sorry for the confusion, the problem is not in fact with nullable ints, but unsigned integers hahaha

[fjcasti1]

Hi @buhrmann, good catch we did not support unsigned integers. Definitely, we can work on that and include it in a future release soon. As for the support for arrow files via our Python SDK, this is not in our roadmap. Our intention with our Python SDK is to support record-at-a-time ingestion and batch ingestion via Pandas.

If you are interested in ingesting your arrow files directly (as well as avro, parquet, etc), I recommend visiting our docs about our fileimporter tool. You can ingest files directly via Drag & Drop, integrations with cloud storage, as well as table integrations. See the section Sending Data Methods