52.0.0 (2024-06-03)
Breaking changes:
- chore: Make binary_mut kernel accept different type for second arg #5833 [arrow] (viirya)
- fix(flightsql): remove Any encoding of
DoPutPreparedStatementResult
#5817 [arrow] [arrow-flight] (erratic-pattern) - Encode UUID as FixedLenByteArray in parquet_derive #5773 (conradludgate)
- Structured interval types for
IntervalMonthDayNano
orIntervalDayTime
(#3125) (#5654) #5769 [parquet] [arrow] (tustvold) - Fallible stream for arrow-flight do_exchange call (#3462) #5698 [arrow] [arrow-flight] (opensourcegeek)
- Update object_store dependency in arrow to
0.10.0
#5675 [parquet] (tustvold) - Remove deprecated JSON writer #5651 [arrow] (tustvold)
- Change
UnionArray
constructors #5623 [arrow] [arrow-flight] (mbrobbel) - Update py03 from 0.20 to 0.21 #5566 [arrow] (Jefffrey)
- Optionally require alignment when reading IPC, respect alignment when writing #5554 [arrow] [arrow-flight] (hzuo)
Implemented enhancements:
- Serialize
Binary
andLargeBinary
as HEX with JSON Writer #5783 [arrow] - Some optimizations in arrow_buffer::util::bit_util do more harm than good #5771 [arrow]
- Support skipping comments in CSV files #5758 [arrow]
parquet-derive
should be included in repository README. #5751- proposal: Make AsyncArrowWriter accepts AsyncFileWriter trait instead #5738 [parquet]
- Nested nullable fields do not get treated as nullable in data_gen #5712 [arrow]
- Optionally support flexible column lengths #5678 [arrow]
- Arrow Flight SQL example server: do_handshake should include auth header #5665 [arrow] [arrow-flight]
- Add support for the "r+" datatype in the C Data interface /
RunArray
#5631 [arrow] - Serialize
FixedSizeBinary
as HEX with JSON Writer #5620 [arrow] - Cleanup UnionArray Constructors #5613 [arrow] [arrow-flight]
- Zero Copy Support #5593
- ObjectStore bulk delete #5591
- Retry on Broken Connection #5589
StreamReader
is not zero-copy #5584 [arrow]- Create
ArrowReaderMetadata
from externalized metadata #5582 [parquet] - Make
filter
infilter_leaves
API propagate error #5574 [arrow] - Support
List
incompare_op
#5572 - Make FixedSizedList Json serializable #5568 [arrow]
- arrow-ord: Support sortting StructArray #5559 [arrow]
- Add scientific notation decimal parsing in
parse_decimal
#5549 [arrow] take
kernel support forStringViewArray
andBinaryViewArray
#5511 [arrow]filter
kernel support forStringViewArray
andBinaryViewArray
#5510 [arrow]- Display support for
StringViewArray
andBinaryViewArray
#5509 [arrow] - Arrow Flight format support for
StringViewArray
andBinaryViewArray
#5507 [arrow] [arrow-flight] - IPC format support for
StringViewArray
andBinaryViewArray
#5506 [parquet] [arrow]
Fixed bugs:
- List Row Encoding Sorts Incorrectly #5807 [arrow]
- Schema Root Message Name Ignored by parquet-fromcsv #5804 [parquet]
- Compute data buffer length by using start and end values in offset buffer #5756 [arrow]
- parquet: ByteArrayEncoder allocates large unused FallbackEncoder for Parquet 2 #5755 [parquet]
- The CI pipeline
Archery test With other arrow
is broken #5742 [arrow] - Unable to parse scientific notation string to decimal when scale is 0 #5739 [arrow]
- Stateless prepared statements wrap
DoPutPreparedStatementResult
withAny
which is differs from Go implementation #5731 [arrow] [arrow-flight] - "Rustdocs are clean (amd64, nightly)" CI check is failing #5725 [parquet] [arrow]
- "Archery test With other arrows" integration tests are failing #5719 [parquet] [arrow]
- parquet_derive: invalid examples/documentation #5687
- Arrow FLight SQL: invalid location in get_flight_info_prepared_statement #5669 [arrow] [arrow-flight]
- Rust Interval definition incorrect #5654 [parquet] [arrow]
- DECIMAL regex in csv reader does not accept positive exponent specifier #5648 [arrow]
- panic when casting
ListArray
toFixedSizeList
#5642 [arrow] - FixedSizeListArray::try_new Errors on Entirely Null Array With Size 0 #5614 [arrow]
parquet / Build wasm32 (pull_request)
CI check failing on main #5565 [parquet] [arrow]- Documentation fix: example in parquet/src/column/mod.rs is incorrect #5560 [parquet]
- IPC code writes data with insufficient alignment #5553 [arrow] [arrow-flight]
- Cannot access example Flight SQL Server from dbeaver #5540 [arrow] [arrow-flight]
- parquet: "not yet implemented" error when codec is actually implemented but disabled #5520 [parquet]
Documentation updates:
- Minor: Improve arrow_cast documentation #5825 [arrow] (alamb)
- Minor: Improve
ArrowReaderBuilder::with_row_selection
docs #5824 [parquet] (alamb) - Minor: Add examples for ColumnPath::from #5813 [parquet] (alamb)
- Minor: Clarify docs on
EnabledStatistics
#5812 [parquet] (alamb) - Add parquet-derive to repository README #5795 (konjac)
- Refine ParquetRecordBatchReaderBuilder docs #5774 [parquet] (alamb)
- docs: add sizing explanation to bloom filter docs in parquet #5705 [parquet] (hiltontj)
Closed issues:
binary_mut
kernel requires both args to be the same type (which is inconsistent withbinary
) #5818 [arrow]- Panic when displaying debug the results via log::info in the browser. #5599 [arrow]
Merged pull requests:
- feat: impl *Assign ops for types in arrow-buffer #5832 [arrow] (waynexia)
- Relax zstd-sys Version Pin #5829 [parquet] (waynexia)
- Minor: Document timestamp with/without cast behavior #5826 [arrow] (alamb)
- fix: invalid examples/documentation in parquet_derive doc #5823 (Weijun-H)
- Check length of
FIXED_LEN_BYTE_ARRAY
foruuid
logical parquet type #5821 [parquet] (mbrobbel) - Allow overriding the inferred parquet schema root #5814 [parquet] (tustvold)
- Revisit List Row Encoding (#5807) #5811 [arrow] (tustvold)
- Update proc-macro2 requirement from =1.0.83 to =1.0.84 #5805 [arrow] [arrow-flight] (dependabot[bot])
- Fix typo continuation maker -> marker #5802 [arrow] (djanderson)
- fix: serialization of decimal #5801 [arrow] (yjshen)
- Allow constructing ByteViewArray from existing blocks #5796 [arrow] (tustvold)
- Push SortOptions into DynComparator Allowing Nested Comparisons (#5426) #5792 [arrow] (tustvold)
- Fix incorrect URL to Parquet CPP types.h #5790 [parquet] (viirya)
- Update proc-macro2 requirement from =1.0.82 to =1.0.83 #5789 [arrow] [arrow-flight] (dependabot[bot])
- Update prost-build requirement from =0.12.4 to =0.12.6 #5788 [arrow] [arrow-flight] (dependabot[bot])
- Refine parquet documentation on types and metadata #5786 [parquet] (alamb)
- feat(arrow-json): encode
Binary
andLargeBinary
types as hex when writing JSON #5785 [arrow] (hiltontj) - fix broken link to ballista crate in README.md #5784 (navicore)
- feat(arrow-csv): support encoding of binary in CSV writer #5782 [arrow] (hiltontj)
- Fix documentation for parquet
parse_metadata
,decode_metadata
anddecode_footer
#5781 [parquet] (alamb) - Support casting a
FixedSizedList<T>[1]
toT
#5779 [arrow] (sadboy) - [parquet] Set the default size of BitWriter in DeltaBitPackEncoder to 1MB #5776 [parquet] (AdamGS)
- Remove harmful table lookup optimization for bitmap operations #5772 [arrow] (HadrienG2)
- Remove deprecated comparison kernels (#4733) #5768 [arrow] (tustvold)
- Add environment variable definitions to run the nanoarrow integration tests #5764 (paleolimbot)
- Downgrade to Rust 1.77 in integration pipeline to fix CI (#5719) #5761 (tustvold)
- Expose boolean builder contents #5760 [arrow] (HadrienG2)
- Allow specifying comment character for CSV reader #5759 [arrow] (bbannier)
- Expose the null buffer of every builder that has one #5754 [arrow] (HadrienG2)
- feat: Make AsyncArrowWriter accepts AsyncFileWriter #5753 [parquet] (Xuanwo)
- Improve repository readme #5752 (alamb)
- Document object store release cadence #5750 (alamb)
- Compute data buffer length by using start and end values in offset buffer #5741 [arrow] (viirya)
- fix: parse string of scientific notation to decimal when the scale is 0 #5740 [arrow] (yjshen)
- Minor: avoid (likely unreachable) panic in FlightClient #5734 [arrow] [arrow-flight] (alamb)
- Update proc-macro2 requirement from =1.0.81 to =1.0.82 #5732 [arrow] [arrow-flight] (dependabot[bot])
- Improve error message for timestamp queries outside supported range #5730 [arrow] (Abdi-29)
- Refactor to share code between do_put and do_exchange calls #5728 [arrow] [arrow-flight] (opensourcegeek)
- Update brotli requirement from 5.0 to 6.0 #5726 [parquet] (dependabot[bot])
- Fix
GenericListBuilder
test typo #5724 [arrow] (Kikkon) - Deprecate NullBuilder capacity, as it behaves in a surprising way #5721 [arrow] (HadrienG2)
- Fix nested nullability when randomly generating arrays #5713 [arrow] (alexwilcoxson-rel)
- Fix up clippy for Rust 1.78 #5710 [parquet] [arrow] (alamb)
- Support casting
StringView
/BinaryView
-->StringArray
/BinaryArray
. #5704 [arrow] (RinChanNOWWW) - Fix documentation around handling of nulls in cmp kernels #5697 [arrow] (Jefffrey)
- Support casting
StringArray
/BinaryArray
-->StringView
/BinaryView
#5686 [arrow] (RinChanNOWWW) - Add support for flexible column lengths #5679 [arrow] (Posnet)
- Move ffi stream and utils from arrow to arrow-array #5670 [arrow] (alexandreyc)
- Arrow Flight SQL example JDBC driver incompatibility #5666 [arrow] [arrow-flight] (istvan-fodor)
- Add
ListView
&LargeListView
basic construction and validation #5664 [arrow] (Kikkon) - Update proc-macro2 requirement from =1.0.80 to =1.0.81 #5659 [arrow] [arrow-flight] (dependabot[bot])
- Modify decimal regex to accept positive exponent specifier #5649 [arrow] (jdcasale)
- feat: JSON encoding of
FixedSizeList
#5646 [arrow] (hiltontj) - Update proc-macro2 requirement from =1.0.79 to =1.0.80 #5644 [arrow] [arrow-flight] (dependabot[bot])
- fix: panic when casting
ListArray
toFixedSizeList
#5643 [arrow] (jonahgao) - Add more invalid utf8 parquet reader tests #5639 [parquet] (alamb)
- Update brotli requirement from 4.0 to 5.0 #5637 [parquet] (dependabot[bot])
- Update flatbuffers requirement from 23.1.21 to 24.3.25 #5636 [arrow] (dependabot[bot])
- Increase
BinaryViewArray
test coverage #5635 [arrow] (alamb) - PrettyPrint support for
StringViewArray
andBinaryViewArray
#5634 [arrow] (alamb) - feat(ffi): add run end encoded arrays #5632 [arrow] (notfilippo)
- Accept parquet schemas without explicitly required Map keys #5630 [parquet] (jupiter)
- Implement
filter
kernel for byte view arrays. #5624 [arrow] (RinChanNOWWW) - feat: encode FixedSizeBinary in JSON as hex string #5622 [arrow] (hiltontj)
- Update Flight crate README version #5621 [arrow] [arrow-flight] (phillipleblanc)
- feat: support reading and writing
StringView
andBinaryView
in parquet (part 1) #5618 [parquet] [arrow] (alamb) - Use FixedSizeListArray::new in FixedSizeListBuilder #5612 [arrow] (tustvold)
- String to decimal conversion written using E/scientific notation #5611 [arrow] (Nekit2217)
- Account for Timezone when Casting Timestamp to Date32 #5605 [arrow] (Lordworms)
- Update prost-build requirement from =0.12.3 to =0.12.4 #5604 [arrow] [arrow-flight] (dependabot[bot])
- Fix panic when displaying dates on 32-bit platforms #5603 [arrow] (ivanceras)
- Implement
take
kernel for byte view array. #5602 [arrow] (RinChanNOWWW) - Add tests for Arrow Flight support for
StringViewArray
andBinaryViewArray
#5601 [arrow] [arrow-flight] (XiangpengHao) - test: Add a test for RowFilter with nested type #5600 [parquet] (viirya)
- Minor: Add docs for GenericBinaryBuilder, links to
GenericStringBuilder
#5597 [arrow] (alamb) - Bump chrono-tz from 0.8 to 0.9 #5596 [arrow] (Jefffrey)
- Update brotli requirement from 3.3 to 4.0 #5586 [parquet] (dependabot[bot])
- Add
UnionArray::into_parts
#5585 [arrow] (mbrobbel) - Expose ArrowReaderMetadata::try_new #5583 [parquet] (kylebarron)
- Add
try_filter_leaves
to propagate error from filter closure #5575 [arrow] (viirya) - filter for run end array #5573 [arrow] (fabianmurariu)
- Pin zstd-sys to
v2.0.9
in parquet #5567 [parquet] (Jefffrey) - Split arrow_cast::cast::string into it's own submodule #5563 [arrow] (monkwire)
- Correct example code for column (#5560) #5561 [parquet] (zgershkoff)
- Split arrow_cast::cast::dictionary into it's own submodule #5555 [arrow] (monkwire)
- Split arrow_cast::cast::decimal into it's own submodule #5552 [arrow] (monkwire)
- Fix new clippy lints for Rust 1.77 #5544 [parquet] [arrow] (alamb)
- fix: correctly encode ticket #5543 [arrow] [arrow-flight] (freddieptf)
- feat: implemented with_field() for FixedSizeListBuilder #5541 [arrow] (istvan-fodor)
- Split arrow_cast::cast::list into it's own submodule #5537 [arrow] (monkwire)
- Bump black from 22.10.0 to 24.3.0 in /parquet/pytest #5535 [parquet] (dependabot[bot])
- Add OffsetBufferBuilder #5532 [arrow] (tustvold)
- Add IPC StreamDecoder #5531 [arrow] (tustvold)
- IPC format support for StringViewArray and BinaryViewArray #5525 [arrow] (XiangpengHao)
- parquet: Use specific error variant when codec is disabled #5521 [parquet] (progval)
- impl
From<ScalarBuffer<T>>
forVec<T>
#5518 [arrow] (mbrobbel)
* This Changelog was automatically generated by github_changelog_generator