Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Update to Apache Lucene 10 for 3.0.0 (#16366)
* Update to Apache Lucene 10 for 3.0.0 Signed-off-by: Andriy Redko <[email protected]> * More Lucene 10 build fixes Core compiles now. (Plugins/modules don't yet.) Major areas covered: 1. MultiTermQuery now takes RewriteMethod via constructor instead of setter. 2. Scorable no longer has docId. 3. ScoreCachingWrappingScorer now takes + returns LeafCollector. 4. MMapDirectory allows configuring preload per file (so we don't have to). 5. Concurrent search can now operate at sub-segment level (but we don't yet). Signed-off-by: Michael Froh <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andrew Ross <[email protected]> * Replace DocValuesFieldExistsQuery with FieldExistsQuery Signed-off-by: Andrew Ross <[email protected]> * Fix signature changes in o.a.l.search.BooleanQuery Signed-off-by: Andrew Ross <[email protected]> * Fix more compilation failures, update shadow Gradle plugin and fix loggerUsageCheck Gradle task Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix analysis-common compilation issues Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * More compilation fixes Signed-off-by: Andrew Ross <[email protected]> * More compilation fixes Signed-off-by: Andrew Ross <[email protected]> * Fix assemble tasks Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures, add CHANGELOG.md Signed-off-by: Andriy Redko <[email protected]> * Fix more compilation failures Signed-off-by: Andriy Redko <[email protected]> * Fixed Lucene latest codec name Signed-off-by: Andriy Redko <[email protected]> * Refactor preload tests in FsDirectoryFactoryTests Also more compilation fixes. Signed-off-by: Andrew Ross <[email protected]> * Fix more compilation failures Signed-off-by: Andrew Ross <[email protected]> * Fix precommit failures Signed-off-by: Andrew Ross <[email protected]> * Fix analysis test cases Signed-off-by: Andriy Redko <[email protected]> * Fix FieldData & ParentJoinAggregator ordinal traversal Signed-off-by: Andriy Redko <[email protected]> * Fix IpFieldMapper & CollapsingDocValuesSource ordinal traversal Signed-off-by: Andriy Redko <[email protected]> * Fix MultiValueMode, GlobalOrdinalValuesSource, BinaryRangeAggregator, DiversifiedOrdinalsSamplerAggregator, GlobalOrdinalsStringTermsAggregator & CardinalityAggregator ordinal traversal Signed-off-by: Andriy Redko <[email protected]> * Fix MultiValueModeTests, BinaryRangeAggregatorTests, IncludeExcludeTests & MissingValuesTests ordinal traversal Signed-off-by: Andriy Redko <[email protected]> * Fix ICUCollationKeywordFieldMapper & VectorFieldTests tests Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.common.xcontent.support.XContentMapValuesTests test cases Signed-off-by: Andriy Redko <[email protected]> * Fix FunctionScoreQuery::CustomBoostFactorWeight::scorerSupplier that may throw NPE Signed-off-by: Andriy Redko <[email protected]> * Fix CodecTests and non-concurrent ContextIndexSearcher::search implementation for aggregations Signed-off-by: Andriy Redko <[email protected]> * Fixed GlobalOrdinalMapping ordinals iteration and SortedWiderNumericSortField implementation of missingValueAsComparableLong Signed-off-by: Andriy Redko <[email protected]> * Fix FieldData and SortedSetDocValuesWriterWrapperTests Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.repositories.azure.AzureBlobStoreRepositoryTests test cases Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.search.aggregations.bucket.composite.CompositeAggregatorTests test cases Signed-off-by: Andriy Redko <[email protected]> * Fix MultiValueMode ordinal traversal Signed-off-by: Andriy Redko <[email protected]> * Fix GlobalOrdinalMapping and GlobalOrdinalsStringTermsAggregator ordinal traversals Signed-off-by: Andriy Redko <[email protected]> * Fix QueryStringQueryBuilderTests and QueryStringQueryBuilder fuzzy query default rewrite method detection logic Signed-off-by: Andriy Redko <[email protected]> * Fix StarTree*DocValueFormatTests The Composite912DocValuesReader was not honoring the IOContext correctly. The metadata file should be opened with READONCE and immediately be closed (on the same thread). The data file needs to be opened with ReadAdvice.NORMAL to ensure that it's closeable on another thread if necessary. Signed-off-by: Michael Froh <[email protected]> * Fix various MultiTermQuery-related tests Now that we use CONSTANT_SCORE_BLENDED_REWRITE by default for most MTQs, we need to update the tests. Also FuzzyQuery should default to TopTermsBlendedFreqScoringRewrite (taken from Lucene). Signed-off-by: Michael Froh <[email protected]> * Fix Spotless Signed-off-by: Michael Froh <[email protected]> * Fix RegEx automaton processing using Operations.determinize(...) Signed-off-by: Andriy Redko <[email protected]> * Fix compilation issues (post merge) Signed-off-by: Andriy Redko <[email protected]> * Fix UnsignedLongComparator::sortableBytesToLong encoding Signed-off-by: Andriy Redko <[email protected]> * Fix HalfFloatComparator::sortableBytesToLong and HalfFloatComparator::missingValueAsComparableLong Signed-off-by: Andriy Redko <[email protected]> * Fix infinite recursion in CollapsingTopDocsCollectorTests Signed-off-by: Michael Froh <[email protected]> * Fix SingleDimensionValuesSourceTests Can no longer mock IndexReader as it's sealed. Signed-off-by: Michael Froh <[email protected]> * Fix a few more MultiTermQuery-related test failures Signed-off-by: Michael Froh <[email protected]> * Fixed QueryPhaseTests#minScoreOptimisation by not delegating setWeight to underlying delegate Signed-off-by: expani <[email protected]> * Fix DerivedFieldMapperQueryTests Needed to add doc values to date and IP fields. Signed-off-by: Michael Froh <[email protected]> * Fix SortedNumericDocValuesWriterWrapperTests Lucene's SortedNumericDocValuesWriter no longer throws an exception if you try to iterate past a document's last value. (Instead, it happily leaks the next document's value.) Signed-off-by: Michael Froh <[email protected]> * Fix IpRangeAggregatorTests Rewrote a for-loop in BinaryRangeAggregator to be less confusing. As a side-effect, it fixed the test. Signed-off-by: Michael Froh <[email protected]> * Fix IpFieldMapperTests I broke these tests as a side-effect for my fix to DerivedFieldMapperQueryTests. Signed-off-by: Michael Froh <[email protected]> * Fix MissingValues ordinal traversal and MultiOrdinalsTests tests Signed-off-by: Andriy Redko <[email protected]> * Prevent cascading test failures in SearchServiceTests I'm not really fixing the failing tests, but ensuring that the two tests that fail don't cause a bunch of other tests to fail by leaving transient settings around. (Clean up the transient settings even if those tests fail.) Signed-off-by: Michael Froh <[email protected]> * Fix DerivedFieldTypeTests I broke these when fixing DerivedFieldMapperQueryTests. Signed-off-by: Michael Froh <[email protected]> * Fix TransferManagerRemoteDirectoryReaderTests Needed to update mocked calls. Signed-off-by: Michael Froh <[email protected]> * Add missing javadoc Signed-off-by: Andriy Redko <[email protected]> * Uploaded Index generated from Lucene 8.x ( OpenSearch 1.3.0 ) to be used by UnitTests Signed-off-by: expani <[email protected]> * Moved path to a constant Signed-off-by: expani <[email protected]> * Update Apache Lucene to 10.1.0 Signed-off-by: Andriy Redko <[email protected]> * Fixed unit tests for lucene 10 upgrade Signed-off-by: expani <[email protected]> * Fix posting format for CompletionFieldMapper Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.search.SearchServiceTests test cases Signed-off-by: Andriy Redko <[email protected]> * Removed extra iteration of DISI and added stored field option to bypass modified check in lucene Field Signed-off-by: expani <[email protected]> * Changed to use advance and init to use docId() instead of nextDoc() Signed-off-by: expani <[email protected]> * Increased number of indexed docs to match segment count Signed-off-by: expani <[email protected]> * Reduced segment count to match assertion instead of increasing docs Signed-off-by: expani <[email protected]> * Revert to make 90_search_after integ pass as hits are not shown Signed-off-by: expani <[email protected]> * Making Half Float failures work with it's conversions Signed-off-by: expani <[email protected]> * Changing max long to signed from unsigned Signed-off-by: expani <[email protected]> * Addressed PR Comments Signed-off-by: expani <[email protected]> * Trying another approach for gracefully handling unsigned long conversions Signed-off-by: expani <[email protected]> * Simplified unsigned to signed long conversion Signed-off-by: expani <[email protected]> * Reverted changes as need to think more on this Signed-off-by: expani <[email protected]> * Fix azure-fixture & s3-fixture Docker images Signed-off-by: Andriy Redko <[email protected]> * Fix 'search/90_search_after/numeric skipping logic with competitive missing value' test cases Signed-off-by: Andriy Redko <[email protected]> * Fix gcs-fixture Docker images Signed-off-by: Andriy Redko <[email protected]> * Updated to use modern Romanian Unicode Signed-off-by: expani <[email protected]> * Fix org.opensearch.wildfly.WildflyIT test cases Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.search.aggregations.bucket.MinDocCountIT test cases Signed-off-by: Andriy Redko <[email protected]> * Fix CardinalityAggregator.OrdinalsCollector ordinals traversal Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.search.query.ScriptScoreQueryIT test cases Signed-off-by: Andriy Redko <[email protected]> * Fix org.opensearch.search.scriptfilter.ScriptQuerySearchIT test cases Signed-off-by: Andriy Redko <[email protected]> * Fix UnsignedLongComparator top/bottom as comparable long conversions Signed-off-by: Andriy Redko <[email protected]> * Fix post-merge (with main) compilation issues Signed-off-by: Andriy Redko <[email protected]> * Fix UnsignedLongComparator with respect to sortable bytes to long conversion Signed-off-by: Andriy Redko <[email protected]> * Fix FlatObjectFieldTypeTests test cases Signed-off-by: Andriy Redko <[email protected]> --------- Signed-off-by: Andriy Redko <[email protected]> Signed-off-by: Michael Froh <[email protected]> Signed-off-by: Andrew Ross <[email protected]> Signed-off-by: expani <[email protected]> Co-authored-by: Michael Froh <[email protected]> Co-authored-by: Andrew Ross <[email protected]> Co-authored-by: expani <[email protected]>
- Loading branch information