Skip to content

Commit

Permalink
Merge pull request #355 from yrodiere/smaller-bulk
Browse files Browse the repository at this point in the history
Reduce side of bulks when indexing in dev, tests and prod
  • Loading branch information
yrodiere authored Nov 7, 2024
2 parents 5eb509c + c24d180 commit 4e3791a
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions src/main/resources/application.properties
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@ quarkus.hibernate-search-standalone.elasticsearch.version=opensearch:2.18
quarkus.elasticsearch.devservices.image-name=opensearch-custom:${maven.version.opensearch}
quarkus.elasticsearch.devservices.java-opts=${PROD_OPENSEARCH_JAVA_OPTS}
# Limit parallelism of indexing, because OpenSearch can only handle so many documents in its buffers.
# This leads to at most 8*20=160 documents being indexed in parallel, which should be plenty
# This leads to at most 8*12=96 documents being indexed in parallel, which should be plenty
# given how large our documents can be.
INDEXING_QUEUE_COUNT=8
INDEXING_BULK_SIZE=20
INDEXING_BULK_SIZE=12
quarkus.hibernate-search-standalone.elasticsearch.indexing.queue-count=${INDEXING_QUEUE_COUNT}
quarkus.hibernate-search-standalone.elasticsearch.indexing.max-bulk-size=${INDEXING_BULK_SIZE}
# We need to apply a custom OpenSearch mapping to exclude very large fields from the _source
Expand Down

0 comments on commit 4e3791a

Please sign in to comment.