Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support giving up on failed individual bulk requests after a number of retries #909

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

support giving up on failed individual bulk requests after a number of retries #909

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions lib/logstash/outputs/elasticsearch/common.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -229,6 +229,8 @@ def retrying_submit(actions)

sleep_interval = @retry_initial_interval

failed_retries = 0

while submit_actions && submit_actions.length > 0

# We retry with whatever is didn't succeed
Expand All @@ -247,6 +249,16 @@ def retrying_submit(actions)
# Everything was a success!
break if !submit_actions || submit_actions.empty?

failed_retries += 1

if @retry_max_failures > 0 && failed_retries == @retry_max_failures
@logger.warn("Giving up on individual bulk actions that failed or were rejected by the previous bulk request.",
:actions => submit_actions.map { |action_type, params, event| [action_type, params, event.to_s] },
:retries => failed_retries,
:count => submit_actions.size)
break
end

# If we're retrying the action sleep for the recommended interval
# Double the interval for the next time through to achieve exponential backoff
Stud.stoppable_sleep(sleep_interval) { @stopping.true? }
Expand Down
3 changes: 3 additions & 0 deletions lib/logstash/outputs/elasticsearch/common_configs.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,9 @@ def self.included(mod)
# Set max interval in seconds between bulk retries.
mod.config :retry_max_interval, :validate => :number, :default => 64

# The number of times we should retry on failed individual bulk actions (0 = infinity)
mod.config :retry_max_failures, :validate => :number, :default => 0

# The number of times Elasticsearch should internally retry an update/upserted document
# See the https://www.elastic.co/guide/en/elasticsearch/guide/current/partial-updates.html[partial updates]
# for more info
Expand Down