Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(java-client): Introduce exponentialBackoff retry mechanism #1822

Merged
merged 10 commits into from
Mar 7, 2024

Conversation

Samunroyu
Copy link
Collaborator

@Samunroyu Samunroyu commented Jan 2, 2024

Introduces a retry mechanism into the Java client to enhance user
convenience and robustness.

It can be enabled by calling retryPolicy(), retryBaseInterval()
and retryMaxTimes() in when construct ClientOptions objects.

@Samunroyu Samunroyu changed the title feat(java-client): add client retry policy feat(java-client): Integrating a retry mechanism to java client Jan 2, 2024
@Samunroyu Samunroyu changed the title feat(java-client): Integrating a retry mechanism to java client feat(java-client): Integrating a retry mechanism into Java client Jan 2, 2024
@acelyc111
Copy link
Member

@acelyc111 acelyc111 requested a review from Apache9 January 3, 2024 12:40
private final int retryMaxTimes;

public ExponentialBackoffRetryPolicy(ClientOptions opts) {
this.retryBaseIntervalMs = opts.getRetryBaseInterval().toMillis();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add some comments and also some checks when setting retry base interval, as here we will convert the timeout to milliseconds, if user specify an interval less than 1 millisecond, we will not perform as expected...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to add some comments and also some checks when setting retry base interval, as here we will convert the timeout to milliseconds, if user specify an interval less than 1 millisecond, we will not perform as expected...

sure. if user specify an interval less than 1 millisecond, it will set to defaultintervalms.

@Samunroyu Samunroyu changed the title feat(java-client): Integrating a retry mechanism into Java client feat(java-client): Introduce retry mechanism into Java client Jan 19, 2024
@Apache9
Copy link
Contributor

Apache9 commented Jan 20, 2024

I tried rerun the failed job but it still failed...

@Samunroyu
Copy link
Collaborator Author

I tried rerun the failed job but it still failed...

the failed job is failed at start pegasus cluster. not java client unit test. Rerun one more time the job successful.

acelyc111
acelyc111 previously approved these changes Mar 1, 2024
@acelyc111 acelyc111 changed the title feat(java-client): Introduce retry mechanism into Java client feat(java-client): Introduce exponentialBackoff retry policy into Java client Mar 1, 2024
@acelyc111 acelyc111 changed the title feat(java-client): Introduce exponentialBackoff retry policy into Java client feat(java-client): Introduce exponentialBackoff retry mechanism Mar 1, 2024
long retryDelayMs =
Math.min(
timeoutMs > 3 ? timeoutMs / 3 : 1, TimeUnit.NANOSECONDS.toMillis(deadlineNanos - now));
return new RetryAction(RetryDecision.RETRY, Duration.ofMillis(retryDelayMs), "");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since here we also pass a Duration in, let's use nanos instead of millis?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since here we also pass a Duration in, let's use nanos instead of millis?

So we should allow the user to specify the time interval for nano units?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we change to use nanos here, then we do not need to change the code here if we allow users to specify timeouts with smaller unit than millis, like micros and nanos in the future.
Whether to allow users to specify timeout with unit smaller than millis can be discussed later.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we change to use nanos here, then we do not need to change the code here if we allow users to specify timeouts with smaller unit than millis, like micros and nanos in the future. Whether to allow users to specify timeout with unit smaller than millis can be discussed later.

I see. i already change the code.pls take a look one more time.

if (now >= deadlineNanos) {
return new RetryAction(RetryDecision.FAIL, Duration.ZERO, "request deadline reached");
}
long normalIntervalMs =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use nanos too.

Apache9
Apache9 previously approved these changes Mar 1, 2024
acelyc111
acelyc111 previously approved these changes Mar 1, 2024
empiredan
empiredan previously approved these changes Mar 1, 2024
@Samunroyu Samunroyu dismissed stale reviews from empiredan, acelyc111, and Apache9 via 5d029a6 March 1, 2024 10:12
@acelyc111 acelyc111 merged commit 0942c4a into apache:master Mar 7, 2024
13 checks passed
@Samunroyu Samunroyu deleted the dev/yjw/retry branch May 30, 2024 03:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants