From fc81a906da88d99c7ea09ffd9fec92ff81925cf6 Mon Sep 17 00:00:00 2001 From: Stephen Crawford <65832608+scrawfor99@users.noreply.github.com> Date: Sun, 28 Apr 2024 19:44:34 -0400 Subject: [PATCH] Restore deleted section (#13415) Signed-off-by: Stephen Crawford --- CONTRIBUTING.md | 11 +++++++++++ 1 file changed, 11 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bce6ca0d49294..0ec0abe535dd0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -9,6 +9,7 @@ - [Changelog](#changelog) - [Review Process](#review-process) - [Tips for Success](#tips) +- [Troubleshooting Failing Builds](#troubleshooting-failing-builds) # Contributing to OpenSearch @@ -180,3 +181,13 @@ We have a lot of mechanisms to help expedite towards an accepted PR. Here are so In general, adding more guardrails to your changes increases the likelihood of swift PR acceptance. We can always relax these guard rails in smaller followup PRs. Reverting a GA feature is much more difficult. Check out the [DEVELOPER_GUIDE](./DEVELOPER_GUIDE.md#submitting-changes) for more useful tips. +## Troubleshooting Failing Builds + +The OpenSearch testing framework offers many capabilities but exhibits significant complexity (it does lot of randomization internally to cover as many edge cases and variations as possible). Unfortunately, this posses a challenge by making it harder to discover important issues/bugs in straightforward way and may lead to so called flaky tests - the tests which flip randomly from success to failure without any code changes. + +If your pull request reports a failing test(s) on one of the checks, please: +- look if there is an existing [issue](https://github.com/opensearch-project/OpenSearch/issues) reported for the test in question +- if not, please make sure this is not caused by your changes, run the failing test(s) locally for some time +- if you are sure the failure is not related, please open a new [bug](https://github.com/opensearch-project/OpenSearch/issues/new?assignees=&labels=bug%2C+untriaged&projects=&template=bug_template.md&title=%5BBUG%5D) with `flaky-test` label +- add a comment referencing the issue(s) or bug report(s) to your pull request explaining the failing build(s) +- as a bonus point, try to contribute by fixing the flaky test(s)