Add logging for tests in RemoteStoreStatsIT to catch assertion failure cause

[Poojita-Raj]

Description

In RemoteStoreStatsIT, for tests (1) testDownloadStatsCorrectnessSinglePrimarySingleReplica and (2) testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards, we see an assertion failure in a similar spot.

The failing assertions check that a successful upload AND no failures took place for an index with remote store enabled. This is before any docs are indexed and after ensureGreen.

Even after days of running this test, we weren't able to reproduce it. Judging by the point of assertion failure, the hypothesis is that we see this failure when the upload didn't take place in time after running refresh. To test this, we are adding in a lightweight log statement before these assertions so we can confirm this is the cause if it fails again.

Previous build failures:

5 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimarySingleReplica (30610,30654,31178,31469,31558)

5 org.opensearch.remotestore.RemoteStoreStatsIT.testDownloadStatsCorrectnessSinglePrimaryMultipleReplicaShards (31085,31091,31140,31369,31441)

Related Issues

#10152
#10193

Check List

• New functionality includes testing.
  • All tests pass
• New functionality has been documented.
  • New functionality has javadoc added
• Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
• Commits are signed per the DCO using --signoff
• Commit changes are listed out in CHANGELOG.md file (See: Changelog)
• Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[mch2]

Can we add TestLogging annotation here as well for the appropriate packages?

[github-actions]

Compatibility status:

Checks if related components are compatible with change 3581b63

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/sql.git]

[github-actions]

✅ Gradle check result for d731afd: SUCCESS

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (16d457d) 71.44% compared to head (3581b63) 71.37%.
Report is 6 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main   #11734      +/-   ##
============================================
- Coverage     71.44%   71.37%   -0.08%     
+ Complexity    59254    59212      -42     
============================================
  Files          4909     4909              
  Lines        278427   278427              
  Branches      40460    40460              
============================================
- Hits         198914   198718     -196     
- Misses        63005    63215     +210     
+ Partials      16508    16494      -14     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[github-actions]

❕ Gradle check result for 3581b63: UNSTABLE

• TEST FAILURES:

      1 org.opensearch.remotestore.RemoteIndexPrimaryRelocationIT.testPrimaryRelocationWhileIndexing

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.