Fix for finalize stress test failing with timeout on AWS

[grusev]

Reference Issues/PRs

What does this implement or fix?

The AWS storage operations are long consuming many times slower than LMDB. The goal is to have hundreds instead of thousands chunks for AWS. That will reduce both time and will still provide knowledge how finalizing behaves in slow environments

https://github.com/man-group/ArcticDB/actions/runs/12548177307/job/34987768673

------------------ generated xml file: /__w/_temp/pytest..xml ------------------
=========================== short test summary info ============================
FAILED tests/stress/arcticdb/version_store/test_stress_finalize_staged_data.py::test_finalize_monotonic_unique_chunks[real_s3-EncodingVersion.V1] - Failed: Timeout >3600.0s
FAILED tests/stress/arcticdb/version_store/test_stress_finalize_staged_data.py::test_finalize_monotonic_unique_chunks[real_s3-EncodingVersion.V2] - Failed: Timeout >3600.0s
==== 2 failed, 75 passed, 1 skipped, 40006 warnings in 11455.91s (3:10:55) =====

Any other comments?

Checklist

Checklist for code changes...

• Have you updated the relevant docstrings, documentation and copyright notice?
• Is this contribution tested against all ArcticDB's features?
• Do all exceptions introduced raise appropriate error messages?
• Are API changes highlighted in the PR description?
• Is the PR labelled as enhancement or bug so it appears in autogenerated release notes?

[maxim-morozov]

instead of putting ifs in the test making it less reliable to the environments, i would suggest to create fixtures and pass iterations to tests. like this you can create a separate test for amazonaws which will be much better visible for the failures and in test results

[grusev]

I am not sure that this is possible with current design of fixtures ... Currently I use fixture called 'basic_arctic_library'. It automatically creates 6 version of the test. 2 of which are S3 AWS with 2 different types of encodings.

We do not have actually a way to combine environments or select only environments we want from a fixtere that has been predefined with such. Or At least I am not aware how this can happen.

I will see what I can do with current set of fixtures we have. Otherwise combining couple of this and that in a new fixture results in huge pile of fixtures which again is going to be very complex to maintain

[maxim-morozov]

You can create fixture that returns library and the sizes in tuple or dictionary, which can be fed into this test. Fixtures can also create multiple params in tuple.

[grusev]

mixing fixture param along with non-fixture param is quite tricky. At least I could not make it work, although I úsed ''indirect'' parameter ...

thus got back to another much simpler way - refactored by dividing into 2 tests for clarity (lmdb and real_s3)

Now instead of 6 times we run it 2 times. Why?

• lmdb and real s3 are only needed 'mem' would be redundant. We test with very fast and very slow storages, no need to do other tests all the time

• encoding versions are not necessary I think at this point. Thus we test only with the default encoding

Thus the time to run tests is now 1/3

Additionally fixed type hints were I was not correct initially Iterrator[Type] for fixture although that passes type checks should be actually be Generator[Type, None, None]