Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #124 Record iteration when recording only zeros #125

Merged
merged 3 commits into from
Nov 11, 2023

Conversation

Carreau
Copy link
Contributor

@Carreau Carreau commented Aug 19, 2023

Fixes #124

As pointed in the comment in the corresponding issues, we want to ensure we trigger the picker logic at least once to keep the invariant of the loop, that is to say that the picking logic has been executed for last_picked_index, which is not the case for zero when there are only zeros.

I added a test that collect the record and check for their count.

Would it be better to check for the following:

assert_eq!(h.iter_recorded().collect::<Vec<_>>(), vec![IterationValue::<u64>::new(0, 1.0, 1.0, 1, 1)]);

It is easy to assume that h.iter_recorded().collect::<Vec<_>>().len() == h.len() and simplify the added test, and I'm unsure how to convey that we really want to collect and check the length of all the record.

@Carreau
Copy link
Contributor Author

Carreau commented Aug 19, 2023

I would have preferred a fix that make the initial value of last_picked_index and invalid index, being a usize it can't be negative, and I don't see a performant and api compatible way to make it something else – but I'm also fairly new to rust so no doubt someone can come up with something better.

--

BTW @jonhoo, if you are reading this, no need to apologize for the time it took you reach this PR, as a Python open source maintainer myself, I know the felling. I'm also personally learning to switch my wording from "Sorry for the delay" to "Thanks for you patience", and it has – I think – made a small but non negligible changes in both the responses I got, and decreased the pressing feeling of need to respond.

@codecov
Copy link

codecov bot commented Aug 22, 2023

Codecov Report

Merging #125 (7429b51) into main (bdfad24) will not change coverage.
The diff coverage is 100.00%.

Files Coverage Δ
src/iterators/mod.rs 83.47% <100.00%> (ø)

@jonhoo
Copy link
Collaborator

jonhoo commented Aug 25, 2023

Thanks for the thorough explanation! I think you're right that more is not to blame (and I agree with you it should probably be renamed one day). I think the way to go here is to make last_picked_index an Option<usize>. That way you can explicitly indicate that it hasn't been set yet. It's not a part of the public API, so you don't have to worry about breaking backwards compatibility. What's also nice is that once you change the type of that value in the struct HistogramIterator type definition, the compiler will guide you to all the places where the code has to be updated to reflect this change!

Fixes HdrHistogram#124

As pointed in the comment in the corresponding issues, we want to ensure
we trigger the picker logic at least once to keep the invariant of the
loop, that is to say that the picking logic has been executed at least
once for last_picked_index, which is not the case for zero when there
are only zeros.

To do so we use and Option, and initialize to None, to make sure the
pick logic is ran at least once.

I added a test that collect the record and check for their count.

Would it be better to check for the following:

```rust
assert_eq!(h.iter_recorded().collect::<Vec<_>>(), vec![IterationValue::<u64>::new(0, 1.0, 1.0, 1, 1)]);
```

It is easy to assume that `h.iter_recorded().collect::<Vec<_>>().len() == h.len()`
and simplify the added test, and I'm unsure how to convey that we
_really_ want to collect and check the length of all the record.
@Carreau
Copy link
Contributor Author

Carreau commented Aug 25, 2023

an Option [...] It's not a part of the public API

Oh, yeah, thanks I thought about it, but still sometime forget that structs fields are private by default, and wasn't sure about the performance implications.

@Carreau Carreau closed this Aug 25, 2023
@Carreau Carreau reopened this Aug 25, 2023
@Carreau
Copy link
Contributor Author

Carreau commented Aug 25, 2023

Ubuntu beta failed, I can't rerun a single test, so closing and reopening.

@Carreau
Copy link
Contributor Author

Carreau commented Oct 17, 2023

Going through my list of opened PRs, is there anything more I can do to help here ?

Thanks !

Copy link
Collaborator

@jonhoo jonhoo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, this looks good, thank you!

tests/histogram.rs Outdated Show resolved Hide resolved
@jonhoo jonhoo merged commit 6de1b9b into HdrHistogram:main Nov 11, 2023
15 of 16 checks passed
@jonhoo
Copy link
Collaborator

jonhoo commented Nov 11, 2023

Released in 7.5.3 🎉

Carreau added a commit to Carreau/HdrHistogram_rust that referenced this pull request Nov 11, 2023
In
HdrHistogram#125 (comment)
a simplification was made without discussion:

```diff
-     h.iter_recorded().collect::<Vec<_>>().len()
+     h.iter_recorded().count()
```

This is incorrect as what this test actually tests was that the iterator
has the correct length. In a perfect world both should be identical but
the PR fix a case where this was not the case as some items would be
skipped over.

In this I revert the change, add a comment and a second (redundant)
check to make sure this is not re-simplified later.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Error when iterating over histogram with only zeros
2 participants