-
Notifications
You must be signed in to change notification settings - Fork 38
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Ranges::from_iter
, Ranges:from_unsorted
and document ranges invariants
#273
Conversation
## Motivation https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232 needs to modify an existing `Ranges`. We don't want to allow direct access to ranges since this allows breaking the version-ranges invariants. This led me to two questions: How do we best construct ranges from a number of versions (#249), does the user need to hold up the invariants or do we sort and merge the given ranges? And: What are our invariants? Given my previous profiling, construction ranges isn't nearly noticeable in benchmarks (but i'm gladly proven otherwise), the input ranges are too small and we do orders of magnitude more ranges constructions in the pubgrub algorithm itself, so the primary problem of https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232 and #249 is ergonomics and zero-to-low overhead construction is a secondary problem, one that i'm tackling only because it would be inconsistent to have a slow API in pubgrub. Currently, we don't have a method for constructing ranges from multiple segments and this makes for an unergonomic API. ## Invariants For the invariants, we need 1), but i'm not clear if we also need 2) and 3) as strong as they are. They are currently invariants in the version ranges, but we could weaken them. What i like about them is that they imply that any instance of ranges is "fully reduced", it can't be simplified further (without knowning the actual versions available). 1. The segments are sorted, from lowest to highest (through `Ord`). 2. Each segment contains at least one version (start < end). 3. There is at least one version between two segments. ## API I added two functions: A `from_iter` where the user has to pass valid segments. This wants to be a `try_collect`, but that's still unstable. It is targeted at https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232, which currently does this in a less secure fashion. There, you'd replace `iter_mut().for_each(...)` with an `.into_iter().map(...).collect()` (With the `IntoIter` i've also added this shouldn't even be an allocation; i'm optimizing this too for consistency's sake but i'd be surprised if it mattered to users). The other is `from_unsorted` which is ergonomics and could be used for https://github.com/astral-sh/uv/blob/0335efd0a901ee2351e663d0488b1ce87fd4cc80/crates/pep508-rs/src/marker/algebra.rs#L638-L645 and https://github.com/astral-sh/uv/blob/e1a8beb64b9f6eb573b3af616ffff680c282a316/crates/uv-resolver/src/pubgrub/report.rs#L1098-L1102 from #249. The user passes arbitrary segments and we're merging them into valid ranges. Please discuss :)
See pubgrub-rs#273, it's split out from that PR, for https://github.com/astral-sh/uv/pull/8797/files. Closes #33
Use astral-sh/pubgrub#34 (pubgrub-rs/pubgrub#273) for safer ranges modification that can't break pubgrub invariants. In the process, I removed the `&mut` references in favor of a direct immutable-to-immutable mapping.
Use astral-sh/pubgrub#34 (pubgrub-rs/pubgrub#273) for safer ranges modification that can't break pubgrub invariants. In the process, I removed the `&mut` references in favor of a direct immutable-to-immutable mapping.
Use astral-sh/pubgrub#34 (pubgrub-rs/pubgrub#273) for safer ranges modification that can't break pubgrub invariants. In the process, I removed the `&mut` references in favor of a direct immutable-to-immutable mapping.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These all look like useful changes!
@Eh2406 what do you think about the invariants and about the functions in general? I've also been thinking that if we handroll the from unsorted variant, we could fast-path already matching inputs so we may not need the panicking function? |
I think those are the invariants we currently maintain. The reason we need a "fully reduced" form or at least a canonicalised form is so that The new functionality seems extremely useful. |
Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249
Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249
Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249
Thank you for all the discussion! I've split this into four PRs now: |
Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249
Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249
* Add `FromIter` for `Ranges` Add a method to construct ranges from an iterator of arbitrary segments. This allows to `.collect()` an iterator of tuples of bounds. This is more ergonomic than folding the previous ranges with the next segment each time, and also faster. Split out from #273 Closes astral-sh#33 Fixes #249 * Fix ascii art alignment * Fix algorithm with new proptest * Sorting comment * Review
Motivation
https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232 needs to modify an existing
Ranges
. We don't want to allow direct access to ranges since this allows breaking the version-ranges invariants. This led me to two questions: How do we best construct ranges from a number of versions (#249), does the user need to hold up the invariants or do we sort and merge the given ranges? And: What are our invariants?Given my previous profiling, construction ranges isn't nearly noticeable in benchmarks (but i'm gladly proven otherwise), the input ranges are too small and we do orders of magnitude more ranges constructions in the pubgrub algorithm itself, so the primary problem of https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232 and #249 is ergonomics and zero-to-low overhead construction is a secondary problem, one that i'm tackling only because it would be inconsistent to have a slow API in pubgrub. Currently, we don't have a method for constructing ranges from multiple segments and this makes for an unergonomic API.
Invariants
For the invariants, we need 1), but i'm not clear if we also need 2) and 3) as strong as they are. They are currently invariants in the version ranges, but we could weaken them. What i like about them is that they imply that any instance of ranges is "fully reduced", it can't be simplified further (without knowning the actual versions available).
Ord
).API
I added two functions: A
from_iter
where the user has to pass valid segments. This wants to be atry_collect
, but that's still unstable. It is targeted at https://github.com/astral-sh/uv/pull/8797/files#diff-17170cd58cc6474d6c6ee0413af4b3b28610aca267bbe7863411627219c26549R232, which currently does this in a less secure fashion. There, you'd replaceiter_mut().for_each(...)
with an.into_iter().map(...).collect()
(With theIntoIter
i've also added this shouldn't even be an allocation; i'm optimizing this too for consistency's sake but i'd be surprised if it mattered to users). The other isfrom_unsorted
which is ergonomics and could be used for https://github.com/astral-sh/uv/blob/0335efd0a901ee2351e663d0488b1ce87fd4cc80/crates/pep508-rs/src/marker/algebra.rs#L638-L645 and https://github.com/astral-sh/uv/blob/e1a8beb64b9f6eb573b3af616ffff680c282a316/crates/uv-resolver/src/pubgrub/report.rs#L1098-L1102 from #249. The user passes arbitrary segments and we're merging them into valid ranges.Please discuss :)