Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add multithreading support to Multi* geometries #1265

Merged
merged 8 commits into from
Nov 6, 2024
Merged

Add multithreading support to Multi* geometries #1265

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
32e0143
Add multithreading support to Multi* geometries
urschrei Nov 6, 2024
6df4c2f
Update multithreading flag docs
urschrei Nov 6, 2024
1137aef
Add multithreading feature to geo-types top-level docs
urschrei Nov 6, 2024
df64658
Fix feature name
urschrei Nov 6, 2024
76200e7
Remove needless comments
urschrei Nov 6, 2024
2c2141d
Add some sanity / ergonomic tests for the multithreading impls
urschrei Nov 6, 2024
7f6b16a
Feature-gate multithreading tests
urschrei Nov 6, 2024
416b5c8
Omit let bindings hated by Clippy
urschrei Nov 6, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions geo-types/CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Unreleased
* Add rstar compatibility for MultiPolygon
* Add multi-threading support to Multi* geometries.
* Feature-gated ('multithreading'), disabled by default, enabled by default when geo-types is used by geo

## 0.7.13

Expand Down
2 changes: 2 additions & 0 deletions geo-types/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ edition = "2021"
[features]
default = ["std"]
std = ["approx?/std", "num-traits/std", "serde?/std"]
multithreading = ["rayon"]
# Prefer `use-rstar` feature rather than enabling rstar directly.
# rstar integration relies on the optional approx crate, but implicit features cannot yet enable other features.
# See: https://doc.rust-lang.org/nightly/cargo/reference/unstable.html#namespaced-features
Expand All @@ -25,6 +26,7 @@ use-rstar_0_11 = ["rstar_0_11", "approx"]
use-rstar_0_12 = ["rstar_0_12", "approx"]

[dependencies]
rayon = { version = "1.10.0", optional = true }
approx = { version = ">= 0.4.0, < 0.6.0", optional = true, default-features = false }
arbitrary = { version = "1.2.0", optional = true }
num-traits = { version = "0.2", default-features = false, features = ["libm"] }
Expand Down
32 changes: 32 additions & 0 deletions geo-types/src/geometry/multi_line_string.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ use alloc::vec::Vec;
#[cfg(any(feature = "approx", test))]
use approx::{AbsDiffEq, RelativeEq};
use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of
/// [`LineString`s](line_string/struct.LineString.html). Can
Expand Down Expand Up @@ -118,6 +120,36 @@ impl<T: CoordNum> MultiLineString<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiLineString<T> {
type Item = LineString<T>;
type Iter = rayon::vec::IntoIter<LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The consuming into implementation makes sense to me.

And I can see how the ref and mut flavors below provide necessary non-consuming functionality. But what does it look like to use it? Can you include some example code in your PR that uses these new methods to make the new interface easier to evaluate?

My guess is that you'd need to do something like

(& my_multi_line_string).into_par_iter()
and
(&mut my_multi_line_string).into_par_iter()
?

Which is a bit rough. I'm afraid people will just end up my_multi_line_string.clone().into_par_iter() out of confusion.

Would it get better if we got rid of these two ref/mut implementations and instead implemented rayon::IntoParallelRefIterator and rayon::IntoParallelRefMutIterator which seem purpose built for this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. These work with the current impls:

    #[test]
    fn test_multithreading_linestring() {
        let multi: MultiLineString<i32> = wkt! {
            MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
        };
        let mut multimut: MultiLineString<i32> = wkt! {
            MULTILINESTRING((0 0,2 0,1 2,0 0), (10 10,12 10,11 12,10 10))
        };
        let _ = multi.par_iter().for_each(|_p| ());
        let _ = multimut.par_iter_mut().for_each(|_p| ());
        let _ = &multi.into_par_iter().for_each(|_p| ());
        let _ = &mut multimut.par_iter_mut().for_each(|_p| ());
    }

I admittedly didn't see the rayon::IntoParallelRefIterator and just adapted our existing iterators to rayon::IntoParallelIterator, but I'm confused by

This trait is automatically implemented for I where &I: IntoParallelIterator. In most cases, users will want to implement IntoParallelIterator rather than implement this trait directly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh splendid! They're already implemented by the blanket impl!

I'm glad that users will be able to do:

let _ = multi.par_iter().for_each(|_p| ());
let _ = multimut.par_iter_mut().for_each(|_p| ());

vs. the slightly more arcane:

let _ = &multi.into_par_iter().for_each(|_p| ());
let _ = &mut multimut.into_par_iter().for_each(|_p| ());

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh yeah the latter two were just to make sure "everything" worked. Thanks for the sanity check!

impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiLineString<T> {
type Item = &'a LineString<T>;
type Iter = rayon::slice::Iter<'a, LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiLineString<T> {
type Item = &'a mut LineString<T>;
type Iter = rayon::slice::IterMut<'a, LineString<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

#[cfg(any(feature = "approx", test))]
impl<T> RelativeEq for MultiLineString<T>
where
Expand Down
32 changes: 32 additions & 0 deletions geo-types/src/geometry/multi_point.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,8 @@ use approx::{AbsDiffEq, RelativeEq};
use alloc::vec;
use alloc::vec::Vec;
use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of [`Point`s](struct.Point.html). Can
/// be created from a `Vec` of `Point`s, or from an
Expand Down Expand Up @@ -85,6 +87,36 @@ impl<'a, T: CoordNum> IntoIterator for &'a mut MultiPoint<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiPoint<T> {
type Item = Point<T>;
type Iter = rayon::vec::IntoIter<Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiPoint<T> {
type Item = &'a Point<T>;
type Iter = rayon::slice::Iter<'a, Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiPoint<T> {
type Item = &'a mut Point<T>;
type Iter = rayon::slice::IterMut<'a, Point<T>>;

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

impl<T: CoordNum> MultiPoint<T> {
pub fn new(value: Vec<Point<T>>) -> Self {
Self(value)
Expand Down
33 changes: 33 additions & 0 deletions geo-types/src/geometry/multi_polygon.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ use alloc::vec;
use alloc::vec::Vec;
#[cfg(any(feature = "approx", test))]
use approx::{AbsDiffEq, RelativeEq};

use core::iter::FromIterator;
#[cfg(feature = "multithreading")]
use rayon::prelude::*;

/// A collection of [`Polygon`s](struct.Polygon.html). Can
/// be created from a `Vec` of `Polygon`s, or from an
Expand Down Expand Up @@ -75,6 +78,36 @@ impl<'a, T: CoordNum> IntoIterator for &'a mut MultiPolygon<T> {
}
}

#[cfg(feature = "multithreading")]
impl<T: CoordNum + Send> IntoParallelIterator for MultiPolygon<T> {
type Item = Polygon<T>;
type Iter = rayon::vec::IntoIter<Polygon<T>>; // Parallel iterator type for Vec<Polygon<T>>

fn into_par_iter(self) -> Self::Iter {
self.0.into_par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Sync> IntoParallelIterator for &'a MultiPolygon<T> {
type Item = &'a Polygon<T>;
type Iter = rayon::slice::Iter<'a, Polygon<T>>; // Parallel iterator type for a slice of Polygon<T>

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter()
}
}

#[cfg(feature = "multithreading")]
impl<'a, T: CoordNum + Send + Sync> IntoParallelIterator for &'a mut MultiPolygon<T> {
type Item = &'a mut Polygon<T>;
type Iter = rayon::slice::IterMut<'a, Polygon<T>>; // Parallel iterator type for a mutable slice of Polygon<T>

fn into_par_iter(self) -> Self::Iter {
self.0.par_iter_mut()
}
}

impl<T: CoordNum> MultiPolygon<T> {
/// Instantiate Self from the raw content value
pub fn new(value: Vec<Polygon<T>>) -> Self {
Expand Down
2 changes: 1 addition & 1 deletion geo/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ default = ["earcutr", "spade", "multithreading"]
use-proj = ["proj"]
proj-network = ["use-proj", "proj/network"]
use-serde = ["serde", "geo-types/serde"]
multithreading = ["i_overlay/allow_multithreading"]
multithreading = ["i_overlay/allow_multithreading", "geo-types/multithreading"]

[dependencies]
earcutr = { version = "0.4.2", optional = true }
Expand Down
Loading