Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

minimal upgrade to PyO3 0.23 (ignoring deprecations) #1556

Merged
merged 2 commits into from
Nov 26, 2024
Merged

Conversation

davidhewitt
Copy link
Contributor

Change Summary

This updates PyO3 to 0.23. To keep things reviewable, I have started here by ignoring deprecations and just fixing the immediate build failures caused by the version bump.

They fit mostly into two categories:

  • PyO3 0.23 introduced a new trait IntoPyObject and changed many methods to use it instead of IntoPy<PyObject> and ToPyObject. This PR tries to avoid switching traits as much as possible, making the changes only where necessary to fix compile failures.
  • PyO3 0.23 requires the #[pyclass] types to be Sync (i.e. thread-safe), so I had to replace RefCell with a Mutex in a couple of internal places.

Needs jiter release pushed before this can be merged.

Related issue number

Related to #1555

Checklist

  • Unit tests for the changes exist
  • Documentation reflects the changes where applicable
  • Pydantic tests pass with this pydantic-core (except for expected changes)
  • My PR is ready to review, please add a comment including the phrase "please review" to assign reviewers

Copy link

codspeed-hq bot commented Nov 26, 2024

CodSpeed Performance Report

Merging #1556 will degrade performances by 16.85%

Comparing dh/pyo3-0.23-1 (9613a64) with main (334e3df)

Summary

❌ 2 regressions
✅ 153 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark main dh/pyo3-0.23-1 Change
test_json_any_list_int 195.7 µs 223 µs -12.26%
test_to_json_list_of_lists 1.7 ms 2.1 ms -16.85%

@davidhewitt
Copy link
Contributor Author

Perf regressions are only part of the varied story; some of the benches showing up to 12% improvement. Overall in PyO3 0.23 because of the freethreading changes as well as IntoPyObject there have been a bunch of adjustments all over the place, and I'm optimistic that 0.24 and so on will have a bunch of optimisations landed on top of the new code.

In summary I am not super worried about the regressions, they probably are due to the Mutex use here and I would love to rework serializer state anyway (maybe one for @sydney-runkle with some design collaboration).

Copy link
Member

@samuelcolvin samuelcolvin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good in principle, I haven't reviewed every line.

@davidhewitt davidhewitt marked this pull request as ready for review November 26, 2024 22:39
@davidhewitt davidhewitt enabled auto-merge (squash) November 26, 2024 22:39
@davidhewitt davidhewitt merged commit 5c96bec into main Nov 26, 2024
28 checks passed
@davidhewitt davidhewitt deleted the dh/pyo3-0.23-1 branch November 26, 2024 22:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants