Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add no_std support to bevy_reflect #16256

Merged
merged 13 commits into from
Dec 5, 2024

Conversation

bushrat011899
Copy link
Contributor

@bushrat011899 bushrat011899 commented Nov 6, 2024

Objective

Solution

  • Added std feature (enabled by default)

Testing

  • CI
  • cargo check -p bevy_reflect --no-default-features --target "x86_64-unknown-none"
  • UEFI demo application runs with this branch of bevy_reflect, allowing derive(Reflect)

Notes

  • The spin crate has been included to provide RwLock and Once (as an alternative to OnceLock) when the std feature is not enabled. Another alternative may be more desirable, please provide feedback if you have a strong opinion here!
  • Certain items (Box, String, ToString) provided by alloc have been added to __macro_exports as a way to avoid alloc vs std namespacing. I'm personally quite annoyed that we can't rely on alloc as a crate name in std environments within macros. I'd love an alternative to my approach here, but I suspect it's the least-bad option.
  • I would've liked to have an alloc feature (for allocation-free bevy_reflect), unfortunately, erased_serde unconditionally requires access to Box. Maybe one day we could design around this, but for now it just means bevy_reflect requires alloc.

@bushrat011899 bushrat011899 added C-Feature A new feature, making something new possible A-Reflection Runtime information about types X-Contentious There are nontrivial implications that should be thought through D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Nov 6, 2024
@bushrat011899 bushrat011899 added this to the 0.16 milestone Nov 6, 2024
@alice-i-cecile alice-i-cecile requested a review from MrGVSV November 6, 2024 01:47
@bushrat011899 bushrat011899 added S-Needs-Review Needs reviewer attention (from anyone!) to move forward S-Needs-SME Decision or review from an SME is required and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Nov 6, 2024
@bushrat011899 bushrat011899 marked this pull request as ready for review November 6, 2024 04:07
@bushrat011899
Copy link
Contributor Author

Tested with my demo UEFI application and confirmed it works as expected, letting me derive Reflect and query for type information at runtime. CI task has been updated to ensure bevy_reflect will continue to compile. I have noticed that because bevy_reflect has a lot of macros, and they aren't checked at compile time in the crate that defines them (only at the compile time of the crate that uses them) we'll probably want to add a no_std example to Bevy to really ensure everything works as expected for end users.

@aecsocket
Copy link
Contributor

IIUC it's possible to make alloc optional, since it's only used in reflect de/serialization. If you feature gate Reflect(De)Serialize, and the logic in TypedReflect(De)Serializer, it should be possible to get no alloc. These are optional features anyway (without alloc you just can't override de/serialization for your types), and I don't think this will be super complex, but I don't know if it's worth it for the extra complexity.

@bushrat011899
Copy link
Contributor Author

I'll give this another look to see if serde can be made optional. Right now it definitely isn't optional, but that would be all that's really needed to allow a no_alloc experience.

@aecsocket
Copy link
Contributor

Actually, reflection uses a lot of Box<dyn PartialReflect> and Box<dyn Reflect>, so alloc may be more necessary than I initially thought. In which case, it might be pointless to feature gate it. Although I suppose there are use cases which don't involve string boxed reflect types.

Copy link
Member

@MrGVSV MrGVSV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great! A lot less involved than I thought it'd be.

Just a few questions (mainly for my own clarity and understanding).

crates/bevy_reflect/src/lib.rs Outdated Show resolved Hide resolved
crates/bevy_reflect/src/tuple.rs Outdated Show resolved Hide resolved
@@ -4,6 +4,9 @@ use core::{
slice::Iter,
};

#[cfg(not(feature = "std"))]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we use this attribute here but not on any of the other alloc imports?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is an interesting one. Basically I added these imports without the feature gate to ensure no_std compiled, and then checked compilation with std. In the cases were Clippy said these imports weren't used I added the feature gate. So in short, I don't have a good explanation as to why sometimes it's needed and sometimes it's not, but the process used to arrive here was very mechanical ("clippy said so")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you Clippy, very helpful

tools/ci/src/commands/compile_check_no_std.rs Show resolved Hide resolved
crates/bevy_reflect/src/impls/std.rs Show resolved Hide resolved
crates/bevy_reflect/src/impls/std.rs Outdated Show resolved Hide resolved
crates/bevy_reflect/derive/src/serialization.rs Outdated Show resolved Hide resolved
bushrat011899 and others added 4 commits November 7, 2024 08:06
Instead of embedding an `extern crate alloc;`

Co-Authored-By: Gino Valente <[email protected]>
Moved `std` feature gate inside `impl_reflect_for_atomic` explicitly for `ReflectDe/Serialize`

Co-Authored-By: Gino Valente <[email protected]>
Copy link
Member

@MrGVSV MrGVSV left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work! Can't wait to have reflection on my smart mirror!

@MrGVSV MrGVSV added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Needs-Review Needs reviewer attention (from anyone!) to move forward labels Nov 6, 2024
@GnomedDev
Copy link

GnomedDev commented Nov 7, 2024

Hey, I've never touched Bevy before but I do have a strong opinion against spin locks. Would it be better to use critical-section instead of a spin lock, so no_std platforms can provide their own locking mechanism?

@bushrat011899
Copy link
Contributor Author

Hey, I've never touched Bevy before but I do have a strong opinion against spin locks. Would it be better to use critical-section instead of a spin lock, so no_std platforms can provide their own locking mechanism?

That looks perfect honestly, thanks for the advice! I'll update this PR to use critical-section instead.

@bushrat011899 bushrat011899 added S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged and removed S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it labels Nov 7, 2024
@bushrat011899
Copy link
Contributor Author

Ok @GnomedDev I've spent a good couple of hours working with critical-section, reading the attached blog-post, and doing some further research into the area. As far as I can tell, I believe a spin-lock is the best we can do without access to the standard library.

critical-section provides a global mutex, where only one thread can access it at any one time. How it achieves this is configurable via features/dependencies by the user. At first that sounds great, since spinning wastes CPU cycles and can be deadlocked. However, there are two major issues with this approach within the context of bevy_reflect:

  1. bevy_reflect needs a RwLock, not a mutex
  2. The way bevy_reflect uses these locks is via a sometimes long-living guard (long being relative to the typical lifespan of a lock).

Point 1 could be addressed by just accepting that we lose the concurrency boost an RwLock provides. But that further exacerbates point 2, since instead of a small number of writes blocking the critical section, we now have a large number of reads doing it too. Effectively, to replicate an RwLock using critical sections, I believe you would:

  1. Use a CS to lock the RwLock into read mode.
  2. Pass out a guard object which will decrement a counter on drop.
  3. Once counter hits zero, use a CS to unlock.

The problem being if a write lock is requested between steps 1 and 3 (remembering that this time-frame can be large, since multiple readers are valid), the CS isn't actually held, so the writing thread would acquire the CS, see that the lock is in use, and then have no way to park execution, other than just spinning.

To confirm this belief, I attempted to find a pre-made RwLock implementation using critical sections, and came up blank, short of this 15 year old Stack Overflow answer basically saying it can't be done.

So, where to from here? In my opinion, a spin::RwLock is as good as I can provide for a no_std port of bevy_reflect without changing its behaviour to not use read-write locks in general. As such, I believe this PR should go ahead as-is. However, if you (or anyone else!) can provide a suitable replacement for RwLock I will gladly incorporate it here and likely use it in other areas of Bevy too! Otherwise, this might be something to be tackled in a followup PR.

@bushrat011899 bushrat011899 added S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it and removed S-Waiting-on-Author The author needs to make changes or address concerns before this can be merged labels Nov 8, 2024
auto-merge was automatically disabled December 3, 2024 20:30

Head branch was pushed to by a user without write access

@alice-i-cecile alice-i-cecile added this pull request to the merge queue Dec 5, 2024
Merged via the queue into bevyengine:main with commit bf765e6 Dec 5, 2024
31 of 32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-Reflection Runtime information about types C-Feature A new feature, making something new possible D-Modest A "normal" level of difficulty; suitable for simple features or challenging fixes S-Needs-SME Decision or review from an SME is required S-Ready-For-Final-Review This PR has been approved by the community. It's ready for a maintainer to consider merging it X-Contentious There are nontrivial implications that should be thought through
Projects
Status: In Progress
Development

Successfully merging this pull request may close these issues.

5 participants