Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch storage strategy #87

Merged
merged 95 commits into from
Oct 21, 2024
Merged

Switch storage strategy #87

merged 95 commits into from
Oct 21, 2024

Conversation

memorycode
Copy link
Contributor

@memorycode memorycode commented Jul 14, 2024

Proposed changes

This PR completely changes the internal storage layout to optimize queries. The goal is to be able to iterate the entities in each archetype in a cache friendly way and with as little indirection as possible

At its core, an archetype stores data as a map from a component -> map from an entity -> component data (fields in the code). However, we assign contiguous indices to each component and to each entity so that the maps are effectively arrays.

All of the supporting structures in an archetype are to make this densely packed layout possible. For instance, idToIndex maps a component ID to its index in fields.

Performance

Iterating ~7,400 entities with 128 archetypes is 7.5x faster. More synthetic benchmarks show 10-15x improvement.

image

Next steps

There are plenty of optimizations that can still be made. Some will come at the expense of readability and have been left out of this implementation to keep the focus clear, while others were omitted simply due to time constraints and the desire to keep this PR as small as possible.

The most obvious and immediate follow-up is to implement an archetype graph so that insertions and removals can be much faster.

Thanks to @Ukendio for guidance and @SanderMertens for the great articles upon which this implementation is based on.

@memorycode memorycode marked this pull request as draft July 14, 2024 04:07
bench.project.json Outdated Show resolved Hide resolved
lib/World.luau Outdated Show resolved Hide resolved
@memorycode memorycode marked this pull request as ready for review October 20, 2024 20:06
Copy link
Member

@Ukendio Ukendio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Ukendio Ukendio merged commit 4c1dca1 into main Oct 21, 2024
6 checks passed
@Ukendio Ukendio deleted the use-real-archetypes branch October 21, 2024 18:18
jackTabsCode added a commit to jackTabsCode/rbxts-matter that referenced this pull request Oct 21, 2024
Ukendio added a commit that referenced this pull request Oct 21, 2024
@memorycode memorycode mentioned this pull request Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants