perf: don't handle SitePage nodes when deleting stale nodes #39161
+371
−100
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gatsbot
bot
added
the
status: triage needed
pieh
added
topic: core
pieh
force-pushed
the
perf/dont-cleanup-sitepage-node-after-source-nodes
branch
2 times, most recently
from
63be5c4 to
370b4b2
Compare
pieh
force-pushed
the
perf/dont-cleanup-sitepage-node-after-source-nodes
branch
from
370b4b2 to
f611677
Compare
pieh
force-pushed
the
perf/dont-cleanup-sitepage-node-after-source-nodes
branch
from
d0b1452 to
c4adf6a
Compare
pieh
commented
Dec 11, 2024
| @@ -445,3 +452,99 @@ export const clearGatsbyImageSourceUrls = | |||
| type: `CLEAR_GATSBY_IMAGE_SOURCE_URL`, | |||
| } | |||
| } | |||
|
|
|||
| // We add a counter to node.internal for fast comparisons/intersections | |||
There was a problem hiding this comment.
the additions here were just moved from public.js file here to make it easier to reuse
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
When we are clearing up stale nodes (nodes that exist from previous build that were not recreated in current build), we are deleting all
SitePagenodes because at this point in time site creation didn't run yet. This results in mass nodes deletion and subsequent mass node creation - other than being wasteful this tends to lead to increase in LMDB datastore file size increasing as managing disk size is not priority for it so it tends to allocate more and more space over time with data mutations even if "logical" data remains at similar volume. This PR mainly looks to avoid wasteful LMDB operations and thus trying to slow down LMDB filesize increases over timeDocumentation
Tests
Related Issues