Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up overzooming in tile-join #147

Merged
merged 14 commits into from
Oct 4, 2023
Merged

Speed up overzooming in tile-join #147

merged 14 commits into from
Oct 4, 2023

Conversation

e-n-f
Copy link
Collaborator

@e-n-f e-n-f commented Sep 27, 2023

Make tile-join faster:

clip.cpp

  • allow creating overzoomed tiles without compression, since tile-join will immediate decompress them
  • allow creating overzoomed tiles from an mvt_tile instead of always decompressing from a protobuf
  • speed up clipping by doing a quick reject by bounding box if possible

tests/ne_110m_ocean/join/joined.mbtiles.json

The change to the test fixture is just because some tiles now have their layers generated in a different sequence because uncompressed (overzoomed) tiles compare earlier than than compressed (direct from mbtiles) tiles.

tile-join.cpp

  • Keep a cache of parent tiles instead of retrieving the parent tile each time to create an overzoomed tile from it
  • Decode the parent tiles to mvt_tile instead of leaving them compressed until it is time to use them
  • Remove the current tile reader from the reader queue before checking whether the next tile on the queue is for the same tile that is currently being handled, which allows the output queue to be processed at the appropriate time instead of leaving everything to be done accumulating in memory until all of the tiles have been read

@e-n-f e-n-f marked this pull request as ready for review October 2, 2023 15:57
tile-join.cpp Outdated Show resolved Hide resolved
tile-join.cpp Outdated Show resolved Hide resolved
tile-join.cpp Outdated Show resolved Hide resolved
@e-n-f e-n-f merged commit cc5c1c7 into main Oct 4, 2023
1 check passed
@e-n-f e-n-f deleted the tile-join-overzoom-speed branch October 4, 2023 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants