Tile-join merging large files is particularly slow

[james1028zhao]

I use the following command，The file is extremely large, over 10GB，
tile-join -o out123.mbtiles in01.mbtiles in02.mbtiles in03.mbtiles in04.mbtiles
Very slow, my memory is only 8GB, virtual memory is a lot
But it takes two hours to finish.
why?How to solve？

[e-n-f]

It is surprising that tile-join uses so much memory, since the only thing in memory at any given moment should be the few tiles that are actually being decoded. Is it actually using that much memory, or is it being slow for some other reason?

[james1028zhao]

Very fast when the amount of data is less than the memory.
Once the amount of data is greater than the memory, it depends on the virtual memory, and it may even fail

[victoraalvarez]

Having this issue still. Need to recursively merge vector tiles into one singular tileset and it begins to get quite slow and fails at a certain point. Does simply increasing RAM fix this?

[e-n-f]

There is a bug in the current https://github.com/felt/tippecanoe tile-join that I am working on fixing in felt/tippecanoe#147. You might want to try that branch and see if it fixes your problem.