Layerization

Reasons for splitting off a new layer

• Bitmap elements don't need to be copied over, they already are directly displayable
• Videos can't even be rendered in Canvas. Not with meaningful performance, at least
• TextFields can be rendered by the browser (bonus: selection for free). Even if rendered into Canvas elements, it can be rendered in HiDPI independently of the rest of the display list
• filters/ blendmodes
• an element moves relative to content below it
• elements in a group that move relative to content below in the same way would create a very sparse layer, so creating individual layers is advantageous

Reasons against splitting off a new layer

• splitting off this particular element would needlessly split a group that could be in a larger (or, well, thicker) layer. This is most relevant to bitmaps
• taking all layerization opportunities would take up too much memory

Ideally, layerization decisions would take the whole display list into consideration. I.e., we should create a list of potential groups to layerize and assign them priorities and costs. Then we discard groups until we stay within our memory budget. Which groups to join into a layer is decided based on their costs.

After initial rendering, we'll only have to re-render groups that contain invalid elements. If multiple consecutive groups are invalid, we can reconsider grouping decisions for them.

Additionally, the priority of whole groups should wear off over time and be reset to the initial value each time the group is invalidated. Once their priority reaches 0, they get re-merged with their background layer. The exception to this are videos, which never get re-merged, and potentially TextFields, depending on how we want to handle them.

Assigning priority and cost to elements/ groups

• for Bitmaps by themselves, the priority should be relatively high, while the size could be set to 0 and the fill ratio to 1 to make them appear free. Once grouped with other elements (even other bitmaps), they get their real area as the size and a low priority
• for Videos, the priority should be positive infinity, with an irrelevant cost: we never ever group them
• for elements with filters/blendmodes, the priority should be positive infinity, with an irrelevant cost: we never ever group them initially
• for TextFields, the priority should be very high: we probably don't want to group them, but we can if we absolutely must. Their cost should be low, but not zero, when by themselves, and normal when grouped with non-TextField elements
• shapes could get a priority based on their path and fill complexity: the more complex, the less we want to re-rasterize. Their cost should just be their area
• for groups of elements, the priority should be the combined priority of their members. To cost should be the combined cost, somehow weighted by the layer fill ratio (but see the first open question)

Open questions

• how do we efficiently determine that a group would create a very sparse layer and split it into sub-groups? I couldn't find a good algorithm for that, but am pretty sure that it can be done in O(n log n) with a fairly high k
• how do we determine our memory budget? On memory-constrained devices, we can't just use up as much memory as we get our hands on: we'd force other applications to get unloaded (somewhat) needlessly