Add support for virtual/sparse/mega textures

[jonbonazza]

Describe the project you are working on:
Large open world games

Describe the problem or limitation you are having in your project:
When working with large, open world games, it's important that level streaming be employed in order to reduce the amount data that is loaded into both RAM and GPU memory at any given time. Level streaming itself already solves this problem by only loading in chunks of a scene that is immediately relevant to the player and unloading those that aren't, and mesh LODs help reduce the strain on the GPU for distant meshes that are still loaded. The former can be implemented today, and the latter will be supported in Godot 4.0, but there is still one problem that hasn't been addressed but is important to tackle for large open world games: textures.

Large open world games often contain A LOT of textures. For hi res games, these textures are often huge as well. Constantly loading and unloading all of these different textures is extremely inefficient and makes the level streaming process slower and more "spiky".

Describe the feature / enhancement and how it helps to overcome the problem or limitation:
A popular solution to this is virtual textures (also called Sparse or Mega textures). Unity and Unreal both offer these already and I believe that Godot could also benefit from them. Essentially, with virtual texturing, you have a few extremely large textures that usually contain multiple textures, but when the texture data is uploaded to the GPU, only the pieces of the texture that are relevant to what is currently being rendered is uploaded. This means that the actual texture data can reside in memory for longer periods of times, reducing loading/unloading contention during level streaming, but significantly reduces the amount of GPU memory that is used when rendering the textures, since only a small portion is actually uploaded.

Related:
#1197
godotengine/godot#17470

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:
A good explanation of virtual textures can be found in this video: https://www.youtube.com/watch?v=MejJL87yNgI

An additional feature of the VT implementation should be the support for asynchronous loading of textures in order to assist in reducing contention when steaming these large textures from disk during level streaming. I think this might already be planned in Godot 4.0 via StreamTextures, but I haven't looked too much into them to really know how relevant they are here.

If this enhancement will not be used often, can it be worked around with a few lines of script?:
No

Is there a reason why this should be core and not an add-on in the asset library?:
Texturing is a core component of Godot's rendering pipeline. In Godot 4.0 it might be something that can be added as a plugin, depending on what the rendering APIs end up looking like, but at least in 3.X, it must be core.

[TechnoPorg]

Texture streaming is going to be implemented in 4.x: #3177. I was watching the talk on virtual textures, and I'm wondering if they could be optimized by implementing them in parallel with #2793, and using the geometry buffer to determine the required texels, rather than doing an extra pass just for virtual textures.

[Calinou]

Texture streaming is going to be implemented in 4.x: #3177. I was watching the talk on virtual textures, and I'm wondering if they could be optimized by implementing them in parallel with #2793, and using the geometry buffer to determine the required texels, rather than doing an extra pass just for virtual textures.

Remember that anything Godot will support needs to have broad hardware support (relative to the API in use). It needs to work on any Vulkan-capable desktop GPU (since mobile is outside the scope of texture streaming). It should not be limited to dedicated GPUs only or recent GPU architectures only.

[Mantissa-23]

Just wanted to do a sanity check here- Are there any proprietary engines that are in current use that still use this technique? I know RAGE and RAGE 2 were famous for this, but they weren't exactly smash-hit titles and I feel like the technology has largely faded from common use.

As far as I'm aware this has largely been replaced with decals, which recent graphics APIs render very efficiently. I know iD Tech for DOOM 2016 and beyond just used decals for any kind of detail or environmental effect.

[equalent]

@Mantissa-23
as far as I understand, it's used extensively. All current desktop GPUs and graphics APIs (D3D11 and newer) have "sparse resources" feature, which maps different virtual areas of a resource (texture or buffer) to physical GPU memory (in a similar manner to the way OS implements virtual memory). This allows for direct use of hardware resources without having to sacrifice performance and memory by relying on additional look-up textures.

Unreal Engine uses it in 2 ways:

• Runtime Virtual Texturing (RVT), the engine generates the virtual texture at runtime
• Streaming Virtual Texturing (SVT), the engine prebakes the virtual texture and streams in parts of it at runtime

As for in-house engines, I'd assume most engines centered on large open-world games are using it but it's very hard to verify without in-depth graphics debugging.