Setup Unreal Engine
Unreal Engine can be easily configured for OpenXR passthrough support.
To enable alpha channel output, go to the Project Settings. In the Rendering section under the Postprocessing header, set Enable alpha channel support in post processing (experimental) to Allow Through Tonemapper.
To be able to control the passthrough, we need a postprocess material. Create a new material. In the material details, set the Material Domain to Post Process.
Below, in the Post Process Material section, enable the Output Alpha option.
The material can now be used to control passthrough by what is written to the alpha output. There are many options, such as using scene depth or a custom stencil to control where passthrough is drawn, but for this example we'll draw it in empty areas where no scene objects have been drawn. When Unreal draws the scene, it fills in the alpha for each pixel drawn, except the alpha is inverted from what OpenXR requires, with white pixels being empty. In order to get the correct output, we invert the alpha received by the scene.
Note: Unreal forces the Alpha Unpremultiplied flag in its OpenXR composition layers, so the alpha should not be premultiplied here.
To enable the material, create a PostProcessVolume in the map. Add the created material to the Postprocess Materials array, and enable the Infinite Extent (Unbound) option.
Running the map should now display passthrough in any empty spaces. Make sure to remove or disable any sky sphere and geometry that may be blocking it.
Unreal only supports controlling environment blend modes through the xr.OpenXREnvironmentBlendMode console variable, with no way of checking which modes are available (the Varjo plugin below can be used for that).
The variable has the following values:
0 - Automatic (use the runtime preferred mode)
1 - Opaque
2 - Additive
3 - Alpha Blended
To toggle passthrough rendering, the variable van be toggled between the 1 and 3 modes using Execute Console Command nodes.
To use the Varjo extensions for depth blending, as well as checking availability of blend modes, the Varjo OpenXR plugin can be used. It does not require a Varjo HMD.
See the Varjo OpenXR documentation for more information and a better tutorial:
https://developer.varjo.com/docs/unreal/ue5/mixed-reality-with-unreal5