-
Notifications
You must be signed in to change notification settings - Fork 249
Generic hud toggle design
(Outdated. It can be done with Reshade 5 now. See: https://discord.com/channels/586242553746030596/677946763239489537/938066945138376735. Basically it means catching the hashes for the shaders we want to block, then deny the drawcalls which bind the shader)
To obtain the shader handle and then discover whether it's bound, use: https://discord.com/channels/586242553746030596/938951838227632218/941333672517304351
This system is based on an interceptor similar to Reshade and 3D Migoto: it wraps the DXGI device and device context and simply intercepts calls to those objects. It's key to have this in a dll which is named d3d11.dll or dxgi.dll. It would be great if it can be merged with reshade, but as reshade doesn't always work it might be a good idea to first try it out for ourselves based on a fork of reshade.
This system intercepts Pixelshaders passed to the device context and blocks them if they match a pre-defined hash. This hash is calculated with code from 3D migoto's util.h, the fnv_64_buf function. This function calculates the hash of a buffer with a length. It contains a subtle bug as it assumes the length is always a multiply of 8, which might not be the case, so modulo the length first and subtract the result from the length to get a length within the data. For hashes this isn't that big of a deal.
There are two wrappers: one for the Device and one for the DeviceContext. The wrapper for the Device intercepts CreatePixelShader(). This interception gets us the buffer data and the length of the buffer so we can calculate the hash for the buffer data and assign it to the ID3D11PixelShader pointed object we receive in return. This is COM so avoid making the mistake the pointer is the value.
The ID3D11PixelShader object returned by CreatePixelShader is stored with the hash calculated from the buffer and length. In PSSetShader the ID3D11PixelShader instance is passed in and we can then look it up in our lookup and see whether its hash matches a list of blocked shaders.
3D Migoto uses for hunting a 'skip' system, where it skips all draw calls which refer to the current hunted shader. This is cumbersome for us as it requires a lot of code. The other way it allows hunting is by replacing it with a pink coloring shader. This is easier and what we need: we'll replace the shader with a simple 'discard' shader for every pixel.
3D migito's HackerContext.cpp, line 497 (BeforeDraw), shows how to switch the shader with the one we'll use for a shader that has to be hidden. We'll also use this system for hunting and for when a shader needs to be blocked, as it's effectively the same thing. Use the above reference to find the info we need to build this.
- user presses a key which marks 'start hunting'. This clears the known hashes of shaders to block in memory
- as we know the pixel shaders currently active (through PSSetShader and CreatePixelShader), we can start with the first of the set as the active shader. The active shader is blocked to make it easy for the user to see what the current shader does
- user presses the 'next' and 'previous' keys to move through the list of active shaders. Each transition to a new shader makes that shader the 'active' one, making it blocked and the user can then see whether that shader is to be blocked
- if the active shader has to be blocked, the user presses the 'mark' key. The hash of the active shader is added to the list of shaders to block
- when the user is done, the user presses the 'end hunting' key and all hashes are written to a file.
- at any time the user can press the 'toggle' key which will simply set the 'block' bool variable so all shaders matching the hashes currently in the list of shaders to block are blocked.
When PSSetShader is called (happens for every draw call every frame), and the passed in shader object's hash value is in the list of shaders to block and the block boolean is true, the call is changed by passing the pointer to the shader object we created ourselves which simply discards any pixel.
Passing null as the pixelshader object simply makes the polygon being rendered with the active default color, so the polygon is simply not shaded. This isn't what we want, even though it might be enough for some games' HUD.
After implementing the above, it's essential that the shader upload call can be intercepted. This is the weak spot, as often a game uploads the shaders for GUI/2D elements right at the start, as they need it for the main menu too. If the code is injected after that, it won't work.
This is a weak spot, and can't be solved. The only way to do this is by intercepting all calls at the Dx level, ala reshade.
Another way to find shaders is by using heuristics of when they're used. The idea is that a 2D shader to e.g. render a HUD element isn't used elsewhere but to render a 2D element. To be able to do that, the code should calculate a 'Draw call Signature', which is built from entropy obtained from the current state of the API. See: https://github.com/FransBouma/InjectableGenericCameraSystem/blob/ShaderTogglerTestCode/DrawCallSignatureElements.txt for a list of entropy elements to use.
Below is a quick description of how it works.
To calculate the draw call signature, the elements described here are used. This results in a value when packed together, and that value is the signature of that call. The idea is that that signature is unique.
The following steps are used to create a hud toggle this way
- Mark mode: the user marks the shaders like in 3D migoto. This collects the pointers as passed to PSSetShader.
- Learn mode: with the marked pointers (as obtained from PSSetShader), the draw call signature of every call which uses the pointers is collected. This could mean there are multiple signatures for each shader pointer.
When the user is done with the learn mode, the dataset can be persisted. This simply means that the collected draw call signatures are persisted.
When the game is restarted, the call signatures are loaded and used to find the shader pointers again:
- per draw call, get the pixelshader pointer from PSSetShader
- check if it's a known pixel shader.
- A: if yes: check if it's blocked. If yes: ignore the draw call. If no, proceed as normal. Done
- B: if no: calculate the draw call signature. If the signature is in the list of signatures loaded, the pixelshader is marked as 'known' and tied to the draw call signature. Then go to A.
If the user invokes the hud toggle (so the tool has to block all shaders marked), all shaders in the list as tied to a draw call signature, are then marked as 'blocked'. If a draw call then uses such a shader, it will end up in 'A'.
If a shader isn't known (so it ends up in 'B') the draw call signature is calculated and if the signature is unknown (so not in the list of signatures loaded from disk) the shader is marked as 'known', so the call signature won't be recalculated over and over. As the shader won't be in the list of blockable shaders, the next time the call comes through it will be seen as a known shader, it's not blocked and thus the call will go through without a problem.