[d3d9] Implement several IDirect3DShaderValidator9 validations

[WinterSnowfall]

Fixes #2045.

(Working) rebase of @misyltoad's former d3d9-shader-val2 branch, which doesn't crash on the main menu with The Void. On top of it I've added a crude adaptation of @gofman's PS input counts validation. It seems to work fine in terms of actually doing what's expected of it , but it doesn't currently fix the crashing issues in the game.

A save file that can reproduce the crashing can be found here.

[WinterSnowfall]

I've had to comment out:

      // + 1 to account for the opcode...
      uint32_t dwordLength = m_ctx->getInstructionContext().instruction.tokenLength + 1;
      if (cdw != dwordLength) {
        return ErrorCallback(pFile, Line, 0x2,
          D3D9ShaderValidatorMessage::BadInstructionLength,
          str::format("Instruction length specified for instruction (", cdw, ") does not match the token count encountered (", dwordLength, "). Aborting validation."));
      }

because on many occasions it seems like:

• the dwordLength for mov, mul, mad and dp4 is -1 vs cdw
• the dwordLength for tex is -2 vs cdw

That causes the game to crash before getting to the main menu. @misyltoad I hope you have a better memory than I do and maybe have a clue what's going on here?

Using: if (cdw < dwordLength) also works, but I'm not sure if that's the intended behavior here.

Nevermind, not even that does. The game crashes later on, so it seems like there's no guarantee that cdw and dwordLength are equal, though most of the time they are.

[WinterSnowfall]

@Blisto91 Temba, his arms wide.

Seems to not crash for me now in the problematic parts. Please also test when you can.

[WinterSnowfall]

Left the BadInstructionLength validations commented out and with a TODO, since I'm not sure what's going on there. Perhaps it's something we can improve upon in the future, if any game actually cares.

I'd welcome more testing, but at this stage it seems to be enough for The Void. The risk of breaking other stuff I think is rather minimal, since all other validations deal with shader type/structure validations and IDirect3DShaderValidator9 call order.

[WinterSnowfall]

Act of War - Direct Action tries to use register input number 11 in a PS and expects that to work, otherwise it hangs before starting a game. Back to the drawing board, I guess.

Edit: The game uses input register index 11 in a V1.1 PS. I guess I should actually restrict the check to PS 3.0, as per intended design, duh.

[WinterSnowfall]

Should be good to go post a review. It's been tested with quite a few games, and at this point I'm fairly confident it shouldn't regress anything. That being said, it's d3d9, so we won't know until it's merged.

[doitsujin]

do we expect this to get called in every game that uses the validation functions?

[WinterSnowfall]

Only in case of actual validation errors, however some of these (like in the case of The Void) are expected behavior. It would be useful to see them in logs however, at least for a while, until we're sure it's not causing unintended validations in other places.

Testing so far has been promising, but you never know, since this is like entirely undocumented.

[doitsujin]

Does this mean the validator instance is not reusable if validation failed? Does this match native behaviour?

[WinterSnowfall]

A good question for @misyltoad . I'm afraid I have no idea how to go about testing this. Some games do seem to recreate the validator instance for each shader from what I've noticed, but I have no idea if this is intended by design.

[WinterSnowfall]

P.S.: According to these tests it does indeed seem that once a validation error is hit, the validator instance must be released and recreated. Great API, I know.