[ROCM] [GFX1030] no output

[IMbackK]

while testing 5a61d6e for #33 i noticed a new problem in master that's unrelated to 9c81167 on GFX1030 but not GFX900 or GFX906:
examples/inference.py, test_inference.py will both not produce any output. There is no error and processing takes a reasonable amount of time for the selected number of tokens, but no tokens will be generated.

I will try to investigate or bisect this problem later.

Rocm 5.7.1
python 3.11.5
pytorch v2.1.0 @ 7bcf7da3a268b435777fe87c7794c382f444e86d

[turboderp]

Any update on this? (ROCm 6.0 and PyTorch 2.3.1 are supported now)

[IMbackK]

this seams fine in a all-latest setup.