Update wgpu-native to v0.19.1.1

[Vipitis]

My first try at updating to the new wgpu release

followed the steps outlined here

not sure if I did it all correct... I did not update the idl spec

• there were no changes _api.py
• changes diagnostics: WGPUStorageReport got replaced by WGPURegistryReport; dx11 removed
• expect errors changed, so I updated them

Examples pass, tests do (apart from the last two, which crash as I don't have lavapipe on this system), additionally the examples and tests in shadertoy work.

If there is more changes in the api that I missed due to the idl, perhaps you can still cherry pick the commits here.

[almarklein]

Thanks for this! Glad to see someone else giving this a go. If you felt the description of the update process can be improved, that'd be worthwhile too!

It looks like black had an update, so just running black . should silence the linting errors of the examples.

There are a few examples that crash/abort on some platforms. The memtests seem to be failing consistently, which is a bit worrysome 🤔. I'll have a closer look somewhere this week.

[Vipitis]

I did forget about me test. Tried to look into the failing cases and they seem to be all assertion errors again. Where 32 is expected but we get 32, 64 but also 32, 62 (?) haven't fully understood the test myself yet.
I also tried to run larger test set from pygfx/shadertoy#15 and run into AttributionError which never happened before, so I will have to look closer for that

As for the linting, I assume it's an update as the files throwing errors are not changed. Is there a pinned version of black and flake8 we use? Perhaps it's also worth considering to switch to ruff like pygfx/shadertoy#12 as its notably faster even on this size of project.

Will find some time later this week

[Vipitis]

little update. I fixed the linting by updating black.

As for the tests failing and aborting... it crashes before the skip for lavapipe is run. While importing the screenshot example.
It gets all the way to device.queue.write_texture where it wgpuQueueWriteTexture: Attempt to use a resource with a different device from the one that created it The abort happens seemingly for the same reason.
Disabling screenshot in triangle_auto.py helps to see how cube.py is failing.
If you run the tests with pytest examples -v -s you get the panic error trace. Not sure why this fail caused the subsequent test to abort. Might be an issue with how the parts area actually destroyed.
As just running the screenshot tests using pytest examples -v -s -k screenshot skips them successfully (also means no fail/abort)
So I put the test_examples_run at the bottom of the file and it passed (skipped correctly) without any fails and aborts. Which seems like a really stupid bodge. (recent commit)

This simply hides the underlying issue tho - hopefully the mem tests lead me to an answer eventually.

[Korijn]

Note that on CI the examples and screenshots tests pass. Only the tests folder fails.

[almarklein]

I think I know why the memtests fail. Just a quick heads-up in case you were planning to dive in too 😉

[Vipitis]

I won't find time today or tomorrow. So feel free to take over.
What I managed to spot is that the expected dictionary shows fewer items than the one we get in between more2... So it can either be fixed by changing the comparison logic a bit - or there is an actual issue with destroying the instances.
Again, my reordering fix for examples/screenshots simply hides the failure.

[almarklein]

Getting close. Pypy fails consistently tho.

[Vipitis]

run into AttributionError which never happened before, so I will have to look closer for that

can confirm this was an issue with my script and is unrelated.

I have also gotten to a situation where I get an Python-CFFI error trace as a OS pop up. It is charmap encoding related, shadertoys have Unicode in comments that might cause this issue.

[almarklein]

Only Pypy left. It fails consistently in test_validate_shader_error3 which is weird, since that does almost the same as test_validate_shader_error2.

I wanted to reproduce by installing Pypy, but ran into another (macos specific) pypy issue: beeware/rubicon-objc#406

[Vipitis]

I setup pypy on my system and it fails for three tests: test_glfw_canvas_del at the very end during tests (needed an gc.collect, but worked on CI...) however during test_wgpu_native_query_set.py::test_query_set the pytest aborts due to the same underlying rust panic that aborted the examples before I changed the order. this test passes on it's own with pypy -m pytest tests -v -s -k query

The other two failures tests (test_gui.py::test_release_canvas_context, test_gui_glfw.py::test_release_canvas_context) are during tests_mem. Those look similar to the one we had originally with the missing items in that dict. So I suspect these need a similar change to the one you implemented.

I do not get any failures for test_validate_shader_error 1 or 2, however the abort I experience is happening directly afterwards. So I suspect something about that previous test might leave leftovers somehow that are pypy specific, need to read more on that topic

Side node: we also have references to Dx11 in the docs. Also in the enums, but I don't think they can be removed.

[Vipitis]

@almarklein now passes all tests on my system and on CI all the pypy fixes are just more gc.collect() spam

[almarklein]

I suppose we have to learn to live with the fact that things are somewhat flaky on pypy. For the record:

• CI uses PyPy 7.3.15 with Python 3.9.18
• I used Pypy 7.3.15 with Python 3.10.13, also tried Pypy 7.3.15 with Python 3.9.18 (same result).

Anyway, the tests are green, so I think we can start wrapping this up. I just saw that some pygfx examples are broken with this update, so I'll have a look into that.

[almarklein]

I just saw that some pygfx examples are broken with this update, so I'll have a look into that.

These are related to the "float32-filterable" feature. It looks like now that "float32-filterable" is available, it is not longer part of "texture_adapter_specific_format_features". Enabling "float32-filterable" instead of "texture_adapter_specific_format_features" in pygfx fixes it!

[almarklein]

Could you also update the release notes? Some points to include, at least:

• Update to wgpu-native 0.19.1.1.
• The feature "float32-filterable" is now available.
• Devices no longer leak memory.

[almarklein]

Looking good to go from my end!

[Vipitis]

I have also gotten to a situation where I get an Python-CFFI error trace as a OS pop up. It is charmap encoding related, shadertoys have Unicode in comments that might cause this issue.

Finally figured out why this happened: I was using tqdm and rerouted stderr to devnull to avoid some error messages from ruining the progress bar. the cffi callback for the logger prints to stderr - and the error somehow traced all the way to cp1252 encoding 🤷