Update wgpu-native to v0.19.1.1 (#458)

* update wgpu-native version

* run codegen

* Fix diagnostics

* Update expected errors in tests

* Run updated black

* Change order to fix tests

* Fix memtests

* Undo change, maybe helps pypy?

* fix typos

* Fix one pypy test

* fix last test with pypy

* fix lint

* Fix memtests for pypy

* remove DX11 mentions from docs

* move extra gc into clean func.

* Update changelog

* Fix typos in changelog

---------

Co-authored-by: Almar Klein <[email protected]>

CHANGELOG.md

@@ -17,6 +17,21 @@ Possible sections in each release:
 * Security: in case of vulnerabilities.
 
 
+### [unreleased]
+
+Added:
+* The feature "float32-filterable" is now available natively.
+* Add list of projects using wgpu-py to [README](README.md#projects-using-wgpu-py). ([#456](https://github.com/pygfx/wgpu-py/pull/456))
+
+Changed:
+* Updated to wgpu-native 0.19.1.1. ([#458](https://github.com/pygfx/wgpu-py/pull/458))
+
+Removed:
+* Shadertoy util is removed from the wgpu-py. It is now available as a separate package: [wgpu-shadertoy](https://github.com/pygfx/shadertoy). ([#455](https://github.com/pygfx/wgpu-py/pull/455))
+
+Fixed:
+* Devices no longer leak memory.
+
 ### [v0.13.2] - 21-12-2023
 
 Added:
@@ -208,7 +223,7 @@ Fixed:
 
 Added:
 
-* The shadertoy util now supports GLSL, so code from the shadertoy website can be direcly copied and run with wgpu (#343)
+* The shadertoy util now supports GLSL, so code from the shadertoy website can be directly copied and run with wgpu (#343)
 
 
 ### [v0.9.1] - 13-02-2023
@@ -514,14 +529,14 @@ Changed:
 
 Changed:
 
-* The backend selection is automatic by default. To force a backend, the `WGPU_BACKEND_TYPE` evironment variable can be set to e.g. "Vulkan". It could be good to do this on Windows to prevent selection of DX12 for now.
+* The backend selection is automatic by default. To force a backend, the `WGPU_BACKEND_TYPE` environment variable can be set to e.g. "Vulkan". It could be good to do this on Windows to prevent selection of DX12 for now.
 
 
 ### [v0.5.3] - 04-06-2021
 
 Added:
 
-* `adapter.properties` now has actual values, allowing inspeciton of the selected
+* `adapter.properties` now has actual values, allowing inspection of the selected
     GPU and backend.
 * Added back support for filtering float32 textures by enabling a certain wgpu feature
   by default.
@@ -585,7 +600,7 @@ Changed:
 * `GPUAdapter.request_device()`: the `extensions` and `limit` args are now `non_guaranteed_features` and `non_guaranteed_limits`.
 * `GPUDevice.default_queue`: is now called `queue`.
 * `GPUDevice.create_compute_pipeline()`: the `compute_stage` arg is now called `compute`.
-* `GPUDevice.create_bind_group_layout()` has changed the required structure of the layout enty dicts.
+* `GPUDevice.create_bind_group_layout()` has changed the required structure of the layout entry dicts.
 * `GPUDevice.create_render_pipeline()` has changed *a lot* in terms of shape of input dicts. See new docs.
 * `GPUTexture.create_view()`: args `mip_level_count` and `array_layer_count` are default `None` instead of `0`.
 * `GPUCommandEncoder.begin_render_pass()`: the `color_attachments` and `depth_stencil_attachment` arguments have their `attachment` field renamed to `view`.
@@ -638,7 +653,7 @@ Removed:
 
 Added:
 
-* The canvase now has a `request_draw` method.
+* The canvas now has a `request_draw` method.
 * More and better docs.
 * The canvas can be passed to `request_adapter` so that the created surface
   can be selected on it.

docs/guide.rst

@@ -212,7 +212,7 @@ Debugging
 If the default wgpu-backend causes issues, or if you want to run on a
 different backend for another reason, you can set the
 `WGPU_BACKEND_TYPE` environment variable to "Vulkan", "Metal", "D3D12",
-"D3D11", or "OpenGL".
+or "OpenGL".
 
 The log messages produced (by Rust) in wgpu-native are captured and
 injected into Python's "wgpu" logger. One can set the log level to

docs/start.rst

@@ -49,7 +49,7 @@ Platform requirements
 Under the hood, wgpu runs on Vulkan, Metal, or DX12. The wgpu-backend
 is selected automatically, but can be overridden by setting the
 ``WGPU_BACKEND_TYPE`` environment variable to "Vulkan", "Metal", "D3D12",
-"D3D11", or "OpenGL".
+or "OpenGL".
 
 Windows
 +++++++

docs/wgpu.rst

@@ -67,7 +67,7 @@ Information on the adapter can be obtained using :func:`wgpu.GPUAdapter.request_
 
 A device is controlled with a specific backend API. By default one is selected automatically.
 This can be overridden by setting the
-`WGPU_BACKEND_TYPE` environment variable to "Vulkan", "Metal", "D3D12", "D3D11", or "OpenGL".
+`WGPU_BACKEND_TYPE` environment variable to "Vulkan", "Metal", "D3D12", or "OpenGL".
 
 The device and all objects created from it inherit from :class:`GPUObjectBase` - they represent something on the GPU.
 

examples/tests/test_examples.py

@@ -13,7 +13,7 @@
 import pytest
 
 
-from testutils import (
+from tests.testutils import (
     can_use_wgpu_lib,
     wgpu_backend,
     is_lavapipe,
@@ -37,14 +37,6 @@
 examples_to_test = find_examples(query="# test_example = true", return_stems=True)
 
 
-@pytest.mark.parametrize("module", examples_to_run)
-def test_examples_run(module, force_offscreen):
-    """Run every example marked to see if they can run without error."""
-    # use runpy so the module is not actually imported (and can be gc'd)
-    # but also to be able to run the code in the __main__ block
-    runpy.run_module(f"examples.{module}", run_name="__main__")
-
-
 @pytest.fixture
 def force_offscreen():
     """Force the offscreen canvas to be selected by the auto gui module."""
@@ -145,6 +137,14 @@ def update_diffs(module, is_similar, img, stored_img):
             path.unlink()
 
 
+@pytest.mark.parametrize("module", examples_to_run)
+def test_examples_run(module, force_offscreen):
+    """Run every example marked to see if they can run without error."""
+    # use runpy so the module is not actually imported (and can be gc'd)
+    # but also to be able to run the code in the __main__ block
+    runpy.run_module(f"examples.{module}", run_name="__main__")
+
+
 if __name__ == "__main__":
     # Enable tweaking in an IDE by running in an interactive session.
     os.environ["WGPU_FORCE_OFFSCREEN"] = "true"

tests/test_gui_glfw.py

@@ -8,10 +8,11 @@
 import time
 import weakref
 import asyncio
+import gc
 
 import wgpu
 from pytest import skip
-from testutils import run_tests, can_use_glfw, can_use_wgpu_lib
+from testutils import run_tests, can_use_glfw, can_use_wgpu_lib, is_pypy
 from renderutils import render_to_texture, render_to_screen  # noqa
 
 
@@ -83,6 +84,8 @@ async def miniloop():
     loop.run_until_complete(miniloop())
     assert ref() is not None
     del canvas
+    if is_pypy:
+        gc.collect()  # force garbage collection for pypy
     loop.run_until_complete(miniloop())
     assert ref() is None
 

tests/test_wgpu_native_errors.py

@@ -141,20 +141,10 @@ def test_parse_shader_error4(caplog):
         Caused by:
             In wgpuDeviceCreateShaderModule
 
-        Shader validation error:
-          ┌─ :1:1
-          │
-        1 │ ╭ fn foobar() {
-        2 │ │     let m = mat2x2<f32>(0.0, 0.0, 0.0, 0.);
-        3 │ │     let scales = m[4];
-          │ │                  ^^^^ naga::Expression [9]
-          │ ╰──────────────────────^ naga::Function [1]
+        Shader '' parsing error: Index 4 is out of bounds for expression [11]
 
 
-            Function [1] 'foobar' is invalid
-            Expression [9] is invalid
-            Type resolution failed
-            Index 4 is out of bounds for expression [7]
+    Index 4 is out of bounds for expression [11]
     """
 
     code = dedent(code)
@@ -185,8 +175,8 @@ def test_validate_shader_error1(caplog):
         }
     """
 
-    expected1 = """Left: Load { pointer: [3] } of type Matrix { columns: Quad, rows: Quad, width: 4 }"""
-    expected2 = """Right: Load { pointer: [6] } of type Vector { size: Tri, kind: Float, width: 4 }"""
+    expected1 = """Left: Load { pointer: [3] } of type Matrix { columns: Quad, rows: Quad, scalar: Scalar { kind: Float, width: 4 } }"""
+    expected2 = """Right: Load { pointer: [6] } of type Vector { size: Tri, scalar: Scalar { kind: Float, width: 4 } }"""
     expected3 = """
         Validation Error
 
@@ -236,7 +226,7 @@ def test_validate_shader_error2(caplog):
         }
     """
 
-    expected1 = """Returning Some(Vector { size: Tri, kind: Float, width: 4 }) where Some(Vector { size: Quad, kind: Float, width: 4 }) is expected"""
+    expected1 = """Returning Some(Vector { size: Tri, scalar: Scalar { kind: Float, width: 4 } }) where Some(Vector { size: Quad, scalar: Scalar { kind: Float, width: 4 } }) is expected"""
     expected2 = """
         Validation Error
 

tests/test_wgpu_native_query_set.py

@@ -1,11 +1,15 @@
 import wgpu.utils
+import gc
 
 from testutils import run_tests, can_use_wgpu_lib
+from tests_mem.testutils import is_pypy
 from pytest import mark
 
 
 @mark.skipif(not can_use_wgpu_lib, reason="Needs wgpu lib")
 def test_query_set():
+    if is_pypy:
+        gc.collect()  # avoid a panic here when using pypy
     shader_source = """
     @group(0) @binding(0)
     var<storage,read> data1: array<f32>;

tests_mem/test_gui_offscreen.py → tests_mem/test_gui.py

@@ -80,6 +80,13 @@ def test_release_canvas_context(n):
     # Check that the canvas objects are really deleted
     assert not canvases
 
+    # This is a bit weird, but somehow this tests produces a dangling
+    # CommandBuffer for reasons likely related to the internals of
+    # wgpu-core. The lines below allocate and release a new
+    # CommandBuffer, which solves the issue :)
+    command_encoder = DEVICE.create_command_encoder()
+    command_encoder.finish()
+
 
 TEST_FUNCS = [test_release_canvas_context]
 

tests_mem/test_gui_glfw.py

@@ -6,10 +6,11 @@
 import weakref
 import asyncio
 
+import wgpu
 import pytest
 import testutils  # noqa
 from testutils import create_and_release, can_use_glfw, can_use_wgpu_lib
-from test_gui_offscreen import make_draw_func_for_canvas
+from test_gui import make_draw_func_for_canvas
 
 
 if not can_use_wgpu_lib:
@@ -22,6 +23,9 @@
     pytest.skip("Asyncio loop is running", allow_module_level=True)
 
 
+DEVICE = wgpu.utils.get_default_device()
+
+
 async def stub_event_loop():
     pass
 
@@ -57,6 +61,10 @@ def test_release_canvas_context(n):
     # Check that the canvas objects are really deleted
     assert not canvases, f"Still {len(canvases)} canvases"
 
+    # Help clear dangling CommandBuffer, see test_gui.py
+    command_encoder = DEVICE.create_command_encoder()
+    command_encoder.finish()
+
 
 if __name__ == "__main__":
     # testutils.TEST_ITERS = 40  # Uncomment for a mem-usage test run

tests_mem/test_gui_qt.py

@@ -5,10 +5,11 @@
 import gc
 import weakref
 
+import wgpu
 import pytest
 import testutils  # noqa
 from testutils import create_and_release, can_use_pyside6, can_use_wgpu_lib
-from test_gui_offscreen import make_draw_func_for_canvas
+from test_gui import make_draw_func_for_canvas
 
 
 if not can_use_wgpu_lib:
@@ -17,6 +18,9 @@
     pytest.skip("Need pyside6 for this test", allow_module_level=True)
 
 
+DEVICE = wgpu.utils.get_default_device()
+
+
 @create_and_release
 def test_release_canvas_context(n):
     # Test with PySide canvases.
@@ -51,6 +55,10 @@ def test_release_canvas_context(n):
     # Check that the canvas objects are really deleted
     assert not canvases
 
+    # Help clear dangling CommandBuffer, see test_gui.py
+    command_encoder = DEVICE.create_command_encoder()
+    command_encoder.finish()
+
 
 if __name__ == "__main__":
     # testutils.TEST_ITERS = 40  # Uncomment for a mem-usage test run

tests_mem/test_meta.py

@@ -9,7 +9,7 @@
 from testutils import can_use_wgpu_lib, create_and_release
 from testutils import get_counts, ob_name_from_test_func
 from test_objects import TEST_FUNCS as OBJECT_TEST_FUNCS
-from test_gui_offscreen import TEST_FUNCS as GUI_TEST_FUNCS
+from test_gui import TEST_FUNCS as GUI_TEST_FUNCS
 
 
 ALL_TEST_FUNCS = OBJECT_TEST_FUNCS + GUI_TEST_FUNCS

tests_mem/test_objects.py

@@ -25,14 +25,11 @@ def test_release_adapter(n):
 
 @create_and_release
 def test_release_device(n):
-    pytest.skip("XFAIL")
-    # todo: XFAIL: Device object seem not to be cleaned up at wgpu-native.
-
     # Note: the WebGPU spec says:
     # [request_device()] is a one-time action: if a device is returned successfully, the adapter becomes invalid.
 
     yield {
-        "expected_counts_after_create": {"Device": (n, n), "Queue": (n, 0)},
+        "expected_counts_after_create": {"Device": (n, n), "Queue": (n, n)},
     }
     adapter = DEVICE.adapter
     for i in range(n):
@@ -82,9 +79,7 @@ def test_release_bind_group_layout(n):
     global _bind_group_layout_binding
     _bind_group_layout_binding += 1
 
-    yield {
-        "expected_counts_after_create": {"BindGroupLayout": (n, 1)},
-    }
+    yield {}
 
     binding_layouts = [
         {
@@ -193,9 +188,13 @@ def test_release_query_set(n):
 
 @create_and_release
 def test_release_queue(n):
-    pytest.skip("XFAIL")
-    # todo: XFAIL: the device and queue are kinda one, and the former won't release at wgpu-native.
-    yield {}
+    # Note: cannot create a queue directly, so we create devices, which gave queue's attached.
+    yield {
+        "expected_counts_after_create": {
+            "Device": (n, n),
+            "Queue": (n, n),
+        },
+    }
     adapter = DEVICE.adapter
     for i in range(n):
         d = adapter.request_device()

tests_mem/testutils.py

@@ -81,8 +81,9 @@ def clear_mem():
     if is_pypy:
         gc.collect()
 
-    device = wgpu.utils.get_default_device()
-    device._poll()
+    wgpu.utils.get_default_device()._poll()
+
+    gc.collect()
 
 
 def get_counts():
@@ -190,7 +191,7 @@ def core_test_func():
             assert ob_name == cls.__name__[3:]
 
             # Give wgpu some slack to clean up temporary resources
-            wgpu.utils.get_default_device()._poll()
+            clear_mem()
 
             # Measure peak object counts
             counts2 = get_counts()
@@ -200,7 +201,9 @@ def core_test_func():
 
             # Make sure the actual object has increased
             assert more2  # not empty
-            assert more2 == options["expected_counts_after_create"]
+            assert (
+                more2 == options["expected_counts_after_create"]
+            ), f"Exepected:\n{options['expected_counts_after_create']}\nGot:\n{more2}"
 
             # It's ok if other objects are created too ...
 

wgpu/backends/wgpu_native/__init__.py

@@ -9,8 +9,8 @@
 
 
 # The wgpu-native version that we target/expect
-__version__ = "0.18.1.3"
-__commit_sha__ = "8561b0d8c0b5af7dfb8631d6f924e5418c92f2ce"
+__version__ = "0.19.1.1"
+__commit_sha__ = "569a2be60d1dc90a660c1c96ffb3722942ada782"
 version_info = tuple(map(int, __version__.split(".")))
 _check_expected_version(version_info)  # produces a warning on mismatch