Concurrency limit for maximal different values per tag

[OneCyrus]

What's the use case?

we have multiple io_managers (duckdb) which should never run in parallel as they will result in file locking issues as they access the same duckdb database (basically multiple dbt projects and dagster duckdb data loading).
as we want to improve the performance we would like to run multiple materializations in parallel. so we are looking for a way to have multiple assets in parallel but only within the same tag-value/io_manager.

Ideas of implementation

applyLimitPerUniqueValue is already pretty close but is inverted from our use case. what would like is something like applyMaxUniqueValuesAtSameTime.

with this config we would like limit the count of unique values at the same time:

run_queue:
  tag_concurrency_limits:
    - key: "use-case"
      value:
        applyMaxUniqueValuesAtSameTime: true
      limit: 1

in the end it should try to queue as max allowed assets with the same tag value at the same time and only start to queue assets with a different value when no asset with another value is running.

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.