Namespace conflict for tests: Two tests gets the same name but only one is executed

[johansjob]

Describe the bug

Test are skipped if the test name is compiled to the same name.

Steps To Reproduce

Create two models:

• base.sql with sql SELECT CAST(NULL AS STRING) AS extension_id
• base_extension.sql SELECT 'NOT_NULL' AS id

Create a schema:

version: 2

models:
    - name: base
       columns:
           - name: extension_id
              tests:
                  - not_null
    - name: base_extension
       columns:
           - name: id
              tests:
                  - not_null

Run:
dbt run dbt test

Current behavior will skip the first test and hence succeed even though this particular scenario should fail.

Using dbt list --output json also only shows one of the tests named 'not_null_base_extension_id'. (I guess this shows why this problem rises in the first place).

Expected behavior

The exepected behaviour would probably be a compilation error or handle this namespace conflict in such a way so both tests are executed.

System information

Which database are you using dbt with?

• [X ] bigquery
  But probably an error for both

The output of dbt --version:
0.16.0

The operating system you're using:
macOS Catalina Version 10.15.3

The output of python --version:
3.8.2

Additional context

Very much a corner case. Thought I report it anyway..

[drewbanin]

Gosh, that's not great, thanks for opening this issue @johansjob!

We have some further thoughts outlined in #1173 that will help us generate unique ids / names for tests in a much more sane/sensible way! This is definitely a corner case, but it's one worth fixing for sure.

Thanks for digging in here!

[jtcohen6]

Closing in favor of #3254