Tests fail consistently (Macbook)

[NiwakaDev]

When running cargo test --workspace, Tests fail, like so:

---- integration_http_file_test::test_catalog_metrics stdout ----
Listening on 127.0.0.1:65371
thread 'integration_http_file_test::test_catalog_metrics' panicked at 'assertion failed: body.contains(\"catalog_schema_count 3\")', tests-integration/tests/http.rs:350:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    integration_http_file_test::test_catalog_metrics

test result: FAILED. 55 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1.51s

error: test failed, to rerun pass `-p tests-integration --test main`

Right now, I am trying the latest develop branch:

commit 4151d7a8eae4d7f86bba7becdc0c98fbf2b40885 (HEAD -> develop, origin/develop, origin/HEAD)
Author: Ning Sun <~>
Date:   Thu May 11 08:54:28 2023 +0000

    fix: allow cross-schema query on information_schema (#1568)

Environment

chip: M2
OS : macOS Ventura 13.3

[evenyag]

Quite strange 😵 . Looks like some tests are easier to fail using cargo test. We use nextest in our CI so didn't reproduce this.

[evenyag]

I guess the cause is that metrics are global variables so different tests might increment the same counter

[sunng87]

It seems metrics won't reset between tests. Let me check if we can reset them

[waynexia]

One way to do this is using nextest, where each case is run as a single binary

cargo nextest run

[NiwakaDev]

I used cargo nextest run instead of cargo test --workspace, and then Tests passed!!

Thanks!!