Mark flaky tests

[mikealfare]

resolves #770

Problem

We have a handful of flaky tests in dbt-redshift that routinely prevent CI from passing. It's not always the same test that fails. After a few runs, all tests have passed at least once, but it's rare that they all pass at the same time.

Solution

• mark flaky tests using pytest.mark (e.g. @pytest.mark.flaky)
• update the existing integration tests to only run non-flaky tests
• add a new set of integration tests that only run flaky tests
  • re-running duration is greatly reduced
• reduce collisions from tests running at the same time
  • run these specifically after the non-flaky tests
  • run them in series according to GHA
  • run them in series according to pytest
• only require non-flaky tests
• only alert on failed non-flaky tests
• identify similar characteristics across flaky tests to troubleshoot the source

Checklist

• I have read the contributing guide and understand what's expected of me
• I have run this code in development and it appears to resolve the stated issue
• This PR includes tests, or tests are not required/relevant for this PR
• This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

[github-actions]

Thank you for your pull request! We could not find a changelog entry for this change. For details on how to document a change, see the dbt-redshift contributing guide.

[mikealfare]

The integration tests workflow is updated as part of this PR. Here's a run against this branch:
https://github.com/dbt-labs/dbt-redshift/actions/runs/8990807294

[mikealfare]

Here's the latest integration tests run against this branch:
https://github.com/dbt-labs/dbt-redshift/actions/runs/9011428584

[github-actions]

The backport to 1.8.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.8.latest 1.8.latest
# Navigate to the new working tree
cd .worktrees/backport-1.8.latest
# Create a new branch
git switch --create backport-813-to-1.8.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b57890e3503928ced7dfd75de6985b348f0e7307
# Push it to GitHub
git push --set-upstream origin backport-813-to-1.8.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.8.latest

Then, create a pull request where the base branch is 1.8.latest and the compare/head branch is backport-813-to-1.8.latest.

[github-actions]

The backport to 1.7.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.7.latest 1.7.latest
# Navigate to the new working tree
cd .worktrees/backport-1.7.latest
# Create a new branch
git switch --create backport-813-to-1.7.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b57890e3503928ced7dfd75de6985b348f0e7307
# Push it to GitHub
git push --set-upstream origin backport-813-to-1.7.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.7.latest

Then, create a pull request where the base branch is 1.7.latest and the compare/head branch is backport-813-to-1.7.latest.

[github-actions]

The backport to 1.6.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.6.latest 1.6.latest
# Navigate to the new working tree
cd .worktrees/backport-1.6.latest
# Create a new branch
git switch --create backport-813-to-1.6.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b57890e3503928ced7dfd75de6985b348f0e7307
# Push it to GitHub
git push --set-upstream origin backport-813-to-1.6.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.6.latest

Then, create a pull request where the base branch is 1.6.latest and the compare/head branch is backport-813-to-1.6.latest.

[github-actions]

The backport to 1.6.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.6.latest 1.6.latest
# Navigate to the new working tree
cd .worktrees/backport-1.6.latest
# Create a new branch
git switch --create backport-813-to-1.6.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b57890e3503928ced7dfd75de6985b348f0e7307
# Push it to GitHub
git push --set-upstream origin backport-813-to-1.6.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.6.latest

Then, create a pull request where the base branch is 1.6.latest and the compare/head branch is backport-813-to-1.6.latest.

[github-actions]

The backport to 1.7.latest failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-1.7.latest 1.7.latest
# Navigate to the new working tree
cd .worktrees/backport-1.7.latest
# Create a new branch
git switch --create backport-813-to-1.7.latest
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 b57890e3503928ced7dfd75de6985b348f0e7307
# Push it to GitHub
git push --set-upstream origin backport-813-to-1.7.latest
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-1.7.latest

Then, create a pull request where the base branch is 1.7.latest and the compare/head branch is backport-813-to-1.7.latest.