Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clean up Stable Diffusion tests #15748

Merged
merged 1 commit into from
Dec 9, 2024
Merged

Clean up Stable Diffusion tests #15748

merged 1 commit into from
Dec 9, 2024

Conversation

esmalTT
Copy link
Contributor

@esmalTT esmalTT commented Dec 5, 2024

Problem description

The Stable Diffusion source code is unorganized and full of dead code. There are several disabled tests contributing to CI instability for this model.

This PR attempts to improve overall model stability, organization, and maintainability.

What's changed

  • Consolidate all SD tests under models/demos/wormhole/stable_diffusion/tests instead of having them scattered around the repository.
  • Re-enable tests that are no longer broken.
  • Remove _new_conv postfix from test names.
  • Add UNet2DConditionModel to post-commit pipeline. EDIT: This fails on CI for some reason, I'm going to merge this without post-commit and investigate in a separate issue:
  • Create a dedicated job in the nightly pipeline for SD to catch model regressions early.

Follow-on work should tackle renaming the files in models/demos/wormhole/stable_diffusion/tt. It seems like there are several un-tested and un-unused modules in that directory.

Checklist

  • Model regression CI testing passes (if applicable) link

@esmalTT esmalTT force-pushed the esmal/sd-cleanup branch 10 times, most recently from 586c185 to e5509e7 Compare December 6, 2024 17:05
@esmalTT esmalTT force-pushed the esmal/sd-cleanup branch 3 times, most recently from cfe0c01 to 219042b Compare December 9, 2024 13:21
Refactor the Stable Diffusion test suite and CI jobs to improve model
stability and maintainability.

 - Move all Stable Diffusion tests into the model directory for better
   organization.

 - Remove unnecessary postfixes from test filenames.

 - Delete dead test code, including tests unrelated to SD modules and
   code for deprecated SD variants using 224x224 input shapes.

 - Re-enable previously broken tests in CI.

 - Update the CODEOWNERS file to reflect moved/deleted directories and
   files.
- model: "wh_b0_unstable"
cmd: ./tests/scripts/single_card/nightly/run_wh_b0_unstable.sh
- model: "stable_diffusion"
cmd: pytest --timeout 900 -n auto tests/nightly/single_card/stable_diffusion
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could just move this into the models section then

@esmalTT esmalTT merged commit 4b6e844 into main Dec 9, 2024
9 checks passed
@esmalTT esmalTT deleted the esmal/sd-cleanup branch December 9, 2024 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants