Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Ray Workflow: Multiple Run Support, Distributed Hyperparameter Tuning, and Consistent Setup Across Local/Cloud #1301

Open
wants to merge 132 commits into
base: main
Choose a base branch
from

Conversation

glvov-bdai
Copy link
Collaborator

@glvov-bdai glvov-bdai commented Oct 25, 2024

Description

This PR adds Ray support, which enables a lot of really cool stuff by leveraging the existing Hydra support, including but not limited to:

  • Several training runs at once in parallel or consecutively with minimal interaction
  • Using the same training setup everywhere (on cloud and local) with minimal overhead
  • Tuning hyperparameters
  • Tuning hyperparameters in parallel on multiple GPUs and/or multiple GPU Nodes
  • Simultaneously tuning model hyperparameters for different environments/agents
  • Resource Isolation

I know this PR seems huge, but most of the code diff is config files / argparser stuff / documentation / comments

My main project at BDAI is changing from RL to LfD effective November 1st, so I'm posting this PR as early as possible so I have as much time as possible to address comments.

It would be much appreciated if the NVIDIA folks can work with me to get this reviewed ASAP. I realize that this is a pretty big PR; but I also think that it adds a lot of cool functionality, and the merging process will go much smoother if I am able to devote time to this while at work. Thanks! ;)

Fixes # (issue#1190), (issue#1213)

Type of change

  • New feature (non-breaking change which adds functionality)
  • This change requires a documentation update

Screenshots

image
image

image

image

Checklist

  • I have run the pre-commit checks with ./isaaclab.sh --format
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the changelog and the corresponding version in the extension's config/extension.toml file
  • I have added my name to the CONTRIBUTORS.md or my name already exists there

@garylvov
Copy link
Contributor

garylvov commented Nov 4, 2024

I also made this walk-through video for this integration. I'd be happy to link it in the ray.rst lmk what you think

https://www.youtube.com/watch?v=z7MDgSga2Ho

Copy link
Collaborator

@jsmith-bdai jsmith-bdai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did another pass through, still need to try it out

source/standalone/workflows/ray/isaac_ray_tune.py Outdated Show resolved Hide resolved
source/standalone/workflows/ray/isaac_ray_util.py Outdated Show resolved Hide resolved
docs/source/features/ray.rst Outdated Show resolved Hide resolved
docs/source/features/ray.rst Outdated Show resolved Hide resolved
source/standalone/workflows/ray/isaac_ray_tune.py Outdated Show resolved Hide resolved
docs/source/features/ray.rst Show resolved Hide resolved
docs/source/features/ray.rst Show resolved Hide resolved
docs/source/features/ray.rst Outdated Show resolved Hide resolved
source/standalone/workflows/ray/isaac_ray_util.py Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants