- Fix interpolation in Hydra config node (#1335)
- Remove
PluginsfromSingleton.get_state()(#1330)
- Support Python 3.9 . (#1062)
- Deprecate support for renaming packages via the command line. (#1140)
- Fixed hydra.job.id and hydra.job.num not getting passed to jobs in multirun (#1270)
- Support
additional_parametersas an optional param in the Submitit launcher plugin. - Add Optuna Sweeper plugin
- Limit OmegaConf depedency to 2.0 versions (#1253)
- Hydra no longer errorenously changes the USER environment variable in pytest unit tests once installed (#1059)
- Allow @ symbol in unquoted values in overrides (#1074)
- Fix a bug where Structured Config has a field like 'list : Optional[List[int]] = None` (#1117)
- Switch Ax Sweeper plugin to use new Hydra command line syntax
- Switch Nevergrad Sweeper plugin to use new Hydra command line syntax
- Revert the use of "_" to separate the path element in keys for the Ax sweeper plugin.
- Fixed interaction between interpolation and instantiate (#1001)
- Fix float parsing for values where the exponent starts with 0 (e.g 1e-05) (#999)
This is a small update improving support for config dataclasses generated by configen
- Overridden parameters annotated as Any in the config are not type checked by instantiate() (#982)
- Add ability to escape special characters in unquoted command line values (#954)
- Add support for merging dicts and lists specified in the commend line (#939)
- Fix appending dict values with new fields to config with + (#946)
- Fix for Override Grammar integer parsing: Integers are now compatible with Python integers ('+' prefix, correct handling of '_') (#954)
- Upgrade to OmegaConf 2.0.2 (#755)
Hydra 1.0 is out! See blog post for high level details.
Major new features in Hydra 1.0
- Config type safety via Structured Configs
- More powerful command line
- New plugins enabling remote launching and hyper parameter optimization
- Improved error reporting
- Reduce nesting levels with config packages
These release notes are covering the changes since 1.0.0rc4.
To see the complete release notes for 1.0 take a look at the release notes of:
- Add support for casting interval to ints and floats (#915)
- Fix to flush logging instead of shutting it down at job end (#833)
- compose() no longer initialized logging subsystem (#833)
- Fix overriding of hydra.job.env_set from the command line (#854)
- Fix tab completion to not output errors in some cases when using --multirun (#856)
- Fix OmegaConf custom resolvers not being propagated to Sweepers properly (#861)
- Interpolation accepts any char between ${ and } (#891)
- Update the signature of hydra.utils.instantiate() allow input configs of Any type (#896)
- Fix instantiate to work correctly when parameters are interpolations into a parent node (#904)
- Apps will now raise underlying exception if env HYDRA_FULL_ERROR=1 (#926)
- Fix integer/float choice from spec in the Nevergrad plugin
- New page in Structured Configs tutorial: Static schema with many configs (#918)
- New page in Structured Configs tutorial: Dynamic schema with many configs (#918)
Be aware that cfg.pretty() is now deprecated and you should use OmegaConf.to_yaml(cfg) instead.
- Upgrade to OmegaConf 2.0.1rc12 (#889)
- Fix bug with bash autocompletion for the users who has
.in their PATH environment variable (#868) - Fix instantiate to return None when None config is passed (#887)
- Extended Override grammar supports range(),glob(),type casts,shuffle, sort and more. (#752)
- Add support for specifying an additional config dir in the command line (#874)
- Allow overriding of the
defaultsandhydranodes in primary Structured Configs that does not mentioneddefaultsandhydra. (#877)
- ObjectConf is deprecated for TargetConf. See upgrade page. (#882)
- All plugins updated to use TargetConf. Be sure to update all your Hydra plugins (#882)
- Instantiated objects no longer have a params node in the conf. keys like hydra.launcher.params.foo are changed to hydra.launcher.foo. Update your configs and overrides if you were overriding such parameters. (#882)
- Fixed parsing of two nested lists with whitespace between them: [[a], [b]] (#836)
- New Extended command line syntax page (#752)
- Introducing a DSL for the command line override syntax (#797)
- Sanitize exception messages by default, showing only interesting stack frames (#646)
- Add a new --info flag to show Hydra debug information without running the user function. (#662)
- Add --package flag, can be used with --cfg to print a specific config package (#674)
- Improve the Compose API initialization methods (#702)
- Switch internal package resource loader from pkg_resources to importlb_resources (#719)
- Allow interpolation in the Hydra config node to access the application config (#771)
- Hydra provides pytest fixtures (hydra_sweep_runner, hydra_task_runner, hydra_restore_singletons) (#687)
- Added contexts to help initialize Hydra in a localized context (Mostly for Unit Tests and Jupyter Notebooks) (#687)
- Prints error when command line override looks like a sweep but --multirun is not specified (#712)
- Frozen Structured Configs are respected during composition command line overrides (#724)
- Hydra argparse flags are now consistently using dash-style. (#661)
- Restructure the configuration of the Submitit launcher (#682)
- Deprecated cls in favor of target in hydra.utils.{instantiate()/call{}} (#721)
- Add hydra/py.typed to pip package to enable type checking in downstream projects (#657)
- Fix initialization of Hydra in a module imported from a Jupyter Notebook (#695)
- Fix handling of main module override via HYDRA_MAIN_MODULE (#717)
- Fix interpolations in overrides to evaluate lazily (#725)
- Fixed handling of multirun when target config node is introduced by composition (#726)
- Sweepers are now validating config composition before launching (#775)
- Save
.hydra/*.yamlbefore task execution (#678)
- Add Redis Queue Launcher plugin (Jan-Matthis)
- Upgrade ax-platform used by Ax sweeper plugin to 0.1.13 or newer
- Add a unit testing example in the Hydra example application (#687)
- New Terminology page (#795)
- New Advanced/Hydra in Jupyter Notebooks page (#795)
- New Advanced/Hydra in Unit Tests page (#795)
- New Advanced/Override Grammar/Basic page (#795)
Hydra 1.0 is a major release introducing many new features and breaking some compatibility.
- Upgrade to OmegaConf 2.0 (Release notes) (#630)
- Optional config type safety via Structured Configs (#629)
- Improve command line and config composition error reporting (#349)
- Hydra config can now be accessed through interpolation using ${hydra:key}, for example ${hydra:job.name} (#325)
- Support for setting environment variable of running job (#7)
- Changes command line processing (requiring + and ~ prefixes for appending and removing items) (#598)
- Introducing
@packageheader for config files (#586) - Add command line override flags for
config_pathandconfig_name(#386) - hydra.main() now take an optional cfg object to passthrough to the function (#575)
- Add hydra.experimental.{initialize_with_file, initialize_with_module} (#574)
- Support for disabling the creation of the
.hydrasubdirectory by overriding "hydra.output_subdir" to "null" (#324) - Add
hydra.utils.call()to call methods and functions as well as instantiate objects. Search module paths more generically. (#498) - Add support for overriding package from command line and defaults list (#235)
- Config source is now abstracted, allowing additional config sources to be used (#257)
- New ConfigSource plugin API allowing configs to be provided by external plugins (#367)
- Add isort to ensure imports are always sorted (#340)
- Codebase is now passing mypy --strict type checking (#342)
- Improve performance of plugin discovery and instantiation (#489)
- Modules whose name starts with "_" are skipped during plugin discovery (#494)
- Add Ax Sweeper plugin (Shagun Sodhani)
- Add Nevergrad Sweeper plugin (Jérémy Rapin)
- Add Joblib Launcher plugin (Jan-Matthis)
- Add Submitit Launcher plugin to launch jobs to SLURM clusters
- Add Fish shell Tab Completion plugin (Binsheng Liu) (#549)
- Drop support Python 2.7 and 3.5 (#313)
- hydra.main() now takes an additional optional config_name and composite-style config_path is deprecated (#395)
- Launcher API launch method now takes an additional initial_job_idx indicating the id of the first job in the batch (#284)
- Singleton metaclass is now exposed at hydra.core.Singleton (#371)
- Moved HydraConfig from hydra.plugins.common.utils to hydra.core (#371)
- Move several formerly internal APIs to hydra/core to ensure plugins does not need to use internal APIs (#371)
- Plugin import now requires explicit name (from hydra.plugins.launcher import Launcher) (#371)
- Object Config "class" field is deprecated in favor of "cls" and will be removed in a future version. (#389)
- Experimental compose API config_file changed to config_name (#395)
- User plugins should be modified to not import twice during plugin discovery. see issue for details. (#482)
- Change hydra.core.plugins.Plugins class to a Singleton. access should be changed to the pattern Plugins.instance().foo() (#489)
- Plugins should now include test fixtures (sweep_runner, task_runner) via a standardized conftest.py (#521)
- Switch Python 3 native namespace packages for plugins (See task for details) (#534)
- Packaged configuration directories now requires an
__init__.pyat their top level (#536) - Appending config groups to the defaults list via the command line now requires a + prefix (#598)
- Removing an item from the defaults list by assigning null (db=null) is deprecated, use ~db instead (#598)
- Installed Hydra applications no longer need have an additional
entry()function on the stack (#92)
- Fix a bug causing sys.exit() error code to not be propagated (#351)
- Shutdown logging subsystem aftter job finishes to ensure log files are flushed and closed (#378)
- Fix a bug with utils.instantiate() failing if params contains interpolated values. (#388)
- Allow hydra.utils.instantiate() to accept non primitive objects for passthrough by name (#400)
- Fix to work when an Hydra app is executed in Jupyter notebook using the %run command (#481)
- Plugins are no longer imported twice during plugin discovery (#482)
- to_absolute_dir(path) now converts relative path to be relative to os.cwd() when used outside of Hydra (#496)
- Working examples are provided for all Hydra plugins in plugins/examples (#253)
- The basic tutorial was rewritten to reflect many changes (#602)
- Add a new tutorial covering Structured Configs (#628)
- Pin Hydra 0.11 to OmegaConf 1.4 to avoid breaking compatibility when OmegaConf 2.0 is released (#334)
- Document a simple Ray example (#317)
- Change website from cli.dev to hydra.cc (#314)
- Fixes --cfg command line flag not working (#305)
- Add hydra.experimental API for composing configuration on demand (hydra.experimental.compose) (#219)
- Add hydra.utils.get_original_cwd() to access original working directory and hydra.utils.to_absolute_path() to convert a path to absolute path (#251)
- Change hydra logging format pattern, example: "[2019-10-22 16:13:10,769][HYDRA] Installed Hydra Plugins" (#254)
- Change --cfg to require config type (one of 'job', 'hydra' or 'all') (#270)
- Upgrade to OmegaConf 1.4.0, see full change log here (#280)
- Experimental support for Jupyter notebooks via compose API (#281)
- Allow configuring override_dirname via hydra.job.config.override_dirname to exclude specific keys or change separator characters (#95)
- Fix a bug that caused out of order composition when mixing config-groups with non-config-group in the defaults block (#261)
- Allow '=' to be used in the value of an override, eg. foo=bar=10 (key foo, value bar=10). (#266)
- Allow composing config files with dot in their name (foo.bar.yaml) (#271)
- hydra-colorlog plugin adds colored log output.
- Document utils.get_original_cwd() and utils.to_absolute_path("foo") (#251)
- Change the default sweep subdir from ${hydra.job.num}_${hydra.job.id} to ${hydra.job.num} (#150)
- App help now contains config groups and generated config App help can now be customized Hydra help is now available via --hydra-help (#1)
- Simplify install and uninstall commands for tab completion (#200)
- hydra.runtime.cwd now contains the original working directory the app was launched from (#244)
- Fix an error with tab completion that occurred when a TAB was pressed after two spaces (#203)
- Fix a bug when sweeping over groups that are used in an interpolation in defaults (#206)
- Fix tab completion for cases where app name looks like foo.par (#213)
- Describe news fragment types in more details in development/contributing (#150)
- Examples dir now mirrors the structure of the website docs (#209)
- Old demos directory was removed and a new tutorial directory as added (#167)
- Make strict mode the default when a config file is specified directly (#150)
- Replace --verbose with a standard override (hydra.verbose) (#161)
- Add IntegrationTestSuite to test_utils for testing Launcher plugins (#168)
- Add support for specifying which config to print in -c, options are job (default), hydra or all. (#176)
- Hydra is now hosted on https://cli.dev! (#75)
- Move all Hydra configuration files to a subfolder (.hydra by default) under the output folder (#77)
- Fix a bug in tab completion of missing mandatory values. (#145)
- Fix a bug with multirun overriding free config groups (not in defaults) when strict mode is enabled (#181)
- Fix a bug that prevented sweeping over an unspecified ('???') default group (#187)
- Updated the primary tutorial on the web site and added a brand new tutorial directory (#58)
- Documented debugging methods in website (#6)
- Move FAIRTask, Submititit and FAIR-cluster-defaults plugins to fairinternal/hydra-fair-plugins repository (#138)
- Remove Fairtask and Submitit example configs from demos as they are no longer needed (#146)
- Created hydra-core pip package, Hydra can now installed with 'pip install hydra-core' (#143)
- Finalized submitit support (#18)
- Add default launcher config for Fairtask and Submitit launcher plugins
- hydra.utils.instantiate() can now take an additional optional kwargs dictionary to pass to constructor of the instantiated object (#142)
- Initial support for bash completion, see documentation for details (#8)
- Fixed Town Crier to generate correct links to issues in generated news file. (#122)
- Fixed bug with overriding hydra defaults from user config file. (#123)
- Fixed Singletons not getting serialized to remote jobs. (#128)
- Fixed a bug that prevented using submitit in strict mode (#129)
- Fixed example submitit config to set the job name (#133)
- Changed Hydra core defaults to be more appropriate for open source use. To get the FAIR cluster defaults install the fair_cluster plugin (#103)
- Changed output directories default to group by day (output/DAY/TIME instead of output/DAY_TIME) (#121)
- Added the ability to prevent a default from being merged it by assigning null to it (hydra/launcher=null) (#106)
- Implemented Plugin discovery mechanism, this can be used to find all installed plugins that implement a specific interface (#119)
- Implemented an API to for manipulating the config search path (#120)
- Fixed config loading to properly support the use case in demos/8_specializing_config (#104)
- Fixed error message when the user config contains defaults that is a mapping and not a list (#107)
- Fixed config loading order to allow properly overriding Hydra config groups from user config (#115)
- New plugin: fair_cluster Change Hydra defaults to be appropriate for the FAIR cluster
- Initial search path and plugins documentation (#103)
- Hydra config groups were moved to the hydra/namespace (#101)
- Removed support for .hydra directory, Hydra can be configured directly from the job config. (#91)
- Config loading rewrite fixed #88 (#88)
- Move non-public APIs into hydra._internal:
- Moved non-API code into hydra._interanl to flag it as private.
- Moved plugin interfaces into hydra.plugins.
- Moved code meant to be used by plugins to hydra.plugins.common. (#78)
- Integrated towncrier to bring you this news! (#45)
- Hydra is now compatible with Windows (#63)
- Hydra apps can now be packaged and installed along with their configuration files. (#87)
- It is now possible to use ${override_dirname} in the output directory of a local job (#31)
- Override_dirname separator changed from : to =, for example: foo/a:10,b:10 => foo/a=10,b=10 (#63)
- Fixed automatic detection of missing copyright headers (#72)
- fixed a bug that caused an empty config to be returned if specifed config file did not have a .yaml extension. (#80)
- Multi change diff:
- Logging config search path in verbose logging to assist debugging of config load issues
- Saving hydra.yaml into the job dir to assist debugging hydra issues
- Fixed a bug caused by fairtask logging change
- Improved integration-tests debuggability by switching hydra to debug logging in them
- Added selective plugin testing to nox using env, for example PLUGINS=fairtask would only test fairtask. (#87)
- Improved the contributing docs (#45)
- Documented Hydra app packaging under Deployment/Application packaging (#87)