Skip to content

Commit

Permalink
fix pre-commit hook errors
Browse files Browse the repository at this point in the history
  • Loading branch information
mcw92 committed Mar 28, 2024
1 parent baaf81f commit 562716d
Show file tree
Hide file tree
Showing 51 changed files with 264 additions and 300 deletions.
2 changes: 1 addition & 1 deletion .figs/hai_kit_logos.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
24 changes: 12 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Contributing to Propulate

Welcome to ``Propulate``! We're thrilled that you're interested in contributing to our open-source project.
By participating, you can help improve the project and make it even better.
Welcome to ``Propulate``! We're thrilled that you're interested in contributing to our open-source project.
By participating, you can help improve the project and make it even better.

## How to Contribute

Expand All @@ -17,13 +17,13 @@ By participating, you can help improve the project and make it even better.
git checkout -b your-feature-name
```

4. **Make Changes**: Make your desired changes to the codebase. Please stick to the following guidelines:
4. **Make Changes**: Make your desired changes to the codebase. Please stick to the following guidelines:
* `Propulate` uses [*Black*](https://black.readthedocs.io/en/stable/the_black_code_style/current_style.html) code style and so should you if you would like to contribute.
* Please use type hints in all function definitions.
* Please use American English for all comments and docstrings in the code.
* `Propulate` uses Sphinx autoapi to automatically create API reference documentation from docstrings in the code.
* `Propulate` uses Sphinx autoapi to automatically create API reference documentation from docstrings in the code.
Please use the [NumPy Docstring Standard](https://numpydoc.readthedocs.io/en/latest/format.html) for your docstrings:

```python
"""
Short Description
Expand Down Expand Up @@ -72,7 +72,7 @@ By participating, you can help improve the project and make it even better.
```
When applicable, please make references to parent modules and classes using ```:class:`ParentClassName` ```
as follows:

```python
"""
This is the docstring for MyClass.
Expand All @@ -97,26 +97,26 @@ as follows:
"""
The docstring for the parent class.
"""

class MyClass(ParentClassName):
"""
The docstring for MyClass.
Parameters
----------
param2 : type
Description of param2.
Attributes
----------
attr2 : type
Description of attr2.
"""
```
In the example above, ``` :class:`ParentClassName` ``` is used to create a reference to the parent class `ParentClassName`.
In the example above, ``` :class:`ParentClassName` ``` is used to create a reference to the parent class `ParentClassName`.
Sphinx autoapi will automatically generate links to the parent class documentation.


5. **Commit Changes**: Commit your changes with a clear and concise commit message:
```bash
git commit -m "Add your commit message here"
Expand Down
88 changes: 41 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,79 +14,79 @@
[![Documentation Status](https://readthedocs.org/projects/propulate/badge/?version=latest)](https://propulate.readthedocs.io/en/latest/?badge=latest)
![](./coverage.svg)

# **Click [here](https://www.scc.kit.edu/en/aboutus/16956.php) to watch our 3 min introduction video!**
# **Click [here](https://www.scc.kit.edu/en/aboutus/16956.php) to watch our 3 min introduction video!**

## What `Propulate` can do for you

`Propulate` is an HPC-tailored software for solving optimization problems in parallel. It is openly accessible and easy
to use. Compared to a widely used competitor, `Propulate` is consistently faster - at least an order of magnitude for a
`Propulate` is an HPC-tailored software for solving optimization problems in parallel. It is openly accessible and easy
to use. Compared to a widely used competitor, `Propulate` is consistently faster - at least an order of magnitude for a
set of typical benchmarks - and in some cases even more accurate.

Inspired by biology, `Propulate` borrows mechanisms from biological evolution, such as selection, recombination, and
mutation. Evolution begins with a population of solution candidates, each with randomly initialized genes. It is an
iterative "survival of the fittest" process where the population at each iteration can be viewed as a generation. For
each generation, the fitness of each candidate in the population is evaluated. The genes of the fittest candidates are
Inspired by biology, `Propulate` borrows mechanisms from biological evolution, such as selection, recombination, and
mutation. Evolution begins with a population of solution candidates, each with randomly initialized genes. It is an
iterative "survival of the fittest" process where the population at each iteration can be viewed as a generation. For
each generation, the fitness of each candidate in the population is evaluated. The genes of the fittest candidates are
incorporated in the next generation.

Like in nature, `Propulate` does not wait for all compute units to finish the evaluation of the current generation.
Instead, the compute units communicate the currently available information and use that to breed the next candidate
Like in nature, `Propulate` does not wait for all compute units to finish the evaluation of the current generation.
Instead, the compute units communicate the currently available information and use that to breed the next candidate
immediately. This avoids waiting idly for other units and thus a load imbalance.
Each unit is responsible for evaluating a single candidate. The result is a fitness level corresponding with that
candidate’s genes, allowing us to compare and rank all candidates. This information is sent to other compute units as
Each unit is responsible for evaluating a single candidate. The result is a fitness level corresponding with that
candidate’s genes, allowing us to compare and rank all candidates. This information is sent to other compute units as
soon as it becomes available.
When a unit is finished evaluating a candidate and communicating the resulting fitness, it breeds the candidate for the
next generation using the fitness values of all candidates it evaluated and received from other units so far.
When a unit is finished evaluating a candidate and communicating the resulting fitness, it breeds the candidate for the
next generation using the fitness values of all candidates it evaluated and received from other units so far.

`Propulate` can be used for hyperparameter optimization and neural architecture search.
It was already successfully applied in several accepted scientific publications. Applications include grid load
`Propulate` can be used for hyperparameter optimization and neural architecture search.
It was already successfully applied in several accepted scientific publications. Applications include grid load
forecasting, remote sensing, and structural molecular biology.

## In more technical terms

``Propulate`` is a massively parallel evolutionary hyperparameter optimizer based on the island model with asynchronous
``Propulate`` is a massively parallel evolutionary hyperparameter optimizer based on the island model with asynchronous
propagation of populations and asynchronous migration.
In contrast to classical GAs, ``Propulate`` maintains a continuous population of already evaluated individuals with a
In contrast to classical GAs, ``Propulate`` maintains a continuous population of already evaluated individuals with a
softened notion of the typically strictly separated, discrete generations.
Our contributions include:
- A novel parallel genetic algorithm based on a fully asynchronized island model with independently processing workers.
- Massive parallelism by asynchronous propagation of continuous populations and migration via efficient communication using the message passing interface.
- Optimized use efficiency of parallel hardware by minimizing idle times in distributed computing environments.

To be more efficient, the generations are less well separated than they usually are in evolutionary algorithms.
New individuals are generated from a pool of currently active, already evaluated individuals that may be from any
New individuals are generated from a pool of currently active, already evaluated individuals that may be from any
generation.
Individuals may be removed from the breeding population based on different criteria.

You can find the corresponding publication [here](https://doi.org/10.1007/978-3-031-32041-5_6):
> Taubert, O. *et al.* (2023). Massively Parallel Genetic Optimization Through Asynchronous Propagation of Populations.
> In: Bhatele, A., Hammond, J., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023.
> Lecture Notes in Computer Science, vol 13948. Springer, Cham.
You can find the corresponding publication [here](https://doi.org/10.1007/978-3-031-32041-5_6):
> Taubert, O. *et al.* (2023). Massively Parallel Genetic Optimization Through Asynchronous Propagation of Populations.
> In: Bhatele, A., Hammond, J., Baboulin, M., Kruse, C. (eds) High Performance Computing. ISC High Performance 2023.
> Lecture Notes in Computer Science, vol 13948. Springer, Cham.
> [doi.org/10.1007/978-3-031-32041-5_6](https://doi.org/10.1007/978-3-031-32041-5_6)
## Documentation

Check out the full documentation at [https://propulate.readthedocs.io/](https://propulate.readthedocs.io/) :rocket:! Here you can find installation
Check out the full documentation at [https://propulate.readthedocs.io/](https://propulate.readthedocs.io/) :rocket:! Here you can find installation
instructions, tutorials, theoretical background, and API references.

**:point_right: If you have any questions or run into any challenges while using `Propulate`, don't hesitate to post an
[issue](https://github.com/Helmholtz-AI-Energy/propulate/issues) :bookmark:, reach out via [GitHub
discussions](https://github.com/Helmholtz-AI-Energy/propulate/discussions) :octocat:, or contact us directly via e-mail
:email: to [[email protected]](mailto:[email protected]).**
**:point_right: If you have any questions or run into any challenges while using `Propulate`, don't hesitate to post an
[issue](https://github.com/Helmholtz-AI-Energy/propulate/issues) :bookmark:, reach out via [GitHub
discussions](https://github.com/Helmholtz-AI-Energy/propulate/discussions) :octocat:, or contact us directly via e-mail
:email: to [[email protected]](mailto:[email protected]).**

## Installation

You can install the latest stable release from PyPI: ``pip install propulate``
You can install the latest stable release from PyPI: ``pip install propulate``
If you need the latest updates, you can also install ``Propulate`` directly from the master branch at you own risk.
Pull and run ``pip install -e .`` or ``python setup.py develop``.
``Propulate`` depends on [``mpi4py``](https://mpi4py.readthedocs.io/en/stable/) and requires an MPI implementation under
Pull and run ``pip install -e .`` or ``python setup.py develop``.
``Propulate`` depends on [``mpi4py``](https://mpi4py.readthedocs.io/en/stable/) and requires an MPI implementation under
the hood. Currently, it is only tested with [OpenMPI](https://www.open-mpi.org/).

## Quickstart
*Below, you can find a quick recipe for how to use `Propulate` in general. Check out the official
[ReadTheDocs](https://propulate.readthedocs.io/en/latest/tut_propulator.html) documentation for more detailed tutorials
*Below, you can find a quick recipe for how to use `Propulate` in general. Check out the official
[ReadTheDocs](https://propulate.readthedocs.io/en/latest/tut_propulator.html) documentation for more detailed tutorials
and explanations.*

Let's minimize the sphere function $f_\text{sphere}\left(x,y\right)=x^2 +y^2$ with `Propulate` as a quick example. The
Let's minimize the sphere function $f_\text{sphere}\left(x,y\right)=x^2 +y^2$ with `Propulate` as a quick example. The
minimum is at $\left(x, y\right)=\left(0,0\right)$ at the orange star.
![](./docs/images/sphere.png)
First, we need to define the key ingredients that define our optimization problem:
Expand All @@ -95,7 +95,7 @@ First, we need to define the key ingredients that define our optimization proble
- A tuple of `float` for a continuous parameter, e.g., `{"learning_rate": (0.0001, 0.01)}`
- A tuple of `int` for an ordinal parameter, e.g., `{"conv_layers": (2, 10)}`
- A tuple of `str` for a categorical parameter, e.g., `{"activation": ("relu", "sigmoid", "tanh")}`

Thus, an exemplary search space might look like this:
```python
search_space = {
Expand All @@ -105,19 +105,19 @@ First, we need to define the key ingredients that define our optimization proble
}
```

The sphere function has two continuous parameters, $x$ and $y$, and we consider $x,y\in\left[-5.12,5.12\right]$. The
The sphere function has two continuous parameters, $x$ and $y$, and we consider $x,y\in\left[-5.12,5.12\right]$. The
search space in our example thus looks like this:
```python
limits = {
"x": (-5.12, 5.12),
"y": (-5.12, 5.12)
}
}
```
- The **loss function**. This is the function we want to minimize in order to find the best parameters. It can be any
- The **loss function**. This is the function we want to minimize in order to find the best parameters. It can be any
`Python` function that
- takes a set of parameters as a `Python` dictionary as an input.
- returns a scalar loss value that determines how good the tested parameter set is.

In this example, the loss function whose minimum we want to find is the sphere function:
```python
def sphere(params: Dict[str, float]) -> float:
Expand All @@ -131,15 +131,15 @@ First, we need to define the key ingredients that define our optimization proble
----------
params: Dict[str, float]
The function parameters.
Returns
-------
float
The function value.
"""
return numpy.sum(numpy.array(list(params.values())) ** 2).item()
```
Next, we need to define the **evolutionary operator** or propagator that we want to use to breed new individuals during the
Next, we need to define the **evolutionary operator** or propagator that we want to use to breed new individuals during the
optimization process. `Propulate` provides a reasonable default propagator via a utility function:
```python
# Set up logger for Propulate optimization.
Expand All @@ -166,7 +166,7 @@ propulator = propulate.Propulator(
checkpoint_path=config.checkpoint,
)
```
Now we can run the actual optimization. Overall, ``generations * mpi4py.MPI.COMM_WORLD.size`` evaluations will be
Now we can run the actual optimization. Overall, ``generations * mpi4py.MPI.COMM_WORLD.size`` evaluations will be
performed:
```python
# Run optimization and print summary of results.
Expand Down Expand Up @@ -219,9 +219,3 @@ Do the following to run the [example script](https://github.com/Helmholtz-AI-Ene
## Acknowledgments
*This work is supported by the Helmholtz AI platform grant.*
![](./.figs/hai_kit_logos.svg)






2 changes: 1 addition & 1 deletion RELEASE.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,4 +5,4 @@ The current workflow for releasing a new version of `Propulate` is as follows:
2. On the master branch, update the version number in `setup.cfg`. We use semantic versioning.
3. Rebase release branch onto current master branch.
4. Make Github release from current master, including corresponding version tag.
5. Push release branch. This will trigger a Github action publishing the new release on PyPI.
5. Push release branch. This will trigger a Github action publishing the new release on PyPI.
2 changes: 1 addition & 1 deletion docs/_static/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -7493,4 +7493,4 @@ span[id*='MathJax-Span'] {
*/
.rst-content dl.field-list > dd {margin-top: 5px !important; margin-bottom: 4px !important}
.rst-content dl.field-list > dt {margin-top: 5px !important; margin-bottom: 4px !important}
.rst-content dl dd > p:last-child {margin-top: 4px !important}
.rst-content dl dd > p:last-child {margin-top: 4px !important}
2 changes: 1 addition & 1 deletion docs/algos_explained.rst
Original file line number Diff line number Diff line change
Expand Up @@ -28,4 +28,4 @@ in ``Propulate``.
overview_pop
eas
pso
cmaes
cmaes
2 changes: 1 addition & 1 deletion docs/cmaes.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ Covariance Matrix Adaption Evolution Strategy
=============================================

.. note::
The documentation for Covariance Matrix Adaption Evolution Strategy (CMA-ES) will be available soon.
The documentation for Covariance Matrix Adaption Evolution Strategy (CMA-ES) will be available soon.
2 changes: 1 addition & 1 deletion docs/eas.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,4 +68,4 @@ Mutation
:width: 76 %
:align: center

|
|
2 changes: 1 addition & 1 deletion docs/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ with ``pip`` or via ``setup.py``:
$ git clone https://github.com/Helmholtz-AI-Energy/propulate
$ pip install -e .
Alternatively:

.. code-block:: console
Expand Down
2 changes: 1 addition & 1 deletion docs/island_model.rst
Original file line number Diff line number Diff line change
Expand Up @@ -41,4 +41,4 @@ from island 2.
:width: 100 %
:align: center

|
|
2 changes: 1 addition & 1 deletion docs/overview_pop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,4 @@ Cuckoo Search (CS)
Cuckoo search simulates the behavior of cuckoo birds' reproduction strategy. Each solution corresponds to a cuckoo
egg, and each nest represents a potential solution. Cuckoos lay eggs in nests, and the nests with better eggs
(solutions) survive and reproduce. The algorithm involves creating new solutions by combining existing ones and
occasionally replacing solutions to maintain diversity.
occasionally replacing solutions to maintain diversity.
2 changes: 1 addition & 1 deletion docs/parallel_pop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,4 +56,4 @@ Worker 2 receives light the blue :math:`\small\mathrm{ind}_{g3}` only after fini
:width: 100 %
:align: center

|
|
2 changes: 1 addition & 1 deletion docs/pop.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,4 +32,4 @@ HOW?
.. warning::

As with all metaheuristics, these algorithms have hyperparameters themselves and their effectiveness strongly depends
on choosing proper hyperparameters and problem-specific considerations.
on choosing proper hyperparameters and problem-specific considerations.
2 changes: 1 addition & 1 deletion docs/pso.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,4 @@ Particle Swarm Optimization
===========================

.. note::
The documentation for particle swarm optimization (PSO) will be available soon.
The documentation for particle swarm optimization (PSO) will be available soon.
2 changes: 1 addition & 1 deletion docs/quickstart.rst
Original file line number Diff line number Diff line change
Expand Up @@ -66,4 +66,4 @@ documentation |:raised_hands:|.


.. Links
.. _subfolder: https://github.com/Helmholtz-AI-Energy/propulate/tree/master/tutorials
.. _subfolder: https://github.com/Helmholtz-AI-Energy/propulate/tree/master/tutorials
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
sphinx-autoapi==3.0.0
sphinx-rtd-theme==1.2.0
sphinxcontrib-napoleon
sphinxemoji
sphinxemoji
1 change: 0 additions & 1 deletion docs/tut_islands.rst
Original file line number Diff line number Diff line change
Expand Up @@ -145,4 +145,3 @@ With ten MPI ranks and two islands with five workers each, the output looks like
[2024-03-13 12:30:46,611][propulate.propulator][INFO] - Top 1 result(s) on island 0:
(1): [{'a': '-2.83E-4', 'b': '1.04E-3'}, loss 1.16E-6, island 0, worker 3, generation 901]
2 changes: 1 addition & 1 deletion docs/tut_multi_rank_worker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -11,4 +11,4 @@ In addition to the already explained functionality, ``Propulate`` enables using
parallelized evaluation of the loss function. This is useful for, e.g., data-parallel training of neural networks during
the hyperparameter optimization, where each individual network is trained on multiple GPUs.

A more detailed explanation of the tutorial will be available soon |:rocket:|.
A more detailed explanation of the tutorial will be available soon |:rocket:|.
4 changes: 2 additions & 2 deletions docs/tut_propulator.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ As the very first step, we need to define the key ingredients that define the op
----------
params: Dict[str, float]
The function parameters.
Returns
-------
float
Expand Down Expand Up @@ -223,4 +223,4 @@ will use your current working directory.
.. warning::
If you start an optimization run from existing checkpoints, those checkpoints must be compatible with your current
parallel computing environment. This means that if you use a checkpoint created in a setting with 20 processing
elements in a different computing environment with, e.g., 10 processing elements, the behaviour is undefined.
elements in a different computing environment with, e.g., 10 processing elements, the behaviour is undefined.
Loading

0 comments on commit 562716d

Please sign in to comment.