Skip to content

Commit

Permalink
Merge pull request #129 from deepskies/development
Browse files Browse the repository at this point in the history
Development
  • Loading branch information
voetberg authored Oct 2, 2023
2 parents 3abe9da + 1690932 commit 14bd102
Show file tree
Hide file tree
Showing 145 changed files with 259,427 additions and 14,128 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/build-bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
run: |
python -m ensurepip
python -m pip install --upgrade pip
python -m pip install poetry==1.1.13
python -m pip install poetry==1.3.2
- name: Install dependencies
shell: bash
run: python -m poetry install
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ jobs:
run: |
python -m ensurepip
python -m pip install --upgrade pip
python -m pip install poetry==1.1.13
python -m pip install poetry==1.3.2
- name: View poetry --help
run: poetry --help
- name: Install dependencies
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test-bench.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
run: |
python -m ensurepip
python -m pip install --upgrade pip
python -m pip install poetry==1.1.13
python -m pip install poetry==1.3.2
- name: View poetry --help
run: poetry --help
Expand Down
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ __pycache__/
# Distribution / packaging
.Python
build/
_build/
develop-eggs/
dist/
downloads/
Expand Down
18 changes: 9 additions & 9 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,15 @@ repos:
- id: requirements-txt-fixer
- id: trailing-whitespace
- id: mixed-line-ending
- repo: https://github.com/asottile/pyupgrade
rev: v3.4.0
hooks:
- id: pyupgrade
- repo: https://github.com/psf/black
rev: 22.1.0
rev: 22.10.0
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v0.940
hooks:
- id: mypy
- repo: https://github.com/asottile/pyupgrade
rev: v2.31.0
hooks:
- id: pyupgrade
# - repo: https://github.com/pycqa/flake8
# rev: 6.0.0
# hooks:
# - id: flake8
17 changes: 17 additions & 0 deletions .readthedocs.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Required
version: 2

# Set the OS, Python version and other tools you might need
build:
os: "ubuntu-22.04"
tools:
python: "3.10"
jobs:
post_create_environment:
- pip install poetry
- poetry config virtualenvs.create false
post_install:
- poetry install

sphinx:
configuration: docs/conf.py
24 changes: 0 additions & 24 deletions CONTRIBUTING.md

This file was deleted.

215 changes: 166 additions & 49 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,82 +1,199 @@
# DeepBench
![GitHub Workflow Status](https://github.com/deepskies/DeepBench/actions/workflows/test-bench.yml/badge.svg?label=test)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![PyPI version](https://badge.fury.io/py/deepbench.svg)](https://badge.fury.io/py/deepbench)



![DeepBench Logo](docs/repository_support/DeepSkies_Logos_DeepBench.png)

### What is it?
Simulation library for very simple simulations to *benchmark* machine learning algorithms.
![DeepBench Logo](/repository_support/images/DeepSkies_Logos_DeepBench.png)



### Why do we need it? Why is it useful?
1. There are very universally recognized scientifically meaningful benchmark data sets, or methods with which to generate them.
2. A very simple data set will have objects, patterns, and signals that are intuitively quanitifiable and will be fast to generate.
3. A very simple data set will be a great testing ground for new networks and for newcomers to practice with the technology.

## Documentation

#### Readthedocs link coming soon!!

#### To build
```
pip install sphinx
cd docs
make html
```

The folder `docs/_build/html` will be populated with the documentation. Navigate to `file:///<Path To DeepBench>/docs/_build/html/index.html` in any web browser to view.

## Requirements
1. python 3.x
* python = ">=3.8,<3.11,"
* numpy = "^1.24.3"
* matplotlib = "^3.7.1"
* scikit-image = "^0.20.0"
* astropy = "^5.2.2"
* autograd = "^1.5"
* pyyaml = "^6.0"



## Install

You can use conda and the 'environment.yml' file to generate a conda environment for this project.
### From PyPi
```
pip install deepbench
```

### From Source

```
git clone https://github.com/deepskies/DeepBench.git
pip install poetry
poetry shell
poetry install
poetry run pytest --cov
```

## General Features
1. very fast to generate
2. Mimics in a very basic / toy way what is in astro images
3. Be fully controllable parametrically

![DeepBench Logo](docs/repository_support/DeepBench.png)

### Included Simulations

1. Astronomy Objects - simple astronomical object simulation
- Galaxy, Spiral Galaxy, Star

2. Shapes - simple 2D geometric shapes
- Rectangle, Regular Polygon, Arc, Line, Ellipse

3. Physics Objects - simple physics simulations
- Neutonian Pendulum, Hamiltonian Pendulum

## Example
![Example Image of pipeline](/repository_support/images/example_simplephysicalimage.png)


## Planned Features
1. Kinds of data to mimic
1. Strong lenses: Arcs, circles
2. Supernovae: light curves
3. Quasars: Skewed Sine wave
4. N-body simulations: Points in 2D and 3D and in lightcones
5. Galaxy clusters: Optical - points and 2d kernels; SZ - blurred circles; X-ray - blurred circles
6. Spectra (stellar, galactic)
7. Noise: photon, psf
2. Dimensions of data
1. point / graph
2. 1D, 2D, 3D, 4D (space-time), 6D (phase space)
3. Shapes: Polygons, Circles, squares, Polyhedrons, Spheres, cubes
3. Top Data Sets for Initial Release:
1. Strong lenses
2. Light curves


## Original Development Team

### Standalone
* Produce 3 instance of a pendulum over 10 different times with some level of noise.
```
import numpy as np
from deepbench.collection import Collection
configuration = {
"object_type": "physics",
"object_name": "Pendulum",
"total_runs": 3,
"parameter_noise": 0.2,
"image_parameters": {
"pendulum_arm_length": 2,
"starting_angle_radians": 0.25,
"acceleration_due_to_gravity": 9.8,
"noise_std_percent":{
"acceleration_due_to_gravity": 0
},
"object_parameters":{
"time": np.linspace(0, 1, 10)
}
}
phy_objects = Collection(configuration)()
objects = phy_objects.objects
parameters = phy_objects.object_parameters
```

* Produce a noisy shape image with a rectangle and an arc

```
import numpy as np
from deepbench.collection import Collection
configuration = {
"object_type": "shape",
"object_name": "ShapeImage",
"total_runs": 1,
"image_parameters": {
"image_shape": (28, 28),
"object_noise_level": 0.6
},
"object_parameters": {
[
"rectangle": {
"object": {
"width": np.random.default_rng().integers(2, 28),
"height": np.random.default_rng().integers(2, 28),
"fill": True
},
"instance": {}
},
"arc":{
"object": {
"radius": np.random.default_rng().integers(2, 28),
"theta1":np.random.default_rng().integers(0, 20),
"theta2":np.random.default_rng().integers(21, 180)
},
"instance":{}
}
]
}
}
shape_image = Collection(configuration)()
objects = shape_image.objects
parameters = shape_image.object_parameters
```


### Fine-Grained Control
* Make a whole bunch of stars
```
from deepbench.astro_object import StarObject
import numpy as np
star = StarObject(
image_dimensions = (28,28),
noise = 0.3,
radius= 0.8,
amplitude = 1.0
)
generated_stars = []
x_position, y_position = np.random.default_rng().uniform(low=1, high=27, size=(2, 50))
for x_pos, y_pos in zip(x_position, y_position):
generated-stars.append(star.create_object(x_pos, y_pos))
```


## Contributions
### Original Team
1. Craig Brechmos
2. Renee Hlozek
3. Brian Nord

### Refactor and Deployment
1. Ashia Livaudais
2. M. Voetberg

## How to contribute
I'm really glad you're reading this, because we need volunteer developers to help this project come to fruition.
### Pendulum Team
1. Becky Nevin
2. Omari Paul

### Testing
## Contributing
[Please view the deepskies contribution guidelines before submitting a code addition](https://github.com/deepskies/.github/blob/main/CONTRIBUTING.md)

### Submitting changes
## Acknowledgement

Please send a [GitHub Pull Request to simplephysicaliage](https://github.com/deepskies/SimplePhysicalImage/pull/new/master) with a clear list of what you've done (read more about [pull requests](http://help.github.com/pull-requests/)). When you send a pull request, we will love you forever if you include examples. We can always use more test coverage. Please follow our coding conventions (below) and make sure all of your commits are atomic (one feature per commit).

Always write a clear log message for your commits. One-line messages are fine for small changes, but bigger changes should look like this:
This work was produced by Fermi Research Alliance, LLC under Contract No. DE-AC02-07CH11359 with the U.S. Department of Energy. Publisher acknowledges the U.S. Government license to provide public access under the DOE Public Access Plan DOE Public Access Plan.
Neither the United States nor the United States Department of Energy, nor any of their employees, makes any warranty, express or implied, or assumes any legal liability or responsibility for the accuracy, completeness, or usefulness of any data, apparatus, product, or process disclosed, or represents that its use would not infringe privately owned rights.

$ git commit -m "A brief summary of the commit
>
> A paragraph describing what changed and its impact."

### Coding conventions

Start reading our code and you'll get the hang of it. We optimize for readability:

* We indent using tabs
* We ALWAYS put spaces after list items and method parameters (`[1, 2, 3]`, not `[1,2,3]`), around operators (`x += 1`, not `x+=1`), and around hash arrows.
* This is open source software. Consider the people who will read your code, and make it look nice for them. It's sort of like driving a car: Perhaps you love doing donuts when you're alone, but with passengers the goal is to make the ride as smooth as possible.
![GitHub Workflow Status](https://img.shields.io/github/workflow/status/AeRabelais/DeepBenchmark/build-bench)
![GitHub Workflow Status](https://img.shields.io/github/workflow/status/AeRabelais/DeepBench/test-bench?label=test)
[![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![PyPI version](https://badge.fury.io/py/deepbench.svg)](https://badge.fury.io/py/deepbench)
# DeepBench
A library for the simulation of useful physics benchmark datasets.
We acknowledge the Deep Skies Lab as a community of multi-domain experts and collaborators who’ve facilitated an environment of open discussion, idea-generation, and collaboration. This community was important for the development of this project.

Loading

0 comments on commit 14bd102

Please sign in to comment.