Skip to content
This repository has been archived by the owner on May 10, 2024. It is now read-only.

Commit

Permalink
Merge pull request #11 from alercebroker/library_refactor
Browse files Browse the repository at this point in the history 
Library refactor: v2.0.0
  • Loading branch information
@ignacioreyes
ignacioreyes authored May 18, 2021
2 parents c226abf + e236d2a commit c9256a6
Show file tree
Hide file tree
Showing 67 changed files with 5,606 additions and 2,260 deletions.
2 changes: 1 addition & 1 deletion LICENSE
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
MIT License

Copyright (c) 2020 Automatic Learning for the Rapid Classification of Events (ALeRCE)
Copyright (c) 2021 Automatic Learning for the Rapid Classification of Events (ALeRCE)

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
Expand Down
46 changes: 29 additions & 17 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,35 +14,47 @@ pip install lc-classifier
For development:

```
pip install -e .
python -m pip install -e .
```

# Functionalities
## How to use the library?

## Augmentation
If you want more samples you can use our implementation of data augmentation. For now you can use ShortTransientAugmenter for some transients.
Check the available Jupyter notebooks in the *examples* directory.

## Classifier
The classifier code contains BaseClassifier (a simple random forest) and HierarchicalRandomForest (a random forest with internal hierarchy), both with methods for fit and predict.
## Functionalities

## Features
For train the models, we use features of astronomical time series. The features are obtained from our different extractors, these receive a DataFrame with detections indexed by `oid`.
### Feature computation
This library provides an extensive number of feature extractors for astronomical
light curves, including period computation, autoregresive models, parametric models,
statistical features, etc. We also provide tools to transform your data into
the format that this library expects (Pandas dataframes).

### Preprocess:
Before to get features, we preprocess the time series with filters and boundary conditions:
- Drop duplicates.
### Augmentation
If you want more samples you can use our ShortTransientAugmenter class.
More data augmentation techniques will be implemented in further releases.

### Classifier
Two classifiers are available: A traditional Random Forest model, and a hierarchical
model made from 4 Random Forest classifiers.

### Preprocessing for ZTF data:
Before computing features, we preprocess the time series with filters
and boundary conditions:
- Drop duplicate observations.
- Discard noisy detections.
- Discard bogus.
- Filter time series with more than 5 detections.
- Discard invalid values (like nans and infinite).

### Extractors:
The extractors are the portion of code with the logic to extract features from time series. Each extractor do only one task, after that our CustomHierarchicalExtractor merge all extractors for get features to train the model.

##### How can I add extractors to library?
You can use inheritance from `base extractors` and use it for create your own extractor. For now you can inherit:
- `FeatureExtractor` is a generic extractor only fill methods.
- `FeatureExtractorSingleBand` is a extractor that compute features by band.
## How can I add my own feature extractors to the library?
Feature extractors extend the following classes:
- `FeatureExtractor`
- `FeatureExtractorSingleBand`. This type of feature extractor runs independently
on each available band.

Check out the existent feature extractors in the directory
*lc_classifier/features/extractors*.


## Profile functionalities
Expand Down
17 changes: 17 additions & 0 deletions data_examples/SN2012aw/sn2012aw.info
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
Name: sn2012aw
Type: SNIIP
Zhel: 0.002595
Redshift_error: 0.000013
Zcmb: 0.003758
Ra: 160.97400
Dec: 11.671639
Epoch: 2000
E(B-V)_MW: 0.024000000
E(B-V)_MW_Ref: Schlafly & Finkbeiner 2011, Appendix; 2011ApJ...737..103S
Host: NGC 3351
Host_type: SB(r)b
Distance(m-M): 30.02
Distance_error(m-M): 0.20
Distance_method: NED
Major_Diameter(arcmin): 3.07
Minor_Diameter(arcmin): 2.86
99 changes: 99 additions & 0 deletions data_examples/SN2012aw/sn2012aw.out_B
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
SN: sn2012aw
Filter: B
Npoints: 95
MJD Mag DMag magsys Telescope Detector Source
56005.82000 13.504 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56006.80000 13.492 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56007.81000 13.457 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56008.76000 13.415 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56010.84000 13.394 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56011.78000 13.411 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56018.63000 13.567 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56024.63000 13.756 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56026.74000 13.875 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56029.71000 14.000 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56036.65000 14.228 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56038.61000 14.284 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56042.75000 14.392 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56046.70000 14.466 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56048.61000 14.491 0.025 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56053.62000 14.574 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56056.61000 14.688 0.024 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56058.65000 14.518 0.022 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56060.68000 14.697 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56061.69000 14.711 0.022 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56064.64000 14.771 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56071.63000 14.817 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56074.66000 14.882 0.022 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56074.65000 14.611 0.023 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56075.63000 14.609 0.025 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56077.65000 14.872 0.031 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56084.64000 14.990 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56087.62000 15.042 0.022 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56089.63000 15.057 0.021 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56094.66000 14.694 0.029 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56103.65000 15.231 0.029 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56105.63000 15.158 0.018 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56216.96000 17.526 0.083 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56217.96000 17.657 0.053 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56223.98000 18.069 0.361 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56224.97000 18.033 0.026 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56226.97000 18.106 0.023 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56230.98000 18.088 0.038 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56233.97000 18.163 0.030 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56235.96000 18.126 0.030 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56238.97000 18.278 0.078 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56241.95000 17.850 0.051 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56242.94000 17.875 0.051 Landolt DFOT-0.130m -- Bose et al. 2013: 2013arXiv1305.3152B
56247.98000 18.264 0.019 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56249.98000 18.289 0.020 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56256.95000 18.323 0.018 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56265.90000 18.325 0.060 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56270.02000 18.415 0.070 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56270.98000 18.429 0.022 Landolt ST-0.104m -- Bose et al. 2013: 2013arXiv1305.3152B
56003.91000 13.790 0.010 Landolt Munari -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56005.07000 13.520 0.050 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56006.21000 13.470 0.120 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56005.91000 13.600 0.010 Landolt Munari -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56005.94000 13.540 0.080 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56007.07000 13.470 0.110 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56006.81000 13.530 0.060 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56008.07000 13.380 0.050 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56007.81000 13.410 0.050 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56009.08000 13.420 0.020 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56008.81000 13.420 0.040 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56010.04000 13.360 0.020 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56009.85000 13.430 0.020 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56009.85000 13.350 0.020 Landolt CAFOS -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56009.86000 13.390 0.030 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56011.04000 13.380 0.030 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56010.86000 13.320 0.030 Landolt CAFOS -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56012.86000 13.430 0.060 Landolt AFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56013.94000 13.430 0.030 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56015.03000 13.510 0.040 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56014.89000 13.500 0.020 Landolt AFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56016.01000 13.480 0.020 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56015.87000 13.460 0.030 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56017.07000 13.610 0.090 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56017.93000 13.530 0.020 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56019.82000 13.580 0.070 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56041.93000 14.410 0.030 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56042.90000 14.410 0.040 Landolt Asiago -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56042.99000 14.430 0.010 Landolt ALFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56048.05000 14.450 0.020 Landolt EFOSC2 -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56050.07000 14.540 0.040 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56052.90000 14.720 0.020 Landolt ACAM -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56059.15000 14.710 0.080 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56069.05000 15.100 0.060 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56073.88000 14.970 0.010 Landolt ALFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56095.91000 15.190 0.020 Landolt ALFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56104.90000 15.450 0.040 Landolt ACAM -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56110.98000 15.310 0.120 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56114.99000 15.450 0.040 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56115.90000 15.470 0.030 Landolt ALFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56116.98000 15.490 0.110 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56117.99000 15.520 0.050 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56125.98000 15.870 0.070 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56127.98000 15.920 0.120 Landolt PROMPT -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56288.20000 18.550 0.020 Landolt ALFOSC -- Dall'Ora et al. 2014:2014arXiv1404.1294D
56335.13000 18.980 0.030 Landolt ACAM -- Dall'Ora et al. 2014:2014arXiv1404.1294D
Loading

0 comments on commit c9256a6

Please sign in to comment.