Skip to content

Commit

Permalink
Remove transform_space requirement from FeatureTransformer and `F…
Browse files Browse the repository at this point in the history
…eaturePipeline` and move the functionality into `Exchange`.
  • Loading branch information
notadamking committed Nov 11, 2019
1 parent 70c3953 commit f8aaf18
Show file tree
Hide file tree
Showing 31 changed files with 791 additions and 469 deletions.
46 changes: 24 additions & 22 deletions docs/source/components/features/FeaturePipeline.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# FeaturePipeline

Feature pipelines are meant for transforming observations from the environment into meaningful features for an agent to learn from. If a pipeline has been added to a particular exchange, then observations will be passed through the `FeaturePipeline` before being output to the environment.

For example, a feature pipeline could normalize all price values, make a time series stationary, add a moving average column, and remove an unnecessary column, all before the observation is returned to the agent.
Expand All @@ -23,47 +24,48 @@ model = Sequential([
])
```


## Class Parameters
* `steps`
* A list of feature transformations to apply to observations.
* `dtype`
* The `dtype` elements in the pipeline should be cast to.

## Properties and Setters
- `steps`
- A list of feature transformations to apply to observations.
- `dtype`
- The `dtype` elements in the pipeline should be cast to.

* `steps`
* A list of feature transformations to apply to observations.
* `dtype`
* The `dtype` that elements in the pipeline should be input and output as.
* `reset`
* Reset all transformers within the feature pipeline.
## Properties and Setters

- `steps`
- A list of feature transformations to apply to observations.
- `dtype`
- The `dtype` that elements in the pipeline should be input and output as.
- `reset`
- Reset all transformers within the feature pipeline.

## Functions

Below are the functions that the `FeaturePipeline` uses to effectively operate.
Below are the functions that the `FeaturePipeline` uses to effectively operate.

### Private
* `_transform`
* Utility method for transforming observations via a list of *make changes here* `FeatureTransformer` objects.
* In other words, it runs through all of the `steps` in a for loop, and casts the response.

- `_transform`
- Utility method for transforming observations via a list of _make changes here_ `FeatureTransformer` objects.
- In other words, it runs through all of the `steps` in a for loop, and casts the response.

**The code from the transform function:**
As you see, it iterates through every step and adds the observation to the dataframe.

```py
for transformer in self._steps:
observations = transformer.transform(observations, input_space)
observations = transformer.transform(observations)
```

At the end the observations are converted into a ndarray so that they can be interpreted by the agent.

### Public

* `reset`
* Reset all transformers within the feature pipeline.
* `transform_space`
* Apply the pipeline of feature transformations to an observation frame.
- `reset`
- Reset all transformers within the feature pipeline.
- `transform`
- Apply the pipeline of feature transformations to an observation frame.

## Use Cases

Expand All @@ -84,4 +86,4 @@ feature_pipeline = FeaturePipeline(steps=[normalize_price,
difference_all])

exchange.feature_pipeline = feature_pipeline
```
```
19 changes: 8 additions & 11 deletions docs/source/components/features/FeatureTransformer.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,23 @@
# FeatureTransformer

As stated before in the [overview](../overview.md), We use an `ABCMeta` abstract hierarchy to handle the transformation calls of each asset. The `FeatureTransformer` is an abstract of all other price transformers available inside of the `tensortrade` library. As such, it has a set of common functions that are called on almost every transformer.
As stated before in the [overview](../overview.md), We use an `ABCMeta` abstract hierarchy to handle the transformation calls of each asset. The `FeatureTransformer` is an abstract of all other price transformers available inside of the `tensortrade` library. As such, it has a set of common functions that are called on almost every transformer.

## Properties and Setters

* `columns`
* A list of column names to normalize

- `columns`
- A list of column names to normalize

## Functions

Below are the functions that the `FeatureTransformer` uses to effectively operate.
Below are the functions that the `FeatureTransformer` uses to effectively operate.

### Private

`None`

### Public

* `reset`
* Optionally implementable method for resetting stateful transformers.
* `transform_space`
* Get the transformed output space for a given input space.
* `transform`
* Transform the data set and return a new data frame.
- `reset`
- Optionally implementable method for resetting stateful transformers.
- `transform`
- Transform the data set and return a new data frame.
36 changes: 14 additions & 22 deletions docs/source/components/features/indicators/TAlibIndicator.md
Original file line number Diff line number Diff line change
@@ -1,44 +1,37 @@
# TAlibIndicator
Adds one or more TAlib indicators to a data frame, based on existing open, high, low, and close column values.

Adds one or more TAlib indicators to a data frame, based on existing open, high, low, and close column values.

![TalibIndicator](../../../_static/images/talib_transform.png)


## Class Parameters
* `indicators`
* A list of indicators you want to transform the price information to.
* `lows`
* The lower end of the observation space. See `spaces.Box` to best understand.
* `highs`
* The lower end of the observation space. See `spaces.Box` to best understand.

## Properties and Setters

* **NONE**
- `indicators`
- A list of indicators you want to transform the price information to.
- `lows`
- The lower end of the observation space. See `spaces.Box` to best understand.
- `highs`
- The lower end of the observation space. See `spaces.Box` to best understand.

## Properties and Setters

- **NONE**

## Functions

Below are the functions that the `TAlibIndicator` uses to effectively operate.
Below are the functions that the `TAlibIndicator` uses to effectively operate.

### Private
* `_str_to_indicator` - Converts the name of an indicator to an actual instance of the indicator. For a list of indicators see list [here](http://mrjbq7.github.io/ta-lib/).

### Public

* `transform_space`
* Get the transformed output space for a given input space.
* `transform`
* Transform the data set and return a new data frame.
- `_str_to_indicator` - Converts the name of an indicator to an actual instance of the indicator. For a list of indicators see list [here](http://mrjbq7.github.io/ta-lib/).

### Public

- `transform`
- Transform the data set and return a new data frame.

## Use Cases:



## Use Cases

**Use Case #1: Selecting Indicators**
Expand All @@ -51,7 +44,6 @@ talib_indicator = TAlibIndicator(["rsi", "ema"])

This runs through the indicators in the list, at runtime and matches them to what is seen inside of TA-Lib. The features are then flattened into the `output_space`, both into the `high` and `low` segment of `space.Box`.


```py
for i in range(len(self._indicators)):
output_space.low = np.append(output_space.low, self._lows[i])
Expand Down
31 changes: 14 additions & 17 deletions docs/source/components/features/scalers/MinMaxNormalizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,34 +4,31 @@ A transformer for normalizing values within a feature pipeline by the column-wis

## Class Parameters


* `columns`
* A list of column names to normalize.
* `feature_min`
* The minimum value in the range to scale to.
* `feature_max`
* The maximum value in the range to scale to.
* `inplace`
* If `False`, a new column will be added to the output for each input column.
- `columns`
- A list of column names to normalize.
- `feature_min`
- The minimum value in the range to scale to.
- `feature_max`
- The maximum value in the range to scale to.
- `inplace`
- If `False`, a new column will be added to the output for each input column.

## Properties and Setters

None

## Functions

Below are the functions that the `MinMaxNormalizer` uses to effectively operate.
Below are the functions that the `MinMaxNormalizer` uses to effectively operate.

### Private
*None*

### Public
_None_

* `transform_space`
* Get the transformed output space for a given input space.
* `transform`
* Apply the pipeline of feature transformations to an observation frame.
### Public

- `transform`
- Apply the pipeline of feature transformations to an observation frame.

## Use Cases:

Expand All @@ -48,4 +45,4 @@ feature_pipeline = FeaturePipeline(steps=[normalize_price,
moving_averages,
difference_all])
exchange.feature_pipeline = feature_pipeline
```
```
34 changes: 14 additions & 20 deletions docs/source/components/features/scalers/StandardNormalizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,37 +4,33 @@ A transformer for normalizing values within a feature pipeline by removing the m

## Class Parameters


* `columns`
* A list of column names to normalize.
* `feature_min`
* The minimum value in the range to scale to.
* `feature_max`
* The maximum value in the range to scale to.
* `inplace`
* If `False`, a new column will be added to the output for each input column.
- `columns`
- A list of column names to normalize.
- `feature_min`
- The minimum value in the range to scale to.
- `feature_max`
- The maximum value in the range to scale to.
- `inplace`
- If `False`, a new column will be added to the output for each input column.

## Properties and Setters

* None
- None

## Functions

Below are the functions that the `StandardNormalizer` uses to effectively operate.

### Private

*None*
_None_

### Public

* `transform_space`
* Get the transformed output space for a given input space.
* `transform`
* Apply the pipeline of feature transformations to an observation frame.
* `reset`
* Resets the history of the standard scaler.

- `transform`
- Apply the pipeline of feature transformations to an observation frame.
- `reset`
- Resets the history of the standard scaler.

## Use Cases:

Expand All @@ -56,5 +52,3 @@ feature_pipeline = FeaturePipeline(steps=[normalize_price,
difference_all])
exchange.feature_pipeline = feature_pipeline
```


Original file line number Diff line number Diff line change
Expand Up @@ -2,53 +2,46 @@

A transformer for differencing values within a feature pipeline by a fractional order. It removes the stationarity of the dataset available in realtime. To learn more about why non-stationarity should be converted to stationary information, please look at the blog [here](https://towardsdatascience.com/preserving-memory-in-stationary-time-series-6842f7581800).


## Class Parameters
* `columns`
* A list of column names to difference.
* `difference_order`
* The fractional difference order. Defaults to 0.5.
* `difference_threshold`
* A type or str corresponding to the dtype of the `observation_space`.
* `inplace`
* If `False`, a new column will be added to the output for each input column.

- `columns`
- A list of column names to difference.
- `difference_order`
- The fractional difference order. Defaults to 0.5.
- `difference_threshold`
- A type or str corresponding to the dtype of the `observation_space`.
- `inplace`
- If `False`, a new column will be added to the output for each input column.

## Functions

Below are the functions that the `FractionalDifference` uses to effectively operate.
Below are the functions that the `FractionalDifference` uses to effectively operate.

### Private

* `_difference_weights`
* Gets the weights for ...
* `_fractional_difference`
* Computes fractionally differenced series, with an increasing window width.
- `_difference_weights`
- Gets the weights for ...
- `_fractional_difference`
- Computes fractionally differenced series, with an increasing window width.

### Public

* `transform_space`
* Get the transformed output space for a given input space.
* `transform`
* Apply the pipeline of feature transformations to an observation frame.
* `reset`
* Resets the history of the standard scaler.

- `transform`
- Apply the pipeline of feature transformations to an observation frame.
- `reset`
- Resets the history of the standard scaler.

## Use Cases:

**Use Case #1: Different Input Spaces**

This `FeatureTransformer` operates differently depending on if we pretransform the observation to an ndarray or keep it as a pandas dataframe.


```py
from tensortrade.features import FeaturePipeline
from tensortrade.features.stationarity import FractionalDifference
price_columns = ["open", "high", "low", "close"]
difference_all = FractionalDifference(difference_order=0.6) # fractional difference is seen here
feature_pipeline = FeaturePipeline(steps=[difference_all])
feature_pipeline = FeaturePipeline(steps=[difference_all])
exchange.feature_pipeline = feature_pipeline
```



Loading

0 comments on commit f8aaf18

Please sign in to comment.