ClaudioFlow is a didactic full-featured Deep Learning framework implemented only using NumPy, with a focus on simplicity and code readability (rather than performance).
Most modules in the code link to a web resource or to pages of the Deep Learning Book by Goodfellow et al. that explains the theory behind them.
ClaudioFlow architecture draws a lot of inspiration from Torch 7's clean approach, as well as Keras interfaces.
model = SequentialModel([
LinearLayer(2, 5),
TanhLayer(),
LinearLayer(5, 1),
TanhLayer(),
])
Similarly to Torch, you just need to implement a forward() and a backward() function, and you get a functioning layer.
class MyNewLayer:
def forward(self, x):
self.x = x
return x**2
def backward(self, delta):
return delta * 2 * self.x
model = SequentialModel([
LinearLayer(2, 5),
MyNewLayer(),
])
model.learn(...)
The only good way to understand modern neural network is to try to implement one from scratch. After that you can use TensorFlow.
- Minibatch learning with train/validation/test sets and "patience"
- Tests with numerical gradient checks
- Softmax Negative Log Likelihood (Cross Entropy)
- Working examples like Multi-layer Perceptron for Mnist classification
- SGD Momentum / AdaGrad / RMSProp
- Sigmoid / Relu / Tanh and more common activation functions
- Recurrent Networks
- LSTM
- Convolutions
- Reinforcement learning
- Automatic differentiation: Theano is perfect if you need that in python.