Neural Network Feature (Wish) List

[Versatilus]

Neural Network Feature (Wish) List

These are the basics:

• Basic 2-layer network with bias
• Activation functions which reuse the activation on the backwards pass (sigmoid, tanh, ReLU)
• MSE cost function
• Adjustable learning rate
• Additional examples and tests
  See Half Adder Example #41 and Add simple NN test #66 and Custom test without node using pure javascript #76.

These would be interesting to add:

• Multiple hidden layers
  Per Multi Hidden Layer Support for nn.js #61.
• Semi-arbitrary activation functions
  Per More activation function options? #70 and Add more Activation Function options.  #75.
• Arbitrary cost functions
  See here.
• Automatically adapting learning rate (Momentum)
  Per Add Learning Rate Decay #65. Also see here via here.
• Multiple initial weighting strategies
  See here.
• Convolution layers
  See here. Also see here via here.
• Simple RNN
  See here via here.
• Advanced optimization
  See here.

[Adil-Iqbal]

The two big ones for me are the additional examples and the multiple hidden layers.

Those two things open the door for lots of other possibilities.
The additional example can be used for testing community contributed features.
The multiple hidden layers open the door for convolution layers and the like.
Which then opens the door for some really cool art.