A Review of Activation Functions in SharpNEAT

[Colin Green]

Hi all,

I have recently carried out some basic research exploring the relative
merits of eight alternative activation functions (+ the logistic
function to make nine total) in a NEAT context.

The motivation for this work was originally to identify activation
functions with increased execution speed relative to the logistic
function, however, the tests highlighted two important qualities of
activation functions other than their speed of execution (1) zero
centering, (2) Unbounded range for positive input values.

More details here:

http://sharpneat.sourceforge.net/research/activation-fn-review/activation-fn-review.html


Happy to answer questions, receive feedback, corrections, suggestions, etc.

Colin