Learnability of periodic activation functions: General results

On-line learning in the presence of continuous periodic activation functions is studied analytically. The effect of the ambiguity (an infinite number of inputs with different local fields can produce the same output) on the learnability is examined. A universal interplay between the general features of the activation function (wave number, parity, etc.) and the critical learning rate is found. Analytical results are extended also to multilayer architectures with nonlinear output units. Results are compared with simulations.