Fisher information matrix of unipolar activation function-based multilayer perceptrons

Fisher information matrix of unipolar activation function-based multilayer perceptrons

The multilayer perceptrons (MLPs) are widely used in many fields, however, singularities in the parameter space may seriously influence the learning dynamics of MLPs and cause strange learning behaviors. Given that the singularities are the subspaces of the parameter space where the Fisher informati...

Full description

Saved in:
Bibliographic Details
Main Authors: Guo, Weili, Ong, Yew-Soon, Zhou, Yingjiang, Hervas, Jaime Rubio, Song, Aiguo, Wei, Haikun
Other Authors: School of Computer Science and Engineering
Format: Article
Language:English
Published: 2020
Subjects:
Engineering::Computer science and engineering
Analytical Form
Fisher Information Matrix (FIM)
Online Access:https://hdl.handle.net/10356/139916
Tags: Add Tag
No Tags, Be the first to tag this record!
Institution: Nanyang Technological University
Language: English
Description
Summary:The multilayer perceptrons (MLPs) are widely used in many fields, however, singularities in the parameter space may seriously influence the learning dynamics of MLPs and cause strange learning behaviors. Given that the singularities are the subspaces of the parameter space where the Fisher information matrix (FIM) degenerates, the FIM plays a key role in the study of the singular learning dynamics of the MLPs. In this paper, we obtain the analytical form of the FIM for unipolar activation function-based MLPs where the input subjects to the Gaussian distribution with general covariance matrix and the unipolar error function is chosen as the activation function. Then three simulation experiments are taken to verify the validity of the obtained results.

Similar Items