Neural networks and computing : learning algorithms and applications /

Preface p. V
Introduction p. 1
Background p. 1
Neuron Model p. 2
Historical Remarks p. 4
Network architecture p. 6
Supervised Neural Networks p. 6
McCulloh and Pitts Model p. 7
The Perceptron Model p. 11
Multi-layer Feedforward Network p. 14
Recurrent Networks p. 15
Unsupervised Neural Networks p. 17
Modeling and Learning Mechanism p. 19
Determination of Parameters p. 20
Gradient Descent Searching Method p. 26
Exercises p. 28
Learning Performance and Enhancement p. 31
Fundamental of Gradient Descent Optimization p. 32
Conventional Backpropagation Algorithm p. 35
Convergence Enhancement p. 42
Extended Backpropagation Algorithm p. 44
Least Squares Based Training Algorithm p. 47
Extended Least Squares Based Algorithm p. 55
Initialization Consideration p. 59
Weight Initialization Algorithm I p. 61
Weight Initialization Algorithm II p. 64
Weight Initialization Algorithm III p. 67
Global Learning Algorithms p. 69
Simulated Annealing Algorithm p. 70
Alopex Algorithm p. 71
Reactive Tabu Search p. 72
The NOVEL Algorithm p. 73
The Heuristic Hybrid Global Learning Algorithm p. 74
Concluding Remarks p. 82
Fast Learning Algorithms p. 82
Weight Initialization Methods p. 83
Global Learning Algorithms p. 84
p. 85
Exercises p. 87
Generalization and Performance Enhancement p. 91
Cost Function and Performance Surface p. 93
Maximum Likelihood Estimation p. 94
The Least-Square Cost Function p. 95
Higher-Order Statistic Generalization p. 98
Definitions and Properties of Higher-Order Statistics p. 99
The Higher-Order Cumulants based Cost Function p. 101
Property of the Higher-Order Cumulant Cost Function p. 105
Learning and Generalization Performance p. 108
Experiment one: Henon Attractor p. 109
Experiment Two: Sunspot time-series p. 116
Regularization for Generalization Enhancement p. 117
Adaptive Regularization Parameter Selection (ARPS) Method p. 120
Stalling Identification Method p. 121
[lambda] Selection Schemes p. 122
Synthetic Function Mapping p. 124
Concluding Remarks p. 126
Objective function selection p. 128
Regularization selection p. 129
Confidence Upper Bound of Approximation Error p. 131
Proof of the Property of the HOC Cost Function p. 133
The Derivation of the Sufficient Conditions of the Regularization Parameter p. 136
Exercises p. 137
Basis Function Networks for Classification p. 139
Linear Separation and Perceptions p. 140
Basis Function Model for Parametric Smoothing p. 142
Radial Basis Function Network p. 144
RBF Networks Architecture p. 144
Universal Approximation p. 146
Initialization and Clustering p. 149
Learning Algorithms p. 152
Linear Weights Optimization p. 152
Gradient Descent Optimization p. 154
Hybrid of Least Squares and Penalized Optimization p. 155
Regularization Networks p. 157
Advanced Radial Basis Function Networks p. 159
Support Vector Machine p. 159
Wavelet Network p. 161
Fuzzy RBF Controllers p. 164
Probabilistic Neural Networks p. 167
Concluding Remarks p. 169
Exercises p. 170
Self-organizing Maps p. 173
Introduction p. 173
Self-Organizing Maps p. 177
Learning Algorithm p. 178
Growing SOMs p. 182
Cell Splitting Grid p. 182
Growing Hierarchical Self-Organizing Quadtree Map p. 185
Probabilistic SOMs p. 188
Cellular Probabilistic SOM p. 188
Probabilistic Regularized SOM p. 193
Clustering of SOM p. 202
Multi-Layer SOM for Tree-Structured data p. 205
SOM Input Representation p. 207
MLSOM Training p. 210
MLSOM visualization and classification p. 212
Exercises p. 216
Classification and Feature Selection p. 219
Introduction p. 219
Support Vector Machines (SVM) p. 223
Support Vector Machine Visualization (SVMV) p. 224
Cost Function p. 229
MSE and MCE Cost Functions p. 230
Hybrid MCE-MSE Cost Function p. 232
Implementing MCE-MSE p. 236
Feature Selection p. 239
Information Theory p. 241
Mutual Information p. 241
Probability density function (pdf) estimation p. 243
MI Based Forward Feature Selection p. 245
MIFS and MIFS-U p. 247
Using quadratic MI p. 248
Exercises p. 253
Engineering Applications p. 255
Electric Load Forecasting p. 255
Nonlinear Autoregressive Integrated Neural Network Model p. 257
Case Studies p. 261
Content-based Image Retrieval Using SOM p. 266
GHSOQM Based CBIR Systems p. 267
Overall Architecture of GHSOQM-Based CBIR System p. 267
Image Segmentation, Feature Extraction and Region-Based Feature Matrices p. 268
Image Distance p. 269
GHSOQM and Relevance Feedback in the CBIR System p. 270
Performance Evaluation p. 274
Feature Selection for cDNA Microarray p. 278
MI Based Forward Feature Selection Scheme p. 279
The Supervised Grid Based Redundancy Elimination p. 280
The Forward Gene Selection Process Using MIIO and MISF p. 281
Results p. 282
Prostate Cancer Classification Dataset p. 284
Subtype of ALL Classification Dataset p. 288
Remarks p. 294
Bibliography p. 291
Index p. 305