Length-Changeable Incremental Extreme Learning Machine

Description

Youxi Wu, Dong Liu, He Jiang. Length-Changeable Incremental Extreme Learning Machine. Journal of Computer Science and Technology. 2017

  

Extreme Learning Machine (ELM) is a learning algorithm for generalized Single-hidden-Layer Feed-forward Networks (SLFNs). In order to obtain a suitable network architecture, Incremental Extreme Learning Machine (I-ELM) is a sort of ELM constructing SLFNs by adding hidden nodes one by one. Although kinds of I-ELM-class algorithms were proposed to improve the convergence rate or to obtain minimal training error, they do not change the construction way of I-ELM or face the over-fitting risk. Making the testing error converge quickly and stably therefore becomes an important issue. In this paper, we proposed a new incremental ELM which is referred to as Length-Changeable Incremental Extreme Learning Machine (LCI-ELM). It allows more than one hidden node to be added to the network and the existing network will be regarded as a whole in output weights tuning. The output weights of newly added hidden nodes are determined using a partial error-minimizing method. We prove that an SLFN constructed using LCI-ELM has approximation capability on a universal compact input set as well as on a finite training set. Experimental results demonstrate that LCI-ELM achieves higher convergence rate as well as lower over-fitting risk than some competitive I-ELM-class algorithms.

 

Source Code


Source code of LCI-ELM(MATLAB version) can be found here.
This code can be used to generate a training set with random noise and a noise-free testing set for approximation of function SinC(x).

Youxi Wu, Dong Liu, He Jiang. Length-Changeable Incremental Extreme Learning Machine. Journal of Computer Science and Technology. 2017

 

Usage

 

[TrainingTime, TestingTime, TrainingAccuracy, TestingAccuracy] = LCIELM(TrainingInput, TrainingTarget, TestingInput, TestingTarget, k, lambda, MaxHiddenNeurons, epsilon, ActivationFunType)

  The function is suitable for regression problems with only one output(target), but it can be modified easily to apply to multi-output regression
problems. All the inputs and outputs of the function are interpreted at follows. For usage, the input of dataset are suggested to be normalized into
[-1,1] and the output of dataset are suggested to be normalized into [0,1].

 

Input:

 

TrainingInput

-

Input of training data set, a N*d matrix, where N is the size of training set and d is the number of input attributes

TrainingTarget

-

Target of training data set, a N*1 vector, where N is the size of training set

TestingInput

-

Input of testing data set, a M*d matrix, where M is the size of testing set and d is the number of input attributes

TestingTarget

-

Target of testing data set, a M*1 vector, where M is the size of testing set

k and lambda

-

parameters of LCI-ELM

MaxHiddenNeurons

-

maximum number of network hidden nodes, parameter for algorithm termination

Epsilon

-

expected training error, parameter for algorithm termination

ActivationFunType

-

Type of activation function: 0 for sigmoidal additive function and 1 for Gaussian radial basis function(more types can be appended if needed)

Output:

 

 

TrainingTime

-

Time spent on training(seconds)

TestingTime

-

Time spent on testing(seconds)

TrainingAccuracy

-

Training RMSE for regression

TestingAccuracy

-

Testing RMSE for regression

 

The comparison algorithms: (ELM-class code)

CI-ELM

EI-ELM

I-ELM (EI-ELM with k = 1)

EM-ELM

EIR-ELM

 

Back