Multi-Layered Perceptron (MLP) / Artificial Neural Network (ANN)

Changing Integration Function:

The nonlinear pattern will not be captured by the mere existence of hidden layers. The activation function that will be employed must not be linear.

Only linear patterns may be captured by using linear or identifiable activation functions inside the neurons of the hidden layer.

A linear activation function is assumed by default by the network if no activation functions are provided in the layers.

360DigiTMG offers the Artificial Intelligence Coaching Institutes in Bangalore to start a career in AI. Enroll now!

List of Activation Functions Include

Regularization Techniques used for Overfitting

L1 regularization / L1 weight decay term

L2 regularization / L2 weight decay term

Want to learn more about AI? Enroll in this Artificial Intelligence Coaching Institutes in Hyderabad to do so.

Error-Change Criterion

• Stop when error isn't dropping over a window of, say, 10 epochs
• Train for a fixed number of epochs after criterion is reached (possibly with lower learning rate)

Weight-Change Criterion

• Compare weights at epochs t-10 & t and test
• Possibly express as a percentage of the weight

Also, check this Artificial Intelligence Coaching Institutes in Pune to start a career in Artificial Intelligence. Looking forward to becoming a Artificial Intelligence expert? Check out the Artificial Intelligence Course and get certified today.

Dropout

This method of model averaging in the Deep Learning Training Phase is interesting: Ignore (zero out) a random subset, p, of nodes (and associated activations) for each hidden layer, training sample, and iteration.

In the test phase, use all activations, but scale them down by a factor p (to make up for the activations that were not present during training).

Choose a selection of nodes at random, then reduce their output to zero.

Randomly select a subset of nodes and force their output to zero.

Drop Connect

But unlike dropout, we deactivate the weights rather than the nodes. The nodes are a little bit active here.

Noise

360DigiTMG the award-winning training institute offers a Artificial Intelligence Coaching Institutes in Pune and other regions of India and become certified professionals.

Batch Normalization:

Input: Values of x over a mini-batch: B = { x1...m };

• Batch Normalization layer is usually inserted before non-linearity layer (after Fully Connected or Dense Layer)
• Reduces the strong dependence on weight initialization

Shuffling Inputs

• Choose examples with maximum information content
• Shuffle the training set so that successive training examples never (rarely) belong to the same class
• Present input examples that produce a large error more frequently than examples that produce a small error. Why? It helps to take large steps in the Gradient descent

Weight Initialization Techniques:

Xavier’s initialization

Caffe implements a simpler version of Xavier’s initialization

He’s initialization