The early model of Neurons, ADALINE

In 1943, Warren McCulloch and Walter Pitss

first model brain cells.

first model brain cells.

Later, in 1958, Frank Rosenblatt added ‘weight’

to McCulloch and Pitss’s brain model

and Donald Olding Hebb’s theory about learning,

and came up with perceptrons.

to McCulloch and Pitss’s brain model

and Donald Olding Hebb’s theory about learning,

and came up with perceptrons.

Today’s story is about a model, rather than a person.

ADAINE, known as the early neural model,

is invented in 1960,

by Bernard Widrow and his student Ted Hoff at Stanford.

ADAINE, known as the early neural model,

is invented in 1960,

by Bernard Widrow and his student Ted Hoff at Stanford.

Perceptron classifies the value in the net input function

according to threshold, and returns -1 or 1.

according to threshold, and returns -1 or 1.

ADALINE adapts weight

in order to minimize the error of misclassifying data.

in order to minimize the error of misclassifying data.

We call this cost function J(w).

Then, sum deviation square in order to calculate the error.

This equation represents curvature(quadratic function)

rather than linear step function, thus, it is differentiable.

Then, sum deviation square in order to calculate the error.

This equation represents curvature(quadratic function)

rather than linear step function, thus, it is differentiable.

Looking into the graph for easier understanding-

This cost function J(w) is a quadratic function.

The J(w) is minimized as the w is updated

until the gradient of this curve converges to 0.

ADALINE uses this gradient descent,

and the equation of gradually updating the w is shown below.

This cost function J(w) is a quadratic function.

The J(w) is minimized as the w is updated

until the gradient of this curve converges to 0.

ADALINE uses this gradient descent,

and the equation of gradually updating the w is shown below.

We solve the derivative of J(w) like this.

The derivative of composite function is solved as follows.

The derivative of composite function is solved as follows.

In short, ADALINE boils down to a problem of finding the cost function

that minimizes the w value, by repeating the equation above.

that minimizes the w value, by repeating the equation above.

Do you need database performance monitoring? Contact us and we will send you a free quote

[email protected]

[email protected]

All Rights Reserved. © 2019 EZIS for Colud