Weight change and momentum

• backpropagation algorithm often takes a long time to learn

• So, the learning rule is often augmented with a so called momentum term

• This consist in adding a fraction of the old weight change

• The learning rule then looks like:

  weight change = some small constant ? error ? input activation + momentum constant ? old weight change