Skip to Main Content
U.S. Forest Service
Caring for the land and serving people

United States Department of Agriculture

Home > Search > Publication Information

  1. Share via EmailShare on FacebookShare on LinkedInShare on Twitter
    Dislike this pubLike this pub
    Author(s): Tai-Hoon Cho; Richard W. Conners; Philip A. Araman
    Date: 1992
    Source: Proceedings, 1991 IEEE International Conference on Systems, Man, and Cybernetics. pp. 1587-1592.
    Publication Series: Miscellaneous Publication
    PDF: View PDF  (65 KB)


    In this paper, several back-propagation (BP) learning speed-up algorithms that employ the ãgainä parameter, i.e., steepness of the activation function, are examined. Simulations will show that increasing the gain seemingly increases the speed of convergence and that these algorithms can converge faster than the standard BP learning algorithm on some problems. However, these algorithms may also suffer from increased instability, i.e., they frequently fail to converge within a finite time. One potential cause for the instability is an inappropriate choice for the initial weights. To overcome the instability resulting from this cause it is proposed that automatic weight reinitialization be used whenever the convergence speed becomes ãvery slowä due to a local minimum or premature saturation. On the simulations performed BP algorithms with larger initial gain (around 2 or 3) and automatic weight reinitialization converged much faster and were more stable than algorithms employing the same gain but not employing automatic weight reinitialization. The simulations performed involved a diverse set of problems including exclusive-or (XOR), encoder, and parity problems.

    Publication Notes

    • You may send email to to request a hard copy of this publication.
    • (Please specify exactly which publication you are requesting and your mailing address.)
    • We recommend that you also print this page and attach it to the printout of the article, to retain the full citation information.
    • This article was written and prepared by U.S. Government employees on official time, and is therefore in the public domain.


    Cho, Tai-Hoon; Conners, Richard W.; Araman, Philip A. 1992. Fast Back-Propagation Learning Using Steep Activation Functions and Automatic Weight. Proceedings, 1991 IEEE International Conference on Systems, Man, and Cybernetics. pp. 1587-1592.

    Related Search

    XML: View XML
Show More
Show Fewer
Jump to Top of Page