The Collection of
Computer Science Bibliographies

Bibliography of the paper "Simplifying Neural networks by Soft Weight-Sharing"

[   About   |  Browse   |   Statistics   ]

Number of references:1526Last update:October 7, 2005
Number of online publications:68Supported:no
Most recent reference:1992

Information on the Bibliography

Authors:
Steven J. Nowlan
Computational Neurobiology Laboratory
The Salk Institute
P.O. Box 85800
San Diego, CA 92186-5800, USA

Geoffrey E. Hinton
Department of Computer Science
University of Toronto
Toronto, Canada M5S 1A4

Abstract:
Abstract of "Simplifying Neural networks by Soft Weight-Sharing":
One way of simplifying neural networks so they generalize better is to add an extra term to the error function that will penalize complexity. Simple versions of this approach include penalizing the sum of the squares of the weights or penalizing the number of non-zero weights. We propose a more complicated penalty term in which the distribution of weight values is modelled as a mixture of multiple gaussians. A set of weights is simple if the weights have high probability densities under the mixture model. This can be achieved by clustering the weights into subsets with the weights in each cluster having very similar values. Since we do not know the appropriate means or variances of the clusters in advance, we allow the parameters of the mixture model to adapt at the same time as the network learns. Simulations on two different problems demonstrate that this complexity term is more effective than previous complexity terms.

Browsing the bibliography

Bibliographic Statistics

Types:
article(637), inproceedings(229), techreport(228), incollection(179), misc(93), book(92), unpublished(35), phdthesis(25), mastersthesis(5), inbook(3)
Fields:
title(1526), year(1524), author(1523), key(984), pages(767), journal(640), volume(631), booktitle(413), publisher(380), address(357), number(234), institution(232), annote(217), note(170), type(147), editor(142), month(122), organization(79), bibdate(70), editors(66), keywords(58), school(30), chapter(10), place(4), edition(3), page(3), howpublished(2), notes(2), addresss(1), comment(1), ken(1), keyword(1)
Distribution of publication dates:
Distribution of publication dates

Valid XHTML 1.1!  Valid CSS!