Statistical Mechanics of Support Vector Networks

Rainer Dietrich, Manfred Opper, and Haim Sompolinsky
Phys. Rev. Lett. 82, 2975 – Published 5 April 1999
PDFExport Citation

Abstract

Using methods of statistical physics, we investigate the generalization performance of support vector machines (SVMs), which have been recently introduced as a general alternative to neural networks. For nonlinear classification rules, the generalization error saturates on a plateau when the number of examples is too small to properly estimate the coefficients of the nonlinear part. When trained on simple rules, we find that SVMs overfit only weakly. The performance of SVMs is strongly enhanced when the distribution of the inputs has a gap in feature space.

  • Received 30 November 1998

DOI:https://doi.org/10.1103/PhysRevLett.82.2975

©1999 American Physical Society

Authors & Affiliations

Rainer Dietrich1, Manfred Opper2, and Haim Sompolinsky3

  • 1Institut für Theoretische Physik, Julius-Maximilians-Universität, Am Hubland, D-97074 Würzburg, Germany
  • 2Department of Computer Science and Applied Mathematics, Aston University, Birmingham B4 7ET, United Kingdom
  • 3Racah Institute of Physics and Center for Neural Computation, Hebrew University, Jerusalem 91904, Israel

References (Subscription Required)

Click to Expand
Issue

Vol. 82, Iss. 14 — 5 April 1999

Reuse & Permissions
Access Options
Author publication services for translation and copyediting assistance advertisement

Authorization Required


×
×

Images

×

Sign up to receive regular email alerts from Physical Review Letters

Log In

Cancel
×

Search


Article Lookup

Paste a citation or DOI

Enter a citation
×