On the equivalence of Hebbian learning and the SVM formalism

We show that it is possible to relate the Support Vector Machine formalism to Hebbian Learning in the context of olfactory learning in the insect brain. Since neurons cannot have negative firing rates, two neurons and synaptic inhibition are required to encode a binary classification problem in a biologically realistic way. We show that the two neuron system with plausible Hebbian learning rules can be mapped to a large margin classifier. Two formalisms are analyzed: regular SVMs and the so-called inhibitory SVMs. The regularization term in regular SVMs brings the synaptic vectors of the two neurons close to each other, while the inhibitory SVM can bring them to 0 resembling the memory loss process in Hebbian learning. Based on the analogy to large margin classifiers we also predict the existence of a negative Hebbian leaning rule for negative reinforcement signals.