Normally a Gaussian will be used as the RBF, Figure shows a two-dimensional version of such a kernel. From Eqn. , the output of the kernel is dependent on the Euclidean distance of from (one of these will be the support vector and the other will be the testing data point). The support vector will be the centre of the RBF and will determine the area of influence this support vector has over the data space.
Figure: The Radius Basis Function kernel
A larger value of will give a smoother decision surface and more regular decision boundary. This is because an RBF with large will allow a support vector to have a strong influence over a larger area. Figures , , and show the decision surface and boundaries for two different values. A larger value also increases the value (the Lagrange multiplier) for the classifier. When one support vector influences a larger area, all other support vectors in the area will increase in -value to counter this influence. Hence all -values will reach a balance at a larger magnitude. A larger -value will also reduce the number of support vectors (Table ). Since each support vector can cover a larger space, fewer are needed to define a boundary.
This means that the estimate of ||w|| will increase. The estimation of the VC dimension of the SVM (Eqn. ) depends on the norm of w and also the radius of the sphere that encompasses all the data (R). From Eqn. , as increases, the value of R will decrease. This will balance the increase in ||w||.
The experiments on the effect of different -values show that the expected risk loosely corresponds to the accuracy of the classifier tested on testing data (Figure and ).
Table: The results for different setting
Figure: Decision surface of small
Figure: Decision surface of large
Figure: The values of Lagrange multipliers for small
Figure: The values of Lagrange multipliers for large
Figure: Decision boundary for small
Figure: Decision boundary for large
Figure: The expected error for different -values
Figure: The accuracy of the classifier with different value