Normally a Gaussian will be used as the RBF, Figure
shows a two-dimensional version of such a kernel. From Eqn.
, the output of the kernel is dependent on the Euclidean distance of
from
(one of these will be the support vector and the
other will be the testing data point). The support vector will be the
centre of the RBF and
will determine the area of influence
this support vector has over the data space.
Figure: The Radius Basis Function kernel
A larger value of
will give a smoother decision surface and
more regular decision boundary. This is because an RBF with large
will allow a support vector to have a strong influence over a
larger area.
Figures
,
,
and
show the decision
surface and boundaries for two different
values.
A larger
value also increases the
value (the
Lagrange multiplier) for the classifier. When one support vector
influences a larger area, all other support vectors in the area will
increase in
-value to counter this influence. Hence all
-values will reach a balance at a larger magnitude. A larger
-value will also reduce the number of support vectors (Table
). Since
each support vector can cover a larger space, fewer are needed to
define a boundary.
This means that the estimate of ||w|| will increase. The estimation
of the VC dimension of the SVM (Eqn.
) depends on
the
norm of w and also the radius of the sphere that
encompasses all the data (R). From Eqn.
, as
increases,
the value of R will decrease. This will balance the increase in
||w||.
The experiments on the effect of different
-values show that
the expected risk loosely corresponds to the accuracy of the
classifier tested on testing data (Figure
and
).
Table: The results for different
setting
Figure: Decision surface of small
Figure: Decision surface of large
Figure: The values of Lagrange multipliers for small
Figure: The values of Lagrange multipliers for large
Figure: Decision boundary for small
Figure: Decision boundary for large
Figure: The expected error for different
-values
Figure: The accuracy of the classifier with different
value