Articles
SVM Interview Questions: C, Gamma, Scaling, and Decision Boundaries
Practice SVM interviews with RBF similarity arithmetic, a C-by-gamma diagnostic grid, leakage-safe scaling, and decision-boundary checks.
“Increase C to fix underfitting” sounds plausible until an unscaled feature dominates every distance. In an SVM interview, the best answer begins with the geometry: what is being separated, which examples define the margin, how feature scaling changes the space, and what C and gamma do within that space. This guide walks through an RBF similarity calculation, a controlled tuning grid, and the diagnostic difference between a model that truly lacks flexibility and one trained on the wrong representation.
Evidence boundary: SVM definitions, parameter effects and preprocessing recommendations follow scikit-learn's official SVM guide and its RBF parameter example. The coordinates and interview cases here are original editorial examples, not candidate reports or measured benchmarks.
Start with the margin and the task
Question: What is a support vector? In a fitted support vector classifier, a subset of training observations determines the separating decision function. For a linear soft-margin classifier, examples on or inside the margin—and some misclassified examples—drive the boundary through the constrained optimization problem. The exact count and role of support vectors depend on the fit; it is inaccurate to say that every training row contributes equally to the final decision. The scikit-learn SVM guide describes SVC and its support vectors.
Before discussing a curved boundary, ask whether the interviewer means classification or regression, linear or kernel SVM, and whether probabilities are required. LinearSVC and SVC(kernel="linear") are related choices but have different implementations and can differ in loss and multiclass handling. gamma applies to kernels such as RBF, not to a plain linear margin. Keeping these contracts distinct prevents the common answer that treats C, gamma, and “the margin” as three interchangeable complexity sliders.
For classification, the sign of the decision score determines the side of a binary boundary. Its distance to zero is not automatically a calibrated probability of the class. If the application needs probabilities or an action threshold, evaluate that separately, including calibration and class imbalance. Accuracy at the default boundary may not represent the cost of false negatives in a medical or fraud task.
What does C actually trade off?
Question: Is a larger C always better? C controls the penalty for margin violations in the soft-margin objective. A larger C puts more weight on fitting training examples; a smaller C tolerates more violations while favoring regularization. This does not make the training boundary automatically “correct,” and it does not guarantee a larger geometric margin. The official SVM guide and SVC reference specify its role.
Imagine two clouds of points separated by a nearly straight line, plus one mislabeled example deep inside the other cloud. With a very large C, fitting that one example may become expensive enough to distort the boundary, particularly with a flexible kernel. With a smaller C, the solution can accept that violation. The right choice depends on validation performance and the cost of different errors, not on whether the plotted training labels look perfectly separated.
Describe C in relation to the exact estimator and objective. A linear SVM trained with a different loss, sample weighting or class weights changes the cost attached to violations. For imbalanced classes, inspect per-class errors before deciding that C alone should be adjusted. If the feature scale changes, a previously selected C may no longer be appropriate because the geometry and effective regularization relationship have changed.
Calculate one RBF similarity to explain gamma
The RBF kernel compares two feature vectors by K(x,z)=exp(-gamma × ||x-z||²). Consider two standardized points one unit apart, so squared Euclidean distance is one. At gamma=0.01, similarity is exp(-0.01)≈0.990; at gamma=10, it is exp(-10)≈0.0000454. The first point influences a broad neighborhood of the second; the second setting makes influence extremely local. These values are direct arithmetic from the kernel equation, not a fitted-model experiment.
The scikit-learn RBF parameter example shows why low gamma can yield overly smooth boundaries and high gamma can create highly local, irregular ones. But “high gamma always overfits” is only a heuristic. C, sample distribution, noise, scaling and the validation task matter together. A useful interview answer says that gamma changes the width of similarity influence while C changes the cost of violations within the resulting hypothesis space.
For a second hand calculation, set squared distance to four. With gamma=0.5, similarity is exp(-2)≈0.135. If a feature transformation doubles every coordinate difference, squared distance becomes 16 and similarity falls to exp(-8)≈0.000335 with the same gamma. That is why an apparently harmless unit change can make an RBF boundary radically different.
Scaling can overwhelm the parameter search
Question: Why did the decision boundary change after standardization? Suppose two observations differ by one unit in feature A and 100 units in feature B. Their squared Euclidean separation is 1²+100²=10001. Even if feature A is scientifically important, feature B dominates an RBF distance measured in these raw units. Standardization changes that geometry and therefore the kernel values. The scikit-learn preprocessing guide explains why SVMs with RBF kernels are sensitive to feature scale.
Do not fit the scaler to the full dataset before cross-validation. Doing so lets validation rows influence the means and variances used for training. Put StandardScaler and SVC in a single scikit-learn Pipeline and cross-validate the pipeline. Each fold then fits its scaler on that fold's training partition. With sparse input, check centering constraints (with_mean=False) rather than blindly copying dense-data code.
Standardization is not universally right. If one feature is already a meaningful binary flag and another is a continuous measurement, consider how the transformed distances should encode business meaning. If a feature has a severe outlier distribution, a robust transformation may be worth testing. If a categorical code is simply mapped to arbitrary integers, scaling those integers cannot repair the invented ordering. Explain what distance should mean before tuning gamma.
Read a C by gamma tuning grid
Original interview exercise: Assume the same stratified train/validation split, fixed standardized features and RBF SVC. The cells below are illustrative outcomes, not observed scores. They show how to reason about a grid, not what any dataset will produce.
| Setting | Training behavior to inspect | Validation concern | First diagnostic |
|---|---|---|---|
Low C, low gamma | Smooth, weakly separated fit | Underfitting | Are both train and validation scores poor? |
High C, low gamma | Tries harder within broad similarity | May still be too smooth | Does train score improve without valid generalization? |
Low C, high gamma | Local similarity but violations tolerated | Mixed behavior | Check boundary and classwise errors. |
High C, high gamma | Can fit small neighborhoods tightly | Overfitting risk | Is train-validation gap large or unstable? |
If both training and validation scores are weak, check the representation, labels and kernel before declaring the model “too simple.” If training is excellent and validation poor, investigate overfitting, leakage, small folds and distribution shift. Select hyperparameters using cross-validation on training data; reserve a final test set for evaluation. When data have groups, repeated customers or time dependence, design the split accordingly. A random stratified split is not enough if near-duplicate users leak between folds.
The default gamma="scale" in current scikit-learn uses a data-dependent calculation, so state whether you used it or an explicit numeric gamma. Do not report “gamma was 1” when a library default chose something else. Record the library version and preprocessing in reproducible work.
Draw, diagnose, and then distrust the 2D picture
Question: What does the decision boundary tell you? In two dimensions, plotting a mesh can reveal obvious issues: a nearly flat boundary through nonlinear data, tiny islands around individual points, or a boundary dominated by one axis. Mark support vectors and misclassified points to explain why it moved. Yet a 2D projection of a high-dimensional model is not the full decision surface. A model can separate in dimensions hidden from the plot, and a projected smooth curve can conceal fragile behavior elsewhere.
A strong answer therefore pairs the drawing with out-of-sample evidence. Examine confusion matrices and calibration where relevant, score variability across folds, and cases close to the decision threshold. Ask whether a small measurement change flips predictions. In a fraud application, a flexible boundary around historical artifacts may fail badly when adversaries change behavior. The decision plot supports a hypothesis; it does not prove generalization.
For large datasets, acknowledge computational constraints. The SVC documentation notes that fit time scales at least quadratically with the number of samples in typical use and suggests alternatives such as LinearSVC or kernel approximation for large data. Do not promise that RBF SVC is a good default for millions of rows simply because a small demo plot looked good.
A concise interview answer with one counterexample
If asked, “How do you tune an SVM?”, answer in this order. Define task, error cost and split. Choose a representation and fit scaling inside cross-validation. Start with a linear baseline if appropriate. If an RBF kernel is justified, search a log-spaced grid for C and gamma jointly, monitor training and validation scores, and inspect classwise errors. Then evaluate once on a held-out test set and check serving cost.
Use the one-unit RBF calculation to explain gamma and the 10001 squared-distance counterexample to explain scaling. If the interviewer claims C=100 should compensate for poor scaling, disagree: C changes the optimization penalty, while scaling changes the kernel similarities themselves. Retuning C cannot restore the intended notion of distance when the wrong feature dominates it.
This is an editorial answer pattern. It is not a statement that a particular company asks these exact numbers. Its value is that each claim can be checked: compute the kernel value, inspect fold-specific scaling, and compare train and validation behavior. If two candidates obtain different boundaries, ask them to show their preprocessing and feature units before debating hyperparameters.