Step by Step
S
The problem — accuracy becomes meaningless
With severe class imbalance (say, 99% one class), a model that always predicts the majority class achieves 99% accuracy while completely failing to detect any minority-class examples — accuracy alone is useless here.
Example: a fraud detection model that always predicts "not fraud" would achieve 99%+ accuracy on typical data, while catching zero actual fraud cases.
M
SMOTE — creating synthetic minority examples
SMOTE (Synthetic Minority Oversampling TEchnique) generates new synthetic examples of the minority class by interpolating between existing minority-class examples, rather than simply duplicating them.
Example: SMOTE might generate a new synthetic fraud example that sits mathematically "between" two existing real fraud examples in feature space.
C
Class weights — an alternative to resampling
Instead of creating synthetic examples, you can tell the algorithm to penalize minority-class errors more heavily during training (often via a simple class_weight='balanced' setting), achieving a similar effect without generating new data.
Example: setting class_weight='balanced' in a classifier so that misclassifying a rare fraud case is penalized much more heavily than misclassifying a common legitimate transaction.
T
The critical rule — resample training data ONLY
Any resampling technique (like SMOTE) must be applied only to the training data, never to the test set — the test set must remain a genuine, untouched sample of the real-world class distribution.
Example: applying SMOTE to balance the training data while leaving the test set's natural 99%/1% imbalance completely untouched, since that reflects the real-world distribution the model will actually face.
Applied Walkthrough
1
A fraud detection dataset has 99% legitimate transactions and only 1% fraud cases, and a naive model reports 99% accuracy.
2
Ask: does this high accuracy actually indicate a good model? No — a model that always predicts "not fraud" would achieve the same 99% accuracy while catching zero actual fraud.
3
The team applies SMOTE to their training data, creating synthetic fraud examples to balance the training set, then retrains the model.
4
Critically, they leave the test set's original 99%/1% imbalance untouched, since applying SMOTE to test data would give a misleadingly optimistic picture of real-world performance — and they evaluate using F1 or AUC rather than accuracy.
Exam Application
Exams test whether you understand why accuracy is a poor metric for imbalanced data (a naive majority-class prediction can score deceptively high) and the critical rule that resampling techniques like SMOTE must only be applied to training data, never to the test set.
⚠ Common Trap
The most common trap is applying SMOTE (or any resampling) to the test set as well as the training set. This artificially changes the test set's class distribution away from the real-world distribution the model will actually face, producing a misleadingly optimistic evaluation.
✓ Quick Self-Check
1. Why is accuracy a poor metric for severely imbalanced data?
Because a model that always predicts the majority class can achieve very high accuracy while completely failing on the minority class.
Tap to reveal / hide
2. What does SMOTE do?
Creates synthetic minority-class examples by interpolating between existing minority-class examples.
Tap to reveal / hide
3. What is an alternative to resampling for handling class imbalance?
Using class weights (like class_weight='balanced') to penalize minority-class errors more heavily during training.
Tap to reveal / hide
4. Should resampling techniques like SMOTE ever be applied to the test set?
No — resampling should only ever be applied to training data, never to the test set.
Tap to reveal / hide
5. What metrics are recommended instead of accuracy for imbalanced data?
F1 score or AUC.
Tap to reveal / hide