Overfitting
Overfitting happens, when a model is complex enough that it can memorize all the inputs (or the significant features for the given set). This leads to a low testing error, but potentially to a high validation error.
In contrast, an underfitted model, is not complex enough to model the characteristics of the data. Ideally your model is complex to model te features but not memorizing the individual features.
To help to prevent overfitting, Regularization can be used.
Where:
- Blue curve
- Training error
- Green curve
- Validation error
- capacity
- Complexity of the NN