Hyperparameter

In general #

A parameter to they way a algorithm works rather than a parameter to the algorithm.

In Machine Learning #

• Parameter to the “learning” as opposed to the Model Parameter which is a parameter to the “execution” of a model

• Parameter to the model itself, like the in k-nearest neighbors

• Additional hyperparameters for NN-Classifier:

  • Distance metrics:
    • L1 distance:
    • L2 distance:

Finding Hyperparameters #

• Split up total corpus of data (often times) into:
  • 60% -> training data
  • 20% -> validation data
  • 20% -> test data
• Use training and validation set to optimize hyperparameters
• When found the best ones, use the model with the best hyperparam
• against the test set
• Test set is not for validation it is for testing once at the end
• it is the one-time final test

-> Out comes the performance measure

• If you use test data for evaluation and improving the model, you are cheating