In machine learning, a model has two types of parameters: those that are learned from data, and hyper-parameters, which are set by the user.

Hyper-parameters are like the settings on a camera; you can change them to get different results.

When using our plugin to fine-tune your model these parameters set automatically. You can view hyper-parameters that has been set for each fine-tune process under the Trainings tab. Please click on Hyper-parameters as shown below.

This will open a popup displaying hyper-parameters that have been used for that specific model.

It will look like this:

And below is the hyper-parameters settings of the GPT models that OpenAI trained. All models were trained for a total of 300 billion tokens.

In summary, Hyper-parameter tuning is the process of systematically searching for the best combination of hyperparameters for a learning algorithm in order to improve its performance. Batch size, token, learning rate, epoch, and prompt loss weight are examples of hyperparameters that can be tuned.

Leave a Reply

Your email address will not be published. Required fields are marked *