AutoML or how to automatically fine-tune the hyper-parameters of your ML pipeline.

Which is the best learning rate for your ML pipeline? Do you need a high or a low dropout rate? How many convolutional layers do you need and how many filters? And the most important question: do we really care about these decisions? Most of the time, the answer is no. Although these hyper-parameters can lead to good or poor performances, most of them are too coupled with the specific dataset we are trying to model and, therefore, the exact value that provides the best performance is not a rich source of knowledge to share.

*Finding the pipeline’s hyper-parameters that lead to the best performances can be an endless road.*

To solve this problem, a technique called Automated Machine Learning (AutoML) has started to gain a lot of relevance during the last few years. AutoML systems are meta-level machine learning algorithms, which use other machine learning solutions as building blocks for finding the optimal ML pipeline structures [1]. These systems automatically evaluate multiple pipeline configurations, trying to improve the performance iteratively. As a consequence, one of the AutoML systems’ drawbacks is that they consume a lot of computing resources.

*Which is the most important hyper-parameter for obtaining good performances? It depend*s!

AutoML systems could provide insights to experienced engineers resulting in better models deployed in a shorter period while allowing inexperienced users to get a glimpse of how such models work. The good news is that, currently, we have at our disposal reliable implementation of AutoML with several libraries:

auto-sklearn (https://automl.github.io/auto-sklearn/master/index.html#)
AutoKeras (https://autokeras.com/)
Keras-Tuner (https://keras.io/keras_tuner/)
Optuna (https://optuna.org/)

In some of the notebooks you can find on the Eden Library, we deeply cover the AutoML technique. They all are working examples ready for your modifications.

Ιf you find this repository interesting, please, help us with a star and share it with your colleagues.

Do you want to plot how two specific hyper-parameters interact as this picture depicts? Check our repository!

[1] Feurer, M., Klein, A., Eggensperger, K., Springenberg, J.T., Blum, M., & Hutter, F. (2015). Efficient and Robust Automated Machine Learning. NIPS.