Optimizing LightGBM for Regression: A Study on Parameter Influence and Performance

The accurate forecasting of demand is a major challenge for production companies, especially for companies that engage in make-to-order production. Accurately anticipating the demand enables companies to derive a robust production program and mitigate over- or underloading the production resources. Light Gradient Boosting Machine (LightGBM) is a Machine Learning (ML) algorithm, developed by Microsoft Research, capable of performing regression, classification and ranking tasks. The algorithm gained attention in recent years due to its ability to efficiently deal with large datasets and a high number of features while still being cheap in terms of computational costs. However, the quality of the results is largely depending on carefully setting the right parameters. Hyperparameter optimization, such as GridSearch, can be used to find suiting parameters, but these methods are very time consuming and require the user to limit the number of parameters and also limit the range of the respective values. This paper aims to research the influence of the parameters of LightGBM on the predictive performance of regression tasks. In order to achieve this task, a total of 2,592 different simulated sales datasets were created each varying in seasonality, seasonal duration, seasonal amplitude, linear growth and random noise. For each of the datasets a LightGBM model was trained, using hyperparameter optimization. The models were then compared using the Root Mean Squared Error (RMSE) as a metric to find the best performing models. The thorough analysis of these parameters provides an insight of the importance of different parameters for regression tasks and can be utilized to speed up hyperparameter optimization of future regression algorithms based on LightGBM.