V. Margot, Towards Data Science, October 27th, 2021.
Abstract: Nowadays, data scientists have many accurate machine learning algorithms. But, the choice of the best models is a complicated task and ensemble learning has proved its efficiency in practice. In previous posts: “How to choose the best model?” and “How to deal with overlapping rules?” I have presented the experts' aggregation theory. A theory that should be used more often for ensemble learning instead of simple averaging. Here, I want to focus on the COBRA method. This method has a very different and original approach to the combination of estimators. For simplicity, I use the terms estimators, predictive models and experts without any distinction. Indeed, in a regression setting, an estimator of the regression function can be used as a predictive model, or it can be used as an expert that makes a prediction for each new observation.