Multi-Target Support Vector Regression Via Correlation Regressor Chains

Abstract

Multi-target regression is a challenging task that consists of creating predictive models for problems with multiple continuous target outputs. Despite the increasing attention on multi-label classification, there are fewer studies concerning multi-target (MT) regression. The current leading MT models are based on ensembles of regressor chains, where random, differently ordered chains of the target variables are created and used to build separate regression models, using the previous target predictions in the chain. The challenges of building MT models stem from trying to capture and exploit possible correlations among the target variables during training. This paper presents three multi-target support vector regression models. The first involves building independent, single-target Support Vector Regression (SVR) models for each output variable. The second builds an ensemble of random chains using the first method as a base model. The third calculates the targets' correlations and forms a maximum correlation chain, which is used to build a single chained support vector regression model, improving the models' prediction performance while reducing the computational complexity. The experimental study evaluates and compares the performance of the three approaches with seven other state-of-the-art multi-target regressors on 24 multi-target datasets. The experimental results are then analyzed using non-parametric statistical tests. The results show that the maximum correlation SVR approach improves the performance of using ensembles of random chains.

Datasets Information

These datasets have been collected from the MULAN, MEKA and LABIC repository websites and they are very varied in their degree of complexity, number of labels, number of attributes, and number of examples. The datasets are available to download.

Algorithms used in the experimental study

The implementation of these algorithms is available from the MULAN library. Download the code for SVR, SVRRC and SVRCC proposed in this paper.

• MORF: Multi-Objective Decision Trees.
• ST: Single-Target.
• MTS: Multi-Target Stacking.
• MTSC: Multi-Target Stacking Corrected.
• RC: Regression Chain.
• ERC: Ensemble of Random Chains.
• ERCC: Ensemble of Random Chains Corrected.
• SVR: Support Vector Regression.
• SVRRC: Support Vector Regression with Random Chains.
• SVRCC: Support Vector Regression with Correlation Chaining.

Results