figshare
Browse
kbs14.pdf (363.57 kB)

Exploiting diversity for optimizing margin distribution in ensemble learning

Download (363.57 kB)
journal contribution
posted on 2017-06-16, 11:08 authored by Qinghua Hu, Leijun Li, Xiangqian Wu, Gerald SchaeferGerald Schaefer, Daren Yu
Margin distribution is acknowledged as an important factor for improving the generalization performance of classifiers. In this paper, we propose a novel ensemble learning algorithm named Double Rotation Margin Forest (DRMF), that aims to improve the margin distribution of the combined system over the training set. We utilise random rotation to produce diverse base classifiers, and optimize the margin distribution to exploit the diversity for producing an optimal ensemble. We demonstrate that diverse base classifiers are beneficial in deriving large-margin ensembles, and that therefore our proposed technique will lead to good generalization performance. We examine our method on an extensive set of benchmark classification tasks. The experimental results confirm that DRMF outperforms other classical ensemble algorithms such as Bagging, AdaBoostM1 and Rotation Forest. The success of DRMF is explained from the viewpoints of margin distribution and diversity.

Funding

This work is supported by the National Program on Key Basic Research Project under Grant 2013CB329304, National Natural Science Foundation of China under Grants 61222210, 61170107, 61073125, 61350004 and 11078010, the Program for New Century Excellent Talents in University (No. NCET-12-0399), and the Fundamental Research Funds for the Central Universities (Grant No. HIT.NSRIF.2013091 and HIT.HSS.201407).

History

School

  • Science

Department

  • Computer Science

Published in

Knowledge-Based Systems

Volume

67

Pages

90 - 104

Citation

HU, Q. ... et al, 2014. Exploiting diversity for optimizing margin distribution in ensemble learning. Knowledge-Based Systems, 67, pp. 90-104.

Publisher

© Elsevier

Version

  • AM (Accepted Manuscript)

Publisher statement

This work is made available according to the conditions of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) licence. Full details of this licence are available at: https://creativecommons.org/licenses/by-nc-nd/4.0/

Publication date

2014

Notes

This paper was accepted for publication in the journal Knowledge-Based Systems and the definitive published version is available at http://dx.doi.org/10.1016/j.knosys.2014.06.005

ISSN

0950-7051

Language

  • en