3 years ago

An efficient gradient-based model selection algorithm for multi-output least-squares support vector regression machines

Multi-output least-squares support vector regression machines (MLS-SVR) is proposed by [Xu, S., An, X., Qiao, X., Zhu, L., Li, L., 2013. Multi-output least-squares support vector regression machines. Pattern Recognition Letters 34, 1078–1084] to handle multi-output regression problems. However, the prohibitive cost of model selection severely hinders MLS-SVR’s application. In this paper, an efficient gradient-based model selection algorithm for MLS-SVR is proposed. Firstly, a new training algorithm for MLS-SVR is developed, which allows one to obtain the solution vector for each output independently by dealing with matrices of much lower order. Based on the new training algorithm, a new leave-one-out error estimate is derived through virtual leave-one-out cross-validation. The model selection criterion is based on the new leave-one-out error estimate and its derivatives with respect to the hyper-parameters are also derived analytically. Both the model selection criterion and its partial derivatives can be obtained straightway once a training process ended. Finally, the hyper-parameters corresponding to the lowest model selection criterion is obtained through gradient decent method. The effectiveness and generalization performance of the proposed algorithm are validated through experiments on several multi-output datasets. Experiment results show that the proposed algorithm can save computational time dramatically without losing accuracy.

Publisher URL: www.sciencedirect.com/science

DOI: S0167865518300278

You might also like
Discover & Discuss Important Research

Keeping up-to-date with research can feel impossible, with papers being published faster than you'll ever be able to read them. That's where Researcher comes in: we're simplifying discovery and making important discussions happen. With over 19,000 sources, including peer-reviewed journals, preprints, blogs, universities, podcasts and Live events across 10 research areas, you'll never miss what's important to you. It's like social media, but better. Oh, and we should mention - it's free.

  • Download from Google Play
  • Download from App Store
  • Download from AppInChina

Researcher displays publicly available abstracts and doesn’t host any full article content. If the content is open access, we will direct clicks from the abstracts to the publisher website and display the PDF copy on our platform. Clicks to view the full text will be directed to the publisher website, where only users with subscriptions or access through their institution are able to view the full article.