The University of Southampton
University of Southampton Institutional Repository

Optimal learning from verified training data

Optimal learning from verified training data
Optimal learning from verified training data
Standard machine learning algorithms typically assume that data is sampled independently from the distribution of interest. In attempts to relax this assumption, fields such as adversarial learning typically assume that data is provided by an adversary, whose sole objective is to fool a learning algorithm. However, in reality, it is often the case that data comes from self-interested agents, with less malicious goals and intentions which lie somewhere between the two settings described above. To tackle this problem, we present a Stackelberg competition model for least squares regression, in which data is provided by agents who wish to achieve specific predictions for their data. Although the resulting optimisation problem is nonconvex, we derive an algorithm which converges globally, outperforming current approaches which only guarantee convergence to local optima. We also provide empirical results on two real-world datasets, the medical personal costs dataset and the red wine dataset, showcasing the performance of our algorithm relative to algorithms which are optimal under adversarial assumptions, outperforming the state of the art.
Machine Learning, Adversarial Machine Learning, Nonconvex Optimisation
NeurIPS
Bishop, Nicholas
e2b8dc1a-a609-4709-84af-9b2455fd73e6
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Gerding, Enrico
d9e92ee5-1a8c-4467-a689-8363e7743362
Larochelle, H.
Ranzato, M.
Hadsell, R.
Balcan, M.F.
Lin, H.
Bishop, Nicholas
e2b8dc1a-a609-4709-84af-9b2455fd73e6
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Gerding, Enrico
d9e92ee5-1a8c-4467-a689-8363e7743362
Larochelle, H.
Ranzato, M.
Hadsell, R.
Balcan, M.F.
Lin, H.

Bishop, Nicholas, Tran-Thanh, Long and Gerding, Enrico (2020) Optimal learning from verified training data. Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F. and Lin, H. (eds.) In Advances in Neural Information Processing Systems 33 (NeurIPS 2020). NeurIPS..

Record type: Conference or Workshop Item (Paper)

Abstract

Standard machine learning algorithms typically assume that data is sampled independently from the distribution of interest. In attempts to relax this assumption, fields such as adversarial learning typically assume that data is provided by an adversary, whose sole objective is to fool a learning algorithm. However, in reality, it is often the case that data comes from self-interested agents, with less malicious goals and intentions which lie somewhere between the two settings described above. To tackle this problem, we present a Stackelberg competition model for least squares regression, in which data is provided by agents who wish to achieve specific predictions for their data. Although the resulting optimisation problem is nonconvex, we derive an algorithm which converges globally, outperforming current approaches which only guarantee convergence to local optima. We also provide empirical results on two real-world datasets, the medical personal costs dataset and the red wine dataset, showcasing the performance of our algorithm relative to algorithms which are optimal under adversarial assumptions, outperforming the state of the art.

Text
Optimal Learning From Verified Data - Accepted Manuscript
Download (277kB)

More information

Accepted/In Press date: 25 September 2020
Published date: 2020
Keywords: Machine Learning, Adversarial Machine Learning, Nonconvex Optimisation

Identifiers

Local EPrints ID: 445489
URI: http://eprints.soton.ac.uk/id/eprint/445489
PURE UUID: 80ba9fd6-0215-43cb-a1dd-32f548871b08
ORCID for Nicholas Bishop: ORCID iD orcid.org/0000-0001-7062-9072
ORCID for Long Tran-Thanh: ORCID iD orcid.org/0000-0003-1617-8316
ORCID for Enrico Gerding: ORCID iD orcid.org/0000-0001-7200-552X

Catalogue record

Date deposited: 11 Dec 2020 17:30
Last modified: 17 Mar 2024 03:03

Export record

Contributors

Author: Nicholas Bishop ORCID iD
Author: Long Tran-Thanh ORCID iD
Author: Enrico Gerding ORCID iD
Editor: H. Larochelle
Editor: M. Ranzato
Editor: R. Hadsell
Editor: M.F. Balcan
Editor: H. Lin

Download statistics

Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.

View more statistics

Atom RSS 1.0 RSS 2.0

Contact ePrints Soton: eprints@soton.ac.uk

ePrints Soton supports OAI 2.0 with a base URL of http://eprints.soton.ac.uk/cgi/oai2

This repository has been built using EPrints software, developed at the University of Southampton, but available to everyone to use.

We use cookies to ensure that we give you the best experience on our website. If you continue without changing your settings, we will assume that you are happy to receive cookies on the University of Southampton website.

×