Optimal learning from verified training data
Optimal learning from verified training data
Standard machine learning algorithms typically assume that data is sampled independently from the distribution of interest. In attempts to relax this assumption, fields such as adversarial learning typically assume that data is provided by an adversary, whose sole objective is to fool a learning algorithm. However, in reality, it is often the case that data comes from self-interested agents, with less malicious goals and intentions which lie somewhere between the two settings described above. To tackle this problem, we present a Stackelberg competition model for least squares regression, in which data is provided by agents who wish to achieve specific predictions for their data. Although the resulting optimisation problem is nonconvex, we derive an algorithm which converges globally, outperforming current approaches which only guarantee convergence to local optima. We also provide empirical results on two real-world datasets, the medical personal costs dataset and the red wine dataset, showcasing the performance of our algorithm relative to algorithms which are optimal under adversarial assumptions, outperforming the state of the art.
Machine Learning, Adversarial Machine Learning, Nonconvex Optimisation
Neural Information Processing Systems Foundation
Bishop, Nicholas
e2b8dc1a-a609-4709-84af-9b2455fd73e6
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Gerding, Enrico
d9e92ee5-1a8c-4467-a689-8363e7743362
2020
Bishop, Nicholas
e2b8dc1a-a609-4709-84af-9b2455fd73e6
Tran-Thanh, Long
e0666669-d34b-460e-950d-e8b139fab16c
Gerding, Enrico
d9e92ee5-1a8c-4467-a689-8363e7743362
Bishop, Nicholas, Tran-Thanh, Long and Gerding, Enrico
(2020)
Optimal learning from verified training data.
Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F. and Lin, H.
(eds.)
In Advances in Neural Information Processing Systems 33 (NeurIPS 2020).
Neural Information Processing Systems Foundation..
Record type:
Conference or Workshop Item
(Paper)
Abstract
Standard machine learning algorithms typically assume that data is sampled independently from the distribution of interest. In attempts to relax this assumption, fields such as adversarial learning typically assume that data is provided by an adversary, whose sole objective is to fool a learning algorithm. However, in reality, it is often the case that data comes from self-interested agents, with less malicious goals and intentions which lie somewhere between the two settings described above. To tackle this problem, we present a Stackelberg competition model for least squares regression, in which data is provided by agents who wish to achieve specific predictions for their data. Although the resulting optimisation problem is nonconvex, we derive an algorithm which converges globally, outperforming current approaches which only guarantee convergence to local optima. We also provide empirical results on two real-world datasets, the medical personal costs dataset and the red wine dataset, showcasing the performance of our algorithm relative to algorithms which are optimal under adversarial assumptions, outperforming the state of the art.
Text
Optimal Learning From Verified Data
- Accepted Manuscript
More information
Accepted/In Press date: 25 September 2020
Published date: 2020
Keywords:
Machine Learning, Adversarial Machine Learning, Nonconvex Optimisation
Identifiers
Local EPrints ID: 445489
URI: http://eprints.soton.ac.uk/id/eprint/445489
PURE UUID: 80ba9fd6-0215-43cb-a1dd-32f548871b08
Catalogue record
Date deposited: 11 Dec 2020 17:30
Last modified: 10 Apr 2024 01:41
Export record
Contributors
Author:
Nicholas Bishop
Author:
Long Tran-Thanh
Author:
Enrico Gerding
Editor:
H. Larochelle
Editor:
M. Ranzato
Editor:
R. Hadsell
Editor:
M.F. Balcan
Editor:
H. Lin
Download statistics
Downloads from ePrints over the past year. Other digital versions may also be available to download e.g. from the publisher's website.
View more statistics