Mechanism design for eliciting costly observations in next generation citizen sensor networks

Abstract

Citizen sensor networks are open information systems in which members of the public act as information providers. The information distributed in such networks ranges from observations of events (e.g. noise measurements or monitoring of environmental parameters) to probabilistic estimates (e.g. projected traffic reports or weather forecasts). However, due to rapid advances in technology such as high speed mobile internet and sophisticated portable devices (from smartphones to hand-held game consoles), it is expected that citizen sensor networks will evolve. This evolution will be driven by an increase in the number of information providers, since, in the future, it will be much easier to gather and communicate information at a large scale, which in turn, will trigger a transition to more commercial applications. Given this projected evolution, one key difference between future citizen sensor networks and conventional present ones is the emergence of self-interested behaviour, which can manifest in two main ways. First, information providers may choose to commit insufficient resources when producing their observations, and second, they may opt to misreport them. Both aspects of this self-interested behaviour are ignored in current citizen sensor networks. However, as their applications are broadened and commercial applications expand, information providers are likely to demand some kind of payment (e.g. real or virtual currency) for the information they provide. Naturally, those interested in buying this information, will also require guarantees of its quality.

It is these issues that we deal with in this thesis through the introduction of a series of novel twostage mechanisms, based on strictly proper scoring rules. We focus on strictly proper scoring rules, as they have been used in the past as a method of eliciting truthful reporting of predictions in various forecasting scenarios (most notably in weather forecasting). By using payments that are based on such scoring rules, our mechanisms effectively address the issue of selfish behaviour by motivating information providers in a citizen sensor network to, first, invest the resources required by the information buyer in the generation of their observations, and second, to report them truthfully.

To begin with, we introduce a mechanism that allows the centre (acting as an information buyer) to select a single agent that can provide a costly observation at a minimum cost. This is the first time a mechanism has been derived for a setting in which the centre has no knowledge of the actual costs involved in the generation of the agents’ observations. Building on this, we then make two further contributions to the state of the art, with the introduction of two extensions of this mechanism. First, we extend the mechanism so that it can be applied in a citizen sensor network where the information providers do not have the same resources available for the generation of their observations. These different capabilities are reflected in the quality of the provided observations. Hence, the centre must select multiple agents by eliciting their costs and the maximum precisions of their observations and then ask them to produce these observations. Second, we consider a setting where the information buyer cannot gain any knowledge of the actual outcome beyond what it receives through the agents’ reports. Now, because the centre is not able to evaluate the providers’ reported observations through external means, it has to rely solely on the reports it receives. It does this by fusing the reports together into one observation which then uses as a means to assess the reports of each of the providers.

For the initial mechanism and each of the two extensions, we prove their economic properties (i.e. incentive compatibility and individual rationality) and then present empirical results comparing a number of specific scoring rules, which includes the quadratic, spherical, logarithmic and a parametric family of scoring rules. These results show that although the logarithmic scoring rule minimises the mean and variance of an agent’s payment, using it may result in unbounded payments if an agent provides an observation of poor quality. Conversely, the payments of the parametric family exhibit finite bounds and are similar to those of the logarithmic rule for specific values of the parameter. Thus, we show that the parametric scoring rule is the best candidate in our setting. We empirically evaluate both extended mechanisms in the same way, and for the first extension, we show that the mechanism describes a family of possible ways to perform the agent selection, and that there is one that dominates all others. Finally, we compare both extensions with the peer prediction mechanism introduced by Miller, Resnick and Zeckhauser (2007) and show that in all three mechanisms the total expected payment is the same, while for both our mechanisms the variance in the total payment is significantly lower.

More information

Identifiers

Catalogue record

Export record