Next generation networks seek to integrate wide-area and local-area wireless and wired systems in order to provide seamless services to the end user, providing freedom of movement between metropolitan/enterprise and indoor/outdoor coverage while maintaining continuity of applications experience. Seamless Mobility, and its vision of seamless service delivery, requires significant changes to existing wired and wireless network management systems. Seamless Mobility is an experiential architecture, predicated on providing mechanisms that enable a user to accomplish his or her tasks without regard to technology. This talk will describe a novel autonomic architecture that uses a variety of different types of agents to synthesize knowledge about the environment, the context of users and their service and resource needs, and the capabilities and constraints placed on the network at any given time.

Agents act according to a unique context-aware policy governance approach, and work autonomously to orchestrate system behavior according to business goals. This talk will conclude with a summary of research directions at Motorola Labs and future work to be undertaken.

Autonomic computing is a perfect application domain for autonomous agents and multiagent systems. Expanding on this theme, I will illustrate it with various prototypes my team has developed. Rather than building on existing agent platforms, we have started from existing products and tried to extend them to have some agent-like properties. Therefore, the agents are somewhat coarse and primitive, and "multi" means 2 or 3 in our prototypes, but these agents have the advantage of being very real and practical. And, even with just two agents, we have run into some nasty issues where the two try to adapt simultaneously and work at cross purposes unless we do something explicit to prevent it. We also have some novel machine learning approaches and applications. I will conclude by describing several research challenges for the agent research community.

For close to 15 years, researchers in agents have provided a growing body of evidence that taskwork and teamwork are separable, and have repeatedly demonstrated the benefits of such separation. I will survey my group's role in investigating this hypothesis with physical robots, the lessons learned, and the challenges and opportunities provided by viewing robots as agents, too.

Agents in open environments lack a central mechanism for controlling their behavior, and they may encounter human decision-makers whose behavior is affected by social and psychological factors. Examples include online markets, patient care-delivery systems; virtual reality and simulation systems used for training and IT systems administration. As open environments increase in number and complexity, they pose significant challenges for agent designers. In this talk, I will discuss several research thrusts that address some of the challenges of automated agent design and evaluation in open environments. These agents are self-interested in the sense that they aim to fulfill their own goals as efficiently as possible. However, they may still cooperate if such behavior can serve either their short-or long-term interests or goals. I will present algorithms for representing and learning agents' goals and capabilities that have been evaluated under diverse settings varying in the extent to which agents are cooperative or competitive. Lastly, I will describe Color Trails, a game infrastructure for investigating negotiation strategies in open environments. Color Trails provides a realistic but modeling-tractable setting that facilitates the design and evaluation of automated decision making by agents.

Recent years are seeing dramatically growing interest in robots, side-by-side, paradoxically, with a worrisome decline in robotics work within the autonomous agent community. This, despite the significant opportunities enabled by viewing robots as agents. In this talk, I will argue for such an inclusive view, by examining multi-robot teams. From an agent perspective, challenges in building such teams are many: Some are related to a particular task (taskwork), and some are related to the interactions between agents (teamwork).

For close to 15 years, researchers in agents have provided a growing body of evidence that taskwork and teamwork are separable, and have repeatedly demonstrated the benefits of such separation. I will survey my group's role in investigating this hypothesis with physical robots, the lessons learned, and the challenges and opportunities provided by viewing robots as agents, too.

Virtual environments are often populated by autonomous synthetic agents capable of acting and interacting with other agents as well as with humans. These virtual worlds also include objects that may have different uses and types of interactions. As such, these agents need to identify possible interactions with the objects in the environment and measure the consequences of these interactions. This is particularly difficult when the agents never interacted with some of the objects beforehand. This paper describes SOTAI - Smart ObjecT-Agent Interaction, a framework that will help agents to identify possible interactions with unknown objects based on their past experiences. In SOTAI, agents can learn world regularities, like object attributes and frequent relations between attributes. They gather qualitative symbolic descriptions from their sensorial data when interacting with objects and perform inductive reasoning to acquire concepts about them. We implemented an initial case study and the results show that our agents are able to acquire valid conceptual knowledge.

Intelligent agents that are intended to work in dynamic environments must be able to gracefully handle unsuccessful tasks and plans. In addition, such agents should be able to make rational decisions about an appropriate course of action, which may include aborting a task or plan, either as a result of the agent's own deliberations, or potentially at the request of another agent. In this paper we investigate the incorporation of aborts into a BDI-style architecture. We discuss some conditions under which aborting a task or plan is appropriate, and how to determine the consequences of such a decision. We augment each plan with an optional abort-method, analogous to the failure method found in some agent programming languages. We provide an operational semantics for the execution cycle in the presence of aborts in the abstract agent language CAN, which enables us to specify a BDI-based execution model without limiting our attention to a particular agent system (such as JACK, Jadex, Jason, or SPARK). A key technical challenge we address is the presence of parallel execution threads and of sub-tasks, which require the agent to ensure that the abort methods for each plan are carried out in an appropriate sequence.

We develop a Belief-Desire-Intention (BDI) style agent-oriented programming language with special emphasis on the semantics of goals in the presence of the typical BDI failure handling present in many BDI systems and a novel account of hierarchical lookahead planning. The work builds incrementally on two existing languages and accommodates three type of goals: classical BDI-style event goals, declarative goals, and planning goals. We mainly focus on the dynamics of these type of goals and, in particular, on a kind of commitment scheme that brings the new language closer to the solid existing work in agent theory. To that end, we develop a semantics that recognises the usual hierarchical structure of active goals as well as their declarative aspects. In contrast with previous languages, the new language prevents an agent from blindly persisting with a (blocked) subsidiary goal when an alternative strategy for achieving a higher-level motivating goal exists. In addition, the new semantics ensures watchfulness by the agent to ensure that goals that succeed or are deemed impossible are immediately dropped, thus conforming to the requirements of basic rational commitment strategy. Finally, a mechanism for the proactive adoption of new goals, other than the mere reaction to events, and a formal account of interaction with the external environment are provided. We believe that the new language is an important step towards turning practical BDI programming languages more compatible with the established results in the area of agent theory.

There is a growing interest in developing technologies for creating interactive dramas [13, 22]. Evaluating them, however, remains an open research problem. In this paper, we present a method for evaluating the technical and design approaches employed in a conversation-centered interactive drama. This method correlates players' subjective experience during conversational breakdowns, captured using retrospective protocols, with the corresponding AI processing in the input language understanding and dialog management subsystems. The methodology is employed to analyze conversation breakdowns in the interactive drama Façade. We find that the narrative cues offered by an interactive drama, coupled with believable character performance, can allow players to interpretively bridge system limitations and avoid experiencing a conversation breakdown. Further, we find that, contrary to standard practice for task-oriented conversation systems, using shallowly understood information as part of the system output hampers the player experience in an interactive drama.

Recognizing and annotating the occurrence of team actions in observations of embodied agents has applications in surveillance and in training of military or sport teams. We describe the team actions through a spatio-temporal correlated pattern of movement, which can be modeled by a Hidden Markov Model. The hand-crafting of these models is a difficult task of knowledge engineering, even in application domains where explicit, natural language descriptions of the team actions are available. The main contribution of this paper is an approach through which the library of HMM representations can be acquired from a small number of hand annotated, representative samples of the specific movement patterns. A series of experiments, performed on a dataset describing a real-world terrestrial warfare exercise validates our method and shows good recognition accuracy even in the presence of noisy data. The speed of the recognition engine is sufficiently fast to allow real time annotation of incoming observations.

For the past few years many new applications are being developed featuring interactive environments populated with autonomous virtual agents capable of acting according to their goals, beliefs and even social relations. Such agents must be able to interact with each other, and more importantly with the user, thus involving the users in an engaging narrative experience. To achieve these, in this paper we describe the essential event structure in an autobiographic memory, event reconstructions in memory retrieval process and the influences of such past events in interpersonal relations.

Most of the research on multiagent systems has focused on the development of rational utility-maximizing agents. However, research shows that emotions have a strong effect on peoples' physical states, motivations, beliefs, and desires. By introducing primary and secondary emotion into BDI architecture, we present a generic architecture for an emotional agent, EBDI, which can merge various emotion theories with an agent's reasoning process. It implements practical reasoning techniques separately from the specific emotion mechanism. The separation allows us to plug in emotional models as needed or upgrade the agent's reasoning engine independently.

Most Interactive Storytelling systems developed to date have followed a task-based approach to story representation, using planning techniques to drive the story by generating a sequence of actions, which essentially "solve" the task to which the story is equated. One major limitation of this approach has been that it fails to incorporate characters' psychology, and as a consequence important aesthetic aspects of the narrative cannot be easily captured by Interactive Storytelling. In this paper, we introduce a new approach to Interactive Storytelling, which aims at reconciling narrative actions with the characters' attributed psychology as stated in the narrative. Our long-term goal is to be able to explore Interactive Storytelling for those narrative genres which are based on the characters' psychology rather than solely on their actions. We used as a starting point the formalisation by Flaubert himself of his novel Madame Bovary, which includes a detailed account of characters' desires and feelings. We describe a prototype in which characters' behaviour is driven by a real-time search-based planning system applying operators whose content is based on a specific inventory of feelings. Furthermore, the actual pattern of evolution of the character's plan, as measured through the variation of the search heuristic, is used to confer a sense of awareness to the characters, which can be used to generate feelings about its overall situation, from feelings of boredom to hope.

We present a new method for carrying out state estimation in multi-agent settings that are characterized by continuous or large discrete state spaces. State estimation in multiagent settings involves updating an agent's belief over the physical states and the space of other agents' models. We factor out the models of the other agents and update the agent's belief over these models, as exactly as possible. Simultaneously, we sample particles from the distribution over the large physical state space and project the particles in time.

For embodied agents to engage in realistic multiparty conversation, they must stand in appropriate places with respect to other agents and the environment. When these factors change, such as an agent joining the conversation, the agents must dynamically move to a new location and/or orientation to accommodate. This paper presents an algorithm for simulating movement of agents based on observed human behavior using techniques developed for pedestrian movement in crowd simulations. We extend a previous group conversation simulation to include an agent motion algorithm. We examine several test cases and show how the simulation generates results that mirror real-life conversation settings.

A negotiation chain is formed when multiple related negotiations are spread over multiple agents. In order to appropriately order and structure the negotiations occurring in the chain so as to optimize the expected utility, we present an extension to a single-agent concurrent negotiation framework. This work is aimed at semi-cooperative multi-agent systems, where each agent has its own goals and works to maximize its local utility; however, the performance of each individual agent is tightly related to other agent's cooperation and the system's overall performance. We introduce a pre-negotiation phase that allows agents to transfer meta-level information. Using this information, the agent can build a more accurate model of the negotiation in terms of modeling the relationship of flexibility and success probability. This more accurate model helps the agent in choosing a better negotiation solution in the global negotiation chain context. The agent can also use this information to allocate appropriate time for each negotiation, hence to find a good ordering of all related negotiations. The experimental data shows that these mechanisms improve the agents' and the system's overall performance significantly.

This paper addresses the problem of recognizing policies given logs of battle scenarios from multi-player games. The ability to identify individual and team policies from observations is important for a wide range of applications including automated commentary generation, game coaching, and opponent modeling. We define a policy as a preference model over possible actions based on the game state, and a team policy as a collection of individual policies along with an assignment of players to policies. This paper explores two promising approaches for policy recognition: (1) a model-based system for combining evidence from observed events using Dempster-Shafer theory, and (2) a data-driven discriminative classifier using support vector machines (SVMs). We evaluate our techniques on logs of real and simulated games played using Open Gaming Foundation d20, the rule system used by many popular tabletop games, including Dungeons and Dragons.

Recent studies have shown that various models can explain the emergence of complex networks, such as scale-free and small-world networks. This paper presents a different model to generate complex networks using a multi-agent approach. Each node is considered as an agent. Based on voting by all agents, edges are added repeatedly. We use four different kinds of centrality measures as a utility functions for agents. Depending on the centrality measure, the resultant networks differ considerably: typically, closeness centrality generates a scale-free network, degree centrality produces a random graph, betweenness centrality favors a regular graph, and eigenvector centrality brings a complete subgraph. The importance of the network structure among agents is widely noted in the multi-agent research literature. This paper contributes new insights into the connection between agents' local behavior and the global property of the network structure. We describe a detailed analysis on why these structures emerge, and present a discussion of the possible expansion and application of the model.

Distributed applications require distributed techniques for efficient resource allocation. These techniques need to take into account the heterogeneity and potential unreliability of resources and resource consumers in a distributed environments. In this paper we propose a distributed algorithm that solves the resource allocation problem in distributed multi-agent systems. Our solution is based on the self-organisation of agents, which does not require any facilitator or management layer. The resource allocation in the system is a purely emergent effect. We present results of the proposed resource allocation mechanism in the simulated static and dynamic multi-server environment.

An organization comprises a group of collaborating agents (individual agents or nested organizations) that exhibit complex behaviors. Of particular interest are dynamic organizations that form and dissolve as their members' needs change. Such organizations are important in many applications, including scientific and business computing. Contracts among autonomous agents have long been used to facilitate their collaboration. This paper provides a contracts-based approach for managing organizations. The proposed approach places organizations within institutions, themselves modeled as specialized organizations. Commitments form the basis of contracts and this paper establishes some important dynamic aspects by providing a commitment life cycle analysis. This approach has been applied in a prototype tool to manage organizations.

This paper presents a protocol for agents engaged in argumentation over access to information sources. Obtaining relevant information is essential for agents engaged in autonomous, goal-directed behavior, but access to such information is usually controlled by other autonomous agents having their own goals. Because these various goals may be in conflict with one another, rational interactions between the two agents may take the form of a dialog, in which requests for information are successively issued, considered, justified and criticized. Even when the agents involved in such discussions agree on all the arguments for and the arguments against granting access to some information source, they may still disagree on their preferences between these arguments.

To represent such situations, we design a protocol for dialogs between two autonomous agents for seeking and granting authorization to access some information source. This protocol is based on an argumentation dialog where agents handle specific preferences and acceptability over arguments.

Multi-Agent Agreement Problems (MAP) - the ability of a population of agents to search out and converge on a common state - are central issues in many multi-agent settings, from distributed sensor networks, to meeting scheduling, to development of norms, conventions, and language. While much work has been done on particular agreement problems no unifying framework exists for comparing MAPs that vary in, e.g., strategy space complexity, inter-agent accessibility, and solution type, and understanding their relative complexities. We present such a unification, the Distributed Optimal Agreement (DOA) framework, and show how it captures a wide variety of agreement problems. To demonstrate DOA and its power we apply it to convention evolution. ¹

In this paper, we address the problem of building a system of autonomous agents for a complex environment, in our case, a museum with many visitors. Visitors may have varying preferences for types of art or may wish to visit different exhibits on multiple visits. Often, these goals conflict. For example, many visitors may wish to see the museum's most popular work, but that could cause congestion, ruining the experience. Thus, our task is to build a set of agents that can satisfy their visitors' goals, while simultaneously providing high quality experiences for all.

We study strategic delay in bargaining under uncertainty. First, we discuss bargaining situations in which dynamics and uncertainty may induce the use of strategic delay by intelligent agents. Then, we analyze strategic delay under various circumstances. Finally, we propose an agent reasoning mechanism to decide when and how to deploy strategic delay.

We present a novel approach to knowledge-based automated one-shot multi-issue bilateral negotiation handling, in a homogeneous setting, both numerical features and non-numerical ones. To this aim we introduce P(N), a propositional logic extended with concrete domains, which allows to: model relations among issues (both numerical and not numerical ones) via logical entailment, differently from well-known approaches that describe issues as uncorrelated; represent buyer's request, seller's supply and their respective preferences as formulas endowed with a formal semantics.

This paper proposes dynamic semantics for agent communication languages (ACLs) as a method for tackling some of the fundamental problems associated with agent communication in open multiagent systems. Based on the idea of providing alternative semantic "variants" for speech acts and transition rules between them that are contingent on previous agent behaviour, our framework provides an improved notion of grounding semantics in ongoing interaction, a simple mechanism for distinguishing between compliant and expected behaviour, and a way to specify sanction and reward mechanisms as part of the ACL itself. We extend a common framework for commitment-based ACL semantics to obtain these properties, discuss desiderata for the design of concrete dynamic semantics together with examples, and analyse their properties.

Making commitments, e.g., through promises and threats, enables a player to exploit the strengths of his own strategic position as well as the weaknesses of that of his opponents. Which commitments a player can make with credibility depends on the circumstances. In some, a player can only commit to the performance of an action, in others, he can commit himself conditionally on the actions of the other players. Some situations even allow for commitments on commitments or for commitments to randomized actions. We explore the formal properties of these types of (conditional) commitment and their interrelationships. So as to preclude inconsistencies among conditional commitments, we assume an order in which the players make their commitments. Central to our analyses is the notion of an extortion, which we define, for a given order of the players, as a profile that contains, for each player, an optimal commitment given the commitments of the players that committed earlier. On this basis, we investigate for different commitment types whether it is advantageous to commit earlier rather than later, and how the outcomes obtained through extortions relate to backward induction and Pareto efficiency.

Commitment protocols formalize interactions among autonomous, heterogeneous agents, leaving the agents' local policies unspecified. This paper studies the problem of agents enacting commitment protocols, which inherently requires that their policies cohere with the given protocols. Specifically, in many important settings, if agents incautiously create and discharge commitments, they can expose themselves to certain risk; conversely, if the agents are (excessively) cautious, a protocol enactment may deadlock. This paper adopts the well-known idea of monotonic concession, but specializes and enhances it with the particular features of commitments. Specifically, this paper formulates inference rules for commitment concession that respect the nature of commitments. Next, it shows how commitments can be systematically revised as the agents incrementally engage each other in enacting their protocol. This paper demonstrates how such rules can be applied in practice, and identifies conditions under which progress and termination of protocol enactment can be guaranteed.

Interactions between agents in an open system such as the Internet require a significant degree of flexibility. A crucial aspect of the development of such methods is the notion of commitments, which provides a mechanism for coordinating interactive behaviors among agents. In this paper, we investigate an approach to model commitments with tight integration with protocol actions. This means that there is no need to have an explicit mapping from protocols actions to operations on commitments and an external mechanism to process and enforce commitments. We show how agents can reason about commitments and protocol actions to achieve the end results of protocols using a reasoning system based on temporal linear logic, which incorporates both temporal and resource-sensitive reasoning. We also discuss the application of this framework to scenarios such as online commerce.

Protocols describe interactions among agents and thus underlie the engineering of multiagent systems. However, protocols are enacted by agents in physical systems. In particular, considerations of communication models and how distributed agents are able to make compatible choices would greatly affect whether a protocol may in fact be enacted successfully. The objective of this paper is to study the conceptual underpinnings of protocol enactment in multiagent systems. It seeks to characterize the operationalization of agents so as to determine whether and when agents may be interoperable.

Formal specifications of protocol-oriented agent interactions have focused mainly on the semantics of the constituent agent communication language (ACL). We argue that a proper theoretical treatment of conversations cannot be simply derived compositionally from the semantics of individual Communicative Acts (CAs). Accordingly, we develop a theory of joint conversations that is independent of its constituent CAs. We treat the process of a group following an interaction protocol as a persistent joint communicative action (JCA) by the group. We define compliance in a joint conversation and we prove salient properties of joint conversations.

Agent-mediated institutions (or e-institutions), introduced in [9], are open agent systems that allow heterogeneous agents to enter and perform tasks. The e-institutions specify the admissible behaviour of the agents by means of norms, which are declarative and abstract by nature. On the one hand this allows for a stable specification suitable for almost any conceivable situation that arises in the institution, but in the other hand the norms hardly give any indication which interaction patterns would guarantee satisfaction of the norms.

Diagrammatic methodologies for the representation of agent interaction protocols can be classified as joint representations that describe an interaction in terms of a single sequential process, or agent-centred representations that provide a distinct description of the interaction protocol for each agent (or role) in the interaction. Here we discuss the process of deriving agent centred representations from joint representations and vice versa, using a variant of UML statecharts.

Commitment protocols have been proposed as a basis for modeling and enacting interactions among agents, such as those needed to carry out business processes. A central idea is that protocols would be developed and shared via libraries, and refined and composed to produce protocols that serve specific needs. Success in this program, therefore, presupposes that individual protocols and their compositions can be formally verified with respect to the properties of interest. This paper outlines an approach for verifying the correctness of commitment protocols and their compositions that exploits the well-known software engineering technique of model checking.

Partially Observable Markov Decision Processes (POMDPs) emerged as the primary framework for decision-theoretic planning in single agent settings. Solutions to POMDPs are optimal plans which are conditional on future observations. Dynamic Influence Diagrams (DIDs) are computational representations of POMDPs which compute solutions for finite time horizons in an on-line fashion. Interactive POMDPs (I-POMDPs) [5] generalize POMDPs to multi-agent settings by including models of other agents in the state space. Interactive DIDs (I-DIDs), presented in this paper, are computational representations of I-POMDPs, and thus generalizations of DIDs. DIDs are themselves temporal generalizations of influence diagrams [6].

We address the problem of standardising the semantics of agent communication. The diversity of existing approaches suggests that no single agent communication language can satisfactorily cater for all scenarios. However, standardising the way in which different languages are specified is a viable alternative. We describe a standard meta-language in which the rules of an arbitrary institution can be specified. In this way different agent communication languages can be given a common grounding. From this starting point, we describe a component-based approach to standardisation, whereby a standard can develop by adding component sets of rules; for example to handle various classes of dialogues and normative relations. Eventually we envisage different agent institutions publishing a specification of their rules by simply specifying the subset of standard components in use in that institution. Agents implementing the meta-language can then interoperate between institutions by downloading the appropriate components.

Communication is a key tool for facilitating multiagent coordination in cooperative and uncertain domains. We focus on a class of multiagent problems modeled as Decentralized Markov Decision Processes with Communication (DEC-MDP-COM) with local observability. The planning problem for computing the optimal communication strategy in such domains is often formulated with the assumption of the knowledge of optimal domain-level policy. Computing the optimal communication policy is NP-complete. There is a need, then, for heuristic solutions that trade-off performance with efficiency. We present a decision theoretic approach for computing optimal communication policies in stochastic environments which uses a branching future representation and evaluates only those decisions that an agent is likely to encounter. The communication strategy computed off-line is used in the more probable scenarios that the agent would face in future. Our approach also allows agents to compute communication policies at run-time in the unlikely event of the agents facing scenarios that were discarded while computing the off-line policy.

The ambitious goal of transfer learning is to accelerate learning on a target task after training on a different, but related, source task. While many past transfer methods have focused on transferring value-functions, this paper presents a method for transferring policies across tasks with different state and action spaces. In particular, this paper utilizes transfer via inter-task mappings for policy search methods (TVITM-PS) to construct a transfer functional that translates a population of neural network policies trained via policy search from a source task to a target task. Empirical results in robot soccer Keepaway and Server Job Scheduling show that TVITM-PS can markedly reduce learning time when full inter-task mappings are available. The results also demonstrate that TVITMPS still succeeds when given only incomplete inter-task mappings. Furthermore, we present a novel method for learning such mappings when they are not available, and give results showing they perform comparably to hand-coded mappings.

This article deals with the problem of collaborative learning in a multi-agent system. Here each agent can update incrementally its beliefs B (the concept representation) so that it is in a way kept consistent with the whole set of information K (the examples) that he has received from the environment or other agents. We extend this notion of consistency (or soundness) to the whole MAS and discuss how to obtain that, at any moment, a same consistent concept representation is present in each agent. The corresponding protocol is applied to supervised concept learning. The resulting method SMILE (standing for Sound Multi-agent Incremental LEarning) is described and experimented here. Surprisingly some difficult boolean formulas are better learned, given the same learning set, by a Multi agent system than by a single agent.

To cope with large scale, agents are usually organized in a network such that an agent interacts only with its immediate neighbors in the network. Reinforcement learning techniques have been commonly used to optimize agents local policies in such a network because they require little domain knowledge and can be fully distributed. However, all of the previous work assumed the underlying network was fixed throughout the learning process. This assumption was important because the underlying network defines the learning context of each agent. In particular, the set of actions and the state space for each agent is defined in terms of the agent's neighbors. If agents dynamically change the underlying network structure (also called self-organize) during learning, then one needs a mechanism for transferring what agents have learned so far before (in the old network structure) to their new learning context (in the new network structure).

In this work we develop a novel self-organization mechanism that not only allows agents to self-organize the underlying network during the learning process, but also uses information from learning to guide the self-organization process. Consequently, our work is the first to study this interaction between learning and self-organization. Our self-organization mechanism uses heuristics to transfer the learned knowledge across the different steps of self-organization. We also present a more restricted version of our mechanism that is computationally less expensive and still achieve good performance. We use a simplified version of the distributed task allocation domain as our case study. Experimental results verify the stability of our approach and show a monotonic improvement in the performance of the learning process due to self-organization.

This paper presents the dynamics of multiple reinforcement learning agents from an Evolutionary Game Theoretic (EGT) perspective. We provide a Replicator Dynamics model for traditional multiagent Q-learning, and we extend these differential equations to account for lenient learners: agents that forgive possible mistakes of their teammates that resulted in lower rewards. We use this extended formal model to visualize the basins of attraction of both traditional and lenient multiagent Q-learners in two benchmark coordination problems. The results indicate that lenience provides learners with more accurate estimates for the utility of their actions, resulting in higher likelihood of convergence to the globally optimal solution. In addition, our research supports the strength of EGT as a backbone for multiagent reinforcement learning.

Classically, an approach to the multiagent policy learning supposed that the agents, via interactions and/or by using preliminary knowledge about the reward functions of all players, would find an interdependent solution called "equilibrium". Recently, however, certain researchers question the necessity and the validity of the concept of equilibrium as the most important multiagent solution concept. They argue that a "good" learning algorithm is one that is efficient with respect to a certain class of counterparts. Adaptive players is an important class of agents that learn their policies separately from the maintenance of the beliefs about their counterparts' future actions and make their decisions based on that policy and the current belief. In this paper, we propose an efficient learning algorithm in presence of the adaptive counterparts called Adaptive Dynamics Learner (ADL), which is able to learn an efficient policy over the opponents' adaptive dynamics rather than over the simple actions and beliefs and, by so doing, to exploit these dynamics to obtain a higher utility than any equilibrium strategy can provide. We tested our algorithm on a substantial representative set of the most known and demonstrative matrix games and observed that ADL agent is highly efficient against Adaptive Play Q-learning (APQ) agent and Infinitesimal Gradient Ascent (IGA) agent. In self-play, when possible, ADL is able to converge to a Pareto optimal strategy maximizing the welfare of all players.

No-regret algorithms have been proposed to control a wide variety of multi-agent systems. The appeal of no-regret algorithms is that they are easily implementable in large scale multi-agent systems because players make decisions using only retrospective or "regret based" information. Furthermore, there are existing results proving that the collective behavior will asymptotically converge to a set of points of "no-regret" in any game. We illustrate, through a simple example, that no-regret points need not reflect desirable operating conditions for a multi-agent system. Multi-agent systems often exhibit an additional structure (i.e. being "weakly acyclic") that has not been exploited in the context of no-regret algorithms. In this paper, we introduce a modification of the traditional no-regret algorithms by (i) exponentially discounting the memory and (ii) bringing in a notion of inertia in players' decision process. We show how these modifications can lead to an entire class of regret based algorithms that provide almost sure convergence to a pure Nash equilibrium in any weakly acyclic game.

This paper investigates the problem of estimating the value of probabilistic parameters needed for decision making in environments in which an agent, operating within a multi-agent system, has no a priori information about the structure of the distribution of parameter values. The agent must be able to produce estimations even when it may have made only a small number of direct observations, and thus it must be able to operate with sparse data. The paper describes a mechanism that enables the agent to significantly improve its estimation by augmenting its direct observations with those obtained by other agents with which it is coordinating. To avoid undesirable bias in relatively heterogeneous environments while effectively using relevant data to improve its estimations, the mechanism weighs the contributions of other agents' observations based on a real-time estimation of the level of similarity between each of these agents and itself. The "coordination autonomy" module of a coordination-manager system provided an empirical setting for evaluation. Simulation-based evaluations demonstrated that the proposed mechanism outperforms estimations based exclusively on an agent's own observations as well as estimations based on an unweighted aggregate of all other agents' observations.

It is known that the complexity of the reinforcement learning algorithms, such as Q-learning, may be exponential in the number of environment's states. It was shown, however, that the learning complexity for the goal-directed problems may be substantially reduced by initializing the Q-values with a "good" approximative function. In the multiagent case, there exists such a good approximation for a big class of problems, namely, for goal-directed stochastic games. These games, for example, can reflect coordination and common interest problems of cooperative robotics. The approximative function for these games is nothing but the relaxed, single-agent, problem solution, which can easily be found by each agent individually. In this article, we show that (1) an optimal single-agent solution is a "good" approximation for the goal-directed stochastic games with action-penalty representation and (b) the complexity is reduced when the learning is initialized with this approximative function, as compared to the uninformed case.

In this paper, we investigate the use of parallelization in reinforcement learning (RL), with the goal of learning optimal policies for single-agent RL problems more quickly by using parallel hardware. Our approach is based on agents using the SARSA(λ) algorithm, with value functions represented using linear function approximators. In our proposed method, each agent learns independently in a separate simulation of the single-agent problem. The agents periodically exchange information extracted from the weights of their approximators, accelerating convergence towards the optimal policy. We present empirical results for an implementation on a Beowulf cluster.

We consider the problem of playing in repeated extensive form games where agents do not have any prior. In this situation classic game theoretical tools are inapplicable and it is common the resort to learning techniques. In this paper, we present a novel learning principle that aims at avoiding oscillations in the agents' strategies induced by the presence of concurrent learners. We apply our algorithm in bargaining, and we experimentally evaluate it showing that using this principle reinforcement learning algorithms can improve their convergence time.

The dominant existing routing strategies employed in peer-to-peer(P2P) based information retrieval(IR) systems are similarity-based approaches. In these approaches, agents depend on the content similarity between incoming queries and their direct neighboring agents to direct the distributed search sessions. However, such a heuristic is myopic in that the neighboring agents may not be connected to more relevant agents. In this paper, an online reinforcement-learning based approach is developed to take advantage of the dynamic run-time characteristics of P2P IR systems as represented by information about past search sessions. Specifically, agents maintain estimates on the downstream agents' abilities to provide relevant documents for incoming queries. These estimates are updated gradually by learning from the feedback information returned from previous search sessions. Based on this information, the agents derive corresponding routing policies. Thereafter, these agents route the queries based on the learned policies and update the estimates based on the new routing policies. Experimental results demonstrate that the learning algorithm improves considerably the routing performance on two test collection sets that have been used in a variety of distributed IR studies.

This paper addresses the problem of identifying the value of information held by a teammate on a distributed, multi-agent team. It focuses on a distributed scheduling task in which computer agents support people who are carrying out complex tasks in a dynamic environment. The paper presents a decision-theoretic algorithm for determining the value of information that is potentially relevant to schedule revisions, but is directly available only to the person and not the computer agent. The design of a "coordination autonomy" (CA) module within a coordination-manager system provided the empirical setting for this work. By design, the CA module depends on an external scheduler module to determine the specific effect of additional information on overall system performance. The paper describes two methods for reducing the number of queries the CA issues to the scheduler, enabling it to satisfy computational resource constraints placed on it. Experimental results indicate the algorithm improves system performance and establish the exceptional efficiency---measured in terms of the number of queries required for estimating the value of information---that can be achieved by the query-reducing methods.

Finding the right agents in a large and dynamic network to provide the needed resources in a timely fashion, is a long standing problem. This paper presents a method for information searching and sharing that combines routing indices with token-based methods. The proposed method enables agents to search effectively by acquiring their neighbors' interests, advertising their information provision abilities and maintaining indices for routing queries, in an integrated way. Specifically, the paper demonstrates through performance experiments how static and dynamic networks of agents can be 'tuned' to answer queries effectively as they gather evidence for the interests and information provision abilities of others, without altering the topology or imposing an overlay structure to the network of acquaintances.

Determining the provenance of data, i.e. the process that led to that data, is vital in many disciplines. For example, in science, the process that produced a given result must be demonstrably rigorous for the result to be deemed reliable. A provenance system supports applications in recording adequate documentation about process executions to answer queries regarding provenance, and provides functionality to perform those queries. Several provenance systems are being developed, but all focus on systems in which the components are reactive, for example Web Services that act on the basis of a request, job submission system, etc. This limitation means that questions regarding the motives of autonomous actors, or agents, in such systems remain unanswerable in the general case. Such questions include: who was ultimately responsible for a given effect, what was their reason for initiating the process and does the effect of a process match what was intended to occur by those initiating the process? In this paper, we address this limitation by integrating two solutions: a generic, re-usable framework for representing the provenance of data in service-oriented architectures and a model for describing the goal-oriented delegation and engagement of agents in multi-agent systems. Using these solutions, we present algorithms to answer common questions regarding responsibility and success of a process and evaluate the approach with a simulated healthcare example.

In this paper we present an advanced bidding agent that participates in first-price sealed bid auctions to allocate advertising space on BluScreen - an experimental public advertisement system that detects users through the presence of their Bluetooth enabled devices. Our bidding agent is able to build probabilistic models of both the behaviour of users who view the adverts, and the auctions that it participates within. It then uses these models to maximise the exposure that its adverts receive. We evaluate the effectiveness of this bidding agent through simulation against a range of alternative selection mechanisms including a simple bidding strategy, random allocation, and a centralised optimal allocation with perfect foresight. Our bidding agent significantly outperforms both the simple bidding strategy and the random allocation, and in a mixed population of agents it is able to expose its adverts to 25% more users than the simple bidding strategy. Moreover, its performance is within 7.5% of that of the centralised optimal allocation despite the highly uncertain environment in which it must operate.

Distributed message relaying is an important function of a peer-to-peer system to discover service providers. Existing search protocols in unstructured peer-to-peer systems either create huge burden on communications or cause long response time. Moreover, these systems are also vulnerable to the free riding problem. In this paper we present an incentive mechanism that not only mitigates the free riding problem, but also achieves good system efficiency in message relaying for peer discovery. In this mechanism promised rewards are passed along the message propagation process. A peer is rewarded if a service provider is found via a relaying path that includes this peer. We provide some analytic insights to the symmetric Nash equilibrium strategies of this game, and an approximate approach to calculate this equilibrium. Experiments show that this incentive mechanism brings a system utility generally higher than breadth-first search and random walks, based on both the estimated utility from our approximate equilibrium and the utility generated from learning in the incentive mechanism.

We derive optimal bidding strategies for a global bidding agent that participates in multiple, simultaneous second-price auctions with perfect substitutes. We first consider a model where all other bidders are local and participate in a single auction. For this case, we prove that, assuming free disposal, the global bidder should always place non-zero bids in all available auctions, irrespective of the local bidders' valuation distribution. Furthermore, for non-decreasing valuation distributions, we prove that the problem of finding the optimal bids reduces to two dimensions. These results hold both in the case where the number of local bidders is known and when this number is determined by a Poisson distribution. This analysis extends to online markets where, typically, auctions occur both concurrently and sequentially. In addition, by combining analytical and simulation results, we demonstrate that similar results hold in the case of several global bidders, provided that the market consists of both global and local bidders. Finally, we address the efficiency of the overall market, and show that information about the number of local bidders is an important determinant for the way in which a global bidder affects efficiency.

The paper deals with on-board planning for a satellite swarm via communication and negotiation. We aim at defining individual behaviours that result in a global behaviour that meets the mission requirements. We will present the formalization of the problem, a communication protocol, a solving method based on reactive decision rules, and first results.

This work proposes a method for allocating temporally interdependent tasks to homogeneous or heterogeneous cooperative agents in dynamic large-scale networks. This method views searching, task allocation and scheduling as an integrated problem that has to be efficiently solved in such networks. Solving the general problem optimally in a decentralized way is very hard and can only be solved by a centralized method, be approximated by means of heuristics, or by relaxations of the original problem. Our method facilitates effective searching through the dynamic assignment of gateway roles to agents and the exploitation of routing indices. In combination to searching, it exploits distributed constraint satisfaction techniques and dynamic re-organization of agent teams to efficiently handle the allocation of complex tasks with interdependent subtasks.

TÆMS is a hierarchical modeling language capable of representing complex task networks with intra-task uncertainties and inter-task dependencies. The uncertainty and complexity of the application domains represented in TÆMS models often lead to very large state spaces, which push the need to design efficient solution algorithms for TÆMS problems. In this paper, we present a solver that integrates selective state space search techniques with state space decomposition techniques. Our experiments demonstrate that the solver can find an (approximately) optimal solution much faster than prior approaches.

In this paper we propose a methodology for the construction of 3D electronic institutions. 3D electronic institutions are normative environments where software and human agents can participate and collaborate in a joint 3D Virtual World. The proposed methodology covers the specification of the institutional rules, as well as the design and visualization of 3D environments for the specified institution. It is also supplied with a set of graphical tools that facilitate the development process on every level, from specification to deployment. The resulting system facilitates the direct integration of human users into Multi-Agent Systems as they participate by driving an avatar in the generated 3D environment.

Criminal behaviour often involves a combination of physical, mental, social and environmental (multi-)agent aspects, such as neurological deviations, hormones, arousal, (non)empathy, targets and social control. This paper contributes a dynamical agent-based approach for analysis and simulation of criminal behaviour, covering the above aspects, illustrated for the case of an Intermittent Explosive Disorder. It involves dynamically generated desires and beliefs in opportunities within the social environment, both based on literature on criminal behaviour.

We investigate a framework where agents search for satisfying products by using referrals from other agents. Our model of a mechanism for transmitting word-of-mouth and the resulting behavioural effects is based on integrating a module governing the local behaviour of agents with a module governing the structure and function of the underlying network of agents. Local behaviour incorporates a satisficing model of choice, a set of rules governing the interactions between agents, including learning about the trustworthiness of other agents over time, and external constraints on behaviour that may be imposed by market barriers or switching costs. Local behaviour takes place on a network substrate across which agents exchange positive and negative information about products. We use various degree distributions dictating the extent of connectivity, and incorporate both small-world effects and the notion of preferential attachment in our network models. We compare the effectiveness of referral systems over various network structures for easy and hard choice tasks, and evaluate how this effectiveness changes with the imposition of market barriers.

Human team members often develop shared expectations to predict each other's needs and coordinate their behaviors. In this paper the concept "Shared Belief Map" is proposed as a basis for developing realistic shared expectations among a team of Human-Agent-Pairs (HAPs). The establishment of shared belief maps relies on inter-agent information sharing, the effectiveness of which highly depends on agents' processing loads and the instantaneous cognitive loads of their human partners. We investigate HMM-based cognitive load models to facilitate team members to "share the right information with the right party at the right time". The shared belief map concept and the cognitive/processing load models have been implemented in a cognitive agent architecture---SMMall. A series of experiments were conducted to evaluate the concept, the models, and their impacts on the evolving of shared mental models of HAP teams.

In the framework of heterogeneous multi-agent systems, this paper presents an implemented cognitive model of cooperative problem-solving, based on a psychological experiment. We are interested in simulating how human subjects elaborate plans in situations where knowledge is incomplete and how they interact to obtain missing information. The system BDIGGY, a concurrent implementation of a planning model and an interaction model, is used to simulate the human processes during cooperative problem solving.

With the advancement of teleconferencing technologies, human users are collaborating online more than ever today. To improve the efficiency and effectiveness of online human coalitions, one needs to support and facilitate collaborations among human users who may or may not know of each other well and of how to work well together as a team or in a team. Here we propose the Integrated Human Coalition Formation and Scaffolding (iHUCOFS) framework. This multiagent framework considers the roles of an agent as both an advisor and a representative to a human user, the tradeoffs between forming and scaffolding human coalitions, and how scaffolding could impact human behaviors for future coalitions. Based on the axioms and design principles of iHUCOFS, we have developed VALCAM---an iterative auction based coalition formation algorithm. To investigate the feasibility and impact of VALCAM, we have conducted an experiment in a computer-supported collaborative learning environment and obtained promising results.

Collaborative construction is a main application in the field of autonomous systems. An interesting subject in the area is the construction of realistic human-like architecture. However, the task of building a human-like architecture is non-trivial since the construction is a real time process without human supervision. In this paper, we present a collective building algorithm based on stigmergy. A swarm of virtual agents construct edifications which resemble basic features in human-like architecture. The algorithm maps sensory information to appropriate building actions.

Coalition formation is a problem of great interest in AI, allowing groups of autonomous, rational agents to form stable teams. Furthermore, the study of coalitional stability concepts and their relation to equilibria that guide the strategic interactions of agents during bargaining has lately attracted much attention. However, research to date in both AI and economics has largely ignored the potential presence of uncertainty when studying either coalitional stability or coalitional bargaining. This paper is the first to relate a (cooperative) stability concept under uncertainty, the Bayesian core (BC), with (non-cooperative) equilibrium concepts of coalitional bargaining games. We prove that if the BC of a coalitional game (and of each subgame) is non-empty, then there exists an equilibrium of the corresponding bargaining game that produces a BC element; and conversely, if there exists a coalitional bargaining equilibrium (with certain properties), then it induces a BC configuration. We thus provide a non-cooperative justification of the BC stability concept. As a corollary, we establish a sufficient condition for the existence of the BC. Finally, for small games, we provide an algorithm to decide whether the BC is non-empty.

Voting is a general method for aggregating the preferences of multiple agents. Each agent ranks all the possible alternatives, and based on this, an aggregate ranking of the alternatives (or at least a winning alternative) is produced. However, when there are many alternatives, it is impractical to simply ask agents to report their complete preferences. Rather, the agents' preferences, or at least the relevant parts thereof, need to be elicited. This is done by asking the agents a (hopefully small) number of simple queries about their preferences, such as comparison queries, which ask an agent to compare two of the alternatives. Prior work on preference elicitation in voting has focused on the case of unrestricted preferences. It has been shown that in this setting, it is sometimes necessary to ask each agent (almost) as many queries as would be required to determine an arbitrary ranking of the alternatives. By contrast, in this paper, we focus on single-peaked preferences. We show that such preferences can be elicited using only a linear number of comparison queries, if either the order with respect to which preferences are single-peaked is known, or at least one other agent's complete preferences are known. We also show that using a sublinear number of queries will not suffice. Finally, we present experimental results.

In an election held in a noisy environment, agents may unintentionally perturb the outcome by communicating faulty preferences. We investigate this setting by introducing a theoretical model of noisy preference aggregation and formally defining the (worst-case) robustness of a voting rule. We use our model to analytically bound the robustness of various prominent rules. The results show that the robustness of voting rules is diverse, with different rules positioned at either end of the spectrum. These results allow selection of voting rules that support preference aggregation in the face of noise.

We formulate the problem of computing equilibria in multi-player games represented by arbitrary undirected graphs as a constraint satisfaction problem and present two algorithms. The first is PureProp: an algorithm for computing approximate Nash equilibria in complete information one-shot games and approximate Bayes-Nash equilibria in one-shot games of incomplete information. PureProp unifies existing message-passing based algorithms for solving these classes of games. We also address repeated graphical games, and present a second algorithm, PureProp-R, for computing approximate Nash equilibria in these games.

Game theoretic reasoning about multi-agent systems has been made more tractable by algorithms that exploit various types of independence in agents' utilities. However, previous work has assumed that a game's precise independence structure is given in advance. This paper addresses the problem of finding independence structure in a general form game when it is not known ahead of time, or of finding an approximation when full independence does not exist. We give an expected polynomial time algorithm to divide a bounded-payout normal form game into factor games that isolate independent strategic interactions. We also show that the best approximate factoring can be found in polynomial time for a specific interaction that is not fully independent. Once known, factors aide computation of game theoretic solution concepts, including Nash equilibria (or ε-equilibria for approximate factors), in some cases guaranteeing polynomial complexity where previous bounds were exponential.

This paper presents a two-sided economic search model in which agents are searching for beneficial pairwise partnerships. In each search stage, each of the agents is randomly matched with several other agents in parallel, and makes a decision whether to accept a potential partnership with one of them. The distinguishing feature of the proposed model is that the agents are not restricted to maintaining a synchronized (instantaneous) decision protocol and can sequentially accept and reject partnerships within the same search stage. We analyze the dynamics which drive the agents' strategies towards a stable equilibrium in the new model and show that the proposed search strategy weakly dominates the one currently in use for the two-sided parallel economic search model. By identifying several unique characteristics of the equilibrium we manage to efficiently bound the strategy space that needs to be explored by the agents and propose an efficient means for extracting the distributed equilibrium strategies in common environments.

This paper proposes the requirements driven agent collaboration. This proposal assumes that there are plenty different service agents distributed in Internet. When a request for accomplishing a particular task occurs, these autonomous agents can recognize the newly emergent requirements and dynamically aggregate together to compete with others for fulfilling the requirements. This paper presents a preliminary framework for the requirement driven agent collaboration based on the automated mechanism design.

Integrating contributions received from other agents is an essential activity in multi-agent systems (MASs). Not only must related contributions be integrated together, but the confidence in each integrated contribution must be determined. In this paper we look specifically at the issue of confidence determination and its effect on developing "principled," highly collaborating MASs. Confidence determination is often masked by ad hoc contribution-integration techniques, viewed as being addressed by agent trust and reputation models, or simply assumed away. We present a domain-independent analysis model that can be used to measure the sensitivity of a collaborative problem-solving system to potentially incorrect confidence-integration assumptions. In analyses performed using our model, we focus on the typical assumption of independence among contributions and the effect that unaccounted-for dependencies have on the expected error in the confidence that the answers produced by the MAS are correct. We then demonstrate how the analysis model can be used to determine confidence bounds on integrated contributions and to identify where efforts to improve contribution-dependency estimates lead to the greatest improvement in solution-confidence accuracy.

In many cooperative multiagent domains, there exist some states in which the agents can act independently and others in which they need to coordinate with their teammates. In this paper, we explore how factored representations of state can be used to generate factored policies that can, with minimal communication, be executed distributedly by a multiagent team. The factored policies indicate those portions of the state where no coordination is necessary, automatically alert the agents when they reach a state in which they do need to coordinate, and determine what the agents should communicate in order to achieve this coordination. We evaluate the success of our approach experimentally by comparing the amount of communication needed by a team executing a factored policy to a team that needs to communicate in every timestep.

There is significant interest in modeling teamwork in agents. In recent years, it has become widely accepted that it is possible to separate teamwork from taskwork, providing support for domain-independent teamwork at an architectural level, using teamwork models. However, existing teamwork models (both in theory and practice) focus almost exclusively on achievement goals, and ignore maintenance goals, where the value of a proposition is to be maintained over time. Such maintenance goals exist both in taskwork (i.e., agents take actions to maintain a condition while a task is executing), as well as in teamwork (i.e., agents take actions to maintain the team). This paper presents mechanisms for collaborative maintenance in both taskwork and teamwork, allowing for flexible selection of the maintenance protocol. The mechanism is integrated and evaluated in two teamwork architectures for situated agent teams: DIESEL, an implemented teamwork and taskwork architecture, built on top of Soar, and BITE, an architecture for physical behavior-based robots. We provide details of these implementations, and the results from experiments demonstrating the benefits of support for collaborative maintenance processes, in several dynamic rich domains. We show that the use of collaborative maintenance leads to significant improvement in task performance in all domains.

We consider the problem of managing schedules in an uncertain, distributed environment. We assume a team of collaborative agents, each responsible for executing a portion of a globally pre-established schedule, but none possessing a global view of either the problem or solution. The goal is to maximize the joint quality obtained from the activities executed by all agents, given that, during execution, unexpected events will force changes to some prescribed activities and reduce the utility of executing others. We describe an agent architecture for solving this problem that couples two basic mechanisms: (1) a "flexible times" representation of the agent's schedule (using a Simple Temporal Network) and (2) an incremental rescheduling procedure. The former hedges against temporal uncertainty by allowing execution to proceed from a set of feasible solutions, and the latter acts to revise the agent's schedule when execution is forced outside of this set of solutions or when execution events reduce the expected value of this feasible solution set. Basic coordination with other agents is achieved simply by communicating schedule changes to those agents with inter-dependent activities. Then, as time permits, the core local problem solving infra-structure is used to drive an inter-agent option generation and query process, aimed at identifying opportunities for solution improvement through joint change. Using a simulator to model the environment, we compare the performance of our multiagent system with that of an expected optimal (but non-scalable) centralized MDP solver.

Decentralized MDPs provide powerful models of interactions in multiagent environments, but are often very difficult or even computationally infeasible to solve optimally. Here we develop a hierarchical approach to solving a restricted set of decentralized MDPs. By forming commitments with other agents and modeling these concisely in their local MDPs, agents effectively, efficiently, and distributively formulate co-ordinated local policies. We introduce a novel construction that captures commitments as constraints on local policies and show how Linear Programming can be used to achieve local optimality subject to these constraints. In contrast to other commitment enforcement approaches, we show ours to be more robust in capturing the intended commitment semantics while maximizing local utility. We also describe a commitment-space heuristic search algorithm that can be used to approximate optimal joint policies. A preliminary empirical evaluation suggests that our approach yields faster approximate solutions than the conventional encoding of the problem as a multiagent MDP would allow and, when wrapped in an exhaustive commitment-space search, will find the optimal global solution.

This paper proposes a new variant of the task allocation problem, where the agents are connected in a social network and tasks arrive at the agents distributed over the network. We show that the complexity of this problem remains NP-hard. Moreover, it is not approximable within some factor. We develop an algorithm based on the contract-net protocol. Our algorithm is completely distributed, and it assumes that agents have only local knowledge about tasks and resources. We conduct a set of experiments to evaluate the performance and scalability of the proposed algorithm in terms of solution quality and computation time. Three different types of networks, namely small-world, random and scale-free networks, are used to represent various social relationships among agents in realistic applications. The results demonstrate that our algorithm works well and that it scales well to large-scale applications.

There were always two separated methods to make agent coordination: individual-local balance perspective and individual-society balance perspective. The first method only considered the balance between individual agents and their local neighbors; the second method only considered the balance between individual agents and the whole multiagent society. However, in reality, the agents will be diffused by their local neighbors as well as influenced by their social contexts simultaneously; therefore, it is necessary to deal with the social performance as well as local performance. To address such problem this paper presents an agent coordination method in an integrative model where we combine the two perspectives together and make trade-off between them. With our presented model, the individual, local and social concerns can be balanced well in a unified and flexible manner. Moreover, the experimental results show that there are often situations in which the two coordination perspectives aren't conflictive but often bring out the better in each other.

In this paper, we address the problem of provisioning unreliable and heterogeneous service providers for the constituent tasks of abstract workflows. Specifically, we deal with unreliable providers by provisioning multiple service providers redundantly for specific tasks, and we employ a local search mechanism to choose among many heterogeneous providers that offer the same type of service. We empirically show that our strategy can achieve significant improvements over current approaches, and we demonstrate that it works well over a range of environments.

Research on organization of Multiagent Systems (M.A.S.) has shown that by adapting its organization, a M.A.S. is better able to operate in dynamic environments. In this paper we describe an experiment with a M.A.S. that consists of agents where the capability to reorganize is integrated in their coordination mechanism. In the RoboCupRescue simulator we have implemented a M.A.S. where work can be coordinated according to three different coordination styles; direct supervision and standardization of skills with and without a reorganization extension. An experiment shows the effects of unknown workload distribution and incomplete information on the performance of the three styles. Results show significant interaction effects between both workload distribution and coordination mechanism, and completeness of information and coordination mechanism. Furthermore, results show that standardization of skills with reorganization performs better and is more robust to heterogeneous workload distribution and incompleteness of information.

This paper presents the ARTS (Agent-oriented Robust Transactional System) model, which applies transaction concepts to provide agent developers with high-level support for agent system robustness and reliability. ARTS abstractly considers agents as executors of encapsulated task entities which comply with a set of execution constraints on both normative execution and compensation (repair) semantics. ARTS then defines the task interface in terms of predictable terminating states to support a contract-like interaction among agents. In conjunction with this encapsulation of task semantics, ARTS defines a model for specifying scoped compensation and exception-handling plans for a given task, and for systematically selecting and executing these plans --- triggered by subtask events --- so that the enclosing task semantics are enforced. These capabilities together define a model that reduces design complexity while increasing system robustness, by allowing an agent developer to compose recursively-defined, atomically-handled tasks.

The coordination of teams of cooperating but autonomous agents is a core problem in multiagent systems research. A team of agents is jointly able to achieve a goal if despite any incomplete knowledge or even false beliefs that they may have about the world or each other, they still know enough to be able to get to a goal state, should they choose to do so. Unlike in the single-agent case, the mere existence of a working plan is not sufficient since there may be several incompatible working plans and the agents may not be able to choose a share that coordinates with the others'.

Running several sub-optimal algorithms and choosing-the optimal one is a common procedure in computer science, most notably in the design of approximation algorithms. This paper deals with one significant flaw of this technique in environments where the inputs are provided by rational agents: such protocols are not necessarily incentive compatible even when the underlying algorithms are. We characterize sufficient and necessary conditions for such best-outcome protocols to be incentive compatible in a general model for agents with one-dimensional private data. We show how our techniques apply in several settings.

Multiagent environments are often not cooperative nor collaborative; in many cases, agents have conflicting interests, leading to adversarial interactions. This paper presents a formal Adversarial Environment model for bounded rational agents operating in a zero-sum environment. In such environments, attempts to use classical utility-based search methods can raise a variety of difficulties (e.g., implicitly modeling the opponent as an omniscient utility maximizer, rather than leveraging a more nuanced, explicit opponent model).

We define an Adversarial Environment by describing the mental states of an agent in such an environment. We then present behavioral axioms that are intended to serve as design principles for building such adversarial agents. We explore the application of our approach by analyzing log files of completed Connect-Four games, and present an empirical analysis of the axioms' appropriateness.

Ranking systems are a fundamental ingredient of multiagent environments and Internet Technologies. These settings can be viewed as social choice settings with two distinguished properties: the set of agents and the set of alternatives coincide, and the agents' preferences are dichotomous, and therefore classical impossibility results do not apply. In this paper we initiate the study of incentives in ranking systems, where agents act in order to maximize their position in the ranking, rather than to obtain a correct outcome. We consider several basic properties of ranking systems, and fully characterize the conditions under which incentive compatible ranking systems exist, demonstrating that in general no such system satisfying all the properties exists.

Agents that must reach agreements with other agents need to reason about how their preferences, judgments, and beliefs might be aggregated with those of others by the social choice mechanisms that govern their interactions. The recently emerging field of judgment aggregation studies aggregation from a logical perspective, and considers how multiple sets of logical formulae can be aggregated to a single consistent set. As a special case, judgment aggregation can be seen to subsume classical preference aggregation. We present a modal logic that is intended to support reasoning about judgment aggregation scenarios (and hence, as a special case, about preference aggregation): the logical language is interpreted directly in judgment aggregation rules. We present a sound and complete axiomatisation of such rules. We show that the logic can express aggregation rules such as majority voting; rule properties such as independence; and results such as the discursive paradox, Arrow's theorem and Condorcet's paradox - which are derivable as formal theorems of the logic. The logic is parameterised in such a way that it can be used as a general framework for comparing the logical properties of different types of aggregation - including classical preference aggregation.

Key idea of this work is to appropriately extend one prominent generic agent architecture, namely InteRRap [8], to the case of a quantum pattern matching (QPM) based type-I quantum search agent (QSA) that is supposed to run on a hybrid quantum computer, and to show it's feasibility by instantiating the respective QuantumInteRRap architecture. For a comprehensive and in-depth introduction to quantum computation (QC) we refer the interested reader to [10]. An extended version of this work can be found at [6].

We investigate quantified interpreted systems, a semantics for multiagent systems in which agents can reason about individuals, their properties, and the relationships among them. We analyse a first-order epistemic language interpreted on this semantics and show soundness and completeness of Q.S5n, an axiomatisation for these structures.

We propose an extension of alternating-time temporal logic, that can be used for reasoning about the behavior and abilities of agents under various rationality assumptions.

Up to now, the way institutions and organizations have been used in the development of open systems has not often gone further than a useful heuristics. In order to develop systems actually implementing institutions and organizations, formal methods should take the place of heuristic ones. The paper presents a formal semantics for the notion of institution and its components (abstract and concrete norms, empowerment of agents, roles) and defines a formal relation between institutions and organizational structures. As a result, it is shown how institutional norms can be refined to constructs---organizational structures---which are closer to an implemented system. It is also shown how such a refinement process can be fully formalized and it is therefore amenable to rigorous verification.

Norms are widely recognised as a means of coordinating multiagent systems. The distributed management of norms is a challenging issue and we observe a lack of truly distributed computational realisations of normative models. In order to regulate the behaviour of autonomous agents that take part in multiple, related activities, we propose a normative model, the Normative Structure (NS), an artifact that is based on the propagation of normative positions (obligations, prohibitions, permissions), as consequences of agents' actions. Within a NS, conflicts may arise due to the dynamic nature of the MAS and the concurrency of agents' actions. However, ensuring conflict-freedom of a NS at design time is computationally intractable. We show this by formalising the notion of conflict, providing a mapping of NSs into Coloured Petri Nets and borrowing well-known theoretical results from that field. Since online conflict resolution is required, we present a tractable algorithm to be employed distributedly. We then demonstrate that this algorithm is paramount for the distributed enactment of a NS.

Norm-governed virtual organizations define, govern and facilitate coordinated resource sharing and problem solving in societies of agents. With an explicit account of norms, openness in virtual organizations can be achieved: new components, designed by various parties, can be seamlessly accommodated. We focus on virtual organizations realised as multiagent systems, in which human and software agents interact to achieve individual and global goals. However, any realistic account of norms should address their dynamic nature: norms will change as agents interact with each other and their environment. Due to the changing nature of norms or due to norms stemming from different virtual organizations, there will be situations when an action is simultaneously permitted and prohibited, that is, a conflict arises. Likewise, there will be situations when an action is both obliged and prohibited, that is, an inconsistency arises. We introduce an approach, based on first-order unification, to detect and resolve such conflicts and inconsistencies. In our proposed solution, we annotate a norm with the set of values their variables should not have in order to avoid a conflict or an inconsistency with another norm. Our approach neatly accommodates the domain-dependent interrelations among actions and the indirect conflicts/inconsistencies these may cause. More generally, we can capture a useful notion of inter-agent (and inter-role) delegation of actions and norms associated to them, and use it to address conflicts/inconsistencies caused by action delegation. We illustrate our approach with an e-Science example in which agents support Grid services.

We propose a new distributed mechanism to enforce norms by ostracizing agents that do not abide by them. Our simulations have shown that, although complete ostracism is not always possible, the mechanism substantially reduces the number of norm violations.

The General Game Playing (GGP) problem is concerned with developing systems capable of playing many different games, even games the system has never encountered before. Successful GGP agents must be able to extract relevant features from the formal game description and construct effective search heuristics. In this article, we present a procedure by which autonomous General Game Playing agents can generate effective and efficient search heuristics from the formal game description. The major aspect of our approach is an innovative technique to automatically extract critical features from the game structure. Our method has been incorporated into a fully implemented system that came in fourth place at the second General Game Playing Competition held at AAAI-06.

Temporal difference reinforcement learning algorithms are perfectly suited to autonomous agents because they learn directly from an agent's experience based on sequential actions in the environment. However, their most common algorithmic variants are relatively inefficient in their use of experience data, which in many agent-based settings can be scarce. In particular, they make just one learning "update" for each atomic experience. Batch reinforcement learning algorithms, on the other hand, aim to achieve greater data efficiency by saving experience data and using it in aggregate to make updates to the learned policy. Their success has been demonstrated in the past on simple domains like grid worlds and low-dimensional control applications like pole balancing. In this paper, we compare and contrast batch reinforcement learning algorithms with on-line algorithms based on their empirical performance in a complex, continuous, noisy, multiagent domain, namely RoboCup soccer Keepaway. We find that the two batch methods we consider, Experience Replay and Fitted Q Iteration, both yield significant gains in sample complexity, while achieving high asymptotic performance.

Reinforcement learning promises a generic method for adapting agents to arbitrary tasks in arbitrary stochastic environments, but applying it to new real-world problems remains difficult, a few impressive success stories notwithstanding. Most interesting agent-environment systems have large state spaces, so performance depends crucially on efficient generalization from a small amount of experience. Current algorithms rely on model-free function approximation, which estimates the long-term values of states and actions directly from data and assumes that actions have similar values in similar states. This paper proposes model-based function approximation, which combines two forms of generalization by assuming that in addition to having similar values in similar states, actions also have similar effects. For one family of generalization schemes known as averagers, computation of an approximate value function from an approximate model is shown to be equivalent to the computation of the exact value function for a finite model derived from data. This derivation both integrates two independent sources of generalization and permits the extension of model-based techniques developed for finite problems. Preliminary experiments with a novel algorithm, AMBI (Approximate Models Based on Instances), demonstrate that this approach yields faster learning on some standard benchmark problems than many contemporary algorithms.

This paper proposes a framework for agent-based distributed machine learning and data mining based on (i) the exchange of meta-level descriptions of individual learning processes among agents and (ii) online reasoning about learning success and learning progress by learning agents. We present an abstract architecture that enables agents to exchange models of their local learning processes and introduces a number of different methods for integrating these processes. This allows us to apply existing agent interaction mechanisms to distributed machine learning tasks, thus leveraging the powerful coordination methods available in agent-based computing, and enables agents to engage in meta-reasoning about their own learning decisions. We apply this architecture to a real-world distributed clustering application to illustrate how the conceptual framework can be used in practical systems in which different learners may be using different datasets, hypotheses and learning algorithms. We report on experimental results obtained using this system, review related work on the subject, and discuss potential future extensions to the framework.

One of the major problems that artificial intelligence needs to tackle is the combination of different and potentially conflicting sources of information. Examples are multi-sensor fusion, database integration and expert systems development. In this paper we are interested in the aggregation of propositional logic-based information, a problem recently addressed in the literature on information fusion. It has applications in multiagent systems that aim at aggregating the distributed agent-based knowledge into an (ideally) unique set of propositions. We consider a group of autonomous agents who individually hold a logically consistent set of propositions. Each set of propositions represents an agent's beliefs on issues on which the group has to make a collective decision. To make the collective decision, several aggregation procedures have been proposed in the literature. Assuming that all propositions in question are factually right or wrong, we ask how good belief fusion is as a truth tracker. Will it single out the true set of propositions? And how does information fusion compare with other aggregation procedures? We address these questions in a probabilistic framework and show that information fusion does especially well for agents with a middling competence of hitting the truth of an individual proposition.

An agent attempting to model market conditions may benefit from considering how various combinations of competitor strategies would impact these conditions. We give an illustration using a prediction task faced by our agent for the Supply Chain Management scenario of the Trading Agent Competition (TAC SCM). We present the learning approach taken, evaluate its effectiveness, and then explore methods of improving predictions through combining multiple sources of data reflecting various combinations of competitor behaviors.

In this paper we propose and investigate the use of Reinforcement Learning in a market-based resource allocation mechanism called Iterative Price Adjustment. Under standard assumptions, this mechanism uses demand functions that do not allow the agents to have preferences over the attributes of the allocation, e.g. the price of the resources. To address this limitation, we study the case where the agent's preferences in the resource allocation are described by utility functions and they learn the demand functions given their utility functions. The approach has been evaluated with extensive experiments.

Transfer learning problems are typically framed as leveraging knowledge learned on a source task to improve learning on a related, but different, target task. Current transfer methods are able to successfully transfer knowledge between agents in different reinforcement learning tasks, reducing the time needed to learn the target. However, the complimentary task of representation transfer, i.e. transferring knowledge between agents with different internal representations, has not been well explored. The goal in both types of transfer problems is the same: reduce the time needed to learn the target with transfer, relative to learning the target without transfer. This work introduces one such representation transfer algorithm which is implemented in a complex multiagent domain. Experiments demonstrate that transferring the learned knowledge between different representations is both possible and beneficial.

This paper presents a formulation and an algorithm for the winner determination problem in auction-based decentralized scheduling. Without imposing a time line discretization, the proposed approach allows bidders to bid for the processing of a set of tasks under release time and due date constraints using an expressive bidding language designed for decentralized scheduling. The proposed winner determination algorithm uses a depth first branch and bound search. The search branches on bids and a constraint directed scheduling procedure is used at each node to verify the feasibility of the allocation. Experiments against a commercial optimization package, CPLEX 10.0, show that the proposed algorithm is more than an order of magnitude faster on average over a set of winner determination problems of decentralized scheduling generated based on a suite of job shop constraint satisfaction benchmark problems previously developed in the literature.

Distributed allocation and multiagent coordination problems can be solved through combinatorial auctions. However, most of the existing winner determination algorithms for combinatorial auctions are centralized. The PAUSE auction is one of a few efforts to release the auctioneer from having to do all the work (it might even be possible to get rid of the auctioneer). It is an increasing price combinatorial auction that naturally distributes the problem of winner determination amongst the bidders in such a way that they have an incentive to perform the calculation. It can be used when we wish to distribute the computational load among the bidders or when the bidders do not wish to reveal their true valuations unless necessary. PAUSE establishes the rules the bidders must obey. However, it does not tell us how the bidders should calculate their bids. We have developed a couple of bidding algorithms for the bidders in a PAUSE auction. Our algorithms always return the set of bids that maximizes the bidder's utility. Since the problem is NP-Hard, run time remains exponential on the number of items, but it is remarkably better than an exhaustive search. In this paper we present our bidding algorithms, discuss their virtues and drawbacks, and compare the solutions obtained by them to the revenue-maximizing solution found by a centralized winner determination algorithm.

In this paper, we examine the behavior of bidding agents that are in direct competition with the other participants in an auction setting. Thus the agents are not simply trying to maximize their own utility, rather they wish to maximize a weighted difference of their own gain to that of their competitors. By so doing, this work significantly extends the existing state-of-the-art results on single unit auctions, by generalizing to the multi-unit case. Specifically, our main result is the derivation of symmetric Bayes-Nash equilibria for these agents in both m^th and (m + 1)^th price sealed bid auctions. Subsequently, we use these equilibria to examine the profits of different agents and show that aiming to beat the competition is more effective than pure self interest in any competitive setting. Finally, we examine how the auctioneer's revenue is affected and find that the weight that agents place in minimizing the opponents' profit determines whether the m^th or the (m + 1)^th price auction yields a higher revenue.

Mixed Multi-Unit Combinatorial Auctions (MMUCAs) allow agents to bid for bundles of goods to buy, goods to sell, and transformations of goods. In particular, MMUCAs offer a high potential to be employed for the automated assembly of supply chains of agents offering goods and services, and in general MMUCAs extend and generalise several types of combinatorial auctions. Here we provide a formalism, based on an extension of Petri Nets, with which MMUCAs, and therefore all auction types subsumed by MMUCAs --- and in particular combinatorial auctions for supply chain formation (SCF)---, can be formally analysed. As a second direct benefit, consequence of the provided mapping to Petri Nets, we manage to dramatically reduce the number of decision variables involved in the optimisation problem posed by MMUCAs from quadratic to linear for a wide class of MMUCA Winner Determination Problems (WDPs). Hence, we also make headway in the practical application of MMUCAs, and in particular to SCF.

One of the most crucial criterion in automated negotiation is how to reach a consensus agreement for all negotiators under any negotiation environment. Currently, most negotiation strategies can work under the static environment only. This paper presents a model for designing negotiation agents that makes adjustable rates of concession by reacting to changing market situations with uncertain and dynamic outside options. This work is based on the model of market-driven agents (MDAs). To determine the amount of the concession for each trading cycle, these market-driven agents are guided by four mathematical functions of trading opportunity, trading competition, trading time and strategy and trading eagerness. The contribution of this paper is designing and developing an extended MDA model with the flexibility to respond to uncertain and dynamic outside options, so as to increase problem solving ability for agent negotiation in broad application domains.

We introduce a distributed and tractable mechanism for the allocation of continuously divisible resources to agents, that locally maximises the (Nash) product of their individual welfare. The mechanism involves specific m-resources-at-a-time multilateral deals over bits of resources, termed infinitesimal Nash transfers. It provides an effective way of building "strong social alliances", where in a social alliance agents fully cooperate for the global interest of society, and a strong social alliance has near-optimal utilitarian and egalitarian social welfare, as understood in social choice and welfare economics. The mechanism is scalable, can be distributed amongst agents and can be used to support, e.g., fair trade.

In this paper, we propose a novel incentive mechanism for eliciting fair ratings of selling agents from buying agents. In our mechanism, buyers model other buyers and select the most trustworthy ones as their neighbors from whom they can ask advice about sellers. In addition, however, sellers model the reputation of buyers. Reputable buyers always provide fair ratings of sellers, and are likely to be neighbors of many other buyers. In marketplaces operating with our mechanism, sellers will increase quality and decrease prices of products to satisfy reputable buyers. In consequence, our mechanism creates incentives for buyers to provide fair ratings of sellers.

During the past few decades, the field of telecommunications has been the subject to a continuous evolution. Kridel [1] associates this evolution to three interrelated phenomena: shifts in regulations, increased competition and technological progress. In addition to Kridels' phenomena, we see higher end user expectations with regards to service quality and price as another dimension to this evolution. Here, we propose a market place approach, called Agent Grid Service Marketplace (AGSM), that addresses those evolutionary factors [2].

Interdependent values (IDV) is a valuation model allowing bidders in an auction to express their value for the item(s) to sell as a function of the other bidders' information. We investigate the incentive compatibility (IC) of single-item auctions for IDV bidders in dynamic environments. We provide a necessary and sufficient characterization for IC in this setting. We show that if bidders can misreport departure times and private signals, no reasonable auction can be IC. We present a reasonable IC auction for the case where bidders cannot misreport departures.

Distributed Constraint Optimization (DCOP) is a general framework that can model complex problems in multiagent systems. Several current algorithms that solve general DCOP instances, including ADOPT and DPOP, arrange agents into a traditional pseudotree structure. We introduce an extension to the DPOP algorithm that handles an extended set of pseudotree arrangements. Our algorithm correctly solves DCOP instances for pseudotrees that include edges between nodes in separate branches. The algorithm also solves instances with traditional pseudotree arrangements using the same procedure as DPOP.

We compare our algorithm with DPOP using several metrics including the induced width of the pseudotrees, the maximum dimensionality of messages and computation, and the maximum sequential path cost through the algorithm. We prove that for some problem instances it is not possible to generate a traditional pseudotree using edge-traversal heuristics that will outperform a cross-edged pseudotree. We use multiple heuristics to generate pseudotrees and choose the best pseudotree in linear space-time complexity. For some problem instances we observe significant improvements in message and computation sizes compared to DPOP.

We consider multiagent systems with stochastic non-linear dynamics in continuous space-time. We focus on systems of agents that aim to visit a number of given target locations at given points in time at minimal control cost. The online optimization of which agent has to visit which target requires the solution of the Hamilton-Jacobi-Bellman (HJB) equation, which is a non-linear partial differential equation (PDE). Under some conditions, the log-transform can be applied to turn the HJB equation into a linear PDE. We then show that the optimal solution in the multiagent scheduling problem can be expressed in closed form as a sum of single schedule solutions.

We present a multiagent coordination technique to maintain throughput of a large-scale agent network system in the face of failures of agents. Failures do not just deteriorate throughput of the system but also create and change bottlenecks in the system. Since loss of bottleneck's capacity degrades the overall system performance, the system should identify bottlenecks dynamically and keep their utilization at a high level. In our system, CABS, information about an agent's urgency of jobs to fulfill demanded throughput and maintain its utilization is passed to upstream agents in the network. Upstream agents utilize this information to identify bottleneck agents and coordinate their actions to provide the bottlenecks with necessary and sufficient jobs for preventing their starvation and congestion. We empirically evaluate CABS using a benchmark problem of the semiconductor fabrication process, which is a good example of a large-scale network system, in comparison with a well-known traditional manufacturing control method.

Exchanging scarce resources during execution among a group of agents is one way to improve the overall performance in multiagent systems with limited shared resources, but implementing optimal sequential resource allocation is often a nontrivial problem in complex systems with uncertainties. In this paper, we present an MILP-based algorithm that can automatically break a large mission into multiple phases and make optimal resource (re)allocations at the entry of each phase. We illustrate our algorithms through several increasingly complex classes of sequential resource allocation problems, and show through experiments that our techniques can increase agents' rewards for varying levels of constraints on resources and constraints on exchanging resources.

In this paper, we focus on the problem of finding a periodic allocation strategy for teams of resource-bounded agents. We propose a real-time dynamic algorithm RTDA*, that exploits two key properties to avoid the exponential increase in the state and action spaces associated with multiagent systems. First, resource allocation at each time period follows an earliest deadline first order (EDF) over agents. Second, the resources are undivided, i.e., the resources allocated to an agent restrict their availability to others over time. We can therefore view each incoming agent as a cyclic individual resource-bounded processing, namely "cyclic progressive reasoning unit" (C-PRU), and solve, off-line, the single agent resource allocation problem. In the on-line phase, our algorithm exploits pre-compiled policies, as heuristic metrics, to build near-optimal joint decisions at each time period.

We are interested in domains where an agent repeatedly negotiates with other agents over shared resources where the demand or utility to the agent for the shared resources vary over time. We propose a protocol that will maximize social welfare if agents reveal their true preferences in every negotiation. The protocol, however, is not truth-revealing and selfish agents have the incentive to artificially inflate preferences. We use a probabilistic reciprocative behavior that discourages the reporting of false preferences. This reciprocative behavior promotes cooperation in repeated negotiations and improves both individual and group long-term payoff. We characterize environmental conditions under which agents can develop and sustain mutually beneficial relationships with similar agents and avoid exploitation by different types of selfish agents.

This paper presents a novel, unified distributed constraint satisfaction framework based on automated negotiation. The Distributed Constraint Satisfaction Problem (DCSP) is one that entails several agents to search for an agreement, which is a consistent combination of actions that satisfies their mutual constraints in a shared environment. By anchoring the DCSP search on automated negotiation, we show that several well-known DCSP algorithms are actually mechanisms that can reach agreements through a common Belief-Desire-Intention (BDI) protocol, but using different strategies. A major motivation for this BDI framework is that it not only provides a conceptually clearer understanding of existing DCSP algorithms from an agent model perspective, but also opens up the opportunities to extend and develop new strategies for DCSP. To this end, a new strategy called Unsolicited Mutual Advice (UMA) is proposed. Performance evaluation shows that the UMA strategy can outperform some existing mechanisms in terms of computational cycles.

Teamwork requires that team members coordinate their actions. The representation of the coordination is a key requirement since it influences the complexity and flexibility of reasoning team-members. One aspect of this requirement is detecting coordination faults as a result of intermittent failures of sensors, communication failures, etc. Detection of such failures, based on observations of the behavior of agents, is of prime importance. Though different solutions have been presented thus far, none has presented a comprehensive and formal resolution to this problem. This paper presents a formal approach to representing multiagent coordination, and multiagent observations, using matrix structures. This representation facilitates easy representation of coordination requirements, modularity, flexibility and reuse of existing systems. Based on this representation we present a novel solution for fault detection that is both generic and efficient for large-scale teams.

Most of the current research on distributed diagnosis in and for multiagent systems focuses on diagnosis of coordination failures. Proposed approaches for this problem are not efficient for diagnosing failures in physical devices. This paper proposes algorithms for distributed troubleshooting of physical devices and processes. The consequences of using distributed representation of the knowledge, ATMS, and distributed reasoning are discussed and algorithms are proposed to deal with the occurrence of conflicts and the computation of the set of candidate diagnosis.

Path planning for mobile robots in stochastic, dynamic environments is a difficult problem and the subject of much research in the field of robotics. While many approaches to solving this problem put the computational burden of path planning on the robot, physical path planning methods place this burden on a set of sensor nodes distributed throughout the environment that can communicate information to each other about path costs. Previous approaches to physical path planning have looked at the performance of such networks in regular environments (e.g., office buildings) using highly structured, uniform deployments of networks (e.g., grids). Additionally, these networks do not make use of real experience obtained from the robots they assist in guiding. We extend previous work in this area by incorporating reinforcement learning techniques into these methods and show improved performance in simulated, rough terrain environments. We also show that these networks, which we term SWIRLs (Swarms of Interacting Reinforcement Learners), can perform well with deployment distributions that are not as highly structured as in previous approaches.

In this paper we introduce Dynamics Based Control (DBC), an approach to planning and control of an agent in stochastic environments. Unlike existing approaches, which seek to optimize expected rewards (e.g., in Partially Observable Markov Decision Problems (POMDPs)), DBC optimizes system behavior towards specified system dynamics. We show that a recently developed planning and control approach, Extended Markov Tracking (EMT) is an instantiation of DBC. EMT employs greedy action selection to provide an efficient control algorithm in Markovian environments. We exploit this efficiency in a set of experiments that applied multi-target EMT to a class of area-sweeping problems (searching for moving targets). We show that such problems can be naturally defined and efficiently solved using the DBC framework, and its EMT instantiation.

The paper presents a new approach for multiagent replanning based on Distributed Constraint Satisfaction (DisCSP) and Graph planning techniques. In this approach, a new distributed refinement strategy is proposed to construct a graph plan for fixing errors occurred during the plan execution. The strategy employs an "max-branching" heuristic that can reduce the final graph plan size and allow faster completion time for the graph construction. The graph plan is then compiled into a DisCSP problem and solved using a multi-variable version of the Asynchronous Backtracking Algorithm. The approach is demonstrated with experiments which show that distributed planning graph and CSP can practically solve the replanning problems in a multiagent environment.

Multi-Objective Multiagent Planning (MOMAP) addresses the problem of resolving conflicts between individual agent interests and the group interests. In this paper, we address this problem by presenting a formal framework to represent objective relationships, a decision model using a Vector-Valued Decentralized Markov Decision Process (2V-DEC-MDP) and an algorithm to solve the resulting 2V-DEC-MDP. The formal framework of a Vector-Valued MDP considered uses the value function which returns a vector representing the individual and the group interests. An optimal policy in such contexts is not clear but in this approach we develop a regret-based technique to find a good tradeoff between the group and individual interests. To do that, the approach we present uses Egalitarian Social Welfare orderings that allow an agent to consider during its local optimization the satisfaction of all criteria and reducing their differences. The obtained result is a good balance between individual and group satisfactions where the local policies can lead to more global satisfying behaviors in some settings. This result is illustrated in many examples and compared to alternate local policies.

We present a powerful new account of multiagent knowledge in the situation calculus and an effective reasoning procedure for handling knowledge queries. Our approach generalizes existing work by reifying the observations made by each agent as the world evolves, allowing for agents that are partially or completely unaware of some of the actions that have occurred. This also enables agents to reason effectively about knowledge using only their internal history of observations, rather than requiring a full history of the world. The result is a more robust and flexible account of knowledge suitable for use in partially-observable multiagent domains.

In hierarchical planning, selecting a plan at an abstract level affects planning performance because an abstract plan restricts the scope of primitive plans. However, if all primitive plans under the selected abstract plan have difficult-to-resolve conflicts with the plans of other agents, the final plan after conflict resolution will be inefficient or of low quality. In this paper, we propose a conflict estimation method to generate quality plans efficiently for multiagent systems by appropriately selecting abstract plans in hierarchical planning. This method enables agents to learn which abstract plans are less likely to cause conflicts or which conflicts will be easy to resolve.

Failures in plan execution can be attributed to errors in the execution of plan steps or violations of the plan structure. While in previous work we have concentrated on the first type of failures, in this paper we introduce the idea of diagnosing violations in the plan structure. The structure of a plan prescribes which actions have to be performed and which precedence constraints between them have to be respected. Especially in multiagent environments violations of plan structure might easily occur as the consequence of synchronization errors. Using a formal framework for plan diagnosis, we show how Model-Based Diagnosis can applied to identify these violations of plan structure specifications and we analyze the computational complexity of the associated diagnostic problems.

This paper presents two models of goal generation which enable a motivated autonomous agent to generate goals in response to changes in its underlying drives or motivations, while it is both planning and executing. A Mars rover domain is used to illustrate the two models: the first model involves goals being generated explicitly in response to changes to the agent's motivations where such goals are then provided to a planner, while the second model involves encoding the motivations and the goals that they may generate as part of the planner's domain model. Results from experiments on integrating the models with different planners suggest that while they may bring the benefits of autonomy that we seek, they also introduce more complexity into the planning problem.

Although agent interaction plays a vital role in MAS, and message-centric approaches to agent interaction have their drawbacks, present agent-oriented programming languages do not provide support for implementing agent interaction that is flexible and robust. Instead, messages are provided as a primitive building block. In this paper we consider one approach for modelling agent interactions: the commitment machines framework. This framework supports modelling interactions at a higher level (using social commitments), resulting in more flexible interactions. We investigate how commitment-based interactions can be implemented in conventional agent-oriented programming languages. The contributions of this paper are: a mapping from a commitment machine to a collection of BDI-style plans; extensions to the semantics of BDI programming languages; and an examination of two issues that arise when distributing commitment machines (turn management and race conditions) and solutions to these problems.

We develop a model of normative systems in which agents are assumed to have multiple goals of increasing priority, and investigate the computational complexity and game theoretic properties of this model. In the underlying model of normative systems, we use Kripke structures to represent the possible transitions of a multiagent system. A normative system is then simply a subset of the Kripke structure, which contains the arcs that are forbidden by the normative system. We specify an agent's goals as a hierarchy of formulae of Computation Tree Logic (CTL), a widely used logic for representing the properties of Kripke structures: the intuition is that goals further up the hierarchy are preferred by the agent over those that appear further down the hierarchy. Using this scheme, we define a model of ordinal utility, which in turn allows us to interpret our Kripke-based normative systems as games, in which agents must determine whether to comply with the normative system or not. We then characterise the computational complexity of a number of decision problems associated with these Kripke-based normative system games; for example, we show that the complexity of checking whether there exists a normative system which has the property of being a Nash implementation is NP-complete.

The social stance advocated by institutional frameworks and most multiagent system methodologies has resulted in a wide spectrum of organizational and communicative abstractions which have found currency in several programming frameworks and software platforms. Still, these tools and frameworks are designed to support a limited range of interaction capabilities that constrain developers to a fixed set of particular, pre-defined abstractions. The main hypothesis motivating this paper is that the variety of multiagent interaction mechanisms -- both, organizational and communicative, share a common semantic core. In the realm of software architectures, the paper proposes a connector-based model of multiagent interactions which attempts to identify the essential structure underlying multiagent interactions. Furthermore, the paper also provides this model with a formal execution semantics which describes the dynamics of social interactions. The proposed model is intended as the abstract machine of an organizational programming language which allows programmers to accommodate an open set of interaction mechanisms.

We propose a new class of representations that can be used for modeling (and model checking) temporal, strategic and epistemic properties of agents and their teams. Our representations borrow the main ideas from interpreted systems of Halpern, Fagin et al.; however, they are also modular and compact in the way concurrent programs are. We also mention preliminary results on model checking alternating-time temporal logic for this natural class of models.

There are many studies on ABS and several frameworks for ABS have already been published. However, there are few frameworks that can enable agent-based simulation using large numbers of agents. We have been developing a Java-based platform for Massive Agent-Based Simulation (MABS) called "Zillions of Agents-based Simulation Environment" or ZASE. The purpose of ZASE is to develop MABS applications on multiple computers. ZASE is designed to host over millions of agents. We introduce ZASE in this paper. We evaluated ZASE for the agent-based auction simulation where the number of agents varied from ten to a million. The results indicate that the number of agents affects the final bid prices and their distributions. Performance measurement results on both an SMP computer environment and a multiple-computer environment are shown.

The IRM4S model (Influence Reaction Model for Simulation) is an adaptation of the formalism of [2] for multiagent based simulations (MABS). The goal of IRM4S is to provide a framework that eases the use of the Influence/Reaction principle within MABS.

This paper discusses some advantages of applying the SOA paradigm for the MAS development, showing a framework whose architecture follows the WSA reference model. We believe that service-oriented paradigm can simplify the MAS development because it demands a much simpler coordination level than the traditional approaches focused on the message-oriented model.

Simulation is an important tool for studying systems' behavior under specific conditions. In recent years, agent technology has been recognized as a promising new approach for developing simulation systems, in particular, Multiagent Systems (MAS) based simulations.

Multiagent systems (MAS) research is well established yet there is little development of agent technology in industry. It has been suggested that this is due, in part, to the unavailability of support tools for Agent Oriented Software Engineering (AOSE). This paper suggests requirements for Integrated Development Environments (IDEs) to support MAS construction. We suggest that an IDE can be built as its own MAS which allows it to be decoupled from any particular agent framework thereby allowing it to be platform independent.

Ant algorithms are one of the main programming paradigms in swarm intelligence. They are built on stochastic decision functions, which can also be found in other types of bio-inspired algorithms with the same mathematical form. However, though this modeling leads to high-performance algorithms, some phenomena, like symmetry break, are still not well understood or modeled at the ant level. This paper proposes an original analysis of the problem : we establish a reactive multiagent system based on logistic nonlinear decision maps, and designed according to the influence-reaction scheme. Our proposition is an entirely novel approach to the mathematical foundations of ant algorithms : contrary to the current stochastic approaches, we show that an alternative deterministic model exists, which has its origin in deterministic chaos theory. The rewriting of the decision functions leads to a new way of understanding and visualizing the convergence behavior of ant algorithms. We apply our approach on a concrete example, namely the binary bridge problem.

This paper proposes programming constructs to improve the practical application of existing BDI-based agent-oriented programming languages that have formal semantics. The proposed programming constructs include operations such as testing, adopting and dropping declarative goals, different execution modes for plans, repairing plans when their execution fail, event and exception handling mechanisms, and interfaces to existing imperative and declarative programming languages.

The fact that multiagent applications are prone to the same faults that any distributed system is susceptible to and the need for a higher quality of service in these systems justify the increasing interest in fault-tolerant multiagent systems. In this article, we propose an original method for providing dependability in multiagent systems through replication. Our method is different from other works because our research focuses on building an automatic, adaptive and predictive replication policy where critical agents are replicated to minimize the impact of failures. This policy is determined by taking into account the criticality of the plans of the agents, which contain the collective and individual behaviors of the agents in the application. The set of replication strategies applied at a given moment to an agent is then fine-tuned gradually by the replication system so as to reflect the dynamicity of the multiagent system. Some preliminary measurements were made to assess the efficiency of our approach and future directions are presented.

In Service-Oriented MAS middle-agents provide different kinds of matchmaking functionalities. If no adequate services are available for a specific request, a planning functionality can be used to build up composite services. In order to take advantage of recent advances in the field of AI planning for this purpose, we propose exploiting organisational information of Service-Oriented MAS to heuristically filter out those services that are probably irrelevant to the planning process. We present a novel framework for service-class based filtering and show how it can be instantiated to a particular MAS domain based on organisational information.

The vision of ubiquitous computing is one in which resource constrained mobile devices form ad-hoc networks to enable the delivery of services that are sensitive and responsive to people. Such networks are dynamic and must be capable of dealing with uncertain information. As emergent system behaviour begins to evolve the complexity of pervasive systems increases. The agent development community have been addressing issues of complexity and uncertainty, within distributed computing, for several years. This paper details the migration process of Agent Factory Micro Edition (AFME), a minimised footprint agent platform for resource constrained mobile devices. The process enables agents to migrate from a resource rich environment to a resource constrained mobile device and vice versa. It requires agents to dynamically alter their form and behaviour so as to adapt to their context.

Many agent-based models predict the future. Nonlinear interactions in most non-trivial domains make predictions useless beyond a certain point (the "prediction horizon"), as agent trajectories diverge. We exhibit this behavior in a simple agent-based model, and discuss how a single agent in such a model can estimate the prediction horizon locally and use this estimate to modulate dynamically how far it gazes into the future.

Building multiagent systems that can scale up to very large number of agents is a challenging research problem. In this paper, we present Distributed Multi Agent System Framework (DMASF), a system which can simulate billions of agents in thousands of seconds. DMASF utilizes distributed computation to gain performance as well as a database to manage the agent and environment state. We briefly present the design and implementation of DMASF and present experimental results. DMASF is a generic and versatile tool that can be used for building massive multi agent system applications.

Over the past few years, peer-to-peer(p2p) unstructured networks have emerged as an attractive paradigm for enabling online interactions between a large number of users in a decentralized manner. However, the decentralized nature of unstructured p2p networks makes load balancing a challenging problem. Specifically, the self-interested nature of users on the nodes of a p2p network and dynamic changes in network topology give rise to an unbalanced distribution of nodes across an unstructured p2p network. This results in network congestion and significant search latencies for all nodes. In this paper, we describe a small-world network model and a Bayesian inference mechanism within a multiagent setting to address these issues. Simulation results for a file sharing p2p application show that our algorithm achieves an exponential reduction in number of messages exchanged and improves load-balancing across the network.

We present an axiomatisation for an extension of a temporal epistemic logic with an epistemic "reset" operator defined on the intersection between epistemic and temporal relations. Additionally we show the logic has the finite model property, hence it is decidable.

Logics of knowledge and belief are often too static and inflexible to be used on real-world problems. In particular, they usually offer no concept for expressing that some course of events is more likely to happen than another. We address this problem and extend CTLK (computation tree logic with knowledge) with a notion of plausibility, which allows for practical and counterfactual reasoning. The new logic CTLKP (CTLK with plausibility) includes also a particular notion of belief. A plausibility update operator is added to this logic in order to change plausibility assumptions dynamically. Furthermore, we examine some important properties of these concepts. In particular, we show that, for a natural class of models, belief is a KD45 modality. We also show that model checking CTLKP is PTIME-complete and can be done in time linear with respect to the size of models and formulae.

This paper studies the relation between agent specification and agent programming languages. In particular, it shows that an agent programming language obeys some desirable properties expressed in an agent specification language, i.e., that any agent implemented by the programming language satisfies the desirable property expressed in the specification language. We study this relation by defining and aligning the semantics of an agent specification language and implementation language, and prove that certain properties expressed in the specification language are satisfied by the implementation language.

Cognitive agents often deliberate about preferences among rules. Consider, for example, an agent with the obligation to travel to Paris next week leading to a desire to travel by train, with the preferences that if the desire to travel by train cannot be met, then there is a desire to travel by plane. Such an agent may reason about preferences among rules as follows:

• The rule leading to a desire to travel by train is preferred to the rule leading to a desire to travel by plane, maybe as a second alternative.

We define a tool-supported design framework that allows to specify an agent goal model and to automatically generate fragments of a BDI agent from it. We devise the design process as a transformation process from platform-independent design models to platform-specific models and then to code. The design framework is demonstrated by referring to the Tropos methodology and to the JADE/Jadex platform. In this short paper, key steps in the process are illustrated through an example.

In human society, almost any cooperative working context accounts for different kinds of object, tool, artifacts in general, that humans adopt, share and intelligently exploit so as to support their working activities, in particular social ones. According to theories in human sciences--Activity Theory and Distributed Cognition are two main examples [5, 4]--and to related disciplines in computer science--such as Computer Supported Cooperative Work (CSCW) and Human-Computer Interaction (HCI)--such entities have a key role in determining the success or failure of the activities, playing an essential function in simplifying complex tasks and--more generally--in designing solutions that scale with activity complexity. Such a perspective can be found also in some works in the context of Distributed Artificial Intelligence [1, 2].

Analogously to the human case, we claim that also (cognitive) multiagent systems (MAS) could greatly benefit from the definition and systematic exploitation of a suitable notion of working environment, composed by different sorts of artifacts, dynamically constructed, shared and used by agents to support their working activities.

Along this line, in this paper first we introduce a conceptual framework called A&A (Agents and Artifacts) which aims at directly modelling and engineering working environments in the context of cognitive multiagent systems; then, we provide a brief overview of the basic technologies that support such an approach, CARTAGO in particular--a Java-based framework for engineering working environments to be integrated with heterogeneous agent platforms.

Such a perspective is strenghtened by recent efforts in AOSE (Agent-Oriented Software Engineering) that remark the fundamental role of the environment for the engineering of MAS [8]. The A&A framework can be considered an instance of such approaches, with some specific peculiarity: (i) abstractions and generality---the aim is to find a basic set of conceptual abstractions and related theory which, analogously to the agent abstraction, could be general enough to be the basis to define concrete architectures and programming environments, but specific enough to capture the essential properties of systems; (ii) cognitive---analogous to designed environment in human society, the properties of such environment abstractions should be conceived to be suitably and effectively exploited by cognitive agents, as intelligent constructors / users / manipulators of the environment.

The work presented in this paper generalises and extends our previous work focussed on coordination artifacts presented at [6], and more recent works about the notion of artifact [7].

In this work, we introduce a framework where defeasible argumentation is used for reasoning about beliefs, desires and intentions. A dialectical filtering process is introduced in order to obtain a subset of the agent's desires containing only those that are actually achievable in the current situation. In our framework, different agents types can be defined and this will affect the way in which current desires are obtained. Finally, intentions will be current desires that the agent may commit to pursue.

We present a simple ant model that solves a discrete foraging problem. We provide simulations and a convergence analysis. We argue that the ant population computes the solutions of some optimal control problems and converges in some well defined sense. We also discuss the rate of convergence with respect to the number of ants: we give experimental and theoretical arguments that suggest that this rate is superlinear with respect to the number of agents.

In the recent literature on multiagent systems there have been several proposals of formal systems for reasoning about delegation. Most of these approaches have dealt with the concept of delegation leaving mental states such as beliefs, goals and intentions out of consideration. The aim of this paper is to develop a formal approach for reasoning about delegation by modeling intentions and beliefs of the delegating agent in an explicit way. We present a logic where it is possible to investigate the relations between the concept of Intention to be and the concept of Delegation.

Merging operators try to define the beliefs of a group of agents according to the beliefs of each member of the group. Several model-based propositional belief merging operators have been proposed which use distance measures and aggregation functions. This paper introduces the notion of Partial Satisfiability which is an alternative way of measuring the satisfaction of a formula since this notion lets us have satisfaction values in the interval [0,1]. Partial Satisfiability allows us to define model-based merging operator. The proposal produces similar results to other merging approaches, but while other approaches require many merging operators in order to achieve satisfactory results for different scenarios this proposal obtains similar results for all these different scenarios with a unique operator. Moreover, unlike most of model-based approaches, this approach considers the case where the belief bases are inconsistent. The framework presented is in a preliminary state and further analysis of its properties is needed in order to characterize the proposed merging operator in terms of postulates.

In this paper, we present a new protocol to address multilateral multi-issue negotiation in a cooperative context. We consider complex dependencies between multiple issues by modelling the preferences of the agents with a multi-criteria decision aid tool, also enabling us to extract relevant information on a proposal assessment. This information is used in the protocol to help in accelerating the search for a consensus between the cooperative agents. In addition, the negotiation procedure is defined in a crisis management context where the common objective of our agents is also considered in the preferences of a mediator agent.

This paper analyzes bilateral multi-issue negotiation between self-interested autonomous agents. The agents have time constraints in the form of both deadlines and discount factors. There are m > 1 issues for negotiation where each issue is viewed as a pie of size one. The issues are "indivisible" (i.e., individual issues cannot be split between the parties; each issue must be allocated in its entirety to either agent). Here different agents value different issues differently. Thus, the problem is for the agents to decide how to allocate the issues between themselves so as to maximize their individual utilities. For such negotiations, we first obtain the equilibrium strategies for the case where the issues for negotiation are known a priori to the parties. Then, we analyse their time complexity and show that finding the equilibrium offers is an NP-hard problem, even in a complete information setting. In order to overcome this computational complexity, we then present negotiation strategies that are approximately optimal but computationally efficient, and show that they form an equilibrium. We also analyze the relative error (i.e., the difference between the true optimum and the approximate). The time complexity of the approximate equilibrium strategies is O(nm/ε²) where n is the negotiation deadline and ε the relative error. Finally, we extend the analysis to online negotiation where different issues become available at different time points and the agents are uncertain about their valuations for these issues. Specifically, we show that an approximate equilibrium exists for online negotiation and show that the expected difference between the optimum and the approximate is O(√m). These approximate strategies also have polynomial time complexity.

The Shapley value is one of the key solution concepts for coalition games. Its main advantage is that it provides a unique and fair solution, but its main problem is that, for many coalition games, the Shapley value cannot be determined in polynomial time. In particular, the problem of finding this value for the voting game is known to be #P-complete in the general case. However, in this paper, we show that there are some specific voting games for which the problem is computationally tractable. For other general voting games, we overcome the problem of computational complexity by presenting a new randomized method for determining the approximate Shapley value. The time complexity of this method is linear in the number of players. We also show, through empirical studies, that the percentage error for the proposed method is always less than 20% and, in most cases, less than 5%.

This paper proposes a unified and general framework for argumentation-based negotiation, in which the role of argumentation is formally analyzed. The framework makes it possible to study the outcomes of an argumentation-based negotiation. It shows what an agreement is, how it is related to the theories of the agents, when it is possible, and how this can be attained by the negotiating agents in this case. It defines also the notion of concession, and shows in which situation an agent will make one, as well as how it influences the evolution of the dialogue.

In this paper we will present an argumentation framework for learning agents (AMAL) designed for two purposes: (1) for joint deliberation, and (2) for learning from communication. The AMAL framework is completely based on learning from examples: the argument preference relation, the argument generation policy, and the counterargument generation policy are case-based techniques. For join deliberation, learning agents share their experience by forming a committee to decide upon some joint decision. We experimentally show that the argumentation among committees of agents improves both the individual and joint performance. For learning from communication, an agent engages into arguing with other agents in order to contrast its individual hypotheses and receive counterexamples; the argumentation process improves their learning scope and individual performance.

Argumentation is a promising approach used by autonomous agents for reasoning about inconsistent knowledge, based on the construction and the comparison of arguments. In this paper, we apply this approach to the classification problem, whose purpose is to construct from a set of training examples a model (or hypothesis) that assigns a class to any new example.

We propose a general formal argumentation-based model that constructs arguments for/against each possible classification of an example, evaluates them, and determines among the conflicting arguments the acceptable ones. Finally, a "valid" classification of the example is suggested. Thus, not only the class of the example is given, but also the reasons behind that classification are provided to the user as well in a form that is easy to grasp.

We show that such an argumentation-based approach for classification offers other advantages, like for instance classifying examples even when the set of training examples is inconsistent, and considering more general preference relations between hypotheses. Moreover, we show that in the particular case of concept learning, the results of version space theory are retrieved in an elegant way in our argumentation framework.

It is well established by conflict theorists and others that successful negotiation should incorporate "creating value" as well as "claiming value." Joint improvements that bring benefits to all parties can be realised by (i) identifying attributes that are not of direct conflict between the parties, (ii) tradeoffs on attributes that are valued differently by different parties, and (iii) searching for values within attributes that could bring more gains to one party while not incurring too much loss on the other party. In this paper we propose an approach for maximising joint gains in automated negotiations by formulating the negotiation problem as a multi-criteria decision making problem and taking advantage of several optimisation techniques introduced by operations researchers and conflict theorists. We use a mediator to protect the negotiating parties from unnecessary disclosure of information to their opponent, while also allowing an objective calculation of maximum joint gains. We separate out attributes that take a finite set of values (simple attributes) from those with continuous values, and we show that for simple attributes, the mediator can determine the Pareto-optimal values. In addition we show that if none of the simple attributes strongly dominates the other simple attributes, then truth telling is an equilibrium strategy for negotiators during the optimisation of simple attributes. We also describe an approach for improving joint gains on non-simple attributes, by moving the parties in a series of steps, towards the Pareto-optimal frontier.

Teamwork requires that team members coordinate their actions. The representation of the coordination is a key requirement since it influences the complexity and flexibility of reasoning team---members. One aspect of this requirement is detecting coordination faults as a result of intermittent failures of sensors, communication failures, etc. Detection of such failures, based on observations of the behavior of agents, is of prime importance. Though different solutions have been presented thus far, none has presented a comprehensive and formal resolution to this problem. This paper presents a formal approach to representing multiagent coordination, and multiagent observations, using matrix structures. This representation facilitates easy representation of coordination requirements, modularity, flexibility and reuse of existing systems. Based on this representation we present a novel solution for fault-detection that is both generic and efficient for large-scale teams.

This paper presents a novel, unified distributed constraint satisfaction framework based on automated negotiation. The Distributed Constraint Satisfaction Problem (DCSP) is one that entails several agents to search for an agreement, which is a consistent combination of actions that satisfies their mutual constraints in a shared environment. By anchoring the DCSP search on automated negotiation, we show that several well-known DCSP algorithms are actually mechanisms that can reach agreements through a common Belief-Desire-Intention (BDI) protocol, but using different strategies. A major motivation for this BDI framework is that it not only provides a conceptually clearer understanding of existing DCSP algorithms from an agent model perspective, but also opens up the opportunities to extend and develop new strategies for DCSP. To this end, a new strategy called Unsolicited Mutual Advice (UMA) is proposed. Performance evaluation shows that the UMA strategy can outperform some existing mechanisms in terms of computational cycles.

Trust is essential when an agent must rely on others to provide resources for accomplishing its goals. When deciding whether to trust, an agent may rely on, among other types of trust information, its past experience with the trustee or on reputations provided by third-party agents. However, each type of trust information has strengths and weaknesses: trust models based on past experience are more certain, yet require numerous transactions to build, while reputations provide a quick source of trust information, but may be inaccurate due to unreliable reputation providers. This research examines how the accuracy of experience- and reputation-based trust models is influenced by parameters such as: frequency of transactions with the trustee, trustworthiness of the trustee, and accuracy of provided reputations. More importantly, this research presents a technique for dynamically learning the best source of trust information given these parameters. The demonstrated learning technique achieves payoffs equal to those achieved by the best single trust information source (experience or reputation) in nearly every scenario examined.

In this paper we develop a novel probabilistic model of computational trust that explicitly deals with correlated multi-dimensional contracts. Our starting point is to consider an agent attempting to estimate the utility of a contract, and we show that this leads to a model of computational trust whereby an agent must determine a vector of estimates that represent the probability that any dimension of the contract will be successfully fulfilled, and a covariance matrix that describes the uncertainty and correlations in these probabilities. We present a formalism based on the Dirichlet distribution that allows an agent to calculate these probabilities and correlations from their direct experience of contract outcomes, and we show that this leads to superior estimates compared to an alternative approach using multiple independent beta distributions. We then show how agents may use the sufficient statistics of this Dirichlet distribution to communicate and fuse reputation within a decentralised reputation system. Finally, we present a novel solution to the problem of rumour propagation within such systems. This solution uses the notion of private and shared information, and provides estimates consistent with a centralised reputation system, whilst maintaining the anonymity of the agents, and avoiding bias and overconfidence.

1. This paper proposes a generic method for identifying elements in a domain that can be used as trust evidences. As an alternative to external infrastructured approaches based on certificates or user recommendations we propose a computation based on evidences gathered directly from application elements that have been recognized to have a trust meaning. However, when the selection of evidences is done using a dedicated infrastructure or user's collaboration it remains a well-bounded problem. Instead, when evidences must be selected directly from domain activity selection is generally unsystematic and subjective, typically resulting in an unbounded problem. To address these issues, our paper proposes a general methodology for selecting trust evidences among elements of the domain under analysis. The method uses presumptive reasoning combined with a human-based and intuitive notion of Trust. Using the method the problem of evidence selection becomes the critical analysis of identified evidences plausibility against the situation and their logical consistency. We present an evaluation, in the context of the Wikipedia project, in which trust predictions based on evidences identified by our method are compared to a computation based on domain-specific expertise.

We have developed an agent-based simulation model based on a real case study in the Sekhukhune district of the Limpopo province in South Africa. The work reported here is part of an ongoing project that deals with social complexity emerging from bottom-up as a result of individuals' interaction. We model these interactions at the individual and the household level. Networks that result from the social processes in this simulation are dynamic and co-evolving. We outline a possible way of comparing simulation snapshots when the population size does not depend on a network's global characteristic.

Theories of multiagent systems (MAS), in particular those based on modal logics, often suffer from a high computational complexity. This is due in part to the combination of agents' individual attitudes (beliefs, goals and intentions), and even more importantly to the presence of group attitudes, such as common belief and collective intention.

Agents for Alignment into strategy Experience (ALEX agents), a type of recommender agent (RA), are proposed here as a means of helping employees to perform tasks in line with the strategy adopted by their particular organization. Such an organization is represented in this approach by Fuzzified Strategic Maps (FSMs). Collaborative filtering (CF), opinion based filtering (OBF) and a Human Values Scale (HVS) are used in the ALEX-based approach to interpret the organization's strategy and actions aligned with it. ALEX agents can send messages to their users (the employees) to guide them towards meeting organizational aims aligned with that strategy and with the behavior of the organization's other employees or recommender agents. A real-life example using members of a research group in a European university is presented.

Referral networks are a kind of P2P system consisting of autonomous agents who seek, provide services, or refer other service providers. Key applications include service discovery and selection, and knowledge sharing. This use of referrals is inspired by human interactions, where referrals are a key basis for judging the trustworthiness of a given service.

The use of referrals enable an agent to control how its request is processed, it also provides an architectural basis for four kinds of interaction policies. InterPol is a language and framework supporting such policies. InterPol has been implemented using a Datalog-based policy engine for each agent. It has been applied on scenarios from a (multinational) health care project. The contribution of this paper is in a general referrals-based framework for privacy and trust management, which is shown to effectively capture a variety of privacy and trust requirements of autonomous users.

multiagent systems designed to work collaboratively with groups of people typically require private information that people will entrust to them only if they have assurance that this information will be protected. Although Distributed Constraint Optimization (DCOP) has emerged as a prominent technique for multiagent coordination, existing algorithms for solving DCOP problems do not adeqately protect agents' privacy. This paper analyzes privacy protection and loss in existing DCOP algorithms. It presents a new algorithm, SSDPOP, which augments a prominent DCOP algorithm (DPOP) with secret sharing techniques. This approach significantly reduces privacy loss, while preserving the structure of the DPOP algorithm and introducing only minimal computational overhead. Results show that SSDPOP reduces privacy loss by 29--88% on average over DPOP.

In this paper, we present a framework for evaluating the security of trust and reputation systems for electronic marketplaces populated with buying and selling agents. Our proposed framework offers a method for researchers to understand the security of their systems, and to provide precise guarantees of the degree of provable security that these systems offer. We demonstrate the viability of our proposed framework by presenting a specific monetary-based trust system known as Trunits, along with an analysis that shows that Trunits provides a guaranteed level of security for buyers.

Norms are shared expectations of behaviours that exist in human societies. Norms help societies by increasing the predictability of individual behaviours and by improving co-operation and collaboration among members. Norms have been of interest to Multiagent Systems (MAS) researchers as software agents may violate norms due to their autonomy. In order to built robust MAS that are norm compliant and systems that evolve and adapt norms dynamically, the study of norms is crucial. Our research focuses on how norms emerge in agent societies. In this paper we propose two mechanisms for norm emergence.

Modeling crowd behavior is an important challenge for cognitive modelers. We propose a novel model of crowd behavior, based on Festinger's Social Comparison Theory, a social psychology theory known and expanded since the early 1950's. We propose a concrete framework for SCT, and evaluate its implementations in several crowd behavior scenarios. Results from task measures and human judges evaluation shows that the SCT model produces improved results compared to base models from the literature.