Conclusion: Balancing data-driven personalisation and law as social systems

Abstract

The contributions of this volume have examined the wide-ranging implications of data-driven personalisation. This conclusion will consolidate them through the use of systems theory. Systems theory identifies some entities as capable of sustaining themselves through adaptive reproduction of their own subcomponents and internal organisational schemas. These self-ordering systems are deemed autopoietic (Greek for ‘self-producing’). While systems theory has its origins in the biological sciences, it has become a prominent means of understanding social systems as well. For social systems, “the basic element…is communication”. The defining feature of an autopoietic social system is that it autonomously reproduces itself through internal communication.
One way of imagining data-driven personalization is as one of the subcomponents of society as a whole. Yet even more intriguing is the conceptualization of data-driven personalization as its own autopoietic system. The idea is an elegant fit: data-driven personalisation consists of a recursive feedback loop between profiled persons who provide a continuously evolving mass data set, and an algorithm that, working only with such data, provides continuously updated recommendations to these persons. Data-driven personalisation system are thus iteratively self-replicating through communication between its two main subcomponents: profiled persons and a corresponding data set.
Imagining data-driven personalisation as a self-contained autopoietic system as well as part of the broader autopoietic system of society illuminates central themes in the volume and connects its diverse contributions. Firstly, such a conceptualisation unifies the inability of consent to adequately give persons control over data-driven personalisation. Because the substance of data-driven personalisation is continually redefined through interaction and adaptation, a static grant of permission is woefully inadequate to enable persons to control their information. Furthermore, insofar as data-driven personalisation redefines persons themselves, such involved persons are not appropriately positioned to monitor and control their role in personalisation. Secondly, recognising that data-driven personalisation is autopoietic explains why it is so difficult to purge the system of its power inequities. Systems theorists recognise that systems have moments of genesis and operate in environments that provide continuous external stimulation. But once an autopoietic system has internalised the prejudices of its creation, these tendencies cannot be purged by deliberate intervention. By its very self-sustaining nature, the system cannot be ‘reprogrammed’ from the outside – a characteristic that explains many of the biases that plague data-driven personalisation.
This chapter concludes with a final thought as to how law can be leveraged to address these challenges of personalisation. Because any data-driven personalisation system is dynamic and self-contained, attempt to tame it through a single legal concept (equality, privacy, non-discrimination) will likely face parallel problems as consent: the system will simply internalise the efforts as environmental ‘shocks’ in its process of recursive self-perpetuation. However, system theorists have also explored another subcomponent of society, the legal system, as autopoietic. The status of law as autopoietic explains its potential as a counterweight to algorithmic data-driven personalisation. By adaptively balancing the invasive and hierarchical potential of personalisation, law may not be able to ‘purify’ personalisation of its pathologies, but it can soften its harmful impacts on society and individuals.

More information

Identifiers

ORCID for Jacob Eisler: orcid.org/0000-0003-4422-5255

Catalogue record

Export record