State and surveillance, and AI surveillance

di Trinità Dei Monti - 1 Ottobre 2019

Stockholm, Sweden

State and surveillance, and AI surveillance


Despite being each other’s raison d’être, the relationship between the state and individuals as its subjects constantly asks to be reconsidered and justified anew. On this issue, Susanne Wigorts Yngvesson (2018), an ethicist in Sweden, provides two scenarios based on M. Foucault’s premises in ‘Discipline and Punishment’. The first begins with assuming that persons X and Y have threatened state security or obstructed the implementation of security measures. As those two persons cannot be identified, it provides the grounds for restricting everyone’s right on the private sphere by establishing proportional agencies for eavesdropping, CCTV surveillance, regulations on information flows, etc. The consequence is that people will believe that their private sphere is being protected by the efforts that the X and Y persons may be identified and prosecuted for misusing their freedom. However, the case is that the overall protection has led to limitation of everyone’s freedom.

The other scenario is based on power through transparency and subordination through providing information. Establishing an environment of total transparency would technically provide conditions for the highest degree of security in a state. One’s protective attitude to their private sphere may be interpreted as if the person has certain hidden intentions. The premise is that one (police, journalists, and other authorities) should be able to access the information which they claim as necessary, in order that security may be provided. It leads to the assumption that once people know everything, the world will be a good place. It is a sort of a theology on power structures from which it becomes suspicious whenever someone wishes to hide themself and its correspondence. Accordingly, it is required that all individuals actively disclose that they have not done anything wrong, because by not doing so provides grounds for suspicion.


Freedom and its relation to the state

The setting presented in the two scenarios is fundamentally what is found in the justification of the state and hence too what constitutes the ‘social contract’. The notion of ‘social contract’ builds upon the assumption that individuals have consented to the state authority, either tacitly or explicitly, or even only hypothetically (Wolff, 2006). It implies entrusting certain aspects of individual freedom to the state in order that it may guarantee the common freedom at the level of a society. In simplified terms, the state is perceived simultaneously as a freedom provider and as a freedom-constraining body. This brings into the discussion the notions of collectivism and personal autonomy. The relation between these two may differ from state to state. Majorly in the Western world, especially in the post-revolutionary period, personal autonomy has been perceived as the essential constituent of liberty.

Björkman (2018) regards the example of Sweden as one which provides a somewhat specific relation between collectivism and individualism. Despite being a highly individualist society, during the 20th century the protection of collectivism and its credibility had a priority over personal autonomy. In such instance, the right to the private sphere is certainly assumed. However, general community sentiment is such that it leads to the circumstances in which transparency is being favoured over the inviolability of individual privacy. Especially so if it is justified by the discourse of securing the conditions for the common good. For that reason, Swedes have been rather welcoming towards the controls and inspections made by the state. On the one hand, such an environment motivates individuals to be open about what they are. On the other, they are being simultaneously motivated to shape ‘what they are’ according to the established principles of collectivism.


God–panopticon–technology relation

The issue of the ‘big brother society’ i.e. the surveillance of manifold life spheres appears to have become a normalcy. Only a few decades ago such an environment could exist only in fiction and its consideration in the context of reality would be regarded as an anomaly. The tendencies of this sort have always been reflected in states’ ambition to keep their subjects under control and have especially been favoured by totalitarian regimes. Thus, it can be argued that the very idea of it is anti-democratic and that it goes against the fundamental principles of freedom. Yngvesson (2018) argues however that this has a very enrooted base in human psyche which has since the very beginnings been established through the channel of religion. The human relationship with God is based on the idea that there is someone who is omnipresent and sees everything and is concerned about everything.

The monitoring by the God has mainly been replaced by technology. Although justifications may differ, the motives are likely essentially the same. They are clearly emphasised in the approach established by Jeremy Bentham and his idea of panopticon, i.e. the idea that people can be improved if they are given the right preconditions. It can be done by discipline and control. Panopticon is the idea of a constant and total monitoring of individuals without them knowing from where and when exactly they are being monitored (Schofield, 2009). It establishes the psychological need for self-discipline in those individuals. In the context of actual legislation this can be interpreted as a conflictual relationship between private sphere and state interests on the basis of credibility, security, welfare, health and morality.


Effects of the ‘big brother’ society

On justifying the state authority over individuals, John Stuart Mill (2011) claimed that “[t]he only purpose for which power can be rightfully exercised over any member of a civilized community, against his will, is to prevent harm to others.” This is not based merely on restricting individual freedom, but rather on the belief that entrusting freedom to the state is the precondition for the common good. The individuals who entrust their freedom to the state expect justice and security in return. Even more importantly, those individuals discipline themselves by determining the scope in which they may exercise their freedom in a just and safe way. Hence, what constitutes the power authority is the discourse which denotes the scope of ‘acceptable’ in which freedom may be exercised. In other words, the source of power is in approving the ‘acceptable’ behaviour and ideas, rather than in imposing direct restrictions to individual autonomy.

By the virtue of being positive and approving, power holds the individuals as its subjects, and thus avoids creating an opposition to its authority. However, by creating the scope of ‘acceptable’, it delineates the boundaries whence begins the field of ‘unacceptable’, which is generally referred to as ‘illegal’, ‘criminal’ or ‘immoral’. Because of the fear of punishment, the said structure of power motivates every individual to set their actions in conformity to the discourse of ‘acceptable’. Hence the aim of every ‘good citizen’ to remain ‘seen’ within that scope. It implies a tacit consent to being surveyed. Furthermore, it can motivate the anticipation of an individual to be surveyed in the occasions when they may be seen as a candid citizen. Such attitude thence does not only approve of external surveillance but imposes a form of internal self-surveillance (Taylor, 2010).


AI surveillance today

The underlying idea of being a ‘good citizen’ hence plays an important role in general approval of surveillance systems and AI surveillance technologies. The effect of the assumption that one is being constantly monitored is that the one imposes self-surveillance upon themself and their actions. Currently the most common forms of surveillance are CCTVs, UAV (e.g. drones), data surveillance. However, the main issue is that those forms have been empowered with the ‘artificial intelligence’ (AI) dimension. Due to essentially being ‘self-learning’ algorithms, the AI surveillance systems provide the basis for a constant and more comprehensive monitoring. The classical surveillance imposes the assumption that one never knows when is being watched and thus motivates self-surveillance in the individual. The AI surveillance on contrary provides technical capacities that one may be actually and constantly monitored simply by surveying their data in various forms.


Justifications and controversies

Now, the dominant reading of this issue is that this leads to a total surveillance society which fundamentally correspond to a totalitarian state. It may indeed lead to questioning the idea of private sphere and thus essentially void one of the main constituents of the notion of liberty. A question which follows from here is whether this implies anything in practical terms when it comes to the lives of individuals. The main difference from the perspective of an individual is foremostly semantical. The case was that one assumed that they might be watched anytime, while now it is the case that one knows that they are being watched at all times. Both cases technically require from the individual to be a ‘good citizen’ and to act in conformity with the discourse of ‘acceptable’.

A concern which is being emphasised about AI surveillance is that it needs to work as a ‘big data collection tool’ in order to work efficiently. Accordingly, it does not only survey what may be deemed as relevant but rather everything surveyable. Arthur Holland Michel, an AI researcher, points out that “[t]he thinking behind big data is that you collect everything with the understanding that most of it is probably not going to be all that relevant but that it is only by collecting everything that you will catch those … ‘unknown unknowns’ that could become relevant down the line.” This thence also affects how the AI systems learn. Giovanni Buttarelli, who served as the European Data Protection Supervisor, emphasised in one of his speeches (EDPS, 2018) a necessity for self-regulation of AI systems. The self-regulation technically means that certain ethical parameters ought to be installed in the AI algorithms.


Ending remarks

Despite that certain implications of AI surveillance appear to be controversial, it is significant to note that the said technology has already undergone tacit approval by the society. One may find it surprising when considers the implications of AI surveillance in abstracted terms. However, by considering it in its genealogical terms, this may be perceived just as another phase in defining the structures of power. Individuals have constantly been approving of the surveillance discourse in order to prove their conformity with the state authority. The reason for that is found in the collective reasoning of the individuals which perceives the state authority as the security and liberty provider. The argument is best reflected in the saying ‘one who has pure flour in bag has nothing to hide’ followed by ‘if you have nothing to hide, you have nothing to fear’.

This article has indeed not discussed the current forms of AI surveillance in detail. It is due to the reason that it is currently being largely discussed by AI experts who are more competent to provide insightful remarks on it. This article has accordingly aimed to provide a concise discussion that could contribute to the understanding of the AI surveillance phaenomenon in genealogical terms. The nature and purpose of the idea of surveillance used by states has remained essentially the same. However, the form of it has changed and thus it has gained arguably a more efficient and potent character. Having said the latter, this article aims too to raise awareness on the implications which AI surveillance may produce in the social context. It primarily concerns the understandings of liberty and democracy which may be perceived and further debated in terms of the ‘social contract’ and in Foucauldian discursive terms.





*Stefan Vucic is the author of this article. He is currently based in Stockholm, Sweden, and is reading Politics and International Relations. As always, we publish our articles to stimulate debates which can lead to integrations and detailed studies.

* The contents and the valuations of the article are a responsibility of the author alone.