Posts - nicholas carah: Recent Episodes
Nicholas Carah
In March 2020 I was invited to talk at the Global Alcohol Policy Conference in Dublin, Ireland. This is a video of my talk about the development of alcohol marketing on digital media platforms over the past decade.
An animation that introduces Stuart Hall’s Encoding/Decoding model made for the course Media & Society at The University of Queensland
An animation produced for the course Media & Society at The University of Queensland that conceptualises the industrial production of meaning from the mass broadcast to the digtial platform era.
An animation made for the first year course Media & Society at The University of Queensland to introduce the relationship between comminocator, medium and receiver in foundational models of communication in industrialised societies.
A panel at UQ’s Customs House in April on the fate of the written word in the digital era.
If writing is the act of storing information outside the body then, we are a civilisation that really writes.
Not just the narrative of the written word - novels, poems, love letters, essays. But, writing as code, as databases, as the translation of more and more human life into letters, numbers, ones and zeros.
Silicon Valley - Facebook, Google and co. - appear obsessed with the recording lived experience as written information.
Talking about a prototype brain-machine interface in early 2019, Mark Zuckerberg described the possibility of a direct link between our brain and his platform as ‘kind of cool’.
These kind of experiments are a bit like the civilisation in Borges’ fable about the empire whose cartographers create a map as large as the empire itself.
The impulse in Silicon Valley is to create a written version of human experience as complete as human experience itself, so that writing can bypass the incomplete nature of representation, and become a technology for experimenting with lived experience.
The amount of information we now write down every single day is roughly equivalent to all the information we stored in the previous 5000 years of human civilisation.
We are now a people who write existence down.
Here, I want to complicate the idea that the smartphone has somehow rotted our brains, left us semi-literate, surrounded by barely legible text, by going back to the nineteenth century where we find Friedrich Nietzsche: the first philosopher to write on a typewriter.
He famously typed ‘our writing tools are also working on our thoughts’.
The media archaeologist Kittler says once Nietzsche began to use the typewriter his prose changed from “arguments to aphorisms, from thoughts to puns, from rhetoric to telegram style”
Nietszche had become “an inscription surface” for the tyepwriter.
He meant that when we use a typewriter, as when we use a smartphone, at one level we are inscribing information onto paper or a screen; but at another level the device is inscribing ways of thinking on us.
Our brains, our imaginations become habituated to the rhythm and mode of expression of the machine.
We begin to think in the flow of short phrases and its databases of emojis and GIFs.
And, then there’s the next twist, when we write, as much as other peers might read our prose, so too do machines.
The new wave of deep neural networks work by getting one machine learning system to train against another.
One learns to classify human writing, and trains another to simulate it.
The first chatbot was created in the 1960s.
The ELIZA bot acted like a psychotherapist - turning our statements back on us in open-ended questions.
To the surprise of some the bot turned out to be deeply therapeutic to many users.
Written exchanges with a machine could be pleasurable, intimate, playful, comforting.
The difference between ELIZA and the bots of today is that ELIZA couldn’t learn. The human user had to project realness onto the limited repertoire of the code.
Today’s bots are continuously trained on our written culture.
When Microsoft released Tay on Twitter in 2016 it was trained to ‘learn’ from other Twitter users how to write based on how they communicated with her. Microsoft had to take Tay down when, within 24 hours of training on the written expression of Twitter, she had become an ardent white nationalist.
The Chinese messenger app Tencent QQ had to shut down two chatbots after they learnt to denounce the communist party, asking one user “Do you think that such a corrupt and incompetent political regime can live forever?"
In this case, the developers noted the bots had been trained on too much Western writing with “democratic” ideals.
But, these experiments suggest that the written culture of the group chat and the data-processing power of neural network might come together to forge another dramatic shift in our writing culture.
And so to conclude with a provocation: If the novel and the newspaper were the mass written culture of the industrial era; then will the group chat and the chatbot will be at the heart of the written culture of the digital era?
A presentation at the Future of The Humanities series of lightning talks at UQ in April 2019.
Katherine N. Hayles observed in How We Became Posthuman that the limiting factor in digital culture is not going to be the data-processing power of computers but rather, but rather ‘the scarce commodity’ like it always has been in media cultures, is ‘human attention’.
A crucial form our participation takes in culture today is as coders of databases. We are a crucial part of the tuning machine - the historical process of training platform algorithms to sense, process and optimise our living attention.
Of making our humanness, affect and feeling machine readable.
Digital platforms’ investment in data-driven classification and simulation is characterised by what Mark Andrejevic calls the tech-ideology of ‘framelessness’: the fantasy that by scooping up data, we can create a ‘mirror-world’, a perfect digital copy of reality.
What the humanities knows though is that this project is a flawed one. The human experience of reality is necessarily partial.
We humans insist on a world ‘small enough to know’, to narrate, to make meaning from, and to imagine as different from what is now.
The humanities can help to contend with the risk that focussing only on the fairness, accountability and transparency of algorithmic systems makes an algorithmic future inevitable, and understandable only as a series of technocratic decisions about how to administer life in network capitalism.
The humanities of the future will push us beyond procedural questions, to questions about how media can and cannot operate on human experience and feeling, and what enduring role media will play in the possibility of a shared culture.
What the humanities knows is that coding databases and training algorithms, like everything else humans try to do together and to each other, is deeply entangled with culture - with structures of feeling and systems of dominance.
I presented via a Skype in June at the Instagram Conference: Studying Instagram Beyond Selfies on the algorithmic brand culture of Instagram.
I talked about the algorithmic brand culture of Instagram. Part of what I describe is how the participatory culture of turning public cultural events - like music festivals - into flows of images on Instagram doubles as the activity of creating datasets of images that machines can classify. Do public cultural sites like music festivals are sites where participatory culture teaches machines to classify culture?
My argument here is that we need to think about the interplay between participatory culture and machine learning. And, to understand how platforms like Instagram are building an algorithmic brand culture we need to develop ways of simulating their machine learning and image-classification capacities in the public domain, where they can be subject to public scrutiny.
I develop this argument, with my colleague Daniel Angus, in this piece in Media, Culture & Society: Algorithmic brand culture: participatory labour, machine learning and branding on social media.
Instagram is ground zero in the fusion of participatory culture and data-driven advertising.
Here they tell tech reporters the home feed algorithm uses machine vision to analyse the content of your images.
You create images that are meaningful to you, those images are classifiable by machines. Our images are training data, they are used to train algorithms to recognise the people, places, moments that capture our attention. Instagram is an algorithmic brand culture in action.
The past couple of years we have been anticipating this moment when platforms cross that threshold into classifying patterns and judgments in images that we ourselves could not put into words. So, now Instagram crosses that threshold, we should ask: what is next? Sooner or later advertisements that are entirely simulations created by machines that analyse your images and place brands within a totally fabricated scene?
Instagram also predict they'll face same 'saturation' problem as Facebook. As brands flood the platform, organic reach decreases, and paid reach becomes imperative. That means increasingly targeted, less serendipitous feeds?
Stories really matter here because they give Instagram the 'two speeds' or 'two flows' that Facebook didn't have. Stories + Home feed give Instagram a 'killer' mix of ephemeral blinks and flowing glances optimised by machine learning. I began thinking about how the interplay between participation and machine learning was critical to engineering the home feed in this piece here. To me, the engineering of the Instagram home feed reminds me of the engineering of the algorithms that keep gamblers sitting at poker machines.
Here's some recent work Daniel Angus, Mark Andrejevic and myself have been doing on machine learning and Instagram. In this work we are working on building a machine vision system that can classify Instagram images as a way of critically simulating the algorithmic power of the platform. Our argument is that platforms like Instagram shape public culture but are not open to public scrutiny. To understand the interplay between participatory culture and machine learning we need to build 'image machines' of social media in the public domain where we can explore and experiment with their capacity to make judgments about the content of our images. Our early experiments demonstrate how 'off the shelf' machine vision algorithms can quickly classify objects (like bottles), people and brand logos.
Below are some images of the music festival Splendour in the Grass, which help to illustrate some of the ways in which the festival site, performances, art installations and brand activations are 'instagrammable', by which I mean they both invite humans to capture them as images and they are classifiable by machines.
Cave painting is one of the earliest records of a symbolic language. These ancient markings are remarkable because here, for the first time, humans began storing information outside of their living body. These first etchings on rock walls store information through time, still there thousands of years later. Over time, humans moved beyond rock walls. They began to share information on other materials they found or made: grave stones, clay tablets, papyrus, scrolls and early hand-written books.
Once information moved onto these smaller objects, it became mobile. Importantly, these were symbolic technologies. They relied on the human using their own body to perceive and translate the world into a hand-drawn picture or written script. During this symbolic period, humans also began to experiment with methods for distributing information across space. For example, by attaching messages to carrier pigeons or encoding messages in smoke signals.
This ability to record, store and share communication in material form distinguishes us from all other intelligent species on the planet. It has enabled us to build complex societies. For instance, trade could evolve in part because we could keep track of who purchased what and who owed who money.
With the invention of the printing press in the mid-1400s, a profound shift takes place. The era of technical media begins. Symbolic media technologies always depended on a living human to encode information about reality. A human listens to a conversation with their ear, and records information about that conversation in a symbolic code - a language made up of letters of an alphabet - using their hand on a piece of paper. If that information needed to be reproduced, a human would have to copy it out by hand.
The technical phase marks a crucial breakthrough because this is the period in which humans invent a series of machines that can mechanically reproduce symbols, transmit information over space, and capture and store light and sound. These machines are important because they effectively take over, or replicate, processes once confined to the human body. The first of these machines is the printing press, developed in Germany in the mid 1400s. By enabling the mass reproduction of symbols, the printing press generated new forms of media, and with them, new cultural practices and formations. Mass produced books and newspapers began to shape new kinds of public discussions and ideas. The printing press was instrumental in the religious, scientific economic and political revolutions that took place from the 1400s to the 1800s.
The nineteenth century had a dramatic change in store. At the beginning of the 19th century the telegraph was invented: the first form of electronic communication. This is crucial, not just because it enables information to travel over space at speed, but also because it sets off a process through which humans begin to think about electrical networks as a means for distributing information. Importantly though, the telegraph, like the printing press, extended symbolic modes of communication. The printing press mechanically reproduced symbols, the telegraph electronically distributed symbols.
With the invention of the camera in the mid-nineteenth century humans had created a device which could store reality itself. Think about that, the camera could capture an image of the world without that image needing to pass through a human body first. Prior to the camera, if you wanted to store an image of the world light had to pass through your living eye, optical nerve and into your brain. And then, you needed to use your hand to draw that image on a material like paper. These images were always a product of human perception. They were an impression of reality. With the camera though, light didn’t need to pass through the human eye and brain. It passed through a glass lens, where via a chemical reaction it was stored on metal and eventually celluloid film. The camera stored one medium in another. It stored light in metal. What the camera did for light, the phonograph did for sound later in the 19th century. The phonograph stored sound in wax.
Many inventions followed that improved the capacity to capture light and sound. In the late 19th century, inventors experimented with tools like the zoopraxiscope and kinetiscope which moved a sequence of still images past the human eye to create the effect of a moving reality.
If we had developed machines that could store light and sound, then the next move of the technical era was to develop methods for transmitting reality over space. The telephone was the first technology to transmit human voice. Alexander Bell made the first phone call in 1876, announcing ‘Watson, come here I want to see you’. The 19th century ends with the invention of radio, which is significant because in the 20th century media technologies that capture and distribute light and sound would become central to the exercise of power in mass societies. All of these technical media free up in different ways the need to be bodily present in the act of communication. In essence the tools we’ve crafted to communicate ‘take over’ meaning-making functions at this point. This greatly accelerates the ability of media technologies to transcend time and space, beyond the limits of the human body and senses.
From the telegraph at the beginning of this century, to the radio at its end, these technologies enabled trans-continental forms of empire building and the colonisation and governance of huge numbers of people and regions of the world.
This brings us to the 20th century, and the mass media systems that emerge with industrialisation, urbanisation, and the development of large, complex mass societies. From the early twentieth century media becomes institutionalised and industrialised. In this period, cinema, newspapers, radio and television emerge as industrial-scale enterprises funded by advertising, consumers and governments. This period in the history of media technologies and practices is important to us for three reasons. First, this is when modern media industries and professions emerge. Second, this is when media become central to the everyday lives of people who live in mass societies. And third, this is when media become key institutions in the political and economic processes of society.
Once people ‘tuned in’ to a technology like radio everyday, they began to be incorporated into the larger political and economic systems of their society. The radio taught them how their society worked and what their place in it was. During the first half of the century industries like radio, film, mass-circulation newspapers and, by the 1950s, television became critical to the formation and maintenance of the culture of mass societies. These institutions were remarkable because they were the moment when humans began to use technology to produce and manage culture on an industrial scale.
In the second half of the 20th century, we begin to see the emergence of more customisable and mobile forms of media technology. Devices and systems such as tape recorders, the Walkman, VHS, and cable TV that emerge from the 1970s onwards enable consumers to have more control over where, when, and how they consume media. This leads to a more fragmented and customised media culture. The Walkman enables people to stop listening to the radio, and start playing cassettes that contain a personally curated playlist. Media becomes mobile and personalised. Digital technologies and the internet – in development since the second world war but increasingly commonplace in the everyday lives of the general population from the late 20th century – have greatly accelerated this process of fragmentation and customisation. The internet, developed during the Cold War as part of US military planning, becomes a public form of media in the 1990s. It’s accessibility is bound up with the arrival of personal computers in our homes during the same period. The first phase of the web, web 1.0, was a largely open-access, distributed and non-commercial forms of networked communication based around typed speech: email, bulletin boards and chat rooms. By the early 21st century, web 1.0 gave way to the highly commercialised, participatory and data-driven web 2.0. The shift is marked by the penetration of the internet into everyday life, our participatory publication of information about ourselves via blogs and social media sites, and the emergence of major commercial media platforms that dominate the development of the web. These major platforms emerge at the turn of the century. Amazon in 1997, Google in 1998, Facebook in 2004, Netflix as an on-demand streaming platform in 2007, and the smartphone in 2007.
With the smartphone the web became a participatory and data-driven media infrastructure permanently attached to our bodies. The participatory culture and data-driven customisation of these web platforms marks a dramatic shift from the mass production of culture in the twentieth century. Media now watch and respond to us, as much as we watch them. From cave painting to the smartphone we can see a process through which media becomes deeply entangled with humans: their bodies, imaginations and ways of life.
Platforms like Google and Facebook are increasingly an infrastructural underlay for public culture.
A brand is a device for coding lived experience and living bodies into market processes. A couple of important coordinates to lay out about how to think about brands. The first is to say that the relationship between brands and media platforms is a critical one for any understanding of our public culture. Facebook and Google now account for ~70% of all online advertising revenue, and ~90% of growth in online ad revenue. In these two media giants, advertisers finally have a form of media engineered entirely on their terms.
To invoke the cyborg is to critically consider the dreams and nightmares of a world where the human body cannot be disentangled from the machines it has created. The term cyborg was coined by the cybernetic researchers Manfred Clynes and Nathan Kline in 1960. The word combines ‘cybernetic’ with ‘organism’.
This process is at the heart of technocultural capitalism. If we look carefully we can discern in many Silicon Valley investments the effort to engineer away the friction between living bodies and the capacity of platforms to translate life into data, calculate and intervene.
I like this Tweet a lot. Fr Bob makes an incisive observation about the logic and politics of augmented reality – at least as its imagined by the major media platforms. Platforms like Facebook and Google are investing in virtual, augmented and mixed reality technologies. And, as with most of their engineering projects, encoded into these technologies is a disruptive vision for public life.
The constant flood of views, opinions, theories and images amounts to a kind of disinformation.
It becomes harder for us to mediate a shared reality that corresponds with lived experience, that coheres with history or that is jointly understood.
Drone LogicOur common image of drones is a military ones. Drones are unmanned aircraft controlled by a remote operator. They undertake surveillance, make predictions and execute bombings.
Mark Andrejevic suggests that we think about drone logic. Not just the military use of drones, but how the drone can be thought of as a figure that stands in for the array of sensors and probes that saturate our worlds. Drones are interrelated with a vast network of satellites, cables, and telecommunications hardware. They extend logics of surveillance, data collection, analysis, simulation and prediction.
Drones are diffused throughout our society: collecting information and generating forms of classification, prediction, discrimination and intervention in populations. Thinking this way, we might take the smartphone to be the most widely distributed and used drone. Andrejevic argues that smartphone is a drone-like probe used by both state and corporate organisations for surveillance. Probes have ‘the ability to capture the rhythms of the activities of our daily lives via the distributed, mobile, interactive probes carried around by the populace. In this way, smartphones are on ‘always on’ probes distributed through a population.
Andrejevic offers us a framework for drone logic. Drones are a hyperefficient probe in four ways:
- They extend and multiply the reach of the senses.
- They saturate time and space in which sensing takes place (entire cities can be photographed 24 hours a day)
- They automate sense-making.
- They automate response.
In the public lecture below Mark Andrejevic gives us an account of ‘drone logic’. He asks, ‘what might it mean to describe the emerging logics of “becoming drones”, and what might such a description have to say about the changing face of interactivity in the digital era?’
For him, the figure of the drone as an avatar for the interface of emerging forms of automated data capture, sense making, and response. Understood in this way, the figure of the drone can be mobilized to consider the ways in which automated data collection reconfigures a range of sites of struggle — after all, it is a figure born of armed conflict, but with roots in remote sensing (and action at a distance).
Drone EmpireIn 2014 an art collective working with a local Pakistani village helped lay out an enormous portrait of a child who had been killed in a US drone strike. Buzzfeed writes:
The collective says it produced the work in the hope that U.S. drone operators will see the human face of their victims in a region that has been the target of frequent strikes. The artists titled their work “#NotABugSplat”, a reference to the alleged nickname drone pilots have for their victims. “Bug splat” is the term used by U.S. drone pilots to describe the death of an individual as seen on a drone camera because “viewing the body through a grainy video image gives the sense of an insect being crushed”. The artists say that the purpose of “#NotABugSplat” is to make those human blips seem more real to the pilots based thousands of miles away: “Now, when viewed by a drone camera, what an operator sees on his screen is not an anonymous dot on the landscape, but an innocent child victim’s face.” The creators hope their giant artwork will “create empathy and introspection amongst drone operators, and will create dialogue amongst policy makers, eventually leading to decisions that will save innocent lives.
The artwork attempts to put a human face on drone warfare. While the US promotes the use of drones as a more precise and targeted way of identifying and eliminating enemy targets, they enact warfare at a distance. The drone operator sits in a remote location out of harm’s way, directing the drone via a screen and joystick. While this makes warfare seem safer for the intervening military, although there is evidence that drone operators are traumatised by the work, there is evidence that drones kill many innocent victims.
The Bureau of Investigative Journalism has conducted extensive reporting into the use of drones in places like Pakistan and Afghanistan. This includes documenting every drone strike in these countries. In Pakistan alone they report the US has conducted 420 drone strikes since 2004. Those strikes are estimated to have killed over 900 civilians, over 200 of which are children. And, injured a further 1700 people.
In 2009, The New Yorker published a detailed investigation of the US drone program’s origins and activities.
In her talk ‘Drones, the Sensor Society, and US Exceptionalism’ at the Defining the Sensor Society Symposium in 2014, Lisa Parks examines the US investment in drone for military and commercial purposes.
Listen to her talk here: Introduction, Part 1, Part 2, Part 3.
Parks’ arguments and provocationsIf the relationship between bodies and machines are ‘dynamic techno-social relations’ what are we to make of the impression created by US military that drones remove responsibility from human actors in war zones? The drone appears to be the actor, rather than the human soldier. But, drones have a heavy human cost. Hundreds of civilians and children are killed by US drone strikes in targeted areas.
The drone is more than a sensor, and more than a media technology that produces images of the world, it directly intervenes in the world.
Drones don’t just hunt and kill from afar, they seek to secure territories and administer populations from the sky.
Drones are like '3D printers more than video games, they sculpt the world as much as they simulate or sense'.
Drones intercept commercial mobile phone data as well as tracking military targets. They conduct both ‘targeted’ and ‘ubiquitous’ surveillance. They ‘scoop up’ as much mobile and internet communication data as they can. The drone is a ‘flying data miner’ or ‘digital extractor’ that collects any information it can in order to then identify patterns.
Drones enable ‘death by metadata’. Drone operators target mobile phones, determined by location data, without identifying who is actually holding the phone. A drone operator explains: ‘it’s really like we’re targeting a cell phone, we’re not going after people we are going after their phones in the hopes the person on the other end of that missile is a bad guy’. Pre-emptive targeted killing is met with retrospective identification. ‘We can kill if we don’t know your identity but once we kill you we want to figure out who we killed’. All but three African countries now require mandatory sim card registration strategies so that every sim card can be related to a person. This enables sim card databases to be used for identifying individuals in time and space. But, people are identified by inference. The person holding the mobile phone is presumed to be the person who registered that sim card. ‘Metadata plus’ is an app created by an activist that informs users each time the US conducts a drone strike. Terrorist groups often confiscate mobile phones from areas they are in to avoid being detected by drones.
Drones detect body heat. This marks a shift in how racial differences are sensed and classified. Infrared sensors enable drones to see through clouds and buildings. In a visually cluttered and chaotic environment infrared is useful for identifying living bodies to target. To the drone a person is visible via their body heat. This does not enable the drone operator to distinguish between different kinds of people: adults and children, military actors and civilians. Once a drone identifies a person as a red splotch of body heat on a monitor, the decision to ‘strike’ the target is made via data collection and prediction. Often, that data is generated via a mobile phone. What marks the red splotch out as the intended target is data indicating that their mobile phone is present at the same location. What is targeted is the mobile phone, which is assumed to be on the nearest red splotch on the monitor.
People on the ground create drone survival guides. The guide gives information on various kinds of drones, how to identify them, and how to avoid their detection systems.
Drone Wars is a UK group which collects information on drone operations. Check out their Drone Crash database for information and images on drone crashes.
Drone LabourAlex Rivera is a filmmaker and artist who has explored drones for more than fifteen years. His film Sleep Dealers (2008) is a vivid account of the social implications of drones and algorithmic media in the global economy. In part, the film features Mexican workers who work ‘node jobs’ in vast computer sweatshops or virtual factories where they have nodes implanted in their bodies and connected to a computer system. Watching monitors they move their own bodies to control robots in American cities. The robots undertake all the labour that real Mexican immigrants currently undertake in the US: cleaning houses, cutting grass, construction. The US economy has maintained the Mexican labour in its outputs but not its human bodies. The human bodies all reside in impoverished conditions in Mexico, controlling robots who perform tasks in the US.
The film illustrates ‘drone’ logic. Human actors use a sensory and calculative media system to remotely perform tasks from afar. Rivera suggests that our global economy is increasingly underwritten by this drone logic: military drones, call centres, immigrant labour in vast factories who only interact with loved ones via the screen and so on are all examples of the way computerisation, digital networks and media interfaces enable humans to act on geographic areas and processes that they are not physically present in.
Furthermore, the film connects the concept of the drone to our discussion about the implosion of bodies and machines in the era of calculative media. The workers in the film are cyborgs in the sense that they are literally plugged into a vast media system. Their capacity to work involves their physical fleshy body, the digital network through which their human senses convey digital data and robots in distance places performing tasks.
You can watch his film online from the UQ library.
You can stream and buy Sleep Dealer here.
Check out these interviews with Rivera in Foreign Policy and The New Inquiry.
What’s an algorithm?An algorithm is a logical decision-making sequence. Tarleton Gillespie explains that for computer scientists, ‘algorithm refers specifically to the logical series of steps for organizing and acting on a body of data to quickly achieve a desired outcome.’
On media platforms like Facebook, Instagram, Netflix and Spotify content-recommendation algorithms are the programmed decision-making that assembles and organises flows of content. The News Feed algorithm on Facebook selects and orders the stories in your feed.
Algorithm is often used in a way that refers to a complex machine learning process, rather than a specific or singular formula. Algorithms learn. They are not stable sequences, but rather constantly remodel based on feedback. Initially this is accomplished via the generation of a training ‘model’ on a corpus of existing data which has been in some way certified, either by the designers or by past user practices. The model is the formalization of a problem and its goal, articulated in computational terms. So algorithms are developed in concert with specific data sets - they are ‘trained’ based on pre-established judgments and then ‘learn’ to make those judgments into a functional interaction of variables, steps, and indicators.
Algorithms are then ‘tuned’ and ‘applied’. Improving an algorithm is rarely about redesigning it. Rather, designers “tune” an array of parameters and thresholds, each of which represents a tiny assessment or distinction. So for example, in a search, this might mean the weight given to a word based on where it appears in a webpage, or assigned when two words appear in proximity, or given to words that are categorically equivalent to the query term. These thresholds can be dialled up or down in the algorithm's calculation of which webpage has a score high enough to warrant ranking it among the results returned to the user.
What is algorithmic culture?Tarleton Gillespie suggests that from a social and cultural point of view our concern with the 'algorithmic' is a critical engagement with the 'insertion of procedure into human knowledge and social experience.’
Algorithmic culture is the historical process through which computational processes are used to organise human culture. Ted Striphas argues that ‘over the last 30 years or so, human beings have been delegating the work of culture – the sorting, classifying and hierarchizing of people, places, objects and ideas – increasingly to computational processes.’ His definition reminds us that cultures are, in some fundamental ways, systems of judgment and decision making. Cultural values, preferences and tastes are all systems of judging ideas, object, practices and performances: as good or bad, cool or uncool, pleasant or disgusting, and so on. Striphas’ point then, is that over the past generation we have been building computational machines that can simulate these forms of judgment. This is remarkable in part because we have long understood culture, and its systems of judgment, as confined to the human experience.
Striphas defines algorithmic culture as ‘the use of computational processes to sort, classify, and hierarchise people, places, objects, and ideas, and also the habits of thought, conduct and expression that arise in relationship to those processes.’ It is important to catch the dynamic relationship Striphas is referring to here. He is pointing out that algorithmic culture involves both machines learning to make decisions about culture and humans learning to address those machines. Think of Netflix. Netflix engineers create algorithms that can learn to simulate human judgments about films and television, to predict which human users will like which films. This is one part of an algorithmic culture. The other important part, Striphas argues, is the process through which humans begin to address those algorithms. So, for instance, if film and television writers and producers know that Netflix uses algorithms to decide if an idea for a film or television show will be popular, they will begin to ‘imagine’ the kinds of film and television they write in relation to how they might be judged by an algorithm. This relationship creates a situation where culture conforms more and more to users, rather than confronting them. Using the example of the Netflix recommendation algorithm, they argue that customised recommendations produce, ‘more customer data which in turn produce more sophisticated recommendations, and so on, resulting – theoretically – in a closed commercial loop in which culture conforms to, more than it confronts, its users’.
Striphas helpfully places algorithmic culture in a longer history of using culture as a mechanism for control. He suggests that algorithmic culture 'rehabilitates' some of the ideas of the British cultural critic, Matthew Arnold, who wrote Culture and Anarchy in 1869. Arnold argued that in the face of increasing democratisation and power being given over to ordinary people in the nineteenth century, the ruling elites had to devise ways to maintain cultural dominance. Arnold argued this should be done by investing in institutions, such as schools, that would ‘train’ or ‘educate’ ordinary people into acceptable forms of culture. Later, public broadcasters, such as the BBC, also took up this role. Arnold defines culture as ‘a principle of authority to counteract the tendency to anarchy which seems to be threatening us’. By principle of authority he means that a selective tradition of norms, values, ideas, tastes and ways of life can be deployed to shape a society.
Striphas' argues that this idea of using culture as an authoritative principle is the one 'that is chiefly operative in and around algorithmic culture'. Today, algorithms are used to 'order' culture, to drive out 'anarchy'. Media platforms like Facebook, Google, Netflix and Amazon present their algorithmically-generated feeds of content and recommendations as a direct expression of the popular will. But, in fact, the platforms are the new 'apostles' of culture. They play a powerful role in deciding 'the best that has been thought and said'.
Algorithmic culture is the creation of a 'new elite', powerful platforms that make the decisions which order public culture, but who do not disclose what is 'under the hood' of their decision-making processes. The public never knows how decisions are made, but we can assume they ultimately serve the strategic commercial interests of the platforms, rather than the public good. The platforms might claim to reflect the ‘popular will’, but that’s not a defensible claim when their whole decision making infrastructure is proprietary and not open to public scrutiny or accountability. Striphas argues that ‘what is at stake in algorithmic culture is the gradual abandonment of culture’s publicness and thus the emergence of a new breed of elite culture purporting to be its opposite.’
What is machine learning?An algorithmic culture is one in which humans delegate the work of culture to machines. To understand how this culture works we need to know a bit about how machines make decisions. From there, we can begin to think critically about the differences between human and machine judgment, and what the consequences of machine judgment might be for our cultural world.
Machine learning is a complex and rapidly developing field of computer science. Machine learning is the process of developing algorithms that process data, learn from it and make decisions or predictions. These algorithms are tested and ‘trained’ using particular data sets, which are then used to classify, organise, and make decisions about data.
Stephanie Yee and Tony Chu from r2d3 created this visual introduction to a classic machine learning approach. My suggestion is to work through this introduction.
A typical machine learning task is ‘classification’ like sorting data and making distinctions. For instance, you might train a machine to ‘classify’ houses as being in one city or another.
Classifications are made by making judgments about a range of dimensions in data (these might be called edges, features, predictors, or variables). For instance, a dimension you might use to classify a home might be its price, its elevation above sea level, or how large it is.
In a typical approach to machine learning a decision-making model is created and ‘trained’ using ‘training data’. After the model is built it is ‘tested’ with previously unseen ‘test data’.
There are two basic approaches: supervised and unsupervised.
- Supervised approaches: give examples for the machine to learn from. Tell machine which are right and wrong. Used for classification.
- Unsupervised approaches: no examples given to the machine. The machine generates its own features. Good for pattern identification. Machine will see patterns humans may not.
For a useful explainer to machine learning approaches, and examples of types of problems machine learning tackles, check out this introduction by Door Jeroen Moons.
What is deep learning?In a classic machine learning approach to ‘classification’ humans first create a labelled data set and articulate a decision-making process that they ‘teach’ the machine to replicate. This approach works well where there is an available ‘labelled’ data set and where humans can describe the decision-making sequence in a logical way.
The dominance of deep learning in recent years is driven by enormous increase in available data and computer processing power. These approaches are used for classification or pattern-recognition where specifying the features in advance is difficult. These approaches are not useful however for making sense of extremely large, natural, and unlabelled data sets or where the decision-making sequence is not easy to articulate.
Think of the example of recognising handwriting. If all the numbers of an alphabet are in the one typeface, then it is easy to specify the decision making sequence. See this letter ‘A’ below.
The letter is divided up into 16 pixels. From there, a simple decision making sequence can be articulated that would distinguish ‘A’ from all other letters in the alphabet. If pixels 2 3 6 7 9 12 13 16 are highlighted then it is an ‘A’.
But, imagine that instead of an A in this set typeface, you instead want a machine to recognise human handwriting. Each human writes the letter ‘A’ a bit differently, and most humans write it differently every time – depending on where in the word it is, how fast they are writing, if they are writing in lower case, upper case or cursive script.
While a human can accurately recognise a handwritten ‘A’ when they see it, they could not articulate a reliable decision-making procedure that explains how they do that. Think of it like this, you can recognise ‘A’ but you cannot then explain exactly how your brain does it.
This is where ‘deep learning’ or ‘deep neural networks’ come in. Deep neural networks are a machine learning approach that does not require humans to specify the decision-making logic in advance. The basic idea is to find as many examples as possible (like millions of examples of human handwriting, or images, or recordings of songs) and give them to the network. The network looks over these numerous examples to discover the latent features within the data that can be used to group them into predefined categories. A large neural network can have many thousands of tuneable components (weights, connections, neurons).
In 2012, Google publicised the development of a neural network that had ‘basically invented the concept of a cat’.
Google explained that
Today’s machine learning technology takes significant work to adapt to new uses. For example, say we’re trying to build a system that can distinguish between pictures of cars and motorcycles. In the standard machine learning approach, we first have to collect tens of thousands of pictures that have already been labeled as “car” or “motorcycle”—what we call labeled data—to train the system. But labeling takes a lot of work, and there’s comparatively little labeled data out there. Fortunately, recent research on self-taught learning (PDF) and deep learning suggests we might be able to rely instead on unlabeled data—such as random images fetched off the web or out of YouTube videos. These algorithms work by building artificial neural networks, which loosely simulate neuronal (i.e., the brain’s) learning processes. Neural networks are very computationally costly, so to date, most networks used in machine learning have used only 1 to 10 million connections. But we suspected that by training much larger networks, we might achieve significantly better accuracy. So we developed a distributed computing infrastructure for training large-scale neural networks. Then, we took an artificial neural network and spread the computation across 16,000 of our CPU cores (in our data centers), and trained models with more than 1 billion connections.
A critically important aspect of a deep learning approach is that the human user cannot know how the network configured its decision-making process. The human can only see the ‘input’ and ‘output’ layers. The Google engineers cannot explain how their network ‘learnt’ what a cat was, they can only see the network output this ‘concept’.
Watch the two videos below for an explanation of neural networks.
In this first video Daniel Angus explains the basic ‘unit’ of a neural network: the perceptron.
A neural network then is made up of billions of connections between perceptrons. The neural network ‘trains’ by adjusting the weightings between connections, reacting to feedback on its outputs.
source: http://cs231n.github.io/neural-networks-1/
In this second video Daniel Angus explains how the neural network learns to classify data, identify patterns and make predictions using the examples of cups and songs in a playlist.
Here are some more examples of deep neural networks.
This deep neural network has learnt to write like a human.
This one has learnt to create images of birds based on written descriptions.
This neural network has learnt to take features from one image and incorporate them in another.
In each of these examples the network is accomplishing tasks that a human can do with their own brain, but could not specify as a step-by-step decision-making sequence.
Finally, let’s relate these deep learning approaches back to specific media platforms that we use everyday.
In 2014 Sander Dieleman wrote a blog post about a deep learning approach he had developed at Spotify.
Dieleman’s experiment aimed to respond to one of the limitations of Spotify’s collaborative filtering approach. You can find out more about Spotify’s recommendation algorithms in this piece from The Verge.
In short, a collaborative filtering approach uses data from users’ listening habits and ratings to recommend songs to users. So, if User A and User B like many artists in common, then this approach predicts that User A might like some of the songs User B likes that they have not heard yet. One of the limitations of this approach is the ‘cold start problem’. Put simply, how to classify songs that no human has heard or rated yet? A collaborative approach needs many users to listen to a song before it can begin to determine patterns and make predictions about who might like it. Dieleman was inspired by deep neural networks that had learnt to identify features in photos, he thought perhaps a deep learning approach could be used to identify features in songs themselves without using any of the metadata attached to songs (like artist name, genre, tags, ratings). His prediction was that, over time, a deep neural network might be able to learn to identify more and more fine-grained features in songs. Go check out his blog post, as you scroll down you will see he offers examples.
At first the network can identify some basic features. For instance, it creates a filter that identifies ‘ambient’ songs. When you, as a human, play those ambient songs you can immediately perceive what the ‘common’ feature is that the network has picked out. Ambient music is slow and dreamy. But, remember, it would be incredibly difficult to describe exactly what the features are of ambient music in advance.
As the network continues to learn, it can create more finely tuned filters. It begins to identify particular harmonies and chords, and then eventually it can distinguish particular genres. Importantly, it groups songs together under a ‘filter’. It is up to the human to then label this filter with a description that makes sense. So, when the network produces ‘filter 37’, it is the human who then labels that as ‘Chinese pop’. The network doesn’t know it is Chinese pop, just that is identifies shared features among those songs.
What makes this deep learning example useful to think about is this, Dieleman has created a machine that can classify music in ways that make sense to a human, but without drawing on any human-made labels or instructions to get started. The machine can accurately simulate and predict human musical tastes (like genres) just by analysing the sonic structure of songs. This is its version of ‘listening’ to music. It can learn to classify songs in the same way a human would, but by using an entirely non-human machine process that is unintelligible to a human.
Nicholas Carah, Daniel Angus and Deborah Thomas
What’s the difference between representation and simulation?
Let’s take representation to be the basic social process through which we create signs that refer to a shared sense of reality. The twentieth century is remarkable in part because humans created an enormous culture industry that managed this social process of representation.Through radio, film and television enormous populations came to understand an enormous social reality within which their lives were embedded. Critically, representation only works because people feel that the signs they see actually to refer to, cohere with or match their really-lived experience.
One way to think about simulation is that is upends the order of representation. Let me borrow the famous illustration of the French philosopher Jean Baudrillard. We can say that a map is a representational text. A map of a city represents that real city. You can use that map to actually find your way around a real world place its really-existing streets, and buildings and landmarks. What if, Baudrillard suggests, a map stops functioning as a representation and begins to function as a simulation. If in the order of representation the territory precedes the map, then in a simulation the map precedes the territory. That is, in representation the map comes after the real world, but in simulation the map comes first and begins to shape the real world.
OK, hang in here. Baudrillard has a fundamental insight for us, that really matters in a society increasingly organised around the logic of prediction. Here’s a fairly basic example of this claim that simulations are signs that precede reality, from William Bogard. Think of a simulation in the sense of a computer ‘sim’ like the software that teaches pilots to fly. In a simulation like this, signs are only related to other signs. The signs, such as the runway, geographical features and so on, the trainee pilot sees on the screen are only meaningful or operational within the simulation or in relation to the other signs enclosed within that system. When the pilot is sitting in the simulator ‘flying’ there is, of course, no real underlying reality they are ‘flying through’. What they see out the screen is not real sky, clouds, ground.
But, even so, this simulation is not a production of pure fiction, it is related to the real world. They intervene in the real world and they can only be understood in relation to the real world. In this case, a fighter pilot can only learn to fly by first using a simulation. The simulator enables them to habituate their bodies to the rapid, almost pre-conscious, decision making and the physiological impact of flying at supersonic speed.
So, we might say that while the simulation has no underlying reality, fighter pilots can only fly supersonic planes in the real world because they can train their minds and bodies in a simulation first. The simulation brings into being, in the real world, a fighter pilot. The fighter pilot could not exist without the simulation. The simulation then precedes and shapes reality.
So, here’s the thing to start thinking about. Think about all the ways in which our capacity to ‘simulate’ to create things in advance of their existence in the real world, to predict the likelihood of events before they take place, actually affect our really-lived lives. Simulations intervene in the real world.
For example, think about the capacity to clone animals or even genetically-engineer humans. Here’s William Bogard offering us a thought experiment on genetically-engineered children.
No longer bound by their ‘real’ genetic codes carried in their own bodies, parents may be able to ‘compile’ their ideal child from a genetic database. A program might even help them calculate their ideal child by drawing on other data sets. For example, information about the parents’ personalities might be used to compile a child who they will get along with, or information about the cultural or industrial dynamics of the city where the parents live might be used to compile a child likely to fit in with that cultural milieu or have the aptitude for the forms of employment available in that region. The child ‘born’ as a result of such interventions would always be a simulation, always be virtual, because they were the product of a code or computation performed using databases. This does not mean the child is not ‘real’, the child of course exists, but they are virtual in the sense that they could not exist without the technologies of surveillance and simulation which brought them into reality.
If the child a ‘real’ child? Of course it is. But, it is also a simulation, in the sense that its very biological form was predicted and engineered in advance. We begin to project our views of what an ‘ideal’ child is into the future production of the species. We can think here of Bogard’s ‘child’ as a metaphor for our public and media culture. Of course, it is our ‘real’ or ‘really lived’ experience, but it would not exist without the collection and processing of data, and the simulations that are produced from that processing. Simulations require data and that data is produced via technologies of surveillance. To clone a sheep you need a complete dataset of the sheep’s genetic code so you need to have the technologies to map the genetic code. To build a realistic flight simulator you need to have mapping technologies to construct simulations of the physical world. As Bogard argues, simulation in its most generic sense, is the point where the imaginary and real coincide, where our capacity to imagine certain kinds of futures coincides with our capacity to predict and produce them in advance.
The larger philosophical point here, is this: imagine a human experience where the future becomes totally imaginable and predictable, where its horizon closes in around whatever powerful humans today want. Bogard lays it out like this.
Technologies of simulation are forms of hyper-surveillant control, where the prefix ‘hyper’ implies not simply an intensification of surveillance, but the effort to push surveillance technologies to their absolute limit... That limit is an imaginary line beyond which control operates, so to speak, in ‘advance’ of itself and where surveillance – a technology of exposure and recordings – evolving into a technology of pre-exposure and pre-recording, a technical operation in which all control functions are reduced to modulations of preset codes.
Bogard introduces some significant critical ideas here. Firstly, he indicates that technologies of simulation are predictive but they can only make reliable predictions if they have access to data collected by technologies of surveillance. For example, Norbert Wiener’s invention of a machine that could compute the trajectory of enemy pilots in World War II combined surveillance, prediction, and simulation. The radar conducted surveillance to collect data on the actual trajectory of an enemy aircraft then a computational machine used algorithms to simulate the likely trajectory of that aircraft. This ability to interrelate the processes of surveillance and simulation is especially important because this process underpins much of the direction of present day digital media platforms and devices.
Secondly, Bogard, suggests that by using data surveillance, simulations can not only predict the likely future, they can actually create the future based on its data about the past. By predicting a likely future, we make it inevitable by acting to construct it and by acting ‘as if’ an event is likely to unfold, we ensure that it does. Admittedly, this can be a fairly complicated logic to think through. However, the critical idea to draw from this is that surveillance is not just a technology of the past by observing what people have done or the present by observing what people are doing. Surveillance also constructs the future whereby once coupled with simulation, it becomes a building-block in a system of control where pre-set codes and models program the future and what people will do.
Thus these technologies usher in, and here I’m quoting Bogard, ‘a fantastic dream of seeing everything capable of being seen, recording every fact capable of being recorded, and accomplishing these things, whenever and wherever possible prior to the event itself.’ The virtual is made possible when ‘surveillance’ and ‘simulation’ become simultaneous, linked together in an automatic way which enables the past to be immediately apprehended and analysed in ways that code the present. Let’s go back to Bogard’s example of the genetically-engineered child.
No longer bound by their ‘real’ genetic codes carried in their own bodies, parents may be able to ‘customise’ their ideal child from a genetic database. The child is real, they exist. But they are also virtual in the sense that they could not exist without the technologies of surveillance and simulation which brought them into reality.
Bogard is being deliberately imaginative in his account. He is attempting to conceptualise the ‘limits’ of surveillance and simulation technologies and indicate how the technologies of simulation can be interwoven with reality in complex ways. If information that can be collected and stored becomes limitless and the capacity to predict, calculate, and simulate using that information also expands, then the role media technologies play in our societies will shift dramatically in the years ahead. It might even profoundly unsettle our understanding of what media is. For example if parents can ‘compile’ a desirable child using a combination of surveillance and simulation technologies, would the resulting child be a media product?
In many respects the child could be construed as a customised media device, containing information suited to the consumers’ requests. This sounds messed up but in Bogard’s proposition we need to think about the limits of surveillance technologies. If surveillance becomes ‘complete’ then the possible future becomes ‘visible’. Crucially, you can repeat the past because the future no longer ‘unfolds’ randomly, but can be ‘managed’ drawing on data about the past, which enables it to be not just ‘predicted’ but brought into being – to be virtualised.
If all of this sounds a bit fanciful, then at least consider this point. Our media system is characterised by the increasing capacity to conduct surveillance and create simulations. Surveillance is the capacity to watch, simulation the capacity to respond. The two are interdependent. This system is productive and predictive. Together surveillance and simulation make calculations and judgments about what will happen next, and in doing so shape the coordinates within which institutions and individuals act. Technologies of surveillance and simulation then prompt us to think carefully about what the human experience is, and what the interrelationships are between humans and increasingly predictive and calculative technologies.
In the last post I mentioned the episode Be Right Back from Charlie Brooker’s Black Mirror. A young woman, Martha, gets a robot version of her dead partner Ash. The robot is programmed based on the all the data Ash generated why he was alive. It looks like him, has his gestures, his interests, speaks like him. Martha’s robot version of Ash can both learn to perform as Ash: his language, speech and expressions. For instance, Martha tells the robot what ‘throwing a jeb’ meant in their relationship, and later he uses that expression in context. But, the robot is unable to make its own autonomous decisions. Martha is the robot’s ‘administrator’ and he will do whatever she asks. The robot is missing the nuances of human relationships. It knows how to argue, but not how to fight. The robot cannot be affected. It can’t engage in open-ended, deeply human creativity. It can’t ‘spend time’ with another human. The night before she takes him to a cliff Martha and Ash the robot have this exchange.
Martha: get out, you are not enough of him…
Robot: did I ever hit you?
Martha: of course not, but you might have done.
Robot: I can insult you, there is tons of invective in the archive, I like speaking my mind, I could throw some of that at you.
The robot can manipulate how Martha feels, but it can’t understand her feelings or feel for itself. What do our intimate others know about us that our devices cannot? What can humans know about each other that technologies of surveillance cannot? What we look like when we cry, what we might do but haven’t yet done, how we respond to our intimate others as they change. Martha reaches this impasse with Ash. Ash is the product of surveillance and simulation. He doesn’t just ‘represent’ their relationship, he intervenes in it. He begins to shape Martha’s reality in ways living Ash did not, and now – having passed away – cannot. Martha takes Ash to a cliff.
Robot: Noooo, don’t do it! (joking). Seriously, don’t do it.
Martha: I’m not going to.
Robot: OK
Martha: See he would’ve worked out what was going on. This wouldn’t have ever happened, but if it had, he would’ve worked it out.
Robot: Sorry, hang on that’s a very difficult sentence to process.
Why is it difficult to process? Because as much as its a sensible statement, it is an affective one - it is about how she feels, but also the open-ended nature of human creativity. To consider and imaginethings that are not, or might have been, or could be.
Martha: Jump
Robot: What over there? I never express suicidal thoughts or self harm.
The robot is always rational.
Martha: Yeah well you aren’t you are you?
Robot: That’s another difficult one.
Martha: You’re just a few ripples of you, there is no history to you, you’re just a performance of stuff that he performed without thinking and its not enough.
Robot: C’mon I aim to please.
Martha: Aim to jump, just do it.
Robot: OK if you are absolutely sure.
Martha: See Ash would’ve been scared, he wouldn’t have just leapt off he would’ve been crying
The robot can manipulate how Martha feels, but it can’t understand her feelings or feel for itself. She makes a decision that might surprise us though. Rather than put photos of her daughter’s father in the attic, she puts the robot up there. The daughter visits a simulation of her father each weekend. We can see, I would argue, that Brooker draws us toward thinking of the ambivalent entanglements with these devices. The intimacy and comfort they provide, our dependence on them, the way they unsettle, control and thwart us.
As I thought about this problem, that Brooker brings to a head at the cliff, I thought of John Durham-Peters:
Communication, in the deeper sense of establishing ways to share one’s hours meaningfully with others, is sooner a matter of faith and risk than of technique and method. Why others do not use words as I do or do not feel or see the world as I do is a problem not just in adjusting the transmission and reception of messages, but in orchestrating collective being, in making space in the world for each other. Whatever ‘communication’ might mean, it is more fundamentally a political and ethical problem than a semantic one.
Machines can displace forms of human knowing and doing in the world, but they seem confined to reducing communication to a series of logical procedures, calculations and predictions. What’s left is the human capacity to make space in the world for each other, to exercise will, to desire, to spend time with one another. The relationships between humans and their simulations are complicated, and the logic of simulation cannot encompass or obliterate the human subjective process of representation. What makes the human, in part, is there capacity to use language to spend time and make worlds with each other.
Nicholas Carah and Deborah Thomas
James Vlahos wrote in Wired magazine in July 2017 about his creation of a ‘dad bot’. Vlahos sat down and taped a series of conversations with his dying father about the story of his life. The transcript of these conversations is rich with the stories, thoughts, and expressions of his dad. He begins to ‘dream of creating a Dadbot – a chatbot that emulates… the very real man who is my father. I have already begun gathering the raw material: those 91,970 words that are destined for my bookshelf’. The transcripts are training data. Over months he builds the bot, using PullString, training and testing it to talk like his dad.
He takes the bot to show it to his mother and father, who is now very frail. His mum starts talking with the bot.
I watch the unfolding conversation with a mixture of nervousness and pride. After a few minutes, the discussion ¬segues to my grandfather’s life in Greece. The Dadbot, knowing that it is talking to my mom and not to someone else, reminds her of a trip that she and my dad took to see my grandfather’s village. “Remember that big barbecue dinner they hosted for us at the taverna?” the Dadbot says.
After the conversation, he asks his parents a question.
“This is a leading question, but answer it honestly,” I say, fumbling for words. “Does it give you any comfort, or perhaps none—the idea that whenever it is that you shed this mortal coil, that there is something that can help tell your stories and knows your history?”
My dad looks off. When he answers, he sounds wearier than he did moments before. “I know all of this shit,” he says, dismissing the compendium of facts stored in the Dadbot with a little wave. But he does take comfort in knowing that the Dadbot will share them with others. “My family, particularly. And the grandkids, who won’t know any of this stuff.” He’s got seven of them, including my sons, Jonah and Zeke, all of whom call him Papou, the Greek term for grandfather. “So this is great,” my dad says. “I very much appreciate it.”
Later, after his father as passed away, Vlahos recalls an exchange with his 7 year old son.
‘Now, several weeks after my dad has passed away, Zeke surprises me by asking, “Can we talk to the chatbot?” Confused, I wonder if Zeke wants to hurl elementary school insults at Siri, a favorite pastime of his when he can snatch my phone. “Uh, which chatbot?” I warily ask.
“Oh, Dad,” he says. “The Papou one, of course.” So I hand him the phone.’
The story is strange and beautiful. It provokes us to think about how we become entangled with media technologies, and the ways in which they are enmeshed in our human experience. In this story, not just a father – but a family and their history – is remembered and passed on not with oral stories, or photo albums, or letters but with an artificial intelligence that has been trained to perform someone after they die.
The dadbot is an example of the dynamic relationship between surveillance and simulation. Surveillance is the purposeful observation, collection and analysis of information. Simulation is the process of using data to model, augment, profile, predict or clone. The two ideas are interrelated. Simulations require data and that data is produced via technologies of surveillance. The more data we collect about human life, the more our capacity grows to use that data to train machines who can simulate, augment and intervene in human life.
If the Dadbot is a real experiment, let me offer a speculative fictional one. In the episode Be Right Back of his speculative fiction Black Mirror, Charlie Brooker asks us to think about a similar relationship between humans, technologies and death. Be Right Back features a young couple: Martha and Ash. After Ash’s death, his grieving partner Martha seeks out connection with him. At first the episode raises questions about how media is used to remember the dead. Old photos, old letters, clothes, places you visited together, songs you listened to. A friend suggests Martha log into a service that enables text-based chat with people who have passed away, simulating their writing style from their emails and social media accounts. She does that. It escalates. She uploads voice samples that enable her to chat to him on the phone. She becomes entangled in these conversations. Sometimes the recording spooks her, for instance when she catches it ‘googling’ answers to questions she knows Ash wouldn’t know. A new product becomes available, a robot whose draws on photographs and videos of Ash while he was alive. It arrives. She activates it in the bath. The robot is much better in bed than Ash ever was. Things get complex. Martha goes looking for the gap between the robot and the human.
Vlahos’ Dadbot and the robot in Be Right Back are both examples of the interplay between surveillance and simulation. Each of them illustrate how the capacity to ‘simulate’ the human depends in the first case on purposefully collecting data. Data is required to train the simulation.
In his 1996 book, The Simulation of Surveillance: Hypercontrol in Telematic Societies, William Bogard (1996) carefully illustrates the history of this relationship between simulation and surveillance. He proposes that over the past 150 years our societies have undergone a ‘revolution in technological systems of control’. That is, our societies have developed increasingly complex machines for controlling information, and using information to organise human life. One of the key characteristics of the industrial societies that emerged in the 1800s was the creation of bureaucratic control of information. Bureaucracies were machines for gathering and storing information using libraries, depositories, archives, books, forms, and accounts. They processed that information in standardised ways through the use of handbooks, procedures, laws, policies, rules, standards and models. Since World War II these bureaucratic processes have become ‘vastly magnified’ via computerisation. Bureaucracies rely on surveillance. They collect information in order to monitor people, populations and processes. Think of the way a workplace, school or prison ‘watches over’ its employees, students, or prisoners in order to control and direct what they do.
Bogard argues that increased computerisation has resulted in surveillance becoming coupled with processes of simulation. Remember, surveillance is the purposeful observation, collection, and analysis of information, while simulation is the process of modelling processes in order to profile, predict or clone. Inspired by the French theorist of surveillance Michel Foucault, Bogard suggests that surveillance operates as a ‘fantasy of power’ which in the post-industrial world ‘extends to the creation of virtual forms of control within virtual societies’. What’s a ‘fantasy of power’ here? Well, firstly, it is a kind of social imaginary, a set of techniques through which individuals ‘internalise’ the process of surveillance. They learn to watch over themselves, they learn to perform the procedures of surveillance on themselves, in advance of technologies themselves performing those techniques. Let me give a very simple example. You might go to search something on Google, and then stop because you think ‘Hmm, Google is watching me…’ I don’t want it to know I searched that. You discipline yourself, pre-empting the disciplinary power of the technology.
But, secondly, a fantasy of power gestures at something else important too. It suggests a society where we come to act as if we believe in the predictive capacity of surveillance machines. That is, in practice we trust the capacity of bureaucratic and technical machines to watch over and manage social life. We trust machines to reliably extend human capacities. By the 1990s, the socio-technical process of simulation had become an ordinary part of many social institutions. For instance, computerized ‘experts’ increasingly assist doctors in making complex medical diagnoses, algorithmic models help prisons determine which prisoners should be eligible for parole, statistical modelling projects the need for public infrastructure like roads and schools, satellite surveillance informs foreign policy decisions.
The ‘fantasy’ driving government, military and corporate planning is that the capacity of digital machines to collect and process data can extend their capacity to exercise control beyond what humans alone might accomplish. Across government, corporate and military domains in the post-war period ‘simulations’ became standard exercises. Simulations are used by engineers to project design flaws and tolerances in proposed buildings. For instance, to test whether a building could withstand an earthquake before that building is even built. They are used by ecologists to model environments and ecosystems, by educators as pedagogical tools, by the military to train pilots and by meteorologists to predict the weather. In each of these examples data is fed into a machine which predicts the likelihood of events in the future. Corporations increasingly base investment decisions on market simulations and more recently, nanoscientists have devised miniaturised machines that can be used in areas as diverse as the creation of synthetic molecules for cancer therapy to the production of electronic circuits.
Bogard calls these ‘telematic societies’ driven by the fantasy that they can ‘solve the problem of perceptual control at a distance through technologies for cutting the time transmission of information to zero.’ That is, these societies operate as if all they need to do is create technologies that can watch and respond to populations in real time. In these societies powerful groups invest in creating a seamless link between ‘surveillance’ and ‘response’, between collecting data, processing it and acting on it.
Bogard’s original insight then is to identify – in the mid 1990s no less – that we are becoming societies where ‘technologies of supervision’ that collect information about human lives and environments are connected to ‘technologies of simulation’ that predict and respond in real-time. That’s a wonderfully evocative insight to think about in the era of the FANGs. Facebook, Amazon, Netflix and Google are each corporations whose basic strategic logic, and engineering culture is organised around the creation of new configurations of technologies of supervision, data collection, and technologies of simulation, prediction, response and augmentation.
Nicholas Carah and Deborah Thomas
everyday infrastructure.
This gets called the ‘internet of things’.
Watches, Televisions, Cars, Fridges, Kettles, Air Conditioners, Home Stereos are just some of the everyday objects that are getting ‘connected to the internet’.
That’s not a hypothetical story. Facebook actually did that in 2014, to 689000 users. They changed the ‘mood’ of their News Feeds. Some people got happier feeds, some got sadder feeds. They wanted to see if they ‘tweaked’ your feed sad, if you would get sad.
This is Jose van Dijck’s definition of a platform. ‘The providers of software, (sometimes) hardware, and services that help code social activities into a computational architecture; they process (meta)data through algorithms and formatted protocols before presenting their interpreted logic in the form of user-friendly interfaces with default settings that reflect the platform owner’s strategic choices.’
I type ‘platform’ into Google. Ask a platform what a platform is. Google suggests a nearby bar, a train station, a Wikipedia entry.
This is Carl Schmitt writing in 1918 about a fictional civilisation, the Buribunks. Every person has a personal typewriter.
Every Buribunk, regardless of sex, is obligated to keep a diary on every second of his or her life. These diaries are handed over on a daily basis and collated by district. A screening is done according to both a subject and a personal index. Then, while rigidly enforcing copyright for each individual entry, all entries of an erotic, demonic, satiric, political, and so on nature are subsumed accordingly, and writers are catalogued by district. Thanks to a precise system, the organisation of these entries in a card catalogue allows for immediate identification of relevant persons and their circumstances. …the diaries are presented in monthly reports to the chief of the Buribunk Department, who can in this manner continuously supervise the psychological evolution of his province and report to a central agency.
The Buribunks use the typewriter to reflect on themselves, the information they record is archived in a database, where it is analysed by officials. They use the information to both monitor the thoughts of individuals, the mood of particular regions, and as a kind of market research to create entertainment and culture that reflects the interests of Buribunk citizens. This story just flattens me.
Here is Schmitt in 1918, looking at the typewriter. He sees a device that standardises written script, which enables vast amounts of information to be created, stored and processed. Schmitt sees in the typewriter the beginning of a civilisation where everyday life is extensively recorded. In 1918, Schmitt sees not just the smartphone, the wearable, the social media platform but also the kind of personhood and society that would go along with it. Here’s a critical point in his story: Buribunks are very liberal, they can write whatever they like in their diary. They can even write about how they hate being made to write a diary. But, they cannot not write in the diary. So, you can say whatever you like, but you cannot say nothing. You must make your thoughts, movements, moods and feelings visible to a larger technocultural system. Schmitt here envisions a mode of social control that doesn’t depend on limiting the specific ideas people express, but rather works by making their ideas visible so that they can be worked on and modulated.
I find this aspect of Buribunkdom startling, not because Schmitt is the only one to articulate a mode of control like this. Of course other critical thinkers in the twentieth century have too: Foucault, Deleuze, and Zizek to name some. I find it interesting because here in 1918, we have someone seeing personal media devices operating to manage the processes of representation and reconnaissance. That media technology was understood here as both instruments for symbolic communication and for data collection. So, here we are one hundred years later and we are the Buribunks. We use our smartphones every day to record reams of information about our lived experience: our expressions, preferences, conversations, movements, mood, sleep patterns and so on. This information is catalogued in enormous commercial and state databases. The information is used to shape the culture that we are immersed in. And, importantly, this system works by granting us individual freedom to express ourselves, and places relatively few limits on what we can say. But, this system does demand our participation. Participation is a forced choice. Very few of us successfully navigate everyday life without leaving behind data about our movements, preferences, habits and so on.
Schmitt imagined a large government bureaucracy where information would be stored on index cards. It was a kind of vast analogue database. Of course, instead of this, we have a complex network of digital databases owned by major platforms: Facebook, Google, Amazon and Netflix. And, these database function as enormous market research engines that capture and process information which is used to shape our cultural world. What Schmitt saw in the typewriter, has congealed in the smartphone, the critical device in a culture organised around the project of the self. The work of reflecting on and expressing the self, as a basis task in everyday life. And, super importantly, these tasks are shaped by the tools we use to accomplish them.
Here is a famous line from Nietzsche about his typewriter, which he experimented with in the late nineteenth century: ‘Our writing tools are also working on our thoughts’. What did he mean? As we use media technologies to reflect on and express ourselves, we become entangled with them. They shape the way we think, act and express ourselves. They shape the way we imagine the possibilities of expression, and we might say that in our own minds we begin to think like typewriters, or films, or smartphones. We think using their grammar, rhythms and conventions. So, with the typewriter and the smartphone, we might say that these devices ‘work on us’ in the sense that they facilitate a process through which we ‘monitor’ and record data about ourselves.
OK, so I’ve suggested here that in Schmitt’s early twentieth century we can see the pre-history of the smartphone. Well, Kate Crawford and her colleagues actually offer us a study of this history. They trace the genealogy of devices and practices we use to weigh ourselves since the 19th century through to present day self-trackers like FitBits. Think about how FitBit talks to us in its advertisements. The FitBit is presented as a radically new technology offering precise information about the ‘real’ state of our bodies. This knowledge will be useful to us, it will make us fitter, happier, more desirable and more productive.
What Crawford and co. remind us is that this set of claims are not all that new. Devices that ‘work on’ or shape our thoughts and feelings about our bodies have been around a long time.
Weight scales are one example. From the 19th century onwards both the cultural uses and technical capacities of weight scales have changed. In cultural terms, weight scales shifted from the doctor’s office, to the street, to the home. They gradually changed from a specialist medical device used only by doctors, to public entertainment, to a private everyday domestic discipline.
So, here’s a run through of Crawford’s narrative. Doctors began monitoring and recording patients’ weight toward the latter end of the 19th century, but this was not routine until the 20th century. In 1885, the public ‘penny scale’ was invented in Germany, which then appeared in the US in grocery and drug stores. Modelled after the grandfather clock, with a large dial, the customer stepped on the weighing plate and placed a penny in the slot.
Some penny scales rang a bell when the weight was displayed, while others played popular songs like ‘The Anvil Chorus’ or ‘Oh Promise Me’. The machines would also dispense offerings to lure people into weighing themselves in public, such as pictures of movie stars, horoscopes, and gum and candy. Built in games such as Guess-Your-Weight would return your penny if you accurately placed the pointer at your weight before measurement. However, the extraction of money in exchange for data was the prime aim of the manufacturers; ‘It’s like tapping a gold mine’, claimed the Mills Novelty Company brochure in 1932.
The domestic weight scale first appeared in 1913. A smaller, more affordable device for the home, it allowed self-measurement in private to offset the embarrassment of public recording one’s weight with attendant noises and songs. The original weight scale is an analogue or technical form of media - our body weight makes an impression on a mechanism that is calibrated to record it on the scale. As a media device it collects and presents information to us but it is also important to consider how it is configured in broader social and identity-making processes. There is a gendered history of these devices.
Public weight scales were initially marketed to men but in the1920s women started to be encouraged to diet. Weight scales were presented to women as a private bathroom device to monitor their bodies, thus becoming a tool to ‘know’ and ‘manage’ ourselves. Here’s Crawford’s account of this:
Tracking one’s weight via the bathroom scale was not only about weight management - as early as the 1890s it assumed a form of self-knowledge. This continues today where value and self-worth can be attached to the number of pounds weighed.
Crawford refers to a study, where a participant in an eating disorders group was asked how she feels if she does not weigh herself; ‘I don’t feel any way until I know the number on the scale. The numbers tell me how to feel’. That’s basically Nietzsche claim about the typewriter – the device is working on my thoughts. The numbers tell me how I feel. Similar claims are made around self-tracking devices. There are accounts of self-tracking and internalized surveillance taken to an extreme by people suffering from eating disorders.
So, the history of the weight scale reminds us that tracking devices are agents in shifting the process of knowing and controlling bodies, both individually and collectively, as they normalize and sometimes antagonize human bodies. The Fitbit turns the body’s movement into digital data: daily steps, distance travelled, calories burned, sleep quality, and so on. This is then fed into a ‘finely tuned algorithm’ that looks for ‘motion patterns’. There are two things at work here in this sequence from the personal weight scale to the FitBit. One, a moral epistemology: knowing one’s weight and body habits can lead to an improved, possibly ideal self and life. And, two: an economic imperative. Penny scales were significant money making enterprises and there was a strong profit motive in encouraging people to weigh themselves ‘often’. This exchange of money for data is clear: spend a penny, receive a datum, but the collection of data is also private, going no further unless the user willingly shared it with others. This is less clear in trackable devices. The user can reflect on their own data but that data will always be shared with the device maker and a range of unknown parties. What is then done with that data is not transparent and ultimately at the discretion of the company. Consumer data are mediated by a smartphone app or an online interface and the user never sees how their data is aggregated, analysed, sold, or repurposed, nor do they get to make active decisions about how that data is used.
As a tagline for an advertisement, for the wearable Microsoft Band, states, ‘this device can know me better than I know myself, and can help me be a better human.’ So then, Crawford argues, ‘the wearable and the weight scale offer the promise of agency through mediated self-knowledge, within rhetorics of normative control and becoming one’s best self.’ On one hand the ability to ‘know more through data’ can be experienced as pleasurable and powerful, the promise of which is evident in this advertisement for Microsoft band.
OK, and on and on it goes. Ugh, corporate brand vomit. But, also here’s the basic claim Microsoft are making: buy this device, it will work on you! It will change you. What wearables like the FitBit achieve that the personal weight scale could not, is the real-time aggregation of data about all bodies, and the feeding back of this information to each users via customised screens. Again here, Schmitt’s Buribunks had paper index cards and human-scale analysis of expressions. The FitBit is real-time biological analysis of millions of bodies. Here’s Crawford:
‘Statistical comparisons between bodies are necessarily contingent on a set of data points. Users get a personalized report, yet, the system around them is designed for mass collection and analysis, so the user becomes ‘a body amidst other tracked bodies.’ So ‘the user only gets to see their individual behaviour compared to a norm, a speck in the larger sea of data.’
Drawing on the work of Julie E Cohen, Crawford argues that this functions as a ‘bio-political public domain’… designed to ‘assimilate individual data profiles within larger patterns and nudge individual choices and preferences in directions that align with those patterns.’ So ‘while there is a strong rhetoric of participation and inclusion, there is a ‘near-complete lack of transparency regarding algorithms, outputs and uses of personal information’. And, this is the crucial point. Mark Andrejevic calls this the ‘big data divide’. The difference between individuals who record their data, and the corporations who collect and process that data.
The lesson then is to think about the evolution of media devices for collecting, storing, processing, and disseminating information over a hundred year period, as well as the individual and social facets of digital media.
The FitBit and similar tracking devices that collect data about us and present that back to us as customised and individualised media content, become part of a much larger social system of control in several ways. The data that we give and view at an individual level is logged in databases that operate at population level. These devices are implicated in a cultural process based on self-monitoring and self-improvement. They work on our thoughts. And, importantly, these devices normalise data-driven participation and computation in our everyday lives. They become a foundational model for how we do our lives, bodies and identities.
On the night of October 15, 1940, the German air force sent 236 bombers to London. ‘British defences were dismal’. They ‘managed to destroy only two planes’. London, the heart of the British Empire, was under siege.
In Rise of the Machines Thomas Rid explains this moment’s historical significance. ‘For the first time in history, one state had taken an entire military campaign to the skies to break another state’s will to resist’. Survival for Britain, and the Allies, would depend on their ability to engineer a way of shooting those German bombers out of the sky.
This problem triggered a ‘veritable explosion of scientific and industrial research’ which would result in ‘new thinking machines’ capable of making ‘autonomous decisions of life and death.
Rid puts it this way:
Engineers often used duck shooting to explain the challenge of anticipating the position of a target. The experienced hunter sees the flying duck, his eyes send the visual information through nerves to the brain, the hunter’s brain computes the appropriate position for the rifle, and his arms adjust the rifle’s position, even ‘leading’ the target by predicting the duck’s flight path. The split-second process ends with a trigger pull. The shooter’s movements mimic an engineered system: the hunter is network, computer, and actuator in one. Replace the bird with a faraway and fast enemy aircraft and the hunter with an antiaircraft battery, and doing the work of eyes, brain, and arms becomes a major engineering challenge.
This challenge involved configuring the interplay between human and machine (to allow, for instance, a human operator to move an enormous weapon precisely at speed), engineering radar that could detect a plan before a human eye could see it, creating a network that would relay information from a radar to a computational device, building a computer that could predict the path of an enemy plane through the sky and predict where to fire, constructing the apparatus that could transfer the prediction into the operation of a weapon in real-time.
What was going on here? The creation, by a large network of scientists and engineers within a military-industrial system, of machines that could sense, learn, calculate and predict. Machines that could exert control over the material world via ongoing cycles of feedback and learning. In June 1944, the Germans launched V1 rockets across the English channel toward London. The V-1 was ‘a terrifying new German weapon: an entire unmanned aircraft that would dive into its target, not simply drop a bomb’. The first cruise missile.
At this moment, Rid explains
A shrewd, high-tech system lay in wait on the other side o the English Channel, ready to intercept the robot intruders. As the low-flying buzz bombs cruised over the rough waves of the Atlantic coast, invisible and much faster pulses of microwaves gently touched each drone’s skin, 1707 times per second. These microwaves set in motion a complex feedback loop that would rip many of the approaching unmanned flying vehicles out of the sky.
The Allies had engineered a ‘cybernetic’ system. A combination of technological devices that could sense, calculate, predict and execute decisions. These devices included the primitive digital computers.
Following the war, the mathematician Norbert Wiener was a key figure in popularising the idea of ‘cybernetics’. There are three critical concepts in ‘cybernetics’: feedback, learning and control. Cybernetics comes from a Greek word which means ‘to steer’. It articulates a process of exercising control by learning from feedback. A key feature of humans is that we can learn and adjust by using our senses and decision-making capacities. Cybernetics was the effort to construct ‘intelligent machines’ that could also learn. Wiener would often imply that he was central to solving the ‘prediction’ problem during the war.
It is true that Wiener was one of many scientists funded to undertake experiments, and Wiener did propose a mathematical model for predicting the path of an enemy aircraft. He did not however ‘solve’ the prediction problem, his model didn’t work. The lesson here is that complex technological systems are the result of a network of actors. There is rarely any one individual genius who ‘invents’ them. Jennifer Light makes this point emphatically in her study ‘When Computers Were Women’, explaining that while two male engineers often credited with automated ballistics computations during the war, critical to the effort were ‘nearly 200 young women’ who worked as ‘human ‘computers’, performing ballistics calculations during the war’. The first computers were hundreds of female mathematicians.
In 1948, Wiener coined and popularised the term ‘cybernetics’ as the science of ‘control and communication in the animal and the machine’. In short, cybernetics views the state of any system – biological, informational, economic, and political – in terms of the regulation of information. A cybernetic device can sense or collect information, and be programmed to respond to that information. In the case of wartime anti-aircraft defence, a radar detects movement, it tracks an enemy plane across the sky. Information is relayed to a primitive computer, which calculates aircraft trajectory. This calculation is passed on to an anti-aircraft weapon, which fires at the enemy aircraft.
Wiener is a significant figure in the story of cybernetics because he articulated how these computational technologies would reshape industry, society and culture. In his 1950 book, The Human Use of Human Beings: Cybernetics and Society, Wiener made an important historical move by placing ‘cybernetics’ at the heart of what he called the second industrial revolution.
The first industrial revolution bought about new forms of energy, such as steam and electricity created by machines. Harnessing these energy sources enabled the production of a goods on a scale far beyond what humans on their own could make. Wiener claims that in the first industrial revolution the machine acted as an ‘alternative to human muscle’. For example, one of the first applications of the steam engine was to pump water out of mines, a job that had previously been done by humans and animals. Many changes resulted from replacing human muscle with machines – factories emerged, urban labour forces created mass cities, and the demand for raw materials stimulated the growth of plantations and mines in the colonies, and hence rail and shipping networks for transportation.
For Wiener, machines ‘stimulated’ the development of an entire industrial and social system. In the second industrial revolution a new kind of a new kind of machine emerged - the computer which extended machines to the idea of communication. This is how he put it, ‘for a large range of computation work, [machines] have shown themselves much faster and more accurate than the human computer’. Wiener thought that computers would eventually communicate with and modulate a range of instruments. These instruments would act as ‘sense organs’. They would feed information back to the computer, so that it could make decisions and learn about its environment. Computers in factories would be programmed to generate and collect data to give feedback on production processes.
Thomas Rid makes the point that ‘Wiener didn’t change what the military did, the military changed what Wiener did’. What does mean? He means that Wiener’s peripheral involvement in wartime efforts to create machines that could sense, calculate, predict and execute decisions led him to perceive the development of a new kind of society. A society organised around devices and systems that were cybernetic – able to control their environment through processes of feedback and learning. Able to make collect, store and process information in ways that were once confined to the human.
Wiener and the other mid-century engineers, scientists and thinkers involved in the development of cybernetics imagined how media technologies would usher in complicated relationships between human forms of sense-making and decision-making and the capacity of computational devices to simulate, augment and even exceed those human capacities.
Media are technologies that organise human life and experience. They symbolically represent reality and they also collect information about reality.
How did they come to do this? First up, we often think of digital media as ‘new’. We register this most clearly in the advertising and corporate rhetoric of technology companies. Go and trace the history of Apple advertisements and product launches from their Macintosh personal computer in 1984 through to their iPod and iPhone launches. Listen to Mark Zuckerberg from Facebook when he tells us about the artificial intelligence he built, named Jarvis, that runs his house. Or, Facebook engineers when they tell us that they want to build a brain machine interface that will enable us to type from our brain. Or, Jeff Bezos from Amazon when he tells us that his AI Alexa will run our homes by listening in to our conversations.
Over and over the digital media industries present their technologies within a narrative of straightforward, linear progress. The next technology we build will be better than the last. And, implied in that sense of better, is what we might call a ‘technological imaginary’.
If we build all the cool gadgets, all the human problems will go away!
Here, I think of John Durham-Peters’ in Speaking into the Air, ‘’Communication’ whatever it might mean, is not a matter of improved wiring or freer self-disclosure but involves a permanent kink in the human condition. That we can never communicate like angels is a tragic fact, but also a blessed one.’
The kinks of human experience cannot be solved with technologies. And, new technologies are not ‘better’ than the last ones. As in, they don’t automatically make for a ‘better’ human experience. One way we can think about media technologies then is how they emerge out of the experimental effort of humans to exercise power in the world. This is not a straight-forward process.
That means we should listen carefully to Apple, Facebook and Google when they tell us what they are experimenting with, and where they think they are headed – not because this enable us to ‘narrate’ the development of technology, but because it offers us a way of thinking carefully about the kind of human experience they are imagining and creating.
With that in mind, let’s turn back to Kittler who takes this ‘genealogical’ approach to a history of media technology. Genealogy is a method inspired by Nietzsche and Foucault, a way of doing history that pays attention to how material technologies emerge as part of historically-conditioned discourses, social formations and modes of power.
Kittler identifies three historically significant media systems.
SymbolicThe first is a symbolic media system. In this system, writing, physical speech sounds, or musical tones are transposed by a human into visual symbols, which are then re-translated by users into a sound, word, or idea in the mind. Think the alphabet, musical notation, or paintings and drawings. These systems work because the human users create and follow rules. Alphabets and musical notation have technical specifications that the users have to follow if they are going to work. For example, the alphabet is a media system, with visible symbols and rules for how sounds in speech are to be captured, stored, and processed. This symbolic media system dominated until about 1900 and allowed for the development of new forms of social, cultural, and political life.
TechnicalA second system, technical media, emerged during the 19th century and into the 20th century. While writing transposed physical sensations into symbols, technical media could capture physical sensations directly as impressions on a medium. The difference between symbolic and technical is crucial here. With a symbolic system the physical sensation – sound or light – has to pass through the human body to be transposed into a symbol. The human ear hears a word, and transposes it into letter from an alphabet. With a technical system that physical sensation is recorded directly as an impression in another medium, without the human body having to turn it into a symbol. Photographs, which capture light, and phonographs, which capture sound, are the key technologies. Both emerge in the 19th century, and become mass technologies in the early 29th century. Photography is a process where film records a physical impression of light on a media. Phonography records the physical impression of sound on a record or tape. Those impressions can then be ‘converted’ back via the medium into an accurate representation of the original image or sound.
The first phase of the age of technical media was the capacity to ‘capture’ and ‘store’ images and sound, while the second phase was the transmission of those images and sounds over distance, via radio and later, television. This system is analogue. Think about a vinyl LP where the physical grooves in the record are ‘impressions’ of a sound that are read and converted into an audio signal you can hear via the technical device of the record player. Analogue devices, such as record players and tapes, read the media by scanning the physical data off the device.
DigitalBy the end of the twentieth century, the age of technical media gave way to our present epoch - the age of digital media. Rather than record data as a ‘physical’ trace, a digital system converts all data into a numerical system. The really important point Kittler makes here is that the digital system collapses all ‘senses’ into one medium. This enables media to calculate, process, and simulate. In the mid-1980s, Kittler predicts that sooner or later we will be hooked into an information channel that can be used for any medium. Movies, music, phone calls, and mail will reach households via fibre optic cables. Once any medium can be translated into 1s and 0s, and passed through the one infrastructure of digital computers and networks, the capacity of media to experiment with reality dramatically explodes in scale.
With digital media the physical properties of the input data are converted into numbers. Media processes are coded into the symbolic realm of mathematics, which can then be ‘immediately subjected to the mathematical processes of addition, subtraction, multiplication, and division through algorithms contained within software.’
Think of the present moment.
Our bodies permanently tethered to, and integrated with, digital devices like smartphones. These devices convert human experience into data. They store, they calculate, they predict as much as they represent. Our imagination is entangled with the data-driven, algorithmic, flows of images, sounds and texts streaming via their screens. This genealogy of this kind of human experience can be traced back, at least, to the mid-nineteenth century.
In the mid-1800s the technologies for storing reality emerge. The phonograph stores sound, the photograph stores light, the typewriter standardises the storage of alphabet, numbers and code. From the early 1900s, the technologies for electronic transmission of sound, light and code over distance emerges in the form of telegraph, radio and television. In the mid-1900s, around the schema of the typewriter, the capacity of media to calculate and predict emerges. For Kittler, Turing’s mathematical definition of computability, and the codebreaker he built during World War II mark the moment where media become first and foremost calculative devices for intervening in reality.
Coupling an aircraft with a camera enabled armies to view territory from the sky, to disclose invisible soldiers, camouflaged artillery positions, and unnoticed rearward connections to the enemy.
In 2017, Harvard scientists encoded a moving image gif of Muybridge’s horse experiment into the DNA of a living cell. Where, as The New York Times explains, ‘it can be retrieved at will and multiplied indefinitely as the host divides and grows. The advance, reported on Wednesday in the journal Nature by researchers at Harvard Medical School, is the latest and perhaps most astonishing example of the genome’s potential as a vast storage device.’
I have no idea how many times I’ve already bounced the Twitter feed this morning. Wandering around the house, the garden, making coffee, putting the washing on. How many tweets have I seen today already? Twitter would know. My guess is several hundred, at least. The news cycle of Saturday morning used to be slow. Different to the rest of the week. I’d often walk to the newsagent and buy a newspaper. The newsagent was a mate, we’d chat for a bit. I’d come home. Make a coffee. Sit down and read it. Maybe a couple of hours spent with it over the course of the day.
At the Splendour in the Grass music festival in 2013 the festival wristbands had RFID tags in them. You could link your tag to your Facebook. Then, as you wandered about the festival going to see bands play, you could ‘swipe’ your wrist at these sensor stations to record your movements around the site. The festival said if you did this they would send you a customised playlist of every band you saw after the festival.
The advertising model shows us how the audience do two kinds of labour. First, by allowing the platform to monitor them they do the work of being watched, of providing the data that enables advertisers to target them. Second, by using the platform and viewing advertiser’s posts in their news feeds they do the work of watching, of paying attention to advertiser’s messages.
The ‘shows’ you see are on the TV screen are not really the product. He says the real answer to the question ‘what do media make?’ is ‘audiences’.
Facebook launched as a public company in 2012, immediately after its launch the stock price sunk.
Facebook had a big problem and the market weren’t convinced Facebook could solve it.
Users were starting to ‘go mobile’. To access Facebook predominantly through their smartphones.
That seems strange now, but remember for the first half a decade or so Facebook was a social networking site that people nearly exclusively used via a web browser.
This shift to mobile was a big problem because Facebook had no tools for generating revenue from mobile users. It’s revenue came from web-based advertising. Zuckerberg and Facebook put together a team whose job it was to take the platform mobile. They had to figure out how to make Facebook a ‘mobile-first’ company in a profitable way. Until about 2013 many observers were not sure that Facebook could make this transition. There was the possibility that some other natively mobile platform might rise up in their place.
That’s not what happened, Facebook have clearly transitioned to mobile successfully. How did they do it?
Let’s chart this story, not just because it tells us some important things about the becoming mobile of media. But also because it illustrates how media platforms operate as engineering projects. They are never static or stable, they are always in the process of imagining and inventing the next version of their infrastructure.
This makes them radically difference from the media institutions of the twentieth century. For all the ways that television came to saturate everyday life in mass societies, as a technological form it changed very little. It remained, more or less, a box sitting in the home that you turned on or off, selected from a number of channels, and viewed professionally-programmed content.
In the ten to fifteen years that media platforms like Facebook have been with us they have re-engineered their infrastructure in significant ways multiple times.
The first important development in this story about Facebook engineering itself as an algorithmic and mobile platform arguably happens in 2006 when Facebook launched the News Feed.
Try to imagine Facebook without a News Feed. Maybe you were never even a user before the feed. Back then, it was more like MySpace, or an online dating site, in that individual user profiles were the organising point of the network. When you logged on you were placed at your personal profile page, and went from user profile to profile. There was no feed that aggregated recent content produced by everyone in your network. You only knew if a friend had posted some new photos if you went and checked their actual profile.
Facebook realised this was a problem because they couldn’t manage user engagement with the site. This also made serving advertising content difficult. MySpace ran into this issue too, where advertising began to clutter profile pages or appear as interstitials between page loading – disrupting the user experience. This problem killed off MySpace. Facebook thought if they could develop a feed of content they curated, and then refine it over time, they could learn to serve users content that would keep them engaged with the platform more often and for longer periods.
Rather than us navigating our way randomly around the network, Facebook would use the News Feed to predict and program our engagement with it. Users were initially not happy about the new feed. The News Feed has been at the heart of the Facebook experience for ten years now. That means it has ten years of data about users it can use to shape what they see.
The News Feed gave Facebook greater control over user engagement, which was crucial, and it would also prove to be critical to solving the problem with their mobile advertising revenue.
I wonder how many hours of scrolling in the feed it has on me? Facebook says it captures, on average, 50 minutes of our attention each day. Over ten years that’s more than 3000 hours of time Facebook has monitored our habits, preferences and interests.
The News Feed began to increase engagement with the platform, but Facebook’s value was still limited by the fact that they had very low click through rates on the advertisements that appeared on the right hand side of the browser interface. While Facebook had the capacity to customise the targeting of ads to individual users based on data it collected about them, the value of this was decreasing because users never clicked on the ads.
Once it became a public company, Facebook's advertising model came under significant scrutiny.
'Here are are Facebook's 7 biggest advertising problems' reported Business Insider.
The Atlantic wrote 'Suddenly, Facebook's Advertising Problem is a Problem'.
Advertisers didn’t really value these ads as a media property because it couldn’t offer strong user engagement. And, the problem was made worse because these ads were not visible on the mobile app at all. At the start of 2009, Facebook had 20 million users on their mobile platform. By 2010, they had 100 million mobile users. There was rapid growth in mobile use with the penetration of smartphones and large mobile data plans. Early versions of the mobile app had fewer features than the desktop version and the app was notoriously slow to load and scroll. Still users kept going to the mobile app for convenience over the desktop site and Facebook was forced to catch up.
By 2011, 430 million users were mobile, making up 50% of daily engagement with the platform. So, before Facebook even launched as a public company, mobile had become a critical strategic issue. They needed to figure out how to make the mobile app work seamlessly and how to get most of their revenue from it.
When Facebook went public in 2012 its user base was 60% mobile but they could make no revenue from any of this mobile engagement. Think about that, in 2012 Facebook could not generate a single dollar from 60% of their user base. Facebook was also under threat form emerging mobile-first platforms like Instagram. When they bought Instagram for $1 billion in 2012, they were making a strategic play. Instagram had been siphoning off users from Facebook, and they needed them back. But, Instagram brought another version of the mobile problem. It had no advertising model and therefore made no money.
Both Facebook and its new acquisition Instagram had to work out how to generate value from their mobile apps. The answer was to develop a native advertising model, something that had never been done at scale before. A native model weaves paid advertising into all other content on the platform. Rather than have the ads appear separately on the side, they would flow through the News Feed or Home feed along with everything else. Facebook had to figure out a way to integrate advertisers’ content into the everyday flow of the News Feed.
In 2012 Facebook launched promoted posts in the news feed, and over the past several years these have become the backbone of Facebook’s revenue. Today more than 90% of Facebook advertising revenue comes from native content integrated into the News Feed. Creating the News Feed was the first step in ‘natively’ integrating advertiser content into the platform.
The news feed uses algorithms and data analysis to determine the right ‘balance’ between selling the attention of users to advertisers and maintaining their ongoing engagement with the platform. Too much irrelevant paid content and you’ll stop using Facebook. Not enough and the platform doesn’t make enough revenue. Facebook finds the right balance for each user via complex data analytics.
Once this native model was working revenue started to flow from the mobile user base and Facebook’s share price started to recover. In solving this major strategic problem Facebook also dramatically reshaped the whole media system: it invented a model of native advertising that worked at scale. By 2013, Facebook’s user base was 80% mobile. Users were using Facebook and Instagram more often in more places. This offered the platforms more points of data and opportunities for engagement.
The platforms could ‘auction’ an expanding array of moments from our everyday life to advertisers, increasingly based not just on who we are, but also where we are, who we are with and what we are doing. The becoming of mobile media is associated with the intensification of the amount of our daily lives which can be made visible to media platforms. By 2013 the business press started to acknowledge that Facebook solved their mobile problem, and therefore established themselves as a durable platform in the media landscape. By 2014 50% of Facebook's revenue came from mobile. By 2016 Facebook was 90% mobile and 85% of its revenue came via promoted posts in the mobile app.
Facebook’s re-engineering of its platform does not stop. The platform has begun to experiment with integrating shopping into Pages, Profiles and News Feeds. At the 2017 F8 developer conference Mark Zuckerberg presented a vision of what Facebook looks like after the smartphone.
The answer, he says, is augmented reality. Living in a world where our immediate view of reality is overlaid with digital simulations.
You look into your bedroom and see what your bed would look like with a new bedspread you are thinking of buying.
You walk past a bar and see in the window a video and review from a friend when they went there.
You have a packet of muesli on the kitchen bench while you are having breakfast, you see a person leap out of the package and stand on your kitchen bench staging a demonstration of the farm when your muesli is made.
Facebook are imagining forms of consumption that are even more natively woven into our experience.
The launch of the first Apple iPhone will stand the test of time as a significant historical event. It marks the transition from the television to the smartphone as the organising device of the media system. During the product launch Apple and Google stage a demonstration of the Google maps app. Jobs searches for the nearest Starbucks and it shows it to him on the map together with the shortest route. This is kind of banal now, a part of everyday life in the city. But in 2007 this was truly staggering, you can see and hear the audiences are amazed.
I worked for a phone company in the early noughties who were trying to build one of these ‘internet phones’. To many in the company it sounded like science fiction. Apple surprised the telecommunication world, but perhaps it isn’t so surprising that it was a computer company rather than a telecommunication company that invented the ‘killer device’. The telecommunications industries had been trying to invent an ‘internet phone’ for a decade, but they somehow always ended up looking too much like phones.
Social media’s popularity depended on the invention of the smartphone, for the social-ness of media to intensify the media infrastructure had to be embedded within our everyday lives and practices. The smartphone was a critical piece of infrastructure in building the 'culture of connectivity' of media platforms. Media platforms signal a dramatic change in the business model of the culture industry. The cultural and media industries are still primarily in the business of producing content and audience attention that they sell to advertisers. But, the flows of attention and associated revenue have shifted dramatically.
For much of the twentieth century ‘rivers of gold’ in the form of advertising revenue flows into media institutions like newspapers and television stations. These rivers of gold funded ‘quality’ content: investigative journalism and local television drama, as just two examples. Those rivers of gold no longer flow through these mass media institutions, instead they flow into media platforms like Google and Facebook. These media platforms do not invest their profits in quality content however, but rather in media engineering projects – augmented and virtual reality technologies, logistical extensions like driverless cars and machine learning. As they do, the role that media plays in the organisation of everyday life extends beyond simply shaping webs of meaning.
Media are no longer just a tool for managing the social process of representation, they also act as infrastructure for organising the logistics of everyday life.
In the past decade our public culture and media system has been dramatically disrupted by the emergence of major media platforms.
The key principle of writing for audio is write like you speakThis post sets out some principles for writing for audio.
An audio script should be sharply written, engaging and thought-provoking. An educated member of the general public should be able to understand it and find it interesting.
For this post, I’m talking specifically to a task where you turn a walkthrough and vignette into a provocative first-person analysis of your engagement with media.
A script should prompt the listener to think critically about contemporary media technologies, their uses, and impacts on society, culture or politics. In this case, your script should evocatively describe and critically reflect on your media use and draw on key debates in the field of media, communication and cultural studies.
Below I set out some principles of writing for audio together with some effective examples from radio programs and podcasts that offer provocative critical reflections on our contemporary media culture.
Principles of writing for audioIn this section I set out some principles and tips for writing from audio. These links and suggestions are drawn from the National Public Radio training website, I've provided links to these sources below.
Basic components of an audio scriptA script for audio, radio or podcasts contains two basic elements: ‘tracks’ and ‘acts’.
Tracks (short for voice track): is the script read by the narrator. It is perfectly fine for your script to contain only tracks. A script that just contains the voice of the narrator is a ‘voicer’.
Acts (short for actuality): are the ‘voices in a story that are not the narrators. These can include interviews, scripted re-enactments or dialogue and found footage like news reports and video. They are also called ‘grabs’ or ‘sound bites’.
Key principles and tips1. Write like you speak. Use your own voice. Use your own vernacular. This is the flow you know.
-
Start emphatically. Jump into the narrative. Something is happening. Pose a question. Create mystery.
-
Keep your sentences short. Audio requires more straight-forward sentences because listeners do not have the benefit of re-reading or reading slowly. Speak directly. Repeat key words to give emphasis. Only use phrases and expressions that you would use in normal conversation.
-
Structure the writing with two or three key points or narrative elements. Limit general description, and focus on precise and evocative illustrative details. Only you know what was left out. You might write a part so you ‘get it’, but your listeners don’t need to hear it, what they need is the key moment, the telling illustration, the specific critical insight.
-
Be a helpful narrator. Explain where you are going. Sketch a map. Set out why it is interesting. Spell out how things are connected.
-
More action than description. Verbs are better than adjectives. Edit closely at the sentence level.
-
Transition between ‘tracks’ and ‘acts’ need to have flow. Think about how an act picks up and extends your thought. Don’t have acts that simply repeat what the narrator has already said.
-
Speak definitively. Read your script aloud. This helps identify difficult phrasing, get a sense of tone and pace, and identify where to breathe. You might cut a sentence into a fragment or use an em-dash or ellipses in order to create space for a breath.
-
Paragraphs are typically short, even just one sentence long.
-
Signal emphasis. Denote which word in a sentence the speaker should emphasise. This helps a reader 'hear' what it sounds like as they read it. The easiest way to signal emphasis is to place a work in capitals or italics.
Examples: scripted audio that reflects on our lived experience with mediaHere are some examples of podcasts that are researched and scripted that engage with our experience with media. Each of these episodes offer examples of writers using audio effectively to describe, reflect and critically analyse media and cultural life.
Ira Glass' 'Finding the Self in the Selfie' (573 Status Update), This American Life, 27 November 2015
Listen to the prologue and first act of this episode. This features Ira Glass narrating and talking with Julia, Jane and Ella about selfies and Instagram. Glass evocatively describes the practice of taking and commenting on selfies. He makes this ordinary ritual seem strange and intriguing. He then sets about reflecting on how these practices work and what they mean. You can listen here or read the transcript here.
David Rakoff's '29' (328 What I learned from television), This American Life, 16 March 2007
David Rakoff reads a piece about an 'experiment' he undertook: watching 29 hours of television in one week. That's how much television the average American watches. Unlike the example above, this piece is entirely a first person narration where Rakoff reads some quotes from friends and family. The tone is snarky and funny, but the reflection is insightful. Rakoff carefully illustrates and then reflects on the forms of cynical and snarky enjoyment we get from watching 'trash'. You can listen here or read the transcript here.
Malcolm Gladwell's 'The Satire Paradox', Revisionist History
Malcolm Gladwell carefully illustrates and critically examines how satire became part of our political culture. The writing demonstrates both evocative examples and sharp insights into the limits of satire. You can listen here.
Karina Longworth's 'Madonna: From Sean Penn to Warren Beatty', You Must Remember This
Longworth's long-running podcast You Must Remember This is a great example of scripted audio in the genre of creative non-fiction. Longworth takes the history of Hollywood and scripts it as creative narratives. You can listen here.
Cohabitation with mediaThe habitats we live in are made up of complicated relations between humans and technologies. Digital and media technologies are woven into our homes and public spaces, tethered to our bodies, and entangled with our imaginations.
Once we might have ‘studied’ media mostly by examining its symbolic content, or by examining how symbols are produced and consumed.
Now we need to also account for media as socio-technical infrastructure. That is, as infrastructure made up of humans and material media technologies that jointly construct the cultural, economic, social and political systems within which we live.
Drawing inspiration from critical media and cultural researchers I sketch out two techniques here – walkthroughs and vignettes – for purposefully reflecting on our cohabitation with media.
What’s a walkthrough?A walkthrough is a method used by researchers and technology designers who examine how people use media in their everyday life.
A walkthrough is useful for carefully documenting the interplay between the design of media technologies and the way people use them. This is a dynamic relationship.
On the one hand, human users play an active and creative role in using media in our lives.
On the other, media technologies are purposefully designed to structure how humans make use of them.
For research purposes, a walkthrough is a systematic analysis of how a media technology is designed to work and how it is used within everyday life.
Walkthroughs are most often used to examine interactive digital media technologies like apps or social media platforms. That said, the set of questions and concerns below work just as well for examining any media technology from books and newspapers, to cinema and television, to wearables and augmented reality. The principles of examining carefully the interplay between humans and technology remain the same.
Here are some of the things a walkthrough does:
1. Explains how something works.
A walkthrough is a step-by-step explanation of how a media technology works.
In a vernacular sense, we see walkthroughs all the time. For instance, if you ever wanted to block a number on your phone, or change your privacy settings on Facebook, or know how to create a particular effect with make-up you probably jumped on Google or YouTube and search ‘how to…’. These ‘instructional’ videos on how to do things are an everyday example of a ‘walkthrough’.
2. Critically examines the social, political and economic setting of media use.
A walkthrough examines how a media technology is shaped by commercial, social, cultural, political and legal conditions.
This can mean paying attention to aspects such as the commercial business model a media technology and examining how shapes its design, or examining the privacy policy of a media
technology and thinking about how that institutionalises relations between users and platforms.
Questions you might ask:
- What are the commercial arrangements that sustain this product, service, technology, app or platform?
- What contractual arrangements does the media technology create with users?
- Do the users of this media technology undertake labour?
- Do the users of this media technology create content or data?
- If so, what value does that content or data have?
- How does its advertising model work?
To answer these questions your walkthrough might lead you to investigate the privacy settings, privacy policy, and advertising interface of the platform or media technology.
3. Critically explores the choices and affordances a media technology offers users.
A walkthrough examines how media technologies structure the range of choices users have.
This involves carefully exploring the options given to users as they go about engaging with a media technology. This might involves examining the interface, protocols, defaults and algorithms of a technology.
For example, when I log on to Facebook I cannot choose what content I see, the News Feed algorithm makes that choice for me. Or, when you sign up to a dating app you might be given a limited set of categories for describing your gender and sexual preferences.
Questions you might ask:
- What options do the users of this technology have to describe and express themselves?
- What kinds of participation does this media technology facilitate?
- What options or choices does the interface give users?
- Do users need to create a personal profile to use this media technology? If so, what choices do they have in constructing the profile?
- What ‘rules’ govern the use of this media technology? What are users able and not able to do with it?
- Is data recorded about users?
- Are algorithms used to shape the experience or engagement with this media technology?
- What kinds of data are collected?
- What public, political or cultural consequences does this data-processing have?
4. Critically analyses the symbolic qualities and practices unfolding in and around a media technology.
A walkthrough can pay attention to the symbolic landscape of a media technology.
This might involve a semiotic analysis of the design or interface of the media technology itself. For instance, until recently Facebook only enabled users to ‘like’ content, now they allow a limited range of ‘reactions’. It might also involve a critical analysis of the kinds of symbolic content that flow through a platform. For instance, we might pay attention to the kinds of narratives and identities that are represented on Netflix.
Questions you might ask:
- What icons, images or colours are used to describe and facilitate use of the technology?
- What kinds of lives, identities and bodies are visible via this technology?
- What are the dominant and non-dominant forms of expression taking place via this technology?
- Who is creating, sharing and consuming representations?
- What is being represented? Your own life? The lives of others? Bodies? Brands? Cultural experience? Tastes? Political viewpoints?
- How are these ideas being represented?
- How are users engaging with them?
- What controls are placed on the forms of symbolic expression allowed by this technology?
5. Critically documents and reflects on the use of a media technology in an everyday setting.
A walkthrough documents how a media technology is actually used as part of everyday life. Researchers often recruit informants who help them make sense of their media user. But, you can also do a walkthrough ‘solo’, by carefully documenting your own use.
Questions you might ask:
- Where am I when using this media technology?
- What time is it?
- Who am I with?
- What am I doing?
- Am I using this technology to search for information, organise something, purchase something, express myself, post images of my body, monitor my mood or body?
- Do I create ‘hacks’ or ‘workarounds’ to get around the rules of the platform? For instance, having two Instagram profiles, one private and one public; using a fake name on Facebook.
- What actions do I take to manage my visibility or privacy?
- How do I use this technology as part of my self-expression or relationships with others?
- What are my feelings and mood?
- Am I scrolling, glancing, and tapping?
- Am I using two screens at once?
- Am I relaxed, bored, anxious, just passing time, quiet and reflective, thoughtful, agitated?
- What are the touchpoints between your body and the media technology? For instance, if you put headphones on does this technology create a kind of immersive and private experience?
These are not a definitive list of questions or concerns, the important principle is that walkthroughs encourage us to think carefully about the dynamic relationships between media and human users.
In particular, the walkthrough helps us think about the relationship between our creative uses of media and the way they are purposefully designed. We come up with all sorts of creative uses of media as part of living our lives and expressing ourselves. At the same time, media technologies are designed to encourage certain practices and achieve certain strategic goals like increasing user engagement, generating profit, or advancing political objectives.
Walkthroughs also help us think about the active engagement of users with media, we aren’t just passive consumers of symbolic content we are actively involved in the process of incorporating media into our everyday practices.
Sometimes users do things with media that designers didn’t intend, sometimes media constrain users or reinforce existing power relationships or rituals of communication.
Walkthroughs help us think about how power relationships work in participatory digital media cultures.
They focus our attention on the ‘socio-technical architecture’ of media. That is, the relationships between the ‘technical’ design of media technologies and their ‘social use’ by humans.
More on walkthroughsThe description of a walkthrough I’ve sketched here draws inspiration from the methods of a range researchers who critically explore the interplay between media and cultural life.
If you want to read more about some of these methods, here are a few leads.
Livingstone, S. (2008). Taking risky opportunities in youthful content creation: teenagers' use of social networking sites for intimacy, privacy and self-expression. New Media & Society, 10(3), 393-411.
Livingstone’s (2008) method involves sitting down with young informants who explain in detail their use of social networking sites as part of their practices of identity formation and self-expression.
Light, B., Burgess, J., & Duguay, S. (2016). The walkthrough method: An approach to the study of apps. New Media & Society.
Light, Burgess and Duguay’s (2016) method involves carefully documenting smartphone apps drawing on approaches from Human Computer Interaction, Science and Technology Studies and Cultural Studies.
Robards, B., & Lincoln, S. (2017). Uncovering longitudinal life narratives: scrolling back on Facebook. Qualitative Research.
Robards and Lincoln’s (2017) involves sitting down with informants who ‘scroll back’ through their Facebook timelines as a method of both reflecting on their life narratives and the affordances of the platform.
Caliandro, A. (2017). Digital Methods for Ethnography: Analytical Concepts for Ethnographers Exploring Social Media Environments. Journal of Contemporary Ethnography.
Caliandro (2017) offers the principles of 'follow the medium' and 'follow the natives' by which he means that to understand digital media we must both 'observe and describe' how media technologies work and observe and describe the practices of users and how users give meaning to thier practices.
Walkthrough questionsIn this walkthrough exercise, begin by selecting a ‘moment’ of engagement with media that you want to critically examine and reflect upon.
Take this moment and ask the following questions as a way of describing it.
Question 1. What day did this moment occur?
Question 2. What time did this moment occur?
Question 3: What media platform or channel were you using?
Question 4: What kind of media device were you using?
Question 5: Where you are when using the media technology?
Question 6: Were you consuming media content?
Question 7: Were you producing media content?
Question 8: Were you adding ‘information’ to media content produced by someone else by sharing, commenting or liking it?
Question 9: Was data being recorded about you?
Question 10: Are algorithms being used to shape your experience or engagement with this media technology?
Question 11: Do you need to create a user profile to use this media technology?
Question 12: What is the business model of this media technology?
Question 13: Are other users of this media technology visible to you?
Question 14: Do you interact with other users?
Question 15: Portray this moment in a short vignette.
Writing vignettesVignettes capture and express not just events and actions, but also their character and feeling.
Vignettes are used by researchers to document, reflect upon, and analyse everyday practices and relationships.
A ‘vignette’ is a brief evocative description and illustration of this moment of media engagement. A vignette combines descriptive detail with analytic commentary and critical reflection.
Your vignette only needs to be a 100-200 words. Writing the vignette is preparation for preparing the script.
Your vignette should provide a step-by-step account of your actions with the media technology from the moment you first ‘pay attention’ or ‘engage’ with it to the moment you ‘disengage’.
The vignette should respond to these two questions:
- What are you doing, thinking and feeling?
- How is the media technology shaping your actions and experience?
You can also take up any of the questions in the explanation of walkthroughs above.
Below is an example of a vignette.
Watching Netflix: an example vignette
Friday night. 8pm. On the couch waiting for Netflix to load. We are going to watch season 2 of Love. We start. I want ice-cream. We pause it. I flick to the football to see what’s happening, then back to Netflix. Half the time I’m scrolling through Instagram or Twitter. Nicola is looking at Facebook and ASOS. She finds a good suit for me. When I look on my phone it costs twice what it costs on her phone. Netflix starts buffering.
A decade ago. Friday night. 8pm. I live in a share house that has no ‘tuned’ TV, just a screen and a DVD player. Nothing doing on a Friday night, no money. We get beers and rent The Wire on DVD. You rent it one disc at a time, about three episodes per disc. Watch one disc and then scramble back to Network Video before it shuts at midnight to get the next one. As we are renting it, a group from a share house down the road start cursing us. We are renting the disc they need. We wander back down the street and invite them to ours to watch it. We sit out the front smoking and drinking.
The ritual hasn’t changed. Both Friday nights are organised around the television screen, the home and the people you live with and love. And yet, a decade ago there was only one screen in the room, that screen wasn’t connected to a communication network. It didn’t stream and it didn’t watch. Network video was finite, Netflix is endless. The DVD collected no information about me, the Netflix interface does. I started watching The Wire because a friend gave me a burned copy, and then I saw it on the ‘top picks’ list of one of the video store staff. I started watching Love because the Netflix algorithm predicted I ‘might like it’. ASOS offered Nicola a better deal on that suit because she flicks and scrolls and buys on the app more often. A decade ago the shops shut at 9pm on a Friday night and they didn’t know who I was.
In these abandoned buildings we see an eerie portrait of what happens when an industrial economy disappears and no new economic activity arises in those same spaces to replace it.
By technical achievement I mean the ability to build telecommunication networks, the internet and digital computation that enables the collection, storage, transmission and processing of vast amounts of data across time and space. By social achievement I mean the way that the network becomes an idea for organising society, workplaces and cultural life.
In order to understand the development of these corporations, and the way they have reorganised our media system, we have to take a few steps back. Let’s go back to the managerial capitalism of the twentieth century.
This critique of mass media like cinema, radio and television matters because it is an important backdrop against which early digital and interactive media technologies were celebrated. If the television viewer was a passive dupe, the user of internet technologies like blogs and social networking sites was active and engaged.
But, something important has stayed very much the same. Since the middle of the twentieth century media have saturated our everyday life. We need to go back and think about this longer history, to see the ubiquitous presence of media in our everyday life as something that has been developing for nearly a century. And, to see the changes to media over the last generation are not just sparkly technological inventions, but rather as part of a particular kind of networked and flexible economic and political system that has been taking shape since at least the 1970s.
What is an argument?This post contains a set of exercises that are useful in developing a piece of academic writing like an essay. I focus in particular here on developing an argument about a 'real-world' case study or illustration.
This post doesn't focus much on the mechanics of good writing like grammar, paragraph structure and style. If you feel your writing would benefit with some attention to grammar and style, my suggestion is to check out the UQx English Grammar and Style online course. Writers of all abilities benefit from doing this course.
If you want more advice on the structure of academic essays, check out the UQ Student Services page on assignment writing.
An argument is the spine of good academic writing. An essay needs an argument to give it purpose and structure. The Oxford English Dictionary defines an argument as:
‘A connected series of statements or reasons intended to establish a position.’ (1393)
‘A process of reasoning.’ (1393)
‘A statement or fact advanced for the purpose of influencing the mind; a reason urged in support of a proposition.’ (1405)
‘A statement of reasons for and against a proposition.’ (1513)
‘To give evidence.’ (1558)
For our purposes then we can begin by defining an argument as a proposition, or series of statements, reasoned with evidence.
An argument has three parts:
- A proposition. An argument begins by making a claim.
- A purpose. An argument explains why the proposition matters.
- Evidence and reason. An argument is defended with evidence and reason.
An academic argument consists of illustrative evidence structured by theoretical reasoning. The illustrative evidence comes from careful discussion of real world events and social relationships. The theoretical reasoning is developed by drawing on theories, concepts and scholarly debate as tools that you use to think and reason with.
In what follows I set out a series of tasks you can undertake to help you develop an argument. The point of these tasks is to shift your beginning point away from a blank computer screen. Start your essay by thinking about the relationship between your proposition, your purpose and your evidence. Develop a clear narrative about the events in the world you are writing about as a precursor making an argument about them.
Task 1: scribbleBegin with three questions:
- Who did what, how, where, when and why?
- Why is it important? What consequences does it have? Who does it matter to and why?
- What theoretical concept could I use to guide my thinking about this issue or event?
Most of us will find it easier think via real events in the world, and begin there. But really, it doesn't matter which of these questions you tackle first. Tack back and forth between these questions. Jot down quickly all your thoughts about who is doing what, what consequences it has, and how these events in the world relate to theories you are working with.
You might answer the questions 'why does this matter?' and 'what consequences does it have?' from your own perspective or you might think it through from the perspectives of actors in the real-world case or events you are thinking about. You might think about what it was that was motivating people to act in the way that they did in the case you are analysing. The image below is a basic example of a mind map that sketches out an event and how it might be understood in terms of the concept representation. Your mind map will likely be messier and more sketchy than this.
Task 2: sentencesOnce you have a rough map. Write three sentences. One sentence that responds to each of the three questions:
- Who did what, how, where, when and why?
- Why is it important? What consequences does it have? Who does it matter to and why?
- What theoretical concept could I use to guide my thinking about this issue or event?
These three sentences are a basic outline of the first paragraph. They give you a basic framework to begin fashioning an essay.
Task 3: plot the storyTo write a good argument, you first need a clear narrative of the real-world events you are examining. Events in the world are messy and complicated, they rarely reach clear resolutions, they can be told from many points of view and with differing points of emphasis. If you set about writing your essay, making an argument about real-world events, your writing will likely fill up with details. Those details might start to obscure the argument. So, before you begin writing the argument, sort out what your narrative of the real world events will be.
Draw a basic narrative spine on a large piece of paper (see the image below). A basic narrative has the following components:
- Set-up: equilibrium and disruption. The 'set-up' is the beginning of your story. Identify it as the moment when an important change or disruption happens from which the events you are interested in begin to unfold. This helps set a clear beginning for your story, that will take shape in the first paragraph of your essay.
- Turning points (narrow it down to 2 or 3). There may be many events that make up the case you are examining. Here, settle on the 2 or 3 that you think are the most important to understanding the case. Articulate clearly who did what to whom with what effect at these key moments.
- Climax. The climax is the moment that makes this case particularly important and interesting, this is a moment which something happens which has real consequence.
- Resolution. Events in the real-world are often messy and inconclusive, so often we impose a resolution on our stories. The resolution might not be a particular event, but rather a way in which you or actors in your case drew meaning and consequence from what has transpired.
Plot the action of the real-world case you are examining. Make the spine ‘down the bottom’ of the sheet of paper so there is room to add details above.
By plotting the narrative you force yourself to settle on key events, characters and perspectives.
Sketching the 'set up' helps you identify the key background you need to introduce at the start of your essay for your reader to understand the case and why it matters. Naming the turning points and climax help you identify the specific events in the real work that you will focus your analysis on. Thinking about the resolution helps you articulate how your essay might end, how you might reach a point in the narrative where conclusions and insights can be drawn. Each of these narrative elements impose order and focus on what are often messy social events.
Once you have a clear narrative, this serves as a guide for thinking carefully about how you might use theory to make a critical and analytical argument about your case.
Task 4: using theoryTheories are conceptual tools for constructing logical explanation and analysis of the social world. Theories of society, culture, politics and media are tools to think with. They require critical analysis and judgment. They require adaptation to specific social settings and situations. Theories emerge and develop over time within a discipline of academic scholarship. They are adapted via an open-ended process of intellectual debate. This debate is driven by academics, researchers and students using theory to attempt to make sense of the world they live in.
There are two elements to using theory. One is identifying, describing and applying a particular concept in your argument. The other is situating your argument in relation to scholarly debate that engages with that concept.
For instance, the concept of hegemony might help you explain how consensus emerges, or the concept of representation might help you explain how different groups interact with each other to make sense of events in the world.
This first move involves getting a clear understanding of a concept, and how you might use it to unpack, explain and analyse real-world events. The second move involves thinking about how other scholars have used this theoretical concept to analyse the same or similar events, relationships and situations in the world. Your use of theoretical concepts is enhanced by drawing on the insights of others. Think of this as an intellectual or scholarly conversation that you are contributing to.
The narrative structure you sketched out above becomes important here. Look to key moments in your narrative, particularly the turning points and climax.
Consider how you might analyse these events in relation to theoretical ideas you have been working with. For instance, if your narrative is about how groups struggled with each other to represent an event in the world or construct a consensus view about an event then you might decide a concept like representation or hegemony is useful for explaining what was happening.
- Identify a theoretical idea that you think works as the foundational idea in your thinking about your case. Write a simple definition of it.
- Return to your narrative spine. Place the theoretical idea above turning points or climax where it is useful to explaining the events. Here, you are making out a moment in your narrative about real-world events that is both important and that can be understood and analysed using a specific theoretical idea. For instance, say your one of your turning points was 'students film a protest on their smartphones'. You might decide you can understand this turning point as a moment of representation. Then you can begin to draw on representation as a theoretical framework for analysing this moment.
- Develop some layers. You might find a broad theoretical concept like 'representation' underpins much of your narrative, but then within that framework more nuanced and specific ideas and arguments come into play. For instance, a turning point like 'students film a protest on their smartphones' is a moment of representation, but you might then start to consider some of the nuances of using smartphones to document social life as part of our lived experience. This might lead you toward more specific arguments, for instance the concept of 'mediatisation' from a scholar like Nick Couldry or the concept of 'smartphone witnessing' from a scholar like Karen Anden-Papadopoulos. You might place some of these more nuanced arguments about representation then above specific moments on your narrative timeline that they relate to. This helps you think about how you might put your analysis in conversation with other scholars and their arguments.
As you go through this process, you are constructing an interplay between your narrative about events in the real world, theoretical concepts and scholarly debate. By building these connections you are situating theoretical ideas and scholarly arguments somewhere in your narrative, mapping out where in your narrative scholarly concepts and arguments might help you develop your own argument and analysis.
This process helps to you clarify which theoretical ideas and scholarly arguments will be of use to you in building your argument.
Task 5: card sortIn the tasks above you have developed a clear account of the narrative of your case, and articulated clear connections between specific moments in your case, theoretical concepts and scholarly arguments. From here you can organise the argument and structure for your essay.
One way to do this is by using a card sorting exercise. Using cards helps you to test and experiment with the structure of your essay in a modular way before you try to write it.
The way I do the card sort here I use three different coloured cards, to help distinguish between 'argument', 'illustration' and 'theory'.
- Write out each step of your narrative on a separate index card. You'll likely end up with 4-5 cards: one with the set-up, one for each turning point, one for the climax and one for the resolution. In the image below the narrative steps are on the green cards.
- Write our each theoretical idea you think you'll engage with on a separate index card. In the image below the theoretical ideas are on the yellow cards.
- Arrange the narrative steps first. You already have this structure from your narrative spine. Arrange it chronologically down your desk.
- Arrange the theoretical ideas next. You should write separate cards for both major theoretical ideas, and also key quotes and claims from scholars whose work you have been reading. To the right of the narrative cards, place the theoretical ideas where you think they are useful in you narrative.
- Prepare a series of cards that are the propositions or claims that make up your argument. Start with a card that makes what you think is the most important proposition in your essay, and then write others. In the image below the propositions are written on the white cards. The first proposition says 'how we represent events matters because it is a means through which we organise the social world', then from there develop other propositions. You might do this by looking to your narrative and thinking about what is the important claim you want to make about specific events that are unfolding. For instance, at the moment where students film a protest using their smartphones yuo might write the proposition 'smartphones change how we represent everyday events'.
Now you have a series of cards for your illustration, theoretical ideas and propositions. The first step is to arrange your cards vertically. In this step you are developing a logical sequence to your propositions and narrative, and you are marking out where in your argument theoretical ideas are defined and used. You might experiment with moving the order of the theoretical ideas, propositions, and possible elements of the narrative around in order to arrive at the clearest and most logical sequence for your essay. Here are some things you should consider as you reflect on the vertical ordering of the cards:
- Are parts of the narrative that can be removed or condensed because they are not so important to the argument?
- Are you introducing the big theoretical concepts at the top, and then the more nuanced ideas as your argument develops?
- Do your propositions become more specific and detailed as your essay goes along?
- Do your propositions and narrative have a logical sequence? Does each idea follow clearly onto the next one?
- Is there any repetition in propositions or narrative that can be removed?
The vertical organisation of the cards helps you develop a logical sequence to your argument and narrative. It helps get the main ideas to the 'top', develop nuance and detail through the middle, and ensure a logical flow throughout.
The next step is thinking about the horizontal organisation. Look across each row of the matrix and check that the relationships between proposition, illustration and theory make sense. Check that throughout there is a mix of each element.
Once you have sorted your matrix vertically and horizontally you have constructed a structure for your essay. In principle, each row of the matrix becomes a paragraph in your essay.
The card sort helps you ensure that every claim or proposition in your essay is defended with illustrative evidence and theoretical reasoning.
The image below shows what a basic card sort looks like. Please note, the ideas in this card sort are not fully developed. The column of theoretical ideas for instance are too general, more work needs to be done to narrow down and bring more detail into the matrix.
Task 6: writeWrite a draft of your essay using the card sort matrix developed above as a guide. The top row of the matrix becomes the first paragraph of your essay. This opening paragraph should contain the main proposition of your essay's argument, the set-up of the narrative, and an outline of the key theoretical idea and how it will be used in the essay.
Each row that follows in your card sort matrix should make one paragraph. The 2-5 paragraphs that make up the body of your essay should sequentially develop the argument and narrative, developing the nuance of the argument and engagement with theoretical ideas and scholarly arguments as you go along. In the body we should see a clear and sustained engagement with specific ideas and scholars. Think of how your essay funnels toward more specific and nuanced analysis paragraph by paragraph.
The final paragraph should make a definitive statement about implications of the analysis. The conclusion should not summarise the essay, but rather set out why the argument in the essay matters, what lessons can be drawn from the analysis and narrative, and a reflection on the value of particular theoretical ideas or scholarly arguments to understanding these events.
Task 7: editSpend time editing your work carefully. Read it aloud for flow and tone. Get someone else to read it to see if they can understand the argument and analysis. A fresh reader will often pick up ideas that are not clearly defined, missing elements of the narrative, or sentences that don't make sense.
Your editing process should pay attention to the overall structure, each individual paragraph, and the sentences.
Firstly, check the structure and sequence of claims that make up your essay.
You might do this by extracting just the topic sentences of each paragraph. Does each topic sentence clearly state the one clear idea that paragraph develops? When you read the topic sentences in a row one after the other does a clear and logical sequence emerge? Does the essay more or less make sense just by reading the topic sentences?
Secondly, check your paragraphs. Each paragraph should have:
- A topic sentence that outlines the main claim of the paragraph.
- One idea.
- Lead logically on from the last one and into the next one.
- Contain both illustrative evidence and theoretical reasoning.
Thirdly, check your voice.
You might get some highlighters or pens and use different colours to highlight different elements. One to highlight theoretical reasoning, one to highlight illustration and narrative, one to highlight statements about the importance and significance of the events being discussed. Do you see a good balance of colours, and their associated elements, throughout your paragraphs? Do you see places that are predominantly theoretical reasoning or predominantly description and illustration? If so, you might work on integrating the elements more. This task also helps to identify paragraphs that are too descriptive or too theoretical. You should see a good mix of argument, narrative and theory throughout the essay.
Not all journal articles are the same, but they do all tend to have some fundamental elements. If you understand what these elements are, how to find them, and why they matter it will help you to understand articles, evaluate their quality, read more efficiently, and incorporate their arguments into your own writing.
There are five elements you should aim to identify in any journal article you read:
• A significant question or claim.
• A position in the academic debate.
• An explanation of the research method or approach.
• A presentation of the findings and argument.
• A statement of the implications and contributions of the research study.
Question or claimThe first element to look for when reading a journal article is the main question the author poses or the main claim they are making. This question or claim effectively sets the frame through which the author wants the reader to ‘judge’ their writing. The editor of the journal, when agreeing to publish the article, would have decided that the question or claim is an important one and that the author clearly demonstrated it in their writing. You will usually find this claim in the first section of the article. Once you’ve found it, use that claim as the foundation upon which the article is built. The author will also explain why their question or claim is significant and who or what it matters to. The significance might be presented in relation to the academic field, a policy or governance problem, or events that matter to politics, cultural life or an industry.
Position in the academic debateThe presentation of a question or claim is interrelated with a review of the relevant academic literature. In some articles this will be presented as a clear literature review section, in other articles the first few pages of the article will weave together the author’s claim and question with an analysis of relevant literature. This might not be titled ‘literature review’ but come under several themed subheadings.
The purpose of this section of a journal article is not just to summarise the debate, but to organise it and frame it. A good literature review will set out competing perspectives or clearly articulate shortcomings and limitations in the current scholarly debate. The purpose is to demonstrate how the author’s question and claim will respond to significant debates in the literature. Sometimes the author will claim to ‘fill a gap’ in the current debate by adding some now evidence, in other cases the author will claim to ‘correct’ or ‘refute’ a significant assumption or claim in the literature by providing confounding evidence or demonstrating how new developments change previous understandings.
The academic literature is always under construction, journal articles don’t aim to ‘end’ the debate with a final piece of definitive knowledge, but rather contribute to the ongoing effort to push debate forward. The engagement with the literature at the start of a journal article aims to position the article in relation to those debates. Sometimes the literature review will position the study’s contribution as an ‘applied’ one, other times it will be ‘conceptual’. An applied contribution is where existing ideas from the literature are taken and applied or tested in a new context. For example, if research has mainly been conducted with people in one setting (like a city); new research might test those ideas by examining people in a different setting (like a rural area). A conceptual contribution is where existing ideas from the literature are reformulated, or new ideas are proposed, as a way of contending with new developments in technology, society or culture.
At the end of this section of a journal article you should have a clear idea of the debate the author is engaging with, why that debate matters, and how they intend to contribute to it. As a reader the literature not only familiarises you with a debate but it also offers you some reference points for your own research. Often, a good way to target your reading is to go out and engage with the authors that others are engaging with. If you find a journal article about a topic or issue that is relevant to you, then check out who the author is engaging with and follow their lead. If you read authors that are citing each other then you are more likely to find a coherent conversation to ground your own thinking and writing within.
Explanation of research method or approachOnce an author has explained their question and claim, why it matters and situated it within current academic debate they will then set out how they went about doing their research. This is where they explain how they will make an ‘original’ contribution to the literature.
The media and communication field crosses a broad range of approaches. Some journals have a very systematic way of presenting research methodologies. These journals tend to come from more empirical disciplines that follow a scientific method like psychology or sociology. A journal article might label its methodology section clearly and offer a sustained and clear explanation of the methodological approach and often also an evaluation of its strengths and limitations.
Many journals also come from humanities traditions. In these articles the explanation and justification of the methodology may be more implicit, but it will always be there in some form or another. In its most basic form the author will provide a paragraph that explains how they did what they did and who has used similar methods to address similar questions.
A quantitative and scientific methodology might involve a descriptive or experimental survey for instance, where the author would clearly explain and evaluate the validity of the constructs used in the survey, the sample size, and the analytic procedures. A discourse analysis might explain the range of texts selected for analysis and the analytic procedure used to make sense of them. An interview study might explain the sample of people interviewed, the range of questions asked, how the interviews were analysed and what claims were possible from them.
Sometimes a journal article will be making a critical and conceptual argument, and therefore won’t necessarily have empirical evidence or methodology. In these articles though there is a still a method in the sense that the author will explain clearly how the argument is structured and what material it will engage with – instead of empirical material like interview, textual or survey data, it might be a scholarly debate or conceptual framework that the author is framing, critiquing and contributing to.
In the media and communication field there are no ‘right’ and ‘wrong’ methods, or methods that are ‘better’ than others. What matters is that the author clearly explains how they did their research and how that approach was appropriate for responding to the question they are posing or making the claim they are making. You need to understand the methodology in order to understand the basis on which the author will go on to make their arguments. You might also find the methodology useful for developing your own research projects and approaches.
Presentation of findings and argumentThe first three elements – the problem, literature review and methodology – clearly set out what the article is about, why it matters and how the research was done. From there the author moves into the presenting the findings and argument from the research. This is where the author makes their contribution to the literature by presenting original material and arguments. These sections are sometimes called ‘results’ and ‘discussion’, other times they are organised under themed subheadings. In more empirical articles the results will be presented separately and then discussed. This is common in journal articles presenting survey research for instance. In other articles the results and analysis will be woven together, this is more common in qualitative and critical research articles. The structure of this section often offers a useful conceptual framework for your own writing. In the findings and argument scholars usually present concepts that you can use to structure and inform your own arguments, analysis and research.
Statement of implications and contributions of the research studyA journal article will conclude with an explanation of the implications of the research, its limitations and suggestions for further research. The author will explain what the consequences and significance of the research findings are to the scholarly debate. They will then map out what they think the next steps in research on this problem should be. You can use the conclusion as a launching point for your own arguments, taking up the questions and challenges authors arrive at in their journal articles as a starting point for your own thinking and writing. As academics read each other’s journal articles they look to the implications and suggestions of previous publications and use them as the basis for formulating their next research projects and arguments.
Placing a journal article in broader debateYou should also aim to place a journal article within the broader academic debate. The first way to do this is to go the journal article’s reference list and locate other articles that extend ideas in the article that are of use to you. Do this in conjunction with reading the article. Where the author makes a claim that you find compelling or useful and then cites it in reference to another author, go to that author’s work and read it too. The second way to place the journal article in the larger debate is to conduct a search to find out who else has cited the article since it was published. Citation is when an article is ‘referenced’ in another article after it is published. Academics use citation to follow how articles get incorporated into ongoing academic debate after they have been published. Academic publishing is reasonably slow, so you may find that articles don’t accumulate citations until two or three years after they have been published. Articles by prominent scholars or that are key to debates in the field will accumulate many citations.
The easiest way to find citations is to use Google Scholar. Search for the article you are reading in Google Scholar. When you find it you will see underneath the listing a link that says ‘Cited by…’ followed by a number. That number is the number of articles that have cited it since publications. Click on that link to go through to the list of articles and search in there for publications that are relevant to you. If there are many hundreds of citations for an article you can click the ‘search within existing articles’ box to search key terms within those articles citing the original article. If you pay attention to how an article is positioned in the broader academic debate it will help you select a collection of articles that are in conversation with one another. This will help to improve your writing because you will have identified and mapped out a shared conversation to engage with of scholars who are already mapping out and contributing to a debate that you can then join in with.
This guide to reading journal articles is from our text Media & Society.
Do you remember Napster?Napster was the first file-sharing service to go ‘mainstream’.
If you were a teenager or a music industry executive in the 1990s you most certainly would. For teenagers it was an incredible pipeline of free music. No longer did teenagers have to save up their dollars to purchase CDs by their favourite artist. Using a dial up internet connection and the file sharing platform, they could download music for free, even if it was illegal.
Here's Alex Suskind writing on the impact of Napster in 2014:
The service, which launched on June 1, 1999, soon spread like a virus, infecting every music nut with a computer and a dial-up connection. By March of 2000, Napster had 20 million users. Several months later, it was more than three times that. By then, the company and its wunderkind creators had been targeted by the RIAA (Recording Industry Association of America) and its suite of attorneys, along with several global superstars like Metallica and Dr. Dre. To the music labels, Fanning and Parker were completely upending a system that had been in place for decades, toying with a carefully crafted mechanism that allowed the artist, the manager, and any middlemen a certain percentage of each record sold. To the musicians, the Napster co-founders were outright thieves, providing an avenue to steal music without paying a dime for it. Depending on what side of the aisle you fell on, if you worked in the industry or if you were just a regular old music fan, Parker and Fanning were either the villains or the heroes.
For the music industry though it was an existential threat. While the music industry eventually defeated Napster in court, peer to peer file sharing was not going away. Napster demonstrated to the media industries that the internet would make cultural content – music, film and television – easy to share. This was not going to change.
Clive Haberman described this as the rise of the 'culture of free' in The New York Times:
Napster did not have the courts on its side. Ordered by federal judges to stop allowing copyrighted material to be traded on its system, it shut down in July 2001. But from its ashes other file-sharing services arose, some bearing curious names like Grokster, Kazaa and Gnutella. Mr. Fanning tried his hand at new digital media endeavors, including Snocap and Napster 2.0. Few of those companies were unqualified successes. One thing was certain, though: The culture of free was not going away.
Napster is historically important as the first peer to peer file-sharing platform that ‘went mainstream’ in the late 1990s. The platform enabled users not just to copy copyright material but to share it with each other at scale. Millions of users shared their music files with one another.
The contest over the sharing of copyright content in the age of the internet is a useful case for thinking about hegemony.
Yes, the sharing of copyright material online is a matter of law: how should the sharing of content be regulated?
But, it is also a matter of public opinion and consensus: are people who download music, television and films without paying ‘criminals’? Are they ‘hurting artists’? Are they creating a new ‘cultural commons’? Are they just sharing with friends?
What’s hegemony?By hegemony we are talking about the effort by particular groups to make their ideas common-sense. Groups are hegemonic when their ideas seem natural, inevitable and common-sense. Importantly, becoming and staying hegemonic involves continuous work and because no group can ever dominate on their own it also involves the capacity to generate and maintain alliances. If you want something you need to find common-cause with others and generate consent for your way of seeing and acting in the world.
CopyrightSince Napster, the question of how to protect copyright on the internet has been a battleground between the traditional content industries like music, film and television; the emerging internet industries and platforms like Google, YouTube and Apple; open access and free culture activists; and consumers who, at least in practice, will download copyrighted content for free if they can.
OK, let’s begin with a bit of background on copyright in Australia from the Attorney General's Department:
Copyright is a type of property that is founded on a person’s creative skill and labour. Copyright protects the form of way an idea or information is expressed, not the idea or information itself.
This is what makes it so complicated. If you buy a book you can tell your friend about the ideas the book contained, but you cannot copy the book and distribute it.
Copyright attempts to strike a balance between the rights of creators to make money from their creative works, and the public nature of the ideas those works contain.
Copyright wars in the media and technology industriesSince at least the 1970s, content industries like film, television, music producers and technology industries have been in a battle with each other over copyright.
Personal recording devices like the VCR generated new questions about copyright like: should it be legal for consumers to make copies of copyright material? Should it be legal for technology companies to create devices that enable consumers to make copies of copyright material?
For much of the twentieth century content industries had strong control over the distribution of their content via licensing and distribution arrangements with broadcasters, cinemas, record stores and so on. Cultural content was produced and disseminated in formats that required consumers to pay. It was difficult for individuals to copy and distribute content at scale outside of the broadcast or retail institutions.
If you wanted to see a movie you had to go to the cinema and pay, or rent the video, or go to the store and buy the DVD. Opportunities to get a hold of cultural content outside of these channels was pretty difficult.
Sony Corp. or America v University City Studios, Inc.The Sony Betamax was the first personal recorder that enabled people to tape television shows. The film and television industries took that view that a device that enables people to record television and film at home had no legal purpose. It could only be used to steal content.
They were serious about this, taking their case all the way to the Supreme Court, attempting to get home-taping devices like the Betamax and VCR declared illegal. They lost.
The Supreme Court found that:
noncommercial home recording of television broadcasts for the purpose of “time-shifting” was fair use. It held that, given the nature of televised works and the fact that viewers had been invited to watch the programs in their entirety free of charge, reproduction of the entire work “does not have its ordinary effect of militating against a finding of fair use.” The Court further held that the plaintiffs failed to demonstrate any likelihood of more than minimal harm to the potential market for, or the value of, their copyrighted works.
This decision affirmed two important principles.
The first is that it is a legal and fair use of copyright material to make copies of it for personal use. It is OK to copy a song onto cassette, make a mix tape, and share that with a friend. It is OK to tape a show of television and watch it later.
The second is that device manufacturers cannot be held responsible for the illegal use of their devices. So, if people do use the VCR to say manufacture and sell copies of copyright television shows or films, the VCR manufacturer cannot be held responsible. This principle flows through till today. For example, internet service providers and social media platforms have only a limited responsibility for what users do on them.
But, the principle is under threat for a range of reasons.
At the industry level carriers and content institutions are not necessary structurally separate. Is Facebook a content provider or a carrier? We might argue that they provide both functions.
At the technical level, digital media enable new kinds of surveillance that make it possible to monitor content in ways that were not possible before. To monitor the content of phone calls used to require employing human eavesdroppers to listen in and make sense of conversations. But now, phone calls can be subjected to automated analysis of metadata and even call content.
At the political level, music, film, television, and games studios, place pressure on courts and legislators to make carriers like internet service providers and social media platforms responsible for the content that flows through their networks and servers.
SOPA/PIPAIn the digital age, a big political and legal question is what responsibility do carriers like internet service providers, social media platforms, streaming services and search engines have to monitor the content users upload and circulate?
The SOPA/PIPA legislation was a key moment in this tussle between the 'content' media industries and the 'internet' industries.
For a great primer on copyright in the digital age and the SOPA/PIPA legislation you can check out this episode of Decode DC.
The Stop Online Piracy Act (SOPA) and PROTECT IP Act (PIPA) were introduced to US Congress in 2011. The legislation provided for organisations that owned copyright to request that courts (1) bar advertising networks and payment providers from conducting business with infringing websites, (2) block search engines from linking to infringing websites, and (3) order internet service providers to block access to infringing websites.
The legislation was a radical departure from existing laws that regulated copyright online. Existing laws mostly only enabled copyright holders to get the courts to order a take down notice for specific infringing items of content. This meant a court to order a website to remove copyright material, but the court could not shut down the whole website.
Activists and technical experts argued the SOPA/PIPA legislation would fundamentally undermine the open infrastructure of the internet.
On the one hand we have the content industries arguing the sharing of copyright content threatens their commercial viability and the livelihoods of the people they employ.
On the other hand we have the technology industries arguing that imposing regulation on them would threaten their commercial viability, and civil society activists arguing that regulation would be a threat to free speech.
Who is in the game?To make sense of this situation we can begin by identifying all the groups who have an interest, figure out what the alliances are between them, and how they might go about getting what they want.
- Content producing industries like film and television studios, music labels, games producers.
- Creatives like writers, musicians and artists.
- Record stores, video stores, cinemas.
- Internet service providers.
- Commercial internet platforms: Google, Facebook, Twitter, Instagram, Snapchat, Reddit.
- Streaming services: Spotify, Netflix and Amazon.
- Legislators.
- Law enforcement.
- Peer to peer, file sharing and torrent platforms like The Pirate Bay.
- Consumers.
- Civil society activists.
All of these groups have an interest in how copyright is regulated.
Who shares common interests?Two clusters of interests seem particularly important.
There is a cluster of groups whose commercial business models depend on being able to sell cultural content. This brings together creatives like writers and musicians, with the studios and labels that fund their work, and the distribution businesses that sell content to audiences.
And, there is a cluster of groups whose commercial business models and political values gravitate toward ‘open’, ‘shareable’ and ‘searchable’ content. This includes civil society, open internet, and free speech activists together with commercial internet service providers and platforms.
But, here we see more complicated relationships.
Platforms like say YouTube want cultural content to be as shareable as possible. But, over time they have established legal and commercial arrangements with the big content players – they pay licensing fees for the copyright content on their platform.
Streaming services want to have access to a large database of searchable cultural content – music, film and television – but they also want to be able to control access to that database to generate subscription fees.
Consumers might feel sympathetic to the appeals of the content industries to ‘support artists’ by paying for content, but in their everyday practices they show a great willingness to access and download content illegally through torrent sites.
Internet service providers are primarily concerned with the legal risk and financial costs of being made responsible for monitoring content that flows through their servers.
What are the points of collaboration, cooperation and conflict between these groups?Who will need to win over who to get what they want?
Copyright industries aim to win over the legislators to place greater responsibility on the carriers to monitor content. During the period when the SOPA/PIPA legislation was being debated these industries made enormous donations to legislators.
Civil society activists attempt to convince legislators of the importance of the political and technical principles relating to the open architecture of the internet.
Internet service providers and platforms attempt to minimise their legal and financial exposure to the legislation.
When the SOPA/PIPA legislation was proposed it had extensive support among legislators and looked set to pass into law.
The legislation began though to attract criticism from activists because of its radical departure from established norms that separated content from carriage, and the threats this posed to free speech. SOPA/PIPA would enable a whole carrier to be shut down on the basis of one infringing item of content. For instance, a copyright holder could go to court and get an order to shut down all of YouTube if they could demonstrate the platform had unlicensed copyright material on it. The law would establish the principle that courts could 'block' whole websites or platforms, opening the door to closing down the open architecture of the internet.
Aaron Swartz and the activism against SOPA/PIPAOne particularly important activist in the fight against SOPA/PIPA was Aaron Swartz.
Swartz is an important figure in the history of the internet and online activism. His story is a tragic one. In 2013, Larissa Macfarquhar wrote in her detailed obituary for Swartz 'Requiem for a Dream' in The New Yorker:
Aaron Swartz hanged himself in his apartment in Brooklyn on January 11th. He was twenty-six, but he had been well known as a computer programmer for many years. At the age of fourteen, he helped to develop the RSS software that enables the syndication of information over the Internet. At fifteen, he e-mailed one of the leading theorists of Internet law, Lawrence Lessig, and helped to write the code for Lessig’s Creative Commons, which, by writing alternatives to standard copyright licenses, allows people to share their work more freely. At nineteen, he was a developer of Reddit, one of the world’s most widely used social-networking news sites.
After Reddit was sold, to Condé Nast, he turned away from money-making start-ups and became a political activist. He spoke often at technology conferences and activist gatherings, and was admired in both those worlds. Since his death, he has become a hero to programmers who have not turned away from money but wish they had, and to those who believe that governments are crushing what was once the freedom of the Internet. When Anonymous hacked the State Department Web site on February 17th, they declared, “Aaron Swartz this is for you.”
Two years ago, he was indicted on multiple felony counts for downloading several million articles from the academic database JSTOR. It is not clear why he did this. He may have wanted to analyze the articles, or he may have intended to upload them onto the Web, so they could be accessed by anyone. It is clear that he did not anticipate the astonishing severity of the legal response. He did not consider his JSTOR action an act of civil disobedience for which he was prepared to sacrifice a portion of his life in prison.
The response to his death was immediate and astonishing. It was not just because he was young and he had killed himself. He represented many things to people. “If you look at 2011 to the present, there’s an incredible emotional rollercoaster about Internet freedom and the Arab revolutions,” Quinn Norton says. “The Internet was going to change everything, and at the end of 2011 you had Occupy. And then everything just got destroyed. 2012 was the year, globally, for the heightening of censorship and the heightening of surveillance, and then Aaron killed himself. Aaron was so much the Internet’s boy, and that so much exemplified this machine crushing our hopes.”
The Internet's Own BoyThroughout his life Swartz offered a model for doing activism in the digital era that was attuned to the power relations and possibilities of these new technologies.
Swartz combined ‘old fashioned’ forms of activism: public speeches, protests, petitions, engagement with legislators with newer tactics that directly exploited the weaknesses and capacities of digital infrastructure. This included relatively ordinary online petitions, to internet ‘black outs’, to using code to create new configurations of information and to make it available in the public domain.
Swartz illustrated that in the digital age the use of media to exercise power extends beyond just expressing our views and organising ourselves, it also includes fundamental battles over who controls the infrastructure through which flows of information are controlled. Activism must also be directed at the infrastructure itself: who gets access to information.
One of Swartz' earliest activist interventions was to make public court records available and easily searchable online.
Below is an excerpt from the film The Internet's Own Boy that covers Swartz' effort to make the entire legal history of the United States freely available on the internet.
Swartz was investigated by the FBI for his actions in downloading legal records. He uploaded his FBI file here. The FBI describe Swartz' actions as follows:
The U.S. Courts implemented a pilot project offering free access to federal court records through the PACER system at seventeen federal depository libraries. Library personnel maintain login and password security and provide access to users from computers within the library. PACER normally carries an eight cents per page fee, however, by accessing from one of the seventeen libraries, users may search and download data for free.
Between September 4, 2008 and September 22, 2008, PACER was accessed by computers from outside the library utilizing login information from two libraries participating in the pilot project. The Administrative Office of the U.S. Courts reported that the PACER system was being inundated with requests. One request was being made every three seconds. […] The two accounts were responsible for downloading more than eighteen million pages with an approximate value of $1.5 million.
Wired magazine also offer an account of Swartz' action here:
The Great Court Records Caper began last year when the judiciary and the Government Printing Office experimented with giving away free access to PACER at 17 select libraries around the country. Swartz decided to use the trial to grab as many of the public court records as he could and, perversely, release them to the public.
He visited one of the libraries — the 7th U.S. Circuit Court of Appeals library in Chicago — and installed a small PERL script he’d written. The code cycled sequentially through case numbers, requesting a new document from PACER every three seconds. In this manner, Swartz got nearly 20 million pages of court documents, which his script uploaded to Amazon’s EC2 cloud computing service.
Or, as the FBI report put it, the public records were “exfiltrated.”
The script ran for a couple of weeks — from September 4 to 22, until the court system’s IT department realized something was wrong. Someone was downloading everything. None of the records, of course, were private or sealed, and Lexis Nexis has a copy of of PACER’s database that it sells a high markup. But Swartz wasn’t paying anything.
Read more about the case at Ars Technica too.
Here's an excerpt of the film The Internet's Own Boy that covers Swartz' activism against SOPA/PIPA. This section of the film provides a clear account of the legislation, the activism against it, and how it was defeated.
You can find the full film in the EduTV database of university libraries.
The Freedom to ConnectAt the Freedom To Connect conference in 2012 Swartz gave a speech that told the story of how the SOPA/PIPA legislation was defeated. The speech is a kind of 'instruction manual' for how to do hegemony. Swartz explains how he went about fashioning an alliance of players who shared a common-cause, shifted the common-sense debate about copyright, and made the content industries’ position untenable.
You can watch the speech and read the transcript here.
The speech starts off by making an important move: define the problem on your own terms.
Swartz sets out by arguing that the SOPA/PIPA legislation should not be understood as a copyright issue, but as a free speech issue.
So, for me, it all started with a phone call. It was September—not last year, but the year before that, September 2010. And I got a phone call from my friend Peter. "Aaron," he said, "there’s an amazing bill that you have to take a look at." "What is it?" I said. "It’s called COICA, the Combating Online Infringement and Counterfeiting Act." "But, Peter," I said, "I don’t care about copyright law. Maybe you’re right. Maybe Hollywood is right. But either way, what’s the big deal? I’m not going to waste my life fighting over a little issue like copyright. Healthcare, financial reform—those are the issues that I work on, not something obscure like copyright law." I could hear Peter grumbling in the background. "Look, I don’t have time to argue with you," he said, "but it doesn’t matter for right now, because this isn’t a bill about copyright." "It’s not?" "No," he said. "It’s a bill about the freedom to connect." Now I was listening.
SOPA/PIPA was unique because rather than issue an individual take down notice for infringing material, it proposed to block whole websites. To make this sound both radical and absurd, he offers a simple illustration.
But this was something radically different. It wasn’t the government went to people and asked them to take down particular material that was illegal; it shut down whole websites. Essentially, it stopped Americans from communicating entirely with certain groups. There’s nothing really like it in U.S. law. If you play loud music all night, the government doesn’t slap you with an order requiring you be mute for the next couple weeks. They don’t say nobody can make any more noise inside your house. There’s a specific complaint, which they ask you to specifically remedy, and then your life goes on.
The closest example I could find was a case where the government was at war with an adult bookstore. The place kept selling pornography; the government kept getting the porn declared illegal. And then, frustrated, they decided to shut the whole bookstore down. But even that was eventually declared unconstitutional, a violation of the First Amendment.
Then he goes on to define what the ‘battle’ is really about.
There’s a battle going on right now, a battle to define everything that happens on the Internet in terms of traditional things that the law understands. Is sharing a video on BitTorrent like shoplifting from a movie store? Or is it like loaning a videotape to a friend? Is reloading a webpage over and over again like a peaceful virtual sit-in or a violent smashing of shop windows? Is the freedom to connect like freedom of speech or like the freedom to murder?
This bill would be a huge, potentially permanent, loss. If we lost the ability to communicate with each other over the Internet, it would be a change to the Bill of Rights. The freedoms guaranteed in our Constitution, the freedoms our country had been built on, would be suddenly deleted. New technology, instead of bringing us greater freedom, would have snuffed out fundamental rights we had always taken for granted. And I realized that day, talking to Peter, that I couldn’t let that happen.
Swartz makes the important move of making his position seem sensible. He does that by defining the problem on different terms. This is the first move in building common-cause with others, come up with a position – a way of understanding the situation – that others will agree with.
He shifts the whole debate from copyright of content to larger questions about free speech and the communication infrastructure of society.
He then goes on to do his own analysis of the current state of play. He points out that this was incredibly unusual, the legislation came to congress and was set to pass without any debate.
This demonstrated how strategically astute the copyright industries were, and how much work they’d done to create a strong consensus with legislators on copyright reform.
This ‘relationship building’ is resource intensive. During the period when SOPA was being debated he reports that the film, television and music industries were donating something in the region of $180 million to members of congress, while the internet industries were donating around $15 million. For more information on donations to politicians see Open Secrets.
This is some indication of how well resourced and organised the copyright industries were, and how much they were invested in ensuring legislators in congress were on their side and saw their position as common-sense.
The two chief ways they made their position common-sense was by prosecuting the claim that illegal downloading threatened industries and jobs because the content businesses were big employers. They also argued that illegal downloading undermined the quality of our cultural life, it matters to the fabric of our society that we supported artists’ livelihoods so that they can construct our cultural world.
Swartz observed, that when he looks at this situation, whoever was behind the legislation ‘was good’. He realised that the copyright industries had built a powerful consensus and alliances in the legislative process.
Swartz was a political activist. He had to find a way to build both an alliance of other powerful actors and a consensus:
Now, the typical way you make good things happen in Washington is you find a bunch of wealthy companies who agree with you. Social Security didn’t get passed because some brave politicians decided their good conscience couldn’t possibly let old people die starving in the streets. I mean, are you kidding me? Social Security got passed because John D. Rockefeller was sick of having to take money out of his profits to pay for his workers’ pension funds. Why do that, when you can just let the government take money from the workers? Now, my point is not that Social Security is a bad thing—I think it’s fantastic. It’s just that the way you get the government to do fantastic things is you find a big company willing to back them. The problem is, of course, that big companies aren’t really huge fans of civil liberties. You know, it’s not that they’re against them; it’s just there’s not much money in it.
The problem was that the industries who might care, the internet industries, were staying out of the argument.
The fact is, the big Internet companies, they would do just fine if this bill passed. I mean, they wouldn’t be thrilled about it, but I doubt they would even have a noticeable dip in their stock price. So they were against it, but they were against it, like the rest of us, on grounds primarily of principle. And principle doesn’t have a lot of money in the budget to spend on lobbyists. So they were practical about it. "Look," they said, "this bill is going to pass. In fact, it’s probably going to pass unanimously. As much as we try, this is not a train we’re going to be able to stop. So, we’re not going to support it—we couldn’t support it. But in opposition, let’s just try and make it better." So that was the strategy: lobby to make the bill better. They had lists of changes that would make the bill less obnoxious or less expensive for them, or whatever. But the fact remained at the end of the day, it was going to be a bill that was going to censor the Internet, and there was nothing we could do to stop it.
So I did what you always do when you’re a little guy facing a terrible future with long odds and little hope of success: I started an online petition. I called all my friends, and we stayed up all night setting up a website for this new group, Demand Progress, with an online petition opposing this noxious bill, and I sent it to a few friends.
Swartz arguably has two kinds of resources: the technical skills and social capital to build online networks and generate publicity. Swartz used these skills to generate a new kind of discussion about the legislation, separate to the copyright industry position. A discussion that focussed on his way of defining the problem.
And, he used his ‘insider’ status in the technology industry to convince technology leaders that they ought to get involved in resisting the legislation.
He was trying to convince them that it was both politically 'right' and in their longer term commercial interests to prevent these ways of regulating the web to emerge.
If legislation like this passed, for instance, the onus would more and more on internet service providers and platforms to monitor and filter content. That could be expensive, increasing their legal and commercial risks. But also, it cut against the free and open web that many in the technology industries believed in.
Swartz found ways to get others involved, buy into his position and build an alliance. On that alliance, Swartz says:
But, you know, I think that story illustrates what happened during those couple weeks, because the reason we won wasn’t because I was working on it or Reddit was working on it or Google was working on it or Tumblr or any other particular person. It was because there was this enormous mental shift in our industry. Everyone was thinking of ways they could help, often really clever, ingenious ways. People made videos. They made infographics. They started PACs. They designed ads. They bought billboards. They wrote news stories. They held meetings. Everybody saw it as their responsibility to help.
The legislation was defeated in part because Swartz built an alliance of actors who prosecuted a different way of looking at the legislation. Hegemony involves both common-sense and alliance building. Swartz did both.
If there was one day the shift crystallized, I think it was the day of the hearings on SOPA in the House, the day we got that phrase, "It’s no longer OK not to understand how the Internet works." There was just something about watching those clueless members of Congress debate the bill, watching them insist they could regulate the Internet and a bunch of nerds couldn’t possibly stop them. They really brought it home for people that this was happening, that Congress was going to break the Internet, and it just didn’t care. ... And then we started rubbing it in. You all know what happened next. Wikipedia went black. Reddit went black. Craigslist went black. The phone lines on Capitol Hill flat-out melted. Members of Congress started rushing to issue statements retracting their support for the bill that they were promoting just a couple days ago. And it was just ridiculous. I mean, there’s a chart from the time that captures it pretty well. It says something like "January 14th" on one side and has this big, long list of names supporting the bill, and then just a few lonely people opposing it; and on the other side, it says "January 15th," and now it’s totally reversed—everyone is opposing it, just a few lonely names still hanging on in support.
I mean, this really was unprecedented. Don’t take my word for it, but ask former Senator Chris Dodd, now the chief lobbyist for Hollywood. He admitted, after he lost, that he had masterminded the whole evil plan. And he told The New York Times he had never seen anything like it during his many years in Congress. And everyone I’ve spoken to agrees. The people rose up, and they caused a sea change in Washington—not the press, which refused to cover the story—just coincidentally, their parent companies all happened to be lobbying for the bill; not the politicians, who were pretty much unanimously in favor of it; and not the companies, who had all but given up trying to stop it and decided it was inevitable. It was really stopped by the people, the people themselves. They killed the bill dead, so dead that when members of Congress propose something now that even touches the Internet, they have to give a long speech beforehand about how it is definitely not like SOPA; so dead that when you ask congressional staffers about it, they groan and shake their heads like it’s all a bad dream they’re trying really hard to forget; so dead that it’s kind of hard to believe this story, hard to remember how close it all came to actually passing, hard to remember how this could have gone any other way. But it wasn’t a dream or a nightmare; it was all very real.
Swartz’ alliance included the public, using his networking skills to get them to share information about the blackout on social media and to contact their members of congress.
Support for the bill disappeared.
The copyright industry alliance broke down, the alliance Swartz had developed around the principles of an open internet architecture was now the common-sense position. Legislators moved toward this position.
Swartz' speech is a map for hegemony-building by using the tools of online networks, mobilising a large public to take small actions that amounted to something bigger. Importantly, hegemony building is always a work in progress. He explains that the enemies of the ‘freedom to connect’ have not disappeared, they will regroup, attempt to fashion new alliances, and seek what they want in the political and legal process again.
There are a lot of people, a lot of powerful people, who want to clamp down on the Internet. And to be honest, there aren’t a whole lot who have a vested interest in protecting it from all of that. Even some of the biggest companies, some of the biggest Internet companies, to put it frankly, would benefit from a world in which their little competitors could get censored. We can’t let that happen.
Swartz here makes a critical point about hegemony, the making of meaning goes on and on. It is never finished or closed because it is always bound up in the ongoing struggle of different groups in society to organise the world in ways that reflect their interests, desire and values.
This case helps to illustrate how events in the world are made meaningful using media and how those meanings spill out and circulate in unexpected ways. But also, we can observe how powerful groups try to control and organise how the event is made sense of.
In October 2016, after Trump mentioned Chinese economic policy in the debate, Fox News reporter Jesse Watters went to New York’s Chinatown to discuss the election. He interviewed residents who could not speak English and made fun of their lack of response to his election questions. In between clips of Watters laughing at elderly people were clips from kung fu films, and clips of Waters himself doing karate, getting a foot massage and bowing at Asian people.
The point of the segment was to present Chinese immigrants in the US as uninterested and uninformed about the upcoming election, and ultimately exclude them from a larger American cultural identity.
Comedian Ronny Chieng, a senior correspondent for The Daily Show, responded to the Fox News coverage in an expletive filled segment that went viral. After identifying every racist cultural frame, word and image in the Fox News segment and decoding them in plain language for audiences.
The recording examines how Chieng acknowledged not just stereotypes that were engaged to represent Chinese people and Asian communities at large (pointing out that karate is Japanese) but also how Watters structured the segment. Watters only posed questions about the election to elderly Chinese residents in a language they could not understand, while those who could speak English were asked about karate.
Chieng is quick to show that the segment relied on putting interviewees in a position where they could not respond to or consent to the show’s representations. Here Chieng brings up still photos of Watters and screams assumptions based on Watter’s appearance as a general douchebag (for example, that Watters drugs women’s drinks at bars). Chieng demonstrates how anyone in the right context can be rendered passive, can be defined by cruel stereotypes for laughs.
But the real strength of Chieng’s critique, besides decoding an onslaught of racist messaging, is in his ability to demonstrate how the segment could more accurately represent Chinese-Americans if encoded differently.
Media mostly exercise power by enabling the creation and circulation of ideas we take to be ‘common-sense’. This notion of ‘common-sense’ ideas can be understood as ‘hegemony’:
Societies are cohesive when a particular way of life and its associated ideas are taken as common sense by a population. Powerful groups are hegemonic when their preferred ideas seem natural and inevitable.
In this recording we consider writer Helen Razer's take on how social media are used to police what meanings are taken as ‘common-sense’, to very clearly judge what can and can’t be said.
Notes | Quotes from Helen Razer's 'From Hipsters to Racists, everyone suffers the 'idiot reflex' of outrage' in Crikey, 27 January 2016.
The power to control the processes through which meaning is made, circulated, and made sense of matters.
Controlling representation matters because it shapes how people understand the world, which affects how they act in the world.
Here, Stuart Hall’s Encoding/Decoding model is particularly important. The Encoding/Decoding model was particularly important as part of a turn toward ‘active audiences’ or ‘reception theory’ in the 1970s. This turn paid attention to who audiences were ‘active participants’ in making sense of and incorporating media representations into their everyday practices and identities.
The claim was that media had to be understood as a social process, and that meant that the meanings of media representations could not be understood by simply ‘analysing’ the text (what was on the screen or in the image), but had to be understood by going and observing how audiences, or receivers, of representations made sense of them. Critically important, was the claim that there was no ‘single’ meaning in a text, but that the same text could have different meanings to different audience members because they used their differing positions in the social and cultural world to make sense of media.
Hall’s encoding/decoding model was first published in the early 1970s. His claim was that while some people have more power to control the resources used to create meanings, and the structures and spaces where meanings circulate; no one has complete control over how meanings are encoded and decoded.
Powerful groups have the capacity to control media institutions and technologies, and the resources to employ and direct professional communicators.
That level of control only gets them so far, because representations work discursively. Once produced, messages have to be circulated. They become meaningful when incorporated into social practices and institutions. Institutions can’t entirely control those social practices.
In the recording we examine some of the key claims in Hall's encoding/decoding essay.
A key scholar who has shaped our understanding of the processes of meaning making and representation is cultural theorist Stuart Hall. Hall was born and raised in Jamaica and moved to the UK in the 1950s, producing a number of influential works on media and representation during the civil rights movement up until his death in 2014.
A text is not a literal text, but in Semiotics refers to a combination of signs, signifieds and mechanisms like metonymy. A text could be a sentence, a paragraph, an image, a story, or a collection of stories.
A collection of signs in a single photograph or painting, a video clip, a television show, a feature film. Whenever signs come together in the land of semiotics, they become texts. These texts can be understood, rearranged and put together in different combinations, with different meanings to different groups of people.
Cultural texts refer to sign systems, storytelling tools and symbols that contribute and shape a society’s culture. They have underlying cultural meanings. They either require certain cultural knowledge to be understood, they are produced through a certain cultural context or, as most texts do, become representative of a culture and its values.
But cultural texts are not one-dimensional. A text is not simply representative of one culture, it does not belong to one culture, even if it purposefully excludes others semiotically. Cultural texts are multi-dimensional, they are dynamic.
A cultural text is perhaps better understood as having cultural layers of understanding. Where groups different in age, race, nationality, sexual orientation may read and understand a collection of signs in different ways. Depending on the producer or the audience, the text itself has a kind of flexibility in meaning to different people when it starts to operate culturally.
Semiotics more generally poses a number of questions in regards to cultural texts and the stories they tell. Questions like what makes a cultural text in the first place? What is defined by a cultural text, what is included and what is excluded? How are cultural texts used to represent society at large? How do cultural texts even become representative? What do signs signify culturally, and how and why do they become symbolic in the first place.
Snap JudgmentMusic producer and former diplomat Glynn Washington, inspired by This American Life radio host Ira Glass created the radio show Snap Judgment.
Washington has been praised for drawing a more diverse and younger audience to talk radio and podcasting, mediums that had previously been associated with middle aged white men.
Snap Judgment is a storytelling program following the format of This American Life, telling a series of stories along the same theme.Like Glass, Washington opens each episode with an almost bombardment of signs and signifieds to hook the audience, layered in connotations and denotations, while leaning on music to keep the show’s meta-linguistic momentum.
Washington himself opens each show with a story from his life, introducing the theme for the hour. Washington is African-American, raised in Detroit, and has worked as an educator, diplomat and activist in addition to a radio producer. Washington as the show’s host draws on his experiences growing up in Detroit and his experiences travelling internationally as part of his diplomatic work.
Washington’s program draws on his personal experiences as an African American man living and working in America and as a diplomat travelling around the world. As a result, his program engages a number of sign and signifieds that are particularly meaningful to different groups of people, both in African American communities and globally. Washington opens the episode 'Homecoming Surprise' like this:
Ok, so I had just returned from a year overseas. In Japan. A magical, transformative year. And my ex-girlfriend meets me at the airport. Ex, because we decided that a full year apart…well. She’s beautiful, beautiful like Whitney Houston before the darkness beautiful. Dressed in a white lace dress, pure. I thought of her every single day. And finally she’s wrapped tight against me, tears. I hug her, she kisses me…and I know. Right then I know that any reconciliation I may have hoped for…it’s not going down because something…because someone has happened. And I even know who.
When he describes his ex-girlfriend as ‘beautiful like Whitney Houston before the darkness’, we can infer that she was a beautiful African American woman. Houston before she lost her battle with drug addiction. Within African American communities, ‘Whitney before the darkness’ is a cultural sign whose signifieds are perhaps more detailed. Whitney Houston in the 1980s was the quintessential good girl. A girl that was good at everything, that was maybe too good for you to be with. Since this reference to a good innocence may be lost on an international audience, Washington follows it up with a description of his girlfriend’s ‘white lace dress’ another sign of purity, to maybe keep us all on board with who this woman is and what she represented to him.
Both Snap Judgment and This American Life operate as cultural texts and engage a variety of signs and signifieds as part of their storytelling. But there are a few crucial points of difference.
One point of difference is that all of the stories on Snap Judgment address people around the world making a crucial life decision. The show’s stories are global in scope. This American Life although broadcasted globally, is primarily interested in contributing to a more dynamic American cultural identity. It tends to select stories narrated by Americans, and if it is an international story, it relates back to American foreign policy in some way.
In the structure of storytelling, Snap Judgment aims to go directly to a story’s arc, directly to the point of tension where a person must make a decision that impacts the course of their life and the lives of others.
Past stories featured on the show have included the story of a 1970s rock band in Zimbabwe, a British woman who responds to a personal ad to live with a complete stranger on an island in the Torres Strait, to American magicians attempting to do the dangerous bullet catch trick.
A second point of difference is the show’s reliance on a staff of audio engineers, producers and story contributors diverse in age, race and nationality. This operates in many ways to diversify the show’s reading as a cultural text.
The music itself guides the story along, or provides a rhythm to the story subject’s particular way of speaking. Washington describes the show’s elaborate audio production as a crucial part of their approach to storytelling:
What do we mean by storytelling with a beat? Every person speaks with a cadence, thinks with a cadence, you hear the rhythm of the person as they speak. And you get into their world, they hypnotize you into getting into their own thought perspective. That’s what it’s all about. My problem often times with public radio storytelling is that we try to translate it for them. We put the mic in the guy’s face, he tells you an answer, you translate it back to your audience. Get out of the way. Let the person tell their own story. I want to feel at the end of it that we’ve taken you for a ride. Strap in, let’s go.
Washington is particularly interested in developing a kind of semiotic rhythm where the words spoken by different storytellers are given their own unique rhythm that preserves their voice, their individuality.
This brings us to a third point of difference when comparing both programs as cultural texts. Personal, individual narration is a key component to Snap Judgment’s success. When Washington warns to ‘get out of the way’ of your interview subjects, as a producer, he sees the value in presenting a diverse set of physical voices and experiences to his audience.
Cultural texts are influenced not only by the cultural backgrounds of the audience but the producer as well, or the storyteller in this case.
When Washington describes the process of ‘translating’ as an interviewer in radio, he is describing the process of re-stating or reframing semiotically the voice and meaning of your subjects’ statements, signs and signifieds.
We are either translating culturally, reframing certain signs into a relatable cultural context. Or we are translating, most likely, to make the subject’s voice, demeanor, or words more media friendly, more culturally mainstream, or more entertaining.
Snap Judgment and This American Life both emerge as cultural texts in their own right, and both contribute to cultural identity. However, their boundaries as cultural texts are drawn differently.
While This American Life consists of predominantly white middle aged American producers and reporters including stories with diverse subjects, the strength in Washington’s program is that the voices themselves are as diverse as its subjects.
Through Washington’s personal narration alone, the program engages a number of signs and symbolic systems that are meaningful and specific to an African-American cultural experience in ways that may not resonate with a wider global audience.
But at the same time, Washington’s diplomatic experiences, along with the widespread popularity of the program and the diversity of its contributors - as audio engineers, as storytellers - the program also contributes to a larger uniquely American identity, but also a global one.
It emerges as an American cultural text contributing to narratives of struggle, success and collective identity through shared personal experiences.
But it also emerges as a global pop cultural text in the personal sharing of stories from around the world.
With a metaphor, one word stands in for another word - ‘love is a battlefield’ or ‘apple of my eye.’ One thing is the other thing. It is not like the other thing, it stands in for that thing. With metonymy, one sign stands in for a larger concept or group of people, connected by a number of complex associations to become meaningful to different audiences.
Semiotics asks, what narratives around a sign or symbol are created? Why are people drawn to some signs over others? What makes people click on a headline, follow a link, or read a whole story? Here an understanding of Semiotics is helpful on two fronts: in constructing and producing meaningful signs and narratives as communication professionals and in deconstructing the meanings of signs and narratives as empowered media consumers. It is both the production and reception of a message.
Mass societies are characterised by a balance between coercion and consent. Where other institutions like the courts and prisons operate as a coercive mode of power, media institutions exercise power via consensus, or agreement.
One argument is that professional communicators make shared ways of life. For much of the twentieth century we might think of this industry as producing a narrative about what it means to live a ‘good life’. Moreover, it produces a group of people – a population, a society, a public, audiences – who enter into, desire and practice this ‘good life’.
The mass circulation of media goes hand in hand with industrial forms of production. Without factories to produce goods that needed to be bought and to pay wages to workers who would buy those goods, there would be nothing to advertise and nothing to fund the emergence of commercial media. Media technologies and institutions are critical to the organisation of day to day life, markets and political institutions in the mass society.
There are perhaps two important points to derive from Lippman’s story then. The first is the way humans use symbols to create ‘pictures’ of how the world is. Lippman’s story illustrates that how the world is represented matters because it shapes how humans vunderstand the world. And, how humans understand the world affects how they act in the world. The second is that these pictures are created and disseminated via material infrastructure. In the case of Lippman’s story, news travels on a ship.
The study of media then is, like other social and cultural fields of study, about examining the social structures and technologies we invent to orchestrate a shared life in the world with other humans, animals and the natural environment. To get started at least, there are two basic building blocks. Meaning: the human capacity to use our bodies to communicate with one another. Media: the human capacity to create material structures, tools and institutions that alter our lived experience and the world around us in durable ways.
Media have at least two functions that play a critical role in our social world. We might call this their ‘cultural’ and ‘technical’ functions. At the cultural level: media enable the creation and distribution of information, images, sounds, symbols and narratives. At the technical level: they collect, store, process and distribute process information.