Dancing with Pixels: Digital Artefacts, Memory and the Beauty of Loss

Pam Cook

The Cine-Files, issue 7 (fall 2014)

My video essay Timeless is part of an open-ended series that explores cinematic memory.[1] It was made by editing together clips from Wong Kar-wai’s 2000 movie In the Mood for Love, downloaded from a YouTube copy that is good quality, even in full screen mode, with few obvious signs of image deterioration.[2] The provenance of the YouTube copy is unknown, but a small, superimposed logo that reads “ECHiZEN” in the top left-hand corner of the front credits suggests there may have been Japanese involvement. I have no way of knowing which generation this copy is, but when I imported the selected clips into iMovie significant image degradation occurred, including colour fading, banding and pixel noise. The problem was exacerbated when I exported the video essay using Quicktime. My initial reaction was one of disappointment and I considered discarding it. I made a second video essay for which I edited together still images from the YouTube copy, which improved matters slightly, but fading and “noise” persisted.[3] With the software available to me, it was not possible to produce better quality images. Because of the aesthetic perfection of Wong’s film, I perceived this as a limitation, a negative effect of the processes of appropriation, importing and exporting involved in digital videography.

However, technical constraints also offer opportunities. My sense of loss provoked me to reflect on the affective resonance of image deterioration and its potential as a conduit for memory and nostalgia. With its poignant meditation on lost time, In the Mood for Love lends itself to this approach. There are other implications, too: the aesthetic of imperfection draws attention to videography’s micro-scale, cut-and-paste mode of production and situates it outside, sometimes in opposition to, mainstream digital output and the aspiration to hyperrealism. Pixel noise can produce painterly effects and textures that recall hands-on artistic activity at the same time as registering the autonomy of technical processes that are beyond personal control. This tension between individual self-expression and the possibilities and limits of the medium underpins the ambivalent relationship between consumers and technology.

Elsaesser and Hagener, among others, have pointed to the transition of agency from human to non-human as a consequence of the shift from analogue to digital production and reception.[4] It is not a huge leap from there to Friedrich Kittler’s pronouncement “Media determine our situation.”[5] Kittler’s suggestion that human beings are not in control of technology but adapt to it is resonant in the digital era, when physical contact between person and computer drives our interactions with media channels that store and process data, defining our perceptions of the world and other people. On one hand, the digital presents augmented user experiences: astonishing special effects; pristine facsimiles; fabulous image and sound quality; speed of access and delivery; tactile relationships with hand-held devices and screens; unprecedented ability to manipulate electronic processes and objects; mobility across platforms, time and distance; and the power to “see” just about everything. Consumers are producers, and often appear less restricted by legal and financial pressures than professional business concerns.

On the other hand, the proliferation of different frames and screen sizes and increased activity between them, the limitations that result from viewing situations and use of peripherals, and technological instability, all fracture, contain and determine user activity. Although human agency is not obsolete (it is vital to merchandising, for example), the ways in which objects interact with one another, generating audio-visual experiences without obvious human intervention, implies a struggle for autonomy between human and machine. This conflict has affective consequences: the failed aspiration to mastery and control may create anger and frustration, leading to the impulse to overcome obstacles and strive for (impossible) perfection. It may also give rise to nostalgia, a yearning to retrieve and visualise the human element in pre-digital modes of production.[6] Or it may result in acceptance of loss and a willingness to engage with the limits of human agency and control. These mixed emotions surface in the melancholy of Wong’s melodrama, in my videographic intervention, and in my reflections here.

Digital artefacts

Digital copying of audio-visual material across different formats often results in degeneration and the appearance of unwanted “artefacts” in the image. Colour fading, banding (visible lines that separate out areas) and pixel noise are some of the issues faced by producers of still and moving digital images. An entire industry has grown up to correct such aberrations, exploiting the widespread belief that digital reproduction supersedes other technologies. That is, it generates high-resolution images of superior clarity and detail and can also restore — indeed, significantly enhance — photographic material damaged by time. The amelioration discourses surrounding such techniques imply that the ravages of time can be overcome via software; however, the signs of its destructive impact can also be artificially created using sepia tones, faded colour, torn edges and photochemical traces. Special effects can produce a fabricated vintage “look” for digital photographs or moving images, testifying to the commercial value of nostalgia in an environment in which progress means visual and auditory perfection.

The preferred narrative of digital technologies is one of elimination of flaws, error and chance, a utopian trajectory predicated on the achievement of solutions and goals rather than failure. Yet malfunction is built into the system, in its networks and infrastructures as well as the technologies available to users. Shortcomings can be exploited, not only in the interests of guerrilla activity against state agencies, but also by those engaged in artistic and pedagogic work.[7] Much of this endeavour is experimental, exploring and analysing the expressive potential of the medium. See, for example, Rosa Menkman’s video essay, Demolish the Eerie Void (2010):

Some of it qualifies as “reflective nostalgia,” as Boym calls it, dwelling on what went before and creating art from the detritus that digital technologies leave behind.[8] The “unwanted” artefacts that result from digital processes can be mobilised to trouble the narrative of elimination and remind us that the marks of loss do not have to be erased.

Although digital artefacts do not have the physical presence in time of, say, the marks of chemical decay on a photograph, they do not simply exist in the present. The transition between platforms and software involved in copying produces a sense of duration and an idea of pastness (before and after) that contributes to the experience of loss. When the digital promise of an ideal object and total control founders, and progression to an error-free zone is thwarted, we are confronted with distracting blemishes that arouse desire for what might have been — in my case, that elusive YouTube copy, or the DVD version that may have produced better results. The memory of the 35 mm print also haunts the copied versions. Digital artefacts scatter texture across the image; for many, this may seem a poor substitute for the subtle nuances of celluloid graininess. This regret is related to the nostalgia for earlier methods of production described by Andrew.[9] Errant pixels are a visible reminder that human creativity has lost out to the machine, that the mastery and self-expression underpinning romantic ideas of artisanal production are an illusion.[10] No wonder that digital processes act as a powerful engine for the imagining of pre-digital Golden Ages. With its drive to retrieve and reorder fragments of cinema’s past, videography can mobilise reflective nostalgia to muse on digital technology’s predisposition to look backwards even as it moves forward.

Banding

Banding is a form of digital artefact that produces visible lines between similar colour areas when video is compressed and transferred between platforms. It can be fixed using post-production software.[11] It produces a “posterisation” effect that makes the transition between colours visible and reveals the layering characteristic of digital composition. It can be used as a modern design tool in art and advertising: a high-profile example is street artist and graphic designer Shepard Fairey’s “Hope” poster, which became iconic during Barack Obama’s 2008 presidential campaign and was the subject of a copyright dispute.[12] Fairey’s artwork, which can be simulated in Photoshop, is a prime example of layered composition and the production of textures using computerised techniques, but in its appropriation and transformation of a news photograph, it also chimes with the process of creating a new object from found footage in videography.[13]

There are many different editions of the poster, which has been reproduced countless times. In one striking version, the fusion between old and new is represented pictorially by offsetting stark blocks of primary colours, hard boundary lines and bold sans serif font (signs of modernity) with delicate superimposed textures that add nuance and an affective dimension to the inspirational message.[14] Some of the textures resemble television’s horizontal resolution lines; others are decorative marks in soft colours that hint at the materiality of textile or wallpaper. The design celebrates state-of-the-art computer techniques while echoing handcrafted screen-printing methods. There is also a suggestion, perhaps ironic, of comic-book art. The textural qualities of the poster are produced by the conscious overlay of multiple layers that “muddy” the areas of flat colour. This creates an aesthetic that resists the smooth, untroubled surfaces of digitally enhanced objects and suggests that hope (the future) resides in technical co-existence rather than elimination. As with accidental banding in digital video, the eye is distracted and impeded, confronted with the ironies of idealism.

Pixel noise

Pixel noise is a random variation of image density in digital images, usually unwanted because it is perceived as diminishing quality and legibility. Pixellation can happen as a result of different compression rates encountered in multiple transfers, or because of varying speeds in broadcast signals. It is often visible as interference: individual pixel squares jump across the screen, giving the appearance of disintegration. However, it also displays as colour bleeding and patching that lend a ghostly quality to the image, particularly in shots that feature movement. Pixel noise in digital video is mobile, creating the impression that parts of the image are animated. Like film grain, it is a textural product of technical processes, and it can be added to photographs or video to engender artistic effects. It can also be eliminated.

Accidental pixel noise disrupts the integrity of the image, rendering it vulnerable to decomposition: the image seems to be on the point of dissolving entirely into noise. See, for example, Jeremy Eichenbaum’s video essay (foil) FACED W/DOWNGRADE:

Such dramatic deterioration evokes the celluloid burnout imagery used in apocalyptic pronouncements that herald the death of cinema, which have intensified in the post-digital era.[15] The instability of the digital image is of a different order than celluloid volatility, but it seems to indicate that the technology holds the capacity for its own destruction within it. It projects a future death/disappearance of the representational image, after which there is only signal noise.[16] In the meantime, the videographer’s primary materials are digital objects of variable quality that render the cinematic past available for re-editing and analysis. Their engagement with digital technologies brings cinema back to life (if, indeed, it is dead), but they are faced with problems that test their ingenuity and technical competence.

In the tradition of moving image studies, academic videographers are motivated by respect for cinema’s history, which is reflected in their choice of source material and reverence for the unity of the image. In many cases, the digital can present them with restored historical objects of unparalleled clarity and detail, but in others, the signs of process (digital artefacts) deface the selected images/sound and interfere with the planned scholarly enquiry. Rather than lament the demise of authority and authorship, video essayists can use the visual and auditory potential of image deterioration creatively to reflect on scholarly research and the operations of videography itself. See, for example, Catherine Grant’s video essay: Mechanised Flights: Memories of HEIDI and Shirley Temple (2014):

This (ad)venture, allied to but different from video art, offers the opportunity to explore the materiality of the medium, its relationship to time and the past, and the psychic and emotional investments involved in videography’s memory work.

In the mood for loss

If the prime source material for the videographer is classic cinema or films with canonical status, their methods have more affinity with the avant-garde. Scholarly videography re-edits, re-orders and re-works fragments of cinema history for purposes of research, learning and pedagogy; this activity has much in common with the educational impetus of the montage experiments of Soviet filmmakers in the 1920s described by Hagener.[17] Hagener detects an iconoclastic drive, shared with other avant-gardes in the period, in the preoccupation with using collage techniques to destroy existing order and create fresh meaning from re-assemblage.[18] Similar destructive urges motivated those post-war American avant-garde filmmakers who used re-edited snippets of found footage to challenge mainstream representation.[19] Many produced dissident work that privileged the dirt, scratches and other marks effaced in commercial narrative films.[20] Wees suggests that in some cases, the procedures of collage, by removing film material from its original context, stress its status as image and overturn the premise of mainstream media depictions that assume an image’s transparent relationship to a referent in the real world.[21] For the avant-garde collagist, as for the videographer, the “real” resides in the fragments of historical material they rearrange for their own purposes. The gathered images and sounds relate to one another rather than reality, fashioning less-obvious meanings from juxtaposition.

The collagist’s manipulation of found objects is at the heart of the ludic mechanisms of reflective nostalgia, in which bits and pieces of the past ignored by official versions collide to generate discontinuous chronicles. For Boym, paraphrasing Nabokov, the meanderings produced by “amateurs of Time” shatter the authority, perfection and coherence of historical accounts that seek to restore a stable, unified past with no signs of decay.[22] She compares the fractured surfaces of reflective nostalgia to cubist painting.[23] Although these dilatory patchworks of memory may not have the didactic or pedagogic motivation of avant-garde film- and video-makers, they share a preoccupation with discarded matter and with dispersal of false certainties, ideal solutions, stable identities and cause and effect logic. With their poetic ruminations on the impact of time, they generate musings on memory and experiences of loss allied to the videographer’s playful reinventions of history.

A conclusion of sorts

My decision to retain and publish Timeless complete with digital artefacts was not unproblematic. I was aware that it would be perceived as technically sub-standard and therefore less valuable and/or less interesting than the sophisticated video essays that now make up an impressive body of academic work. To many, digital artefacts are irritating or ugly, and I sympathise with that assessment. However, as a research tool, videography is a learning experience; although Timeless could be used as an example of the pitfalls that face videographers, it opens up intriguing issues of affective response to the nature of the medium, some of which I have explored here. It also raises questions for what is considered suitable pedagogical material for scholarly videography.

The iconoclast within me learned to love the digital artefacts. The dancing pixels fascinated me, luring my eye to them and enticing me to view the screen as a living painting. Together with the faded colours and banding, they render Wong’s images, already dense with the textures and patina of a vanished past, more distant and obscure. This added opacity is appropriate to a film that takes its cue from Liu Yi-Chang’s short story about memory: “Those bygone days were something he could only look at through a dusty window pane, something he could see, but couldn’t touch. And everything he saw was blurred and indistinct.”[24] The reference to touch is telling: almost all the clips in Timeless feature touching in some form, whether of everyday objects and surfaces or between characters. Viewers, on the other hand, cannot touch what they see, and their view is often obstructed. Their visual and tactile perception is compromised, disturbing their sense of mastery through haptic technology. The elusive quality of Wong’s film is intensified by the presence of the digital artefacts that foreshadow its dissolution into noise.

Perhaps more unsettling, the mobile pixels animate the image so that characters and settings are visibly digital figments, shifting affect from the actors’ bodily performance and the meticulous reconstructions of production design to the technological processes that produce them. Again, this fits with Wong’s depiction of protagonists ensnared by forces beyond their control, but it has wider implications in the challenge it presents to human agency. If the human is ancillary to technology, then we are less in control of our world than we imagine.[25] Moreover, the animated image foregrounds the graphic properties of digital representations, questioning their relationship to reality.

The technological constraints I encountered in making Timeless faced me with choices as a videographer: whether to jettison it as a failed enterprise or rethink my role and the place of accidents in digital video essays. I chose the second route, partly because I consider that it raises aesthetic, ethical and theoretical issues for scholarly videography. In retrospect, I could have made more of the affective impact of image deterioration, perhaps by interweaving better quality clips from the DVD copy to offset those dramatically affected by loss, but as it stands, the video essay is more troubling than it might have been had I reclaimed mastery by cleaning it up. Rather than provide a mirror for my supremacy, Timeless put me in my place, asserting its autonomy through interception of my intentions.

Pam Cook is Professor Emerita in Film at the University of Southampton. She is author of numerous publications on moving image history and culture and editor of The Cinema Book Third Edition (2007). Her research spans feminism and film, memory and nostalgia in cinema, national cinemas, visual design, authorship, stardom and performance. Her latest book is a study of Nicole Kidman as global star phenomenon (2012), and she has recently extended her work into videography.

Notes

[1] “Memory” series, http://profpamcook.com/video-essays-2/. “Timeless” (2014), https://vimeo.com/108562747.

[2] “In the Mood for Love with English Subtitle,” https://www.youtube.com/watch?v=p0Q1bHqQT0E, accessed December 3, 2014.

[3] “Non sequitur” (2014), http://profpamcook.com/2014/05/11/non-sequitur-lost-time-in-in-the-mood-for-love/.

[4] Thomas Elsaesser and Malte Hagener, Film Theory: An Introduction Through the Senses (Abingdon and New York: Routledge, 2010); Lucy Fife Donaldson, Texture in Film (Houndmills and New York: Palgrave Macmillan, 2014); Dudley Andrew, “A Film Aesthetic to Discover,” Cinémas: revue d’études cinématographiques/Cinémas: Journal of Film Studies 17 (2-3), (2007).

[5] Friedrich A. Kittler, Gramophone, Film, Typewriter (Stanford: Stanford University Press, 1999), xxxix. For a discussion of Kittler’s often problematic work see Geoffrey Winthrop-Young and Annie van den Oever, “Rethinking the Materiality of Technical Media: Friedrich Kittler, Enfant Terrible with a Rejuvenating Effect on Parental Discipline – A Dialogue,” in Technē/Technology: Researching Cinema and Media Technologies – Their Development, Use, and Impact, ed. Annie van den Oever (Amsterdam: Amsterdam University Press, 2014), 240-60.

[6] In opposition to the prevailing trend to use digital technologies to produce ever more perfect images, independent filmmaker Baz Luhrmann used digital special effects to create an aesthetic of imperfection in Moulin Rouge! (2001), and added a hand-crafted, painterly quality to images in Australia (2008). See Pam Cook, Baz Luhrmann (London: Palgrave/BFI, 2010), 92-4, 125.

[7] See for example the collection of videos on Vimeo tagged “lossy,” exploring image loss, accessed December 3, 2014.

[8] Svetlana Boym, The Future of Nostalgia (New York: Basic Books, 2001), 348.

[9] Andrew, “A Film Aesthetic to Discover,” 54.

[10] See Pam Cook, “The Point of Self-Expression in Avant-garde Film,” in Theories of Authorship, ed. John Caughie (London and New York: Routledge and Kegan Paul, 1981), 271-81.

[11] See Greyscalegorilla, “How to Remove Banding Artifacts in After Effects,” http://greyscalegorilla.com/blog/tutorials/how-to-remove-banding-artifacts-in-after-effects/, accessed October 27, 2014.

[12] Fairey defended his work as fair use, a claim that was discredited. See Nick Allen, “Shepard Fairey, Barack Obama ‘Hope’ Poster Artist, Found Guilty of Lying,” http://www.telegraph.co.uk/news/worldnews/barackobama/9531809/Shepard-Fairey-Barack-Obama-Hope-poster-artist-found-guilty-of-lying.html, accessed October 27, 2014. See also Fairey’s statement on his Obey Giant website, http://www.obeygiant.com/headlines/an-important-message-from-shepard-fairey, accessed October 27, 2014.

[13] See Ash Davies, “Photoshopping Obama ‘Hope’ Posters,” https://vimeo.com/6661988, accessed December 3, 2014.

[14] Available online, “Beyond Obama’s Hope: the work of Shepard Fairey,” http://www.theguardian.com/artanddesign/gallery/2012/oct/17/obamas-hope-the-work-of-shepard-fairey?index=1, accessed October 28, 2014.

[15] See André Gaudréault, “The Future History of a Vanishing Medium,” in Technē/Technology, ed. van den Oever, 261-71. Ironically, it is possible to reverse such imagery by rewinding DVD footage, or to recreate celluloid meltdown through digital special effects, inscribing the memory of traditional cinema in the digital landscape.

[16] The end of representation is discussed by Elsaesser and Hagener in their conclusion to Film Theory: An Introduction Through the Senses.

[17] Malte Hagener, “Re-editing as Psychotechnique: Montage and Mediality in Early Soviet Cinema,” in Technē/Technology, ed. van den Oever, 177-84.

[18] Hagener, “Re-editing as Psychotechnique,” 180.

[19] For example Bruce Conner’s A Movie (1958), https://www.youtube.com/watch?v=4FMjBtvsx2o, accessed December 3, 2014.

[20] For example George Landow [aka Owen Land], Film in Which There Appear Edge Lettering, Sprocket Holes, Dirt Particles Etc. (1966), https://www.youtube.com/watch?v=Vryg0DE7L70, accessed December 3, 2014.

[21] William C. Wees, Recycled Images: The Art and Politics of Found Footage Films (New York: Anthology Film Archives, 1993), 46.

[22] Boym, The Future of Nostalgia, 49, cites Vladimir Nabokov, “On Time and Its Texture,” in Strong Opinions (New York: Vintage International, 1990), 185-6.

[23] Boym, The Future of Nostalgia, 51.

[24] Liu Yi-chang, “Intersection,” trans. Nancy Li, Renditions 29/30, Spring/Autumn 1988, available online, https://drive.google.com/file/d/0Bz2RL4R29lF2Q1NhWmo2SWJaMG8/view?usp=sharing .

[25] Elsaesser and Hagener discuss the implications of digital animation in the conclusion to Film Theory: An Introduction Through the Senses.