Portal de Periódicos da CAPES

The Great Automatic Grammatizator: writing, labour, computers

2017; Wiley; Volume: 59; Issue: 3 Linguagem: Inglês

10.1111/criq.12359

1467-8705

Martin Paul Eve,

Digital Humanities and Scholarship

Writing is pre-eminently the technology of cyborgs – Donna Haraway1 The series of temporal inventions of the object is intelligible only afterwards – Recurrent neural network trained on Textual Practice Even before the advent and mass uptake of the word processor, authors and publishers often imagined their own erasure at the hands of machines that can write. For instance, as Matthew Kirschenbaum has recently charted, Stephen King famously penned a story – ‘Word Processor of the Gods’ – in which the ‘delete’ function of his computer allows him to erase reality; a clear metaphor for a fear of redundancy in the face of the machine's power.2 William Gibson's self-encrypting (and therefore self-erasing) digital poem, Agrippa (A Book of the Dead) (1992), also betrays such an anxiety of obsolescence in its self-undoing. Of all the functions that the word processor introduced, ‘cut’, ‘delete’, and ‘execute’ seemed to cause the loudest noise. Among the more widely circulated of these fearful prophecies, though, is Roald Dahl's imagined ‘Great Automatic Grammatizator’, from his 1953 collection, Someone Like You – a story that features a machine that quantifies human creativity through the mathematicisation of language. A world away from surrealist conceptions of ‘automatic’ writing in the early twentieth century, Dahl's machine is a dark device akin to an organ that a human operator ‘plays’ with the stops set to inject the desired sentiment at any point during the unfolding narrative arc. The most important feature of Dahl's short story, though, is the focus on material textual production and its remuneration. That is, Dahl recognises that the terror of such a machine is predominantly to do with the symbolic economics of authors' names as brands: the ‘author-function’ as Michel Foucault might term it. In Dahl's tale, these names are re-minted as coinage within hierarchies of prestige, akin to those recently explored by James F. English and Ted Underwood.3 Of course, as with all the symbolic economies described by Pierre Bourdieu, these virtualised currencies map onto real economies (if, that is, we can even use the word ‘real’ to describe an economy). As Dahl's protagonist feels his ‘own hand creeping closer and closer to that golden contract', he asks for the strength to maintain human creativity in the face of financial ruin, a strength to value an autonomous art over material circumstances, a strength (expressed with Dahl's customary shock hyperbole) ‘to let our children starve', even while it is ambiguous as to whether this text is itself a product of the Great Automatic Grammatizator.4 Capital has not quite yet built a technology at this level of competence, so far as we know. However, it has already devised factory-like environments where many anonymous authors produce texts under a single name (most notably in recent days, James Patterson) in order to dominate the literary market. It is also certainly the case that there are definable mathematical (and measurable) properties of language, the core of Dahl's machinery. Zipf's law, for instance, tells us that in any text, roughly speaking, the frequency of each word is inversely proportional to its rank in the frequency table (put otherwise: the most frequent word occurs roughly twice as frequently as the next-most frequent, and so on). We also know, from the work of John Burrows, that a type of textual fingerprint can be deduced for a work by calculating the ‘Manhattan distance’ between the multi-dimensional plots of the z-scored word frequencies of texts.5 It is not surprising that mathematics, as a self-referential formal system of representation, is able to represent language, a self-referential formal system of representation, within its own logics. Scholarly debate around computer-generated poetry stretches back to the 1970s, when it was frequently invoked in debates on author intentionality.6 Did it matter, commentators asked, whether a poem was written by a human or a machine in the age of poststructuralist readings? It was also clear at this time that applications of humanities computing (the precursor term for ‘digital humanities’) existed for the study and teaching of poetry, which at least one contemporary found ‘analogous’ to hypertext.7 Most importantly, though, ‘poetry’ written by computers is usually found to be lacking an author. For instance, P.D. Juhl claims that when we read machine-written poetry ‘we are not dealing with anyone's use of the words', although he concedes that the words may ‘possibly’ be ‘the programmer's’.8 Even the most recent comprehensive surveys of computational poetics continue to note that this authorial deletion lies at the heart of machine-authored poetry: ‘contemporary technology radically challenges the creative process of poetry authorship'.9 What is most significant here, though, is that the roots of academic concern about computer-written poetry are far less materialist than those in the popular imagination. For ‘it is important’, writes David Johnston, ‘that poets (and not technologists/linguists) interrogate what the practice of poetry is in a big data/cloud world', a re-inscription of the poet as a valid and distinct labour specialist at the heart of such an enterprise but framed in terms of sentiment and taste.10 Certainly, to return to Dahl, the ‘Great Automatic Grammatizator’ gives us a sense that what is lost to the machine is art-for-art's sake, the ‘creative urge’.11 Yet markets and business saturate the story; it is all about the labour and remuneration of writing and publishing. Of course, Dahl's satire is not actually about computer writing. Despite the predictable nature of Dahl's own brand of shock-twist short story, ‘The Great Automatic Grammatizator’ is instead aimed at formulaic genre fiction that is decried as the output of older writers who have ‘run out of ideas’ but who comprise ‘seventy per cent’ of the work accepted by publishers.12 In turn, this feels akin to John W. Aldridge's formulation of an ‘assembly-line fiction’; for Dahl's protagonist, Adolph Knipe, is surely a swipe at Alfred Knopf, the publisher of Someone Like You.13 That Dahl's target is not truly computational writing does not mean, though, that there is nothing to say about the conjunction of publisher markets and machine prose to which Dahl draws attention. Instead, I will here go on to argue, the fundamental crux that we continue to elide in the space of electronic literature and machine writing is the locus of different labour functions that underwrite their production, reception, circulation, and preservation. For the digital space is often imagined as infinitely abundant. The ability to copy any extant artefact at a near-infinitesimal cost leads the digital imagination to perceive of limitations of labour as a technical, rather than a social, problem. In fact, digital abundance rests upon scarce material labour and requires additional forms of technical expertise to develop and maintain electronic infrastructures. The ability to harness abundant digital potentialities is restricted by an underlying material economy. In this way, I argue, the digital space provides us with a new commodity fetishism, in which we focus upon our technical relationships with the digital prostheses with which we all now write, instead of our labour relationships between people that underwrite such technologies. What I am interested in here addressing, then, is the question that comes out of the work of Jerome McGann in his writing on the information age.14 If the work of literature is a social text, or event, then what forms of labour are invested in the technological tool chains that contribute to its creation but that often lie unrecognised by our contemporary systems of authorship? I want to push the question asked by McGann – ‘where is information technology driving literary and cultural studies?’15 – to its limit by asking what labour underpins such a textual socialisation when, in the current age of books in the making, we believe that computers can write. What does it actually mean to say that computers can write? What are the labour forms that underpin such authorship? The metaphor of ‘writing’ certainly runs throughout computational terminology. Forms of computer storage, from processor registers through random access memory to solid-state and hard drives, are ‘read’ and ‘written’ via minute physical magnetic manipulations and reflections. Computational media are deemed read- or write-protected in some instances. Yet this metaphor is not the type of writing of which we speak when we claim that computational writing is on the rise. This metaphorical reading and writing of physical media in a computational environment is more akin to a type of microscopic palm-reading where a claimed expert can sense that which is hidden to the untrained human eye. Yet even this metaphor is not strictly accurate; to comprehend fully computational writing and reading in this mode would require a sort of ‘alien phenomenology' of the kind detailed by Ian Bogost.16 However, most authors are also now used to writing with computers, albeit not quite in the way that Dahl imagined. The process of fabricating the material codex has been digitally intermediated for many years now.17 Even those contemporary authors – those Don DeLillos and Jennifer Egans – who cling to typewriters and pens and paper will have their words re-wrought into various digital forms by others in an often-gendered division of labour.18 Yet what we talk about when we talk about computational writing is the production of text that appears as though it was generated directly and immediately by humans even while this is not the case. That is, the precise selection of sequential words was decided neither by an individual person nor by that individual working in conjunction with an editor or co-authors. As with so-called ‘artificial intelligence’, the benchmark of success is the exact mimicry, or even out-performance, of human characteristics of intelligence. This is to say that ideas of artificial intelligence and computational writing are saturated with anthropocentric thought. In order to succeed, attempts at producing artificial intelligence and computational writing must, therefore, strive to transcend a mechanistic logic through a type of incomprehensibility; a free will or vitalism should animate the process and produce work that is indistinguishable (by humans) from those created by a human imagination. Yet the criteria for success at computer writing sit on a spectrum of evaluation. Computers can write badly or they can strive to pass Turing tests, but in both cases the computer is ‘writing’. A run of Evolution at generation 554 Evolution, though, also aspires to a type of print bookishness, a material textuality in a digital space. For while Evolution is not called a ‘book’ by its creators (it is referred to as an ‘application’ and an ‘online artwork-in-progress’), it has been assigned an ISBN and further appears in an extremely limited print form. The web page itself on which the software is displayed is styled in the guise of a codex that even incorporates a page-staining effect (or a de-generation). By way of compositional analysis, Evolution's codebase consists of two components: a front-facing HTML and Javascript library that controls the playback and a back-end server-based component. The frontend fetches a set of formatting and music playback instructions from the server. The server returns information about 100 ‘generations’ at a time and is accessed by the Javascript client at locations on the Amazon Elastic Compute Cloud.20 Each set of generations is grouped under a string of text that serves as a ‘sequence’ identifier for random seed data, which range from ‘cups of coffee per episode of Twin Peaks’ through to ‘atlas of extratropical storm tracks (1961–1998)’. A generation itself is composed of a set of instructions encoded in a JSON data format: for example, ‘{"word":"night","age":2577,"index":9,"delta":false}’.21 These instructions are created by a server-side application that deploys an evolutionary algorithmic strategy for stochastic text selection – another reason for the piece's name – based on the work of Andrei Markov and Ingo Rechenberg and selected by ‘a semi-deterministic random seed […] derived from atmospheric data, visual imagery, space observations and popular culture’.22 Evolution is, in some ways, just the latest version of a form that overlaps with concrete poetry and that Bronaċ Ferran, following Haroldo de Campos, has dubbed ‘typoetical’, emanating from the print-publisher networks of Hansjörg Mayer, Max Bense, and Dieter Roth among others over the past six decades.23 This form incorporates iterative process, overlay, and spatial layout as key to its composition, strongly resonating with various algorithmic Oulipo techniques. This model has also been called ‘kinetic poetry’ by Christopher Funkhouser, a mode in which ‘images can be a mélange of fragments of words complemented or replaced by imagistic forms’.24 Let us be clear, though: Evolution is not going to pass a Turing test any time soon. In fact, Evolution is not even going to pass itself off as a substitute for Heldén's own poetry. While it may be true that its computational processes result in an ur-version of Heldén's poetics, this ur-version lacks the specificity and coherence of his earlier work, as in the 2013 Terraforming. For even radical poetry is rarely stochastic. Evolution represents, then, an abstraction of the mathematics of language – as Dahl prophesied – but its techniques do not countenance linguistic sense in the way that Heldén does when he writes. Evolution may yield a semantically empty mathematical average of Heldén's poetry, his layout, and his musical essence but it also points, I will argue, to a set of infrastructures and labours that are its own conditions of possibility. Before turning back to this core of my argument, I want to move to a second example at the bleeding edge of human language emulation: character-based recurrent neural networks (RNNs). Recurrent neural networks are software simulations of biological neurons, in which many small processing units are passed the output from other ‘neurons’, all of which have a memory of input that they have processed before and which they use to modify their output. In short, the machine adapts by passing output from its different processing units as input back in to itself. Character-based recurrent neural networks take text as input and build a statistical matrix of the most-likely next character in any sequence. Unlike teaching a human to read or write, this approach does not focus on words but rather on single characters and their statistical likelihood of occurring in any sequence run. Also unlike teaching a human, character-based recurrent neural networks that are not run on high-performance computing hardware have only a limited number of neurons, somewhat more akin to the capacity of a worm than a person. How well can a worm write when it is taught to predict characters?25 Over a twenty-four-hour period, I trained a torch-rnn model using the entire corpus (until 2016) of the literary studies journal Textual Practice and then sampled 5,000-character chunks from its saved checkpoints.26 The machine learned to produce text that certainly feels emblematic of the journal and that might unnerve others in its uncanny proximity to Alan Sokal's 1996 faux-pomo prose (‘faux-mo’, perhaps?).27 It told me that ‘the series of temporal inventions of the object is intelligible only afterwards’ but that ‘in the early twentieth century, these recognitions are contingent’. In one of its more poetic moments, the network claimed that ‘the world was right to have to introduce its choice: that meaning was a palimpsestuous scholarship, the literary moment’. Without any knowledge of the English language, the network also became proficient at generating bibliographic and footnote items, including ‘Slavoj Žižek, Live Fiction, trans. Rushdie and Jean-Luc Nancy (London: Bohestock Press, 1994)’; ‘John Spottisley, “The privatized climax”. (1929), p. 4, emphasis in original’; and the instruction to ‘see David Pillar, New Bibliography, ed. Donald Davis (London: Lawrence & Wishart, 1979)’. The network learned the capitalisation structure of English proper nouns, the formatting of references, common names, publishers and date structures, and the likely labour functions of editors and translators. This was all achieved simply through probabilistic modelling of the character sequences already present within Textual Practice, using fewer processing units than those inside the neural system of a nematode. Of course, even when it accidentally distils nuggets of truth, the network has no motivation towards communication and no epistemological goal except to achieve ever more perfection in its stylistic mimicry of the articles in Textual Practice. As it noted in one of my samplings, in a remark that could apply well to itself, ‘I shall find our intellectual values, by rewriting their very ties’. For the machine is one of pure textual practice; even while it knows to include footnotes, its references are dead ends and subversions of traditional academic epistemologies.28 They ‘provide the fraud of the epistemological practices of knowledge’; another generation of the network. Taken together with the faux aesthetics of Evolution, these two models of linguistic aping contain within them contradictory logics of artificial intelligence that continually point to social labour. ‘The problem’, as the network aptly phrased it, ‘is that the poem is a construction of the self as a strategy of self-consciousness and context’. Digital literary aesthetics presuppose human readers encountering works after their production, although we can also imagine outputs directed solely at computational systems, much as in the intermediate feedback stage of directed cycle neural networks. That is, their existence presupposes, as Alan Liu put it, ‘a scene of encounters’.29 Indeed, the dynamic temporal inventions of the machine are intelligible only afterwards to a human reader. Yet, where is the line in textual creation between the machine as tool and the machine as author? As of 2017 we have already witnessed the rise of computer-generated business and sports journalism.30 The formalised, highly generic prose style of this work – similar to that at which Dahl directed his ire – lends itself to repetitious statistical natural language generation. (There is in itself another article to be written about the evolution of the term ‘natural language generator’ as opposed to ‘artificial neural network’ and the ongoing erosion of this artificial/natural binary when both systems are underwritten by people as naturecultures.) Small-scale studies have even demonstrated that human audiences are unable to discriminate between this machine-written prose and articles written by people.31 In this case, adjusting sentiment sliders much as Dahl imagined, statistical reporting on the stock market and soccer games can be automatically churned out for mass consumption. Admittedly, there is something alarming in such a trend; it feels connected to a decentring of the human in the production of written language. Yet companies such as ‘Narrative Science’ – corporate specialists in this field – claim that their job lies in ‘humanizing data like never before, with technology that interprets your data’, and that ‘then transforms it into Intelligent Narratives at unprecedented speed and scale’. That is, the organisation paradoxically seeks to humanise through a chiastic mode of mechanisation. Their software also, clearly, requires human calibration and operation. The profusion of the concept of ‘narrative’ beyond the walls of academic literary criticism – and as nothing less than an apparent ‘science’ – is alarming. It undoubtedly cedes what literary critics and journalists, among other groups, have known for many years: that narrative possesses a power worthy of study. In its corporate excess and buzz-speak, this movement also gestures towards the large-scale population-manipulation through narrative that is a feature of most contemporary news media and that undoubtedly played a role in the ascent of democratically elected neo-authoritarians around 2016. At the same time, though, in its utilitarian mobilisation through companies such as Narrative Science, there are other worrying aspects to this growth of computational narrative. These anxieties can be grouped under two headings: first, as a means of eradicating or re-situating labour through mechanisation; and second, in its dividing naturalisation of a realm of scientific data that apparently sit apart from narrative (as though scientific hermeneutics were not, themselves, an interpretation and narrativisation), opposed to a ‘humanised’, narrative version of those data. However, on this second point, we might also ask what the difference is between such a piece of guided ‘helper’ software and the existing systems of word processing that are in broad circulation. Is the use of an automated spellchecker a machine writing? It certainly changes the word that an author may have typed. What about a thesaurus that suggests wholly different words? Grammatical checking that alters sentence structure? My word processor, LibreOffice Writer, even provides automatic completions for words based on the characters that I begin to type, conditioning future possibilities through suggestion. As William Winder has put it, ‘formatters, spell checkers, thesauri, grammar checkers, and personal printers support our writing almost silently’.32 For Winder, the question comes down to whether, in our use of such prostheses, computers are ‘typists or writers’.33 Or, put otherwise: is the Great Automatic Grammatizator different by type or degree from other forms of writing aid? We certainly find that ‘our machines are disturbingly lively, and we ourselves frighteningly inert’, as Donna Haraway put it many years ago.34 Evolution implies, by its very title and mission statement, that its efforts are in competition with human writers and are on the same plane. As a survival of the fittest comes into play, the piece proclaims, the human author will stop writing poetry and the machine will take over; a process of unnatural selection or ‘uncreative writing’, to quote Kenneth Goldsmith.35 Likewise, injecting structural flow components into the decision-making portions of recurrent neural networks would allow argumentative progression, overcoming many of the claimed objections about computational mastery of narrative form. Yet, by their respective modelling on the works of Heldén and by their directed cyclical structures for training, these models of language and aesthetics are inherently conservative. Of course, even in human writing there is an interplay between the individual talent (a progressive randomness) and traditions (a conservatism). It is also frequently argued that there is nothing new under the sun and that all writing is a working through of permutations of a grand set of master narratives, an almost Kabbalistic approach to permuting the name of God. This inward-looking approach to language generation by people is, further, clearly reflected in my neural network's accidental pronouncement that the poem is a construction of the self as a strategy of self-consciousness and context. The self that it uses, in this case, is an aggregate of human selves. Whether or not it has such a self-consciousness, though, is a different matter. However, the absolute history of computer writing rests upon this human writing and labour. Were the human race to die out but the machines to keep on writing, they would continue to produce ever more conservative texts, training themselves upon their own regurgitated outputs with only semi-deterministic random seeds to aid progress and foster change. Of course, were the human race to die out and the machines to continue writing, this would be a remarkable occurrence. This is because of the vast infrastructures that underpin our technologies and the substantial volumes of labour that are necessary for their perpetuation. Evolution gestures towards this challenge of digital preservation in a post-human (in the sense of ‘after human’) era. Its ‘pages’ are stained as though the digital fabric has been damaged by light exposure, thereby calling attention to the enormous global technologies of preservation that we have constructed for the retention of print: libraries. However, because this aspect sits within a digital framework, it also calls attention to matters of digital preservation. Digital preservation is a good space within which to examine such issues of labour since it is dogged by a series of challenges that are, at core, all social rather than technical. Given infinite resources it would be possible to preserve the vast majority of digital artefacts produced today. However, we are not given infinite resources. There is a scarcity of remuneration available within our systems of economic exchange that itself causes a cascade of other problems. For instance, if we cannot preserve everything because we have insufficient resources, how do we decide where to invest our preservation efforts, given that our abilities to forecast value fare extremely poorly under experimental conditions?36 This is exemplary of the core difficulty of scarcity against abundance in the digital space. The ability to copy infinitely leads to the belief that virtual environments are ripe for proliferation, be that in file formats or volume of material. Yet without underlying remuneration for human labour, there is a problem in the long-term retention and ability to access or execute arbitrary binary data. Works like Evolution gesture towards this problem. For, on the one hand, Evolution is an artwork about proliferation, as is the natural language generation of the recurrent neural network. Both programs promise ever-evolving sets of textual permutations, offering an abundance of inscription. Yet, conversely, both programs also rest upon vast quantities of computer scientific research. They both require infrastructures of material production to manufacture silicon chips, to run power facilities, to educate their operators, to debug their software, and so on. Evolution's infrastructure even requires Amazon's hosting facilities for its server components. That is, it relies upon what is not only the greatest ‘virtualiser’ but also the most miraculous materialiser that the world has seen in recent years. With the click of a virtual button at Amazon, it seems, objects appear in the mailbox. Yet we also know that Amazon works only by drawing upon vast reservoirs of poorly paid warehouse staff and by pricing its artefacts as cheaply as possible in order to achieve market domination even while not turning a profit. In other words, a material scarcity underpins such infrastructures. This dichotomy is also apparent in the structure of Evolution. For the work's algorithms run not on text alone but on text and whitespace, on abundance and scarcity.37 Indeed, Evolution samples not only the words of the poet that it is meant to replace, but also the blanks. Like music, which always includes silence with sound, Evolution continually points towards the importance of emptiness. In fact, the blankness and space – that is, of course, not really blank, but actually a falsely stained ‘page’, thereby drawing attention to its own quasi-absence – that sit behind the text are metaphorically indicative of the very problem that I am attempting to draw out. Even while the space of computational writing is seen as one of proliferation (‘computers can write!’) it remains bound to a scarcity – a blankness in recognition – of labour forms that underwrite its possibilities. The print volume of Evolution pushes this even further, oscillating between black background and white foreground for computer code against ‘human’ exegesis with a white background and black text (the data component of Evolution's print book is presented with a white background and black foreground). Yet even this binary reduction to a black-and-white print format contains within it the seeds of a material critique: that print economics can determine, shape, and limit the contrasts of form that are available to poets, be they computational or human. That we continue to refer to computer poetry and literature as lacking an author seems, therefore, somewhat strange. Many labour forms were as integral to its creation as the above-listed labours will be to its preservation. Yet at what point between the spellchecker and the recurrent neural network does the author disappear? The question cannot be boiled down to a percentage of the labour involved; it is conceivable that a text could be written in which the spellchecker was used to correct every single term but still we would not give a byline to the author of the software. There are also historical precedents for this division between the labour of manufacturing the tool as opposed to the output. Thoreau, certainly, did not fully credit his own family's pencil-making industry in the authorship of Walden. Yet the pencil is an ‘advanced technology’.38 Academic publishing has also encountered the dilemma of representing labour, even while efforts continue to use computers to mine papers at high volume (‘distant-reading’).39 High-energy physics experiments such as those conducted at the Large Hadron Collider or the Laser Interferometer Gravitational-Wave Observatory require diverse types of labour forms in order to conduct their work. However, since academic systems of hiring, promotion, and tenure are geared towards authorship of research outputs as their primary measure, we arrive at the somewhat curious state of papers with over 5,000 authors, as in the case of the recent Higgs Boson experiment, credited to G. Aad et al. (where listing the ‘et al.’ consumes twenty-four pages of the article's thirty-three-page total).40 What is further remarkable about the increasing accomplishments of computational writing prostheses is that their success at imitating human writing leads to an imagination of a post-anthropocentric era. There is a temporality of ‘afterwardness’ inherent in computational natural language processing and generation. That is, in achieving a mimesis of human writing – remember, a measure of intelligence formed only by anthropocentric reference to the human – computational writing asks us to imagine a world in which there are no more humans undertaking such labour. Such thinking only emerges, though, in the imagined substitution of the human with human-like automata. This imagined world is both a post-anthropocentric world for writers and a world in which a writing machine that is legitimated by human-like characteristics is inscribed at the centre. It is concurrently a world in which we have no benchmark of contemporary writing success, but one that is nonetheless dominated by machines that meet that nostalgic target. What, then, of the Great Automatic Grammatizator? Have our hands already crept to the other side of the desk, seeking to avoid the starvation of our children? Are our brands – those hollow outlines of action – all that is left when our labours are consigned to the technological dustbins of history? Can you identify which portions of this article should be attributed to me and which portions to the artificial neural network and, hence, to the software authors in some mediated sense? The words from the network do not all appear in quotation marks. Or should we instead be more concerned that we seem unwilling to represent the vast quantities of human labour that have already been invested in the creation of our technological writing prostheses? Ever more frequently, vast volumes of computational labour – programming, infrastructure, and communications labours – underpin our social textual production. As we do not credit them now, I would like to ask: what meagre credit can we expect for our authorial inputs once the literary market has fallen under Knipe's malign influence?