Portal de Periódicos da CAPES

Breaking Up (with) AI Ethics

2023; Duke University Press; Volume: 95; Issue: 2 Linguagem: Inglês

10.1215/00029831-10575148

1527-2117

Luke Stark,

Psychology of Moral and Emotional Judgment

The ethics of artificial intelligence (AI) have become a matter of public concern. According to a recent Stanford report, the number of research papers in the area given at major conferences such as the annual Conference on Neural Information Processing Systems has increased fivefold since 2014, and ethics officers now abound at global technology firms (Moss and Metcalf 2020). Such major institutions as the US government, the United Nations, and the Vatican have articulated visions for so-called ethical AI.By AI ethics here I mean the study of how human values both shape and are shaped by the development of AI technologies. This definition is capacious: it includes the design and deployment of these systems with human values in mind; assessments and activism around the societal impacts of said technologies and their imbrications within existing asymmetries of power, justice, and equality; and the wider relationship between computing technologies and humans as ethical and moral creatures, for instance, through such phenomena as human emotions. Work in these areas is done by trained "ethicists" only infrequently, rarely involves what a member of the public would first think of when asked to describe AI, and sounds outré yet is all too relevant to contemporary social policy and societal inequity.The definition I offer is expansive, perhaps too much so. However, any definition in this field is perilous. The term AI is a leaky discursive umbrella sheltering heterogeneous and often contradictory ideas and practices. It is a quintessential boundary object of the ideal type, "plastic enough to adapt to local needs and constraints of the several parties employing [it], yet robust enough to maintain a common identity across sites" (Star and Griesemer 1989: 393). Those identifying with the term AI ethics might be expected to at least signal some vague acknowledgment that the development and deployment of AI technologies involve normative stakes or impacts. However, a welter of methods, interests, and political positions operate uneasily within this shallow consensus; given its shortcomings, some scholars working on what would colloquially be understood as "AI ethics" eschew the word ethics entirely.Here, I aim to disaggregate AI ethics discourse through reviews of three recent books whose authors grapple in various ways with its rise and prominence. Those seeking an overview would benefit from consulting the first listed: The Alignment Problem: Machine Learning and Human Values, written for a general audience by Brian Christian. A science journalist, Christian grounds the book in dozens of interviews with academics and practitioners and frames it around the titular "alignment problem": how to design machine learning (ML) systems "in alignment" with the intentions of their creators, ones which "capture our norms and values, understand what we mean or intend, and, above all, do what we want" (13). This "alignment problem" is presented as an engineering one, a framing that takes as a given the ongoing development and deployment of AI systems and implies it is possible to ameliorate these technologies sufficiently through various technical improvements.The Alignment Problem provides useful background on the contemporary technical landscape for those not already immersed in the field. When picturing an AI, the public might think of the psychotic HAL 9000 of Kubrick's 2001: A Space Odyssey or Lt. Commander Data of Star Trek, but today's AI systems are neither sentient or nor particularly charismatic. Christian points to the three main subfields of contemporary ML: unsupervised learning, in which an ML system is provided a mass of data and set to identify statistical patterns within it; supervised learning, in which an ML system takes a mass of already categorized data and uses the correlations it finds there to predict into which categories some new set of data should be sorted; and reinforcement learning, in essence a virtual Skinner box, an environment in which an artificial agent is assigned parameters for reward and punishment and then set to maximizing the former and minimizing the latter.Ready to command a starship, AI is not, but the field has always involved fantasy in search of a practical method. The computer scientists who participated in a now-famous inaugural seminar on the topic at Dartmouth College in 1956 were inspired by "the conjecture that every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it" (McCarthy et al. [1955] 2006). The human mind was a computer, their thinking went, and so a computer could be built to equal or surpass a human mind. These researchers spent the ensuing decades seeking the most effective computational means and methods to simulate intelligence and prove their conjecture correct. ML was developed in parallel but subordinate to other past technical paradigms in AI research, such as those built on logical symbols. As early as the 1950s, researchers developed computational pattern recognition: systems that could use cameras to identify repeating patterns in large amounts of data (Jones 2018; Mendon-Plasek 2020). Christian highlights one of the most famous of these early systems, Frank Rosenblatt's Perceptron, based on a simple artificial network of simulated neurons, but efforts were rife in industrial, military, and other applied settings. Today's ML systems, such as the Large Language Models (LLMs) powering products like Open AI's ChatGPT, are built on "deep" neural nets with many layers of simulated neurons.An understanding of how AI technologies like deep learning work is critical to identifying which of these technologies' societal impacts, present and future, are most pressing and problematic. In The Alignment Problem, Christian distinguishes between two groups. The first consists of scholars, practitioners, and activists concerned with the already existing impacts of ML-based automated decision-making systems, in areas such as policing and incarceration, hiring, and social assistance. The second consists mostly of technologists preoccupied with longer-term AI safety, a euphemism for the hypothetical dangers of a future "artificial general intelligence," or a machine able to perform equally well as or superior to a human being in all respects. Despite being bundled together under the banner of AI ethics, these two groups have very different concerns and are frequently at odds. Since contemporary deep learning technologies are not remotely close to supporting artificial general intelligence, those concerned with AI safety would seem to be barking up the wrong tree. However, it is in the interest of these systems' promoters both to give the prospective, future-focused gloss of science fiction to AI ethics and to the broader field of ML, and to imply subtly to the comfortable that the disruptive social impacts of AI systems are safely in the future. A focus on AI safety satisfies these ideological goals admirably, so the term is appearing more and more frequently in AI ethics contexts.Indeed, The Alignment Problem might focus more pointedly on the history of the term AI ethics itself, and the effects of bundling all contemporary public discussions about human social mores, values, and the societal impacts of AI systems under the banner of "ethics." High-level overviews of the topic define ethics broadly, as "the rational and systematic study of the standards of what is right and wrong" (Kazim and Koshiyama 2021: 3). Computer ethics as a defined field developed out of engineering ethics in the 1980s and at its inception possessed many of the same fault lines as AI ethics discourse today (Moor 1985, 2001). Engineering ethics often prioritizes a focus on material problems and their solution through improved design. One of the first textbooks on the subject, Deborah G. Johnson's Computer Ethics (1985), included intellectual property law as applied to software, the unique threat posed to human privacy by computing technologies, and the ethical responsibilities of computing professionals. Yet most scholarly references to the specific notion of AI ethics prior to around 2015 did not involve applying ethics as a branch of philosophy to studying the context of AI's potential uses. Instead, AI ethics was most often invoked in metaphysical speculations about the status of machines as autonomous ethical agents (what today would be an "AI safety" topic).An article by well-known Silicon Valley journalist John Markoff, titled "How Tech Giants Are Devising Real Ethics for Artificial Intelligence" and published in the New York Times in early September 2016, signaled the discursive shift toward contemporary AI ethics talk. Markoff reported that industry researchers from several large Silicon Valley companies (including Microsoft, where this author was once employed) sought to develop "a standard of ethics around the creation of artificial intelligence," one meant to "ensure that A.I. research is focused on benefiting people, not hurting them." The article's framing anticipates several of the elements that have characterized AI ethics discourse in the years since: statements of lofty humanitarian ambition used to justify industry aspirations to self-regulation, the contention that policy makers would inevitably lag in understanding AI systems, and an insistence that government oversight of AI would be both undesirable and ineffective. Perhaps most crucially, the piece suggested that the development of AI technologies was as inevitable as their effects would be widespread and disruptive: social scientists and philosophers needed to be put "in the loop" to help computer scientists manage the effects of AI's undoubtedly epochal impacts.Business and professional ethics were two of the most direct antecedents for today's AI ethics discourse as developed and propagated in corporate spaces (Greene, Hoffmann, and Stark 2019: 2124). The sociologist Gabriel Abend (2014) has developed the idea of the "moral background" to describe second-order assumptions about what problems or questions count as of ethical concern. Abend and others have noted that professional ethics codes in fields like engineering implicitly work to distinguish members of a particular profession from outsiders through recognition of their skills or expertise and by an emphasis on obligations to colleagues and clients, as well as the general welfare, and on enforcement based on public visibility (Abbott 1983). The implicit moral background of today's professionalized AI ethics is latent in Markoff's piece; it matches the analysis by my colleagues Daniel M. Greene, Anna Lauren Hoffmann, and myself (2019) of the then nascent genre of AI vision statements. This background presents a deterministic vision of AI's development and deployment, in which the adoption of these technologies cannot be stopped and the ethics of which are best addressed through certain narrow kinds of technical and design expertise. More recently coined industry terms such as AI safety and responsible AI reflect this worldview, and many of the various existing or proposed mechanisms for the ethical oversight of AI systems are easily co-opted into broader forms of neoliberal governance and capitalist accumulation (Stark, Greene, and Hoffmann 2021).It is crucial, then, that AI ethics include as a possibility that some applications of deep learning never be designed, built, or used at all. One way to respond to the "alignment problem" is thus to interrogate exactly whose values technologists presume deserve alignment with AI systems. Such critique has been led by activists and scholars trained in critical race theory, race and technology studies, gender and sexuality studies, and related fields. In the academy, this work is grounded on informed refusal in justice-based bioethics (Benjamin 2016) and on recognition of the genealogical continuities between contemporary AI systems and white supremacy (Golumbia 2009; Benjamin 2019; Katz 2020), patriarchy and misogynoir (Browne 2015; Noble 2018), and binary gender norms (Scheuerman, Paul, and Brubaker 2019).The activist work of groups such as the Our Data Bodies collective, Data for Black Lives, and the Algorithmic Justice League, to name three American organizations among many hundreds worldwide, has been even more central to the advancement of critical AI discourse. These organizations support what the AI and social justice organizer, advocate, poet, and author Tawana Petty describes as "visionary resistance" (Petty 2014). Such resistance entails mobilizing and working with local communities, particularly racialized, low-income, or otherwise marginalized ones, to document the impacts that the deployment of AI systems are having today. Resisting these technologies and their backers on all fronts also entails advancing a positive vision of justice and equality, doing "the work of creating the world we wish we live in" (Lewis et al. 2018: 83).The Alignment Problem is full of accounts detailing the baleful results of putatively misaligned ML systems and attempts to fix the problem through improved technical know-how. Activists are given short shrift, and Christian's narrative often assumes such technical fixes are both possible and sufficient. Only in the book's conclusion does Christian pause to question whether the technological solutionism he implicitly celebrates is enough to address the deleterious social impacts of the ML applications he documents. "We must take great care not to ignore the things that are not easily quantified or do not easily admit themselves into our models," he admits (326). AI ethics conversations that focus on technical definitions and benchmarks for values like fairness are not well suited in isolation to support such care, or broader conversations about AI's social impact. Such technical work is necessary but ought not to be at the center of our civic debates about which, or whether, AI systems should be designed and deployed.That technical solutions are often touted as the answer to AI's inadequacies by the same companies developing and profiting from these systems in the first place suggests AI alignment is the wrong way to think about the broader questions at stake. If AI technologies are aligned solely with the values and priorities of wealthy, technocratic, and often reactionary Silicon Valley tycoons, what space is there for an AI ethics the rest of us can live with?▪ ▪ ▪Critiques of computing's role in social and moral life predate the current AI ethics boom. The mathematician Norbert Wiener, coiner of the term cybernetics, wrote widely about the potential dangers of computing machines and of their misuse by the powerful and the callous (see Wiener [1950] 1954). In the 1960s, civil rights leaders spoke out against automation as a mechanism intended to roll back the labor and employment gains Black Americans had just begun to make (McIlwain 2019). The AI researcher Joseph Weizenbaum, who in 1966 had developed one of the first interactive conversational computer programs, the simulated Rogerian psychotherapist ELIZA, wrote in the 1976 book Computer Power and Human Reason that "some acts of thought ought to be attempted only by humans" (13). In the 1970s, fields such as the philosophy of technology and science and technology studies increasingly critiqued the social impacts of computing technologies, including those used to automate human decision making. And organizations like Computer Professionals for Peace in the 1970s and Computer Professionals for Social Responsibility in the 1980s highlighted the dangers of computing technologies used by the American military and advocated for appropriate and democratic use of computing in public administration and policy making. The 1980 bibliographic source book The Impact of Computers on Society and Ethics (Abshire 1980) runs to almost 120 pages.It has never been possible to separate cleanly the egregious uses of computing machines from their technical design. In an important early paper, computer scientist Batya Friedman and philosopher Helen Nissenbaum (1996) described three categories of bias, defined as systematic and unfair discrimination, in computer systems: preexisting bias, "with its roots in social institutions, practices, and attitudes; technical bias, aris[ing] from technical constraints or considerations; and emergent bias stemming from 'context of use'" (331). These categories bled into one another in the process of conceptualizing, designing, building, and deploying a digital system. Moreover, Friedman and Nissenbaum wondered, "what ought they [designers] do if a client actually wants a bias to be present?" (345). As ML-based automated decision-making systems are used today to determine access to social assistance benefits, assess a convicted criminal's risk of recidivism, or predict an individual's emotional state in the context of education or hiring, where does the sociopolitical end and the ethical begin?The philosopher Shannon Vallor seeks answers to this question in her incisive and magisterial Technology and the Virtues: A Philosophical Guide to a Future Worth Wanting. Vallor's aim is to help individuals grapple with the ethical impacts of novel technologies. As suggested by its title, Technology and the Virtues seeks to describe a virtue ethics for living with tools like AI. Vallor argues that humans should collectively cultivate "technomoral virtues": character traits "most likely to increase our chances of flourishing together" in a digitally mediated and interconnected world (119).Ethics as a branch of contemporary Western philosophical inquiry has several subdivisions. Consequentialist or utilitarian ethics entails weighing one's actions regarding the pleasure or displeasure of the maximum/minimum number of persons. Deontological or duty-based ethics asks whether a person's actions conform to a normative code of rules or laws. Rights-based ethics suggests ethical or moral entitlement by virtue of belonging to a particular class. And virtue ethics involves "development of the character of an individual and actions that result as a consequence of good character" (Kazim and Koshiyama 2021: 4). Consequentialist and deontological perspectives have dominated philosophical accounts of ethics, including in computing; indeed, one 1985 reviewer of Johnson's Computer Ethics faulted the text for giving short shrift to rights, justice, and virtue in ethical theory. "Students [may] believe," the review complained, "that addressing ethical issues amounts either to calculating the greatest good or urging that persons be respected . . . [which] is little more than all sides invoking motherhood and apple pie on their behalf" (Bowie 1985:321). To the extent that contemporary AI ethics discussions often center on professional ethics codes or the exigencies of capitalist accumulation, this critique of the field still resonates.Through comparative analysis of the "classical" virtue traditions of Aristotelian, Buddhist, and Confucian ethics, Vallor seeks to bring virtue ethics to bear on digital technologies; other global ethical traditions might be added to the conversation, such as the sub-Saharan African philosophy of Ubuntu of relational personhood (Mhlambi 2020). Vallor compares the tenets of these philosophies to develop a common schema for "the practice of moral self-cultivation" (118). This framework, which she terms technomoral wisdom, is one that can lead to a heightened capacity for phronesis, or practical wisdom in both the design and use of digital technologies. The elements of this framework include practices of moral habituation (66–67) and the intentional self-direction of one's moral development (91–93), strengthening one's "moral muscles" through repeated conscious ethical choice; relational understanding (76–77) and reflective self-examination (84–85), whereby individuals conceive themselves as existing within a rich web of relationships and interrogate their own self within it; and appropriate moral attention (99–100), prudent judgment (105–106), and extension of moral concern to others (110–111). In combination, these practices support phronesis, expressed through such virtues as care, humility, civility, and magnanimity (120).Vallor's refinement of virtue ethics practice in the context of digital technologies resonates with scholarship in science and technology studies and critical human-computer interaction exploring how to design digital tools with human values in mind. Phoebe Sengers et al.'s (2005: 50) reflective design methodology, for instance, is grounded in critical reflection, "an essential tool to allow people to make conscious value choices in their attitudes and practices." Reflective design was in turn inspired by critical technical practice, an attempt to design symbolic AI reflexively developed by AI researcher Philip Agre (1997b, 1997a). More recently, Nassim JafariNaimi, Lisa Nathan, and Ian Hargraves (2015) argue for designers to understand human values as hypotheses to inspire a phronetic approach to design. These scholars are part of what might described as an implicit virtue ethics tradition within the philosophy of technology and critical design studies. Vallor's book has much to contribute to these debates and is a fruitful point of entry for transdisciplinary conversations around digital technologies, values, and practice.In the second half of Technology and the Virtues, Vallor applies her framework for cultivating technomoral virtues to several areas of current technological development, including social media, digital surveillance, and robotics—all fields in which AI-based automated decision systems are ubiquitous. Like Johnson, Vallor suggests that not all philosophical questions concerning AI are ethical ones. Ethical decisions, or ones about "how to live well" with AI, are for Vallor distinct from metaphysical, economic, political, or existential concerns (209). Yet the longer tradition of work exploring values in technical design underscores the fuzziness of this distinction. Individual decisions about how to live well are to some degree structured by general conditions of living. While personal phronetic judgment is valuable in widely variable situations, its exercise cannot be separated from the communal and the collective, including the mediating conditions under which social life takes place. Personal or institutional ethical choices can have wide-reaching effects on the terrain of choice available to the broader citizenry as they seek to live ethically with AI technologies—and, perhaps more important, with the interests and ambitions of AI's developers and financial backers.Vallor's normative claim echoes an astute observation of Gilles Deleuze (1990: 180), that "[it] isn't to say that their machines determine different kinds of society but that they express the social forms capable of producing them and making use of them." Indeed, Technology and the Virtues ends with a meditation on knowing what sort of society we ought to strive for. Vallor notes that technical artifacts are but ought not be ends in themselves: "Their ultimate ends," she observes, "must exist outside the technological sphere" (147). Whether Vallor's framework for the cultivation of the technomoral self can sit easily within putatively liberal, uneasily technocratic, and explicitly neoliberal capitalist societies is perhaps the most urgent open question posed by Technology and the Virtues. Vallor acknowledges both the need for and challenges inherent in cultivating collective technomoral virtue, calling for "new cultural investments in moral education and practice adapted for technosocial life" (245). However, it is worth asking whether the fundamentals of computing can conceptualize human beings as agents capable of moral judgment in the way Vallor hopes.▪ ▪ ▪How the developers of AI systems conceptualize human emotion, central to both empathy and phronesis, provides a case study for the challenge of reconciling virtue ethics with today's AI. Computer science largely ignored emotion for many decades. Early AI researchers did not articulate human emotion as either an "aspect of learning" or a "feature of intelligence." Why this omission? Andrew McStay's Emotional AI: The Rise of Empathic Media is one of the few books that even asks this question. "The historical emphasis on AI has been on intellect rather than feeling," McStay observes, and this statement—understatement, really—is certainly true for the field's dominant research paradigms (18). Cognitivism, which dominated both cognitive psychology and philosophy of mind for much of the latter twentieth century, was resolutely focused on the nature of thought, not feeling (Dupuy 2009). And Western philosophical ethics traditions have engaged with emotions sporadically at best.Yet over the past two decades, digital systems that collect data about human emotional expression, analyze it, and seek to simulate it have become increasingly common, part of facial recognition systems and automated chatbots like ChatGPT. McStay's purpose in Emotional AI is to explore this variegated technical landscape and these technologies' impacts.As McStay describes, the developers of emotional AI (EAI) systems have sought to exploit as many quantifiable proxies for emotional expression as possible in their pursuit of verisimilitude. Emotional AI contains chapters on sentiment analysis, biofeedback, facial emotion coding, voice analysis, and virtual reality; all these technologies use a particular proxy (text or images, biorhythms, facial expressions, tone and pitch, and movement) to putatively extrapolate insights about a person's interior emotional state. The conceptual model of human emotion adopted by researchers shapes what conclusions they can glean from this wide array of proxy data. This EAI "offers the appearance of understanding" human emotional expression. McStay terms this approach neobehaviorist because these technologies "'simply' observe, classify, allocate, adapt and modify their behavior" (4). The computational capture of emotional expression should therefore be understood in its double sense: as processes of both literal datafication and metaphorical force, pushing aside subjective accounts of emotion in favor of quantitative data collected by the mediated eye or ear.McStay describes the activities of EAI system as "a form of empathy," which he defines as "to understand another person's condition by means of what we survey, measure, and remember" (4). Such simulated empathy, McStay suggests, "we can simply judge by effectiveness" (5). Compare this definition of empathy with Vallor's in Technology and the Virtues—"a form of co-feeling, or feeling with another, synonymous with compassion" (133)—and at least one sort of alignment problem immediately presents itself. Does empathy require virtue, or can it be automated? McStay does recognize the dangers of defining human emotion "in biomedical terms that suit technology [and] industrial categorization" (186). He argues that the question posed at the beginning of Emotion AI—"is machinic empathy that different from human empathy?" (5)—hinges on distinguishing empathy from sympathy. Machines have the former, but not the latter.Machinic empathy is grounded on what McStay calls the categorical approach to understanding what human emotional expression represents. Often described as basic emotion theory (BET), this view has "practical advantages for technologists, managers, marketers, and sales teams" (58). BET posits that there a "number of primary basic emotions hardwired in our brains and universally recognized" (56). Another way to understand this theory is as "motivational" or "anti-intentional": affective response in this view prefigures conscious intention (Leys 2011), meaning that our "true" emotional signals leak out into the world even as we seek to suppress them. Such an approach contrasts with a second school of thought on human emotion, which posits emotions as variously "dimensional," "evaluative," or "intentional." In this view, individual emotions are grounded in a combination of physiological arousal, affective valence, and social, cultural, and personal experience. They perform, in the words of the sociologist Arlie Russell Hochschild (2003), a "signal function" and can be reflected on, modified, and shaped according to personal and cultural context.This conceptual disagreement over the nature of emotion bears strongly on contemporary emotion recognition technologies. McStay avers that he is "not in the position to conclude whether or not basic emotions exist" (72). Yet if they do not, the theoretical underpinnings of EAI are severely weakened. If human emotions are anti-intentional or categorical, this makes them easy to track and easy to present as incontrovertible proof of an individual's inner state. If they are intentional or dimensional, AI-driven inference about our feelings becomes much more difficult and less computationally tractable. "Although industry prefers the categorical approach," "the nature of emotional life [is] by no means settled" (56), McStay notes drily. The debate does seem to be tilting in a direction unflattering to industry: a recent metareview found little evidence for the efficacy of facial emotion coding systems grounded in BET (Barrett et al. 2019).The nature of human emotion pertains directly to the ethics of responding to, living with, and developing AI technologies. The premise of Vallor's virtue ethics, "the practice of moral self-cultivation" (118), will always be incomplete if human emotions are intrinsically motivational and anti-intentional. Phronesis (applied wisdom or judgment) is possible only if BET is false and humans are understood to be able to reflect on and master their feelings. Such mastery is itself neither an innocent notion nor a panacea: hierarchies of rationality used to justify the subjugation of women and nonwhite people have often been expressed through appeals to who and who is not is mature enough to accomplish this mastery (Schuller 2018). Indeed, a lack of attention to human emotion in computer science stems from in large part the former's association with irrationality and immaturity in the history of Western technoscience (Schuller 2018): irrationality ascribed to the politically marginalized was understood as simultaneously symptom and cause of low status. But an intentionalist theory of emotion at least allows for some degree of subjective freedom and, by extension, space for collective solidarity.McStay concludes his book with a plea for attention to the ethics of EAI systems, including asking whether "citizens . . . are best served by this passive surveillance of emotional life" (191). Read alongside Christian's and Vallor's, McStay's implicit message must surely be no. I respectfully disagree with McStay, therefore, when he suggests that "there is nothing inherently wrong with technologies that detect, learn and interact with emotions" (2). At issue, as Christian articulates belatedly in The Alignment Problem, is not so much that a motivational theory of emotion like BET is true (for it is not) as that such a model is useful for the beneficiaries of AI's contemporary technocratic, deterministic moral background, leaving little conceptual space for moral agency. In computing, tautology and bigotry are playmates like ones and zeros.Can any flavor of communitarian ethics withstand the digitally mediated imperatives of a materialist scientism working in the service of patriarchy, parochialism, and profit? AI systems and their backers tell, teach, and habituate humans to understand themselves as lacking the emotional reflexivity required for phronesis. AI ethics—indeed, all normative life—risks becoming a matter of aligning and complying with two conjectures that for their faithful cannot be falsified: that all in the world can be reduced to number alone, and that fellow feeling can be dispensed with until it makes a buck.