Computer Systems and Values
    (Helen Nissenbaum, 5/2001)

    by Douglas Dixon

Does technology have anything to do with values? Can we develop technology for its own sake, independent of its impact on society? Or do information systems actually embody values?

"Information technology changes the world, and some of these changes challenge previous commitments to values and principles," writes Helen Nissenbaum, research associate at Princeton University's Center for Human Values. "Yet the idea of values embodied in computer and information systems suggests motion in the opposite direction, from values to technology. Values affect the shape of technologies."

In an article in IEEE Computer magazine, March 2001, titled "How Computer Systems Embody Values," Nissenbaum calls for "engineering activism" to "advocate on behalf of values" in order to influence the inevitable incorporation of values into computer systems.

Nissenbaum is spending this year at the Institute for Advanced Study's School of Social Science. Next year she is moving to NYU to join the Department of Culture and Communications. Her areas of expertise are the social, ethical, and political dimensions of science and technology, with an emphasis on information and communications technology. She has written on issues relating to privacy, property rights, electronic publication, accountability, and values embedded in the design of computer systems. She is the author of the book Emotion and Focus (University of Chicago Press), and co-editor (with D. J. Johnson) of Computers, Ethics and Social Values (Prentice-Hall). She is one of four founding editors of the international journal, Ethics and Information Technology.

"Trained as a philosopher," Nissenbaum writes, "I am nevertheless increasingly drawn toward the science and engineering of information technology in my work on its ethical, social, and political dimensions."

She traces her interest to a research project on bias in computer systems published in 1996: "A compelling and mysterious idea emerged from this project: Computer and information systems can embody values. I found this idea so compelling that it has all but hijacked the path of my work since then. Its mystery lies in seeing values as part of technology, a perspective not usually adopted by scholars and researchers who study the social, ethical, and political aspects of information technology."

Nissenbaum originally studied mathematics and philosophy at the University of Witwatersand, Johannesburg, and earned a B.A. with honors in philosophy in 1976. She then moved to Stanford University, and earned an M.A. in education in 1978 and a Ph.D. in philosophy in 1983. She held a postdoctoral fellowship at the Center for the Study of Language and Information and was assistant director of the Symbolic Systems Program at Stanford. She then moved to Princeton in 1991 as the associate director of Princeton University's Center for Human Values.

Nissenbaum organizes her work on the ethical dimension of technological change into two categories, depending on whether the focus is on the social change caused by technology, or on the values themselves embodied in the technology.

Social Changes

"Information technology has radically altered our lives and even our selves," Nissenbaum writes. "The radical effects of the process have extended to institutions, social processes, relationships, power structures, work, play, education, and beyond." Society then struggles with the resulting social changes:

Computer systems replace human decision-making. 
In situations where "computer systems replace humans who act in positions of responsibility -- prescribing drugs, making investment decisions, controlling aircraft," the concern is that "lines of accountability and responsibility will be disturbed and possibly erased. Where once we could hold someone responsible for failure and its consequences, now there is a vacuum."

The digital divide. 
Disparities in access to new technology between the dot-com elite and the disadvantaged also raises the possibility that "information technology will cause even greater social injustice than we currently experience."

Values Embodied In Technology

Even more, Nissenbaum argues that we must accept that "systems have moral or political properties." In fact, these are some of the most contentious issues that have been playing out in recent headlines:

Intellectual property and Napster 
"Because intellectual production has been so profoundly affected by information technology," Nissenbaum writes, "it strikes at the heart of previously settled ideas and valuations of intellectual property." We have seen this in the "raging indignation on both sides of Napster," where the core of the technological development of peer-to-peer music sharing is a fundamental threat to the foundations of the music business, as well as other forms of content creation and licensing, including movies and books.

Intel Pentium III chip with embedded serial number 
Intel's decision to stamp its new processor with a digital serial number as a security and copy protection mechanism caused a public furor over privacy concerns, and Intel eventually disabled the feature by default.

Nissenbaum is interested in the link between values and design in this case, wondering why Intel had decided to add the serial number: "Had it overlooked the privacy implications, merely hoped no one would notice, or made a considered judgment that the potential security benefits out-weighed privacy concerns? Had there been deliberation behind closed doors after some project manager, designer, engineer, or marketing executive alerted company executives to the hazard? Was the decision a sign of carelessness, arrogance, or mere misjudgment? Was Intel out of touch with prevailing values, or did it assume that the company carried enough clout to shape them?"

Addressing Values

Nissenbaum argues that these issues must be addressed: "We cannot simply align the world with the values and principles we adhered to prior to the advent of technological challenges. Rather, we must grapple with the new demands that changes wrought by the presence and use of information technology have placed on values and moral principles."

But the idea of systems embodying values is disquieting for both social scientists and technologists. So her article in IEEE Computer challenges engineers to face an unfamiliar obligation: "To perceive not only the usual set of properties that the systems they build or design may embody, but those systems' moral properties as well: bias, anonymity, privacy, security, and so on. The challenge of building computer systems is transformed into a forum for activism -- engineering activism."

Nissenbaum admits that it may be difficult to address such questions, "because factors in the real world -- such as bosses, share-holders, regulations, competitors, and resource limits -- can prove hostile to yet another layer of constraints."

"Yet tempting as it may be to ignore value properties," she writes, "doing so will not make them go away. Systems and devices will embody values whether or not we intend or want them to. Ignoring values risks surrendering the determination of this important dimension to chance or some other force."

"Act," she challenges engineers, and anyone else involved in designing information systems, "make, build, or design the necessary changes, if doing so is within your power."

References 

Helen F Nissenbaum

NYU 1/04 - www.nyu.edu/projects/nissenbaum/index.html