Lev Manovich - The Language of New Media



Manovich, Lev. The Language of New Media. Cambridge, Mass. : MIT Press, c, 2000. Leonardo.

notes under cut



What is new media?
...

In section “Media and Computation” I show that new media represents a
convergence of two separate historical trajectories: computing and media
technologies. Both begin in the 1830's with Babbage's Analytical Engine and
Daguerre's daguerreotype. Eventually, in the middle of the twentieth century, a
modern digital computer is developed to perform calculations on numerical data
more efficiently; it takes over from numerous mechanical tabulators and
calculators already widely employed by companies and governments since the
turn of the century. In parallel, we witness the rise of modern media technologies
which allow the storage of images, image sequences, sounds and text using
different material forms: a photographic plate, a film stock, a gramophone record,
etc. The synthesis of these two histories? The translation of all existing media into
numerical data accessible for computers. The result is new media: graphics,
moving images, sounds, shapes, spaces and text which become computable, i.e.
simply another set of computer data. In “Principles of New Media” I look at the
key consequences of this new status of media. Rather than focusing on familiar
categories such as interactivity or hypermedia, I suggest a different list. This list
reduces all principles of new media to five: numerical representation, modularity,
automation, variability and cultural transcoding. In the last section, “What New
Media is Not,” I address other principles which are often attributed to new media.
I show that these principles can already be found at work in older cultural forms
and media technologies such as cinema, and therefore they are by themselves are
not sufficient to distinguish new media from the old.
44


This principle can be called "fractal structure of new media.” Just as a fractal has
the same structure on different scales, a new media object has the same modular
structure throughout. Media elements, be it images, sounds, shapes, or behaviors,
are represented as collections of discrete samples (pixels, polygons, voxels,
characters, scripts). These elements are assembled into larger-scale objects but
they continue to maintain their separate identity. The objects themselves can be
combined into even larger objects -- again, without losing their independence. For
example, a multimedia "movie" authored in popular Macromedia Director
software may consist from hundreds of still images, QuickTime movies, and
sounds which are all stored separately and are loaded at run time. Because all
elements are stored independently, they can be modified at any time without
having to change Director movie itself. These movies can be assembled into a
larger "movie," and so on. Another example of modularity is the concept of
“object” used in Microsoft Office applications. When an object is inserted into a
document (for instance, a media clip inserted into a Word document), it continues
to maintain its independence and can always be edited with the program used
originally to create it. Yet another example of modularity is the structure of a
HTML document: with the exemption of text, it consists from a number of
51

separate objects — GIF and JPEG images, media clips, VRML scenes,
Schockwave and Flash movies -- which are all stored independently locally
and/or on a network. In short, a new media object consists from independent parts
which, in their turn, consist from smaller independent parts, and so on, up to the
level of smallest “atoms” such as pixels, 3D points or characters.
World Wide Web as a whole is also completely modular. It consists from
numerous Web pages, each in its turn consisting from separate media elements.
Every element can be always accessed on its own. Normally we think of elements
as belonging to their corresponding Web sites, but this just a convention,
reinforced by commercial Web browsers. Netomat browser which extract
elements of a particular media type from different Web pages (for instance, only
images) and display them together without identifying the Web sites they come
from, highlights for us this fundamentally discrete and non-hierarchical
organization of the Web (see introduction to “Interface” chapter for more on this
browser.)
In addition to using the metaphor of a fractal, we can also make an
analogy between modularity of new media and the structured computer
programming.
...
In fact, the modular structure of new media makes such deletion and
substitution of parts particularly easy. For example, since a HTML document
consists from a number of separate objects each represented by a line of HTML
code, it is very easy to delete, substitute or add new objects. Similarly, since in
Photoshop the parts a digital image are usually placed on separate layers, these
parts can be deleted and substituted with a click of a button.

52



4.3. The information about the user can be used by a computer program to
automatically customize the media composition as well as to create the elements
themselves. Examples: Web sites use the information about the type of hardware
and browser or user's network address to automatically customize the site which
the user will see; interactive computer installations use information about the
user's body movements to generate sounds, shapes, and images, or to control
behaviors of artificial creatures.
4.4. A particular case of 4.3 is branching-type interactivity (sometimes
also called menu-based interactivity.) This term refers to programs in which all
the possible objects which the user can visit form a branching tree structure.
When the user reaches a particular object, the program presents her with choices
and let her pick. Depending on the value chosen, the user advances along a
particular branch of the tree. For instance, in Myst each screen typically contains
a left and a right button, clicking on the button retrieves a new screen, and so on.
In this case the information used by a program is the output of user's cognitive
process, rather than the network address or body position. (See “Menus, Filters,
Plug-ins” for more discussion of this principle.)
4.5. Hypermedia is another popular new media structure, which
conceptually is close to branching-type interactivity (because quite often the
elements are connected using a branch tree structure). In hypermedia, the
multimedia elements making a document are connected through hyperlinks. Thus
the elements and the structure are independent of each other --rather than hardwired
together, as in traditional media. World Wide Web is a particular
implementation of hypermedia in which the elements are distributed throughout
the network . Hypertext is a particular case of hypermedia which uses only one
media type — text. How does the principle of variability works in this case? We
can conceive of all possible paths through a hypermedia document as being
different versions of it. By following the links the user retrieves a particular
version of a document.
57

In particular, such popular new media structures as branching
(or menu) interactivity and hypermedia can be seen as particular instances of
variability principle (4.4 and 4.5, respectively). In the case of branching
interactivity, the user plays an active role in determining the order in which the
already generated elements are accessed. This is the simplest kind of interactivity;
more complex kinds are also possible where both the elements and the structure
of the whole object are either modified or generated on the fly in response to
user's interaction with a program. We can refer to such implementations as open
interactivity to distinguish them from the closed interactivity which uses fixed
elements arranged in a fixed branching structure. Open interactivity can be
implemented using a variety of approaches, including procedural and objectoriented
computer programming, AI, AL, and neural networks
..
Hypermedia, the other popular structure of new media, can also be seen
as a particular case of the more general principle of variability. According to the
definition by Halacz and Swartz, hypermedia systems “provide their users with
the ability to create, manipulate and/or examine a network of informationcontaining
nodes interconnected by relational links.”

Since in new media the individual media elements (images, pages of text, etc.) always retain their
individual identity (the principle of modularity), they can be "wired" together into more than one object. Hyperlinking is a particular way to achieve this wiring. A
59

hyperlink creates a connection between two elements, for example between two words in two different pages or a sentence on one page and an image in another, or two different places within the same page. The elements connected through hyperlinks can exist on the same computer or on different computers connected on a network, as in the case of World Wide Web.
60


The Myth of Interactivity
We have only one principle still remaining from the original list: interactivity. As
with “digital,” I avoid using the word “interactive” in this book without qualifying
it,. for the same reason -- I find the concept to be too broad to be truly useful.
70

Used in relation to computer-based media, the concept of interactivity is a
tautology. Modern human-computer interface (HCI) is by its very definition
interactive. In contrast to earlier interfaces such as batch processing, modern HCI
allows the user to control the computer in real-time by manipulating information
displayed on the screen. Once an object is represented in a computer, it
automatically becomes interactive. Therefore, to call computer media interactive
is meaningless -- it simply means stating the most basic fact about computers.
Rather than evoking this concept by itself, in this book I use a number of
other concepts, such as menu-based interactivity, salability, simulation, imageinterface,
and image-instrument, to describe different kinds of interactive
structures and operations. The already used distinction between “closed” and
“open” interactivity is just one example of this approach.
While it is relatively easy to specify different interactive structures used in
new media object, it is much more difficult to theoretically deal with user
experiences of these structures. This remains to be one of the most difficult
theoretical questions raised by new media. Without pretending to have a complete
answer, I would like to address some aspects of this question here.
All classical, and even more so modern art, was already "interactive" in a
number of ways. Ellipses in literary narration, missing details of objects in visual
art and other representational "shortcuts" required the user to fill-in the missing
information.
41
Theater, painting and cinema also relied on the techniques of
staging, composition and cinematography to orchestrate viewer's attention over
time, requiring her to focus on different parts of the display. With sculpture and
architecture, the viewer had to move her whole body to experience the spatial
structure.
Modern media and art pushed each of these techniques further, putting
new cognitive and physical demands on the viewer. Beginning in the 1920s new
narrative techniques such as film montage forced the audiences to quickly bridge
mental gaps between unrelated images. New representational style of semiabstraction
which, along with photography, became the “international style” of
modern visual culture, required the viewer to reconstruct the represented objects
from the bare minimum -- a contour, few patches of color, shadows cast by the
objects not represented directly. Finally, in the 1960s, continuing where Futurism
and Dada left of, new forms of art such as happenings, performance and
installation turned art explicitly participational. This, according to some new
media theorists, prepared the ground for interactive computer installations which
appeared in the 1980s
When we use the concept of “interactive media” exclusively in relation to
computer-based media, there is danger that we interpret "interaction" literally,
equating it with physical interaction between a user and a media object (pressing a
button, choosing a link, moving the body), at the sake of psychological
interaction. The psychological processes of filling-in, hypothesis forming, recall
71

and identification, which are required for us to comprehend any text or image at
all, are mistakenly identified with an objectively existing structure of interactive
links.

72

Interactive computer media perfectly fits this trend to externalize and
objectify mind’s operations. The very principle of hyperlinking, which forms the
basis of much of interactive media, objectifies the process of association often
taken to be central to human thinking. Mental processes of reflection, problem
solving, recall and association are externalized, equated with following a link,
moving to a new page, choosing a new image, or a new scene. Before we would
look at an image and mentally follow our own private associations to other
images. Now interactive computer media asks us instead to click on an image in
order to go to another image. Before we would read a sentence of a story or a line
of a poem and think of other lines, images, memories. Now interactive media asks
us to click on a highlighted sentences to go to another sentence. In short, we are
asked to follow pre-programmed, objectively existing associations. Put
diffidently, in what can be read as a new updated version of French philosopher
Louis Althusser's concept of "interpellation," we are asked to mistake the
structure of somebody's else mind for our own.
74


Also like Blade Runner, GUI vision also came to influence many other
areas of culture. This influence ranges from purely graphical (for instance, use of
GUI elements by print and TV designers) to more conceptual. In the 1990s, as the
Internet progressively grew in popularity, the role of a digital computer shifted
75

from being a particular technology (a calculator, a symbol processor, an image
manipulator, etc.) to being a filter to all culture, a form through which all kinds of
cultural and artistic production is being mediated. As a window of a Web browser
comes to replace cinema and television screen, a wall in art gallery, a library and
a book, all at once, the new situation manifest itself: all culture, past and present,
is being filtered through a computer, with its particular human-computer
interface.
In semiotic terms, the computer interface acts as a code which carries
cultural messages in a variety of media. When you use the Internet, everything
you access — texts, music, video, navigable spaces — passes through the
interface of the browser and then, in its turn, the interface of the OS.
...

The interface shapes how the computer
user conceives the computer itself. It also determines how users think of any
media object accessed via a computer. Stripping different media of their original
distinctions, the interface imposes its own logic on them. Finally, by organizing
computer data in particular ways, the interface provides distinct models of the
world. For instance, a hierarchical file system assumes that the world can be
organized in a logical multi-level hierarchy. In contrast, a hypertext model of the
World Wide Web models the world as a non-hierarchical system ruled by
metonymy. In short, far from being a transparent window into the data inside a
computer, the interface bring with it strong messages of its own.
76


The term human-computer interface (HCI) describes the ways in which the user
interacts with a computer. HCI includes physical input and output devices such a
monitor, a keyboard, and a mouse. It also consists of metaphors used to
conceptualize the organization of computer data. For instance, the Macintosh
interface introduced by Apple in 1984 uses the metaphor of files and folders
arranged on a desktop. Finally, HCI also includes ways of manipulating this data,
i.e. a grammar of meaningful actions which the user can perform on it. The
example of actions provided by modern HCI are copy, rename and delete file; list
the contents of a directory; start and stop a computer program; set computer’s date
and time.
...
By the end of the decade, as Internet use became commonplace, the computer's
public image was no longer that of tool but also that a universal media machine,
used not only to author, but also to store, distribute and access all media.
As distribution of all forms of culture becomes computer-based, we are
increasingly “interfacing” to predominantly cultural data: texts, photographs,
films, music, virtual environments. In short, we are no longer interfacing to a
computer but to culture encoded in digital form. I will use the term "cultural
interfaces" to describe human-computer-culture interface: the ways in which
computers present and allows us to interact with cultural data. Cultural interfaces
include the interfaces used by the designers of Web sites, CD-ROM and DVD
titles, multimedia encyclopedias, online museums and magazines, computer
games and other new media cultural objects.
80

Cultural interfaces rely on our familiarity with the "page interface" while
also trying to stretch its definition to include new concepts made possible by a
computer.....

The conceptual development of the page in computer media can also be
read in a different way — not as a further development of a codex form, but as a
return to earlier forms such as the papyrus roll of ancient Egypt, Greece and
Rome. Scrolling through the contents of a computer window or a World Wide
Web page has more in common with unrolling than turning the pages of a modern
book. In the case of the Web of the 1990s, the similarity with a roll is even
stronger because the information is not available all at once, but arrives
sequentially, top to bottom, as though the roll is being unrolled.

84

Just as a rectangular frame of painting and photography presents a part of
a larger space outside it, a window in HCI presents a partial view of a larger
document. But if in painting (and later in photography), the framing chosen by an
artist was final, computer interface benefits from a new invention introduced by
cinema: the mobility of the frame. As a kino-eye moves around the space
revealing its different regions, so can a computer user scroll through a window's
contents.