Gesture, or the body without organs of speech

For most people, gesture refers to a primitive, non-intellectual expression of feeling, a rudimentary and not too important kind of communication. The very idea of humans pointing and waving parts of their bodies at each other, themselves, or their gods, appears atavistic, a return to a time before speech; to the origin of the species, to simian chatter or cavemen grunts, to modes of sociality and sensemaking overtaken by the development of language. In relation to the spoken word and the visual image, gesture is a poor, unsophisticated third; pre-rational and retrograde, belonging with certain ceremonies, rituals of the body, ancient dance practices, festivals, sacrifices to the gods; offering little to contemporary discourse beyond a minor anthropological and perhaps artistic interest.

But gesture, it seems, is being re-evaluated, re-cognized and in some sense, reclaimed. Thus, despite (and because of) its supposed primitivism, and for several motives, gesture is increasingly the object of attention and interest -- precisely in relation to the question of the 'human' -- from different directions within the contemporary, logos-driven scene. First, the recognition that the gestural systems used throughout the world by the deaf to commune with each other are full-blown (visual) languages, on a par with and in some respects superior to the (auditory) languages of human speech; one consequence of which is a reconfiguring of established ideas of 'language' and communication (human and animal) and the re-emergence of theories proclaiming the gestural affiliations and origins of speech. Second, the development of audio and visual technologies of speech recognition/simulation and motion capture that, by digitizing the body's activities and making gestures objects of consciousness, are heralding a new, post-documentist mode of re-presenting human movement, of 'writing' the body to allow gesture to be used as a new mode of human-computer interface. Third, the ongoing re-evaluation of the body itself as the site of material and discursive social-historical practices, as an object constructed at the shifting boundaries between the bio-medical and cultural, the actual and virtual, and the concomitant transformations taking place in how we think of, narrate, understand, experience -- become -- what we have for so long (blithely and for the most part without reflection) simply called the human 'body'. Certainly, whatever else it may be, gesture is a body thing; part of the body's shape, its envelope, its presence to itself and others. And just as certainly, a new era of the body is upon us, wherein the creative effects -- spiritual, epistemological, ecological, aesthetico-ethical, ontological -- of our corporeal finitude are being cognized (as if for the first time). "We have just only begun", as Gilles Deleuze paraphrasing Baruch Spinoza has it, "to understand what the body is capable of". An era that recognizes that we have/are embodied minds, enfleshed psyches and that our inner states -- thinking and imagining, dreaming, feeling, remembering, self-experiencing -- cannot (except in the misinformed fantasies of certain techno-transcendentalists) be disengaged from the body. The body-self, increasingly assembled at different places, times and speeds according to heterogeneous imperatives and agencies, is becoming distributed and collectivized: corporeality, being/having a body and a self, is going plural, parallel, simultaneous and multilevel, against old forms of being singular, linear, unitary and sequential. (1) Difficult to conceive the status of gestures -- which are made by and make the body -- could remain the same, that one could think gesture at the present time without re-thinking it, without being led to construct a gesturology. I can't do that here. Nor do I wish to: it would be premature and not appropriate; better to gesture towards it, to point to the project by trying to say why at this moment in its history gesture might be significant, productive, intriguing enough to deserve, at the least, an ology devoted to it. Behind this claim for its significance is an implicit valorization of gesture's so-called primitiveness: the fact that it is more ancient than speech, pre- and counter-intellectual in character, the source of the imagistic-holistic dimension of thought. Without question, gesture -- the work that it does -- reaches deep into human sociality, to its affective and ancient core. It is as essential to the ongoing making of the human (from the proffered breast to the turn-taking which inducts infants into language to the maintenance of innumerable symbolic practices), as it is inseparable from the wordless empathy without which what sociologist Michel Mafesoli calls puissance or the 'will to live' would not be possible; therein -- in its primitiveness, linguistic silence, and aboriginality -- lies its importance and value.

What do I intend by gesture? The mathematician in me insists on starting with a definition. To that end, call any body-movement a gesture that can be identified, repeated, and assigned a meaning in relation to at least one of the three regimes or modes categorized as significance, function or experience. The movement in question can be already finished: I include all supposedly static positions, poses, postures, tableaux, attitudes since they are the frozen endproducts of gestural movements -- striking a pose, taking an attitude or position, assuming a posture, and so on. A gesture can be a bodily antic, an expressive practice, a routine, a mediation, or performance; any iterable pattern the body creates and manifests, any re-cognizable kinesis. The following can be gestures: raising an eyebrow, blowing a kiss, puckering the lips, shrugging, exiting in high dudgeon, any kind of dancing, playing the violin, genuflecting, giving the finger, clenching the fists, beating a rhythm, nodding the head, making an ASL sign, altering volume, pitch or tone of voice, laughing, sobbing, marching, jumping on the spot, sighing, hammering in a nail, ... , painting, drawing, scratching on a surface, daubing, tapping on a keyboard, turning a screw, stroking a cat, slicing an apple, and so on, and so forth. Gestures are co-extensive with bodily movement; to examine them it is necessary to make distinctions. We can, as already mentioned above, identify three modalities of gesture. They correspond to three modes of embodiment or aspects of being embodied: the semiotic (signifying body), the instrumental (functioning body), and the immersive (experiencing body). In the semiotic, the body expresses, communicates, speaks, gives signs, uses language or the apparatus of codes to construct, convey or mediate meanings; in the instrumental -- pounding a nail, turning a wheel, slicing an apple -- the body is captured by a machinic circuit, it becomes a mechanical apparatus (more accurately part of a transduction), the source of what Andre Leroi-Gourhan calls "technics" ; in the immersive mode, the body enacts, presents or performs or experiences itself through immersive and participatory activities, as in beating a rhythm or dancing or marching. Observe that the three modes can differ according to who or what is the source or recipient of the gesture. If one asks, for example, about the movements, postures, and gestures of dance, or of acting, or music -- are they semiotic or experiential? -- the answer will depend on the agent, on the viewpoint. "There are two musics", Roland Barthes, tells us, "the music one listens to, the music one plays. ... Two different arts, each with its own history, its own sociology, its own aesthetics, its own erotic". In just the same way, dance as spectacle, as an art form of bodies choreographed for an audience and designed to express, signify, and encapsulate meanings and affects for observers, can hardly escape the semiotic; whereas dance for the dancer immersed in these same movements and gestures and participating in their production is within the experiential; and likewise, the same doubleness pervades the gestures of all theatrical performance. In what follows, my concern here will be entirely within the semiotic -- the body as meaning, representation, and significance. The topics of the participating body immersed in the construction of experience, and the instrumental body captured within a machinic circuit, I leave for another occasion.

Gestural signs, or 1000 ways of saying nothing
Within the restriction to the semiotic, to gesture as sign, I shall further narrow the focus by considering gesture primarily in its relation to speech. This, it turns out, is quite enough to chew on, and is in any case how most of us encounter gesture. And consequently disparage it: since, in relation to speech, gesture either appears as pop psychologyÕs secret 'body language' that simplistically promises to reveal all those disreputable urges hidden from speech; or as pantomime (mimetic, impoverished, and reduced) appropriate to those with little or no speech, such as the deaf and the dumb (in both senses), or to children or the childlike (Italians gesture a lot); or it is disregarded, dismissed as a vestige of a pre-human repertoire of signs displaced by the evolution of spoken language. Body language, deaf-and-dumb show, vestigial code: gesture appears as an inferior adjunct or trace of pre-speech, crude bodily expression having little connection to anything as elevated as the mind or thought or the spoken word. There have been, of course, alternative evaluations of gesture. Rhetoric and acting manuals have long recognized its importance in relation to the expression and concealment of thought and feeling. For dance it is of course central, whether in the classical Indian forms, where mudras hand gestures play an essential role, or in various forms of movement and posture recognized in Western dance from the Baroque to the present. In the 18th century, Condillac famously argued for gesture as the origin of speech (an idea currently revived as the claim that gesture contains the origins of syntax), and a figure like John Bulwer (inventor of the first fingerspelling alphabet) could attempt in the 17th century to construct an entire gesturology, one that taxonomized/anatomized significant body movement in terms of a close analysis -- a "dissection" -- of the body's muscles and argued for the expressive adequacy (if not superiority) of gesture to speech. From a different direction (and at a level prior to the tripartite division of gestures embraced here), haptic gestures, those involving touch, seem especially important; as if the possibility of contact with the body of the self or the other (kissing, hugging, clapping, slapping the back, shaking hands, punching, grabbing and stroking oneself, squeezing the shoulder, wringing oneÕs hands, and so on) gives such gestures a particular intensity or potency; as if the creation of a circuit of reciprocity, a doubling or folding over to form a new inside/outside, functioned as the site of origination not available to non-haptic gestures. Let me briefly cite two examples. According to more than one neurological account of the mind, a form of internal touching, a virtual auto-hapticity, is a prefiguring or rehearsal or condition for the possibility of self-consciousness. The idea rests on the fact that most of the brain is connected not to sensors measuring the world but to other parts of the brain -- including parts which contain a map of the body on the surface of neo-cortex. This makes a form of self-touching or monitoring inside the brain possible, in which the brain, by exciting/inhibiting regions of itself, achieves a kind of phantom or virtual proprioception, a primitive template of the self becoming (auto)aware. And a quite different example of haptic gesture, touching an external social other rather than an inner neurological self, comes from the work of psychologist Robin Dunbar, who hypothesizes that a particular form of prolonged haptic gesture -- chimpanzee grooming -- gave rise to speech. The idea being that when primate bands got too big, too disseminated, too complex as a group for grooming, a surrogate for it -- gossip -- which served the social purpose of grooming but more efficiently, was made available through the development of spoken language. But speculations on the origins of consciousness and the evolution of spoken language are not my concern here. I'm interested in the relation between speech and gesture in their present day conception, specifically in the hierarchy of speech over gesture; my aim is to dissolve this ranking, to deny that gesture is a mere supplement to speech, to pull it out from the shadow of speech and re-conceive it as an autonomous and interestingly complex attribute of the body. In a sense, what is offered here will be a deconstruction, a working out of the two-fold process that Derrida called the logic of the supplement, whereby a supposedly ordered binary, a major term and a supplement to it, is inverted and displaced. For Derrida, the principal hierarchy was that of speech over writing: speech was primary, the site of presence to oneself, and the ground on which the supplement of writing, seen as that which comes after speech as a technique of notating it, operated. On the contrary, Derrida argued, writing comes before speech in at least two ways. First, writing or graphism, a general mode of signifying operating throughout cultural production, was always more and other than a device of speech inscription; in fact, speech itself relies on features -- spacing, for example -- of writing in this larger sense. Second, the very conception we have of the structure and constituents of speech which are assumed to have preceded writing cannot be divorced from its action; "The syllableÓ, as David Olson observes, "is as much a product of graphic system as a prerequisite for it." (x) For us, the hierarchy in question is speech over gesture: gesture being a semiotic supplement and outmoded accompaniment of spoken language. We shall invert and displace this in two ways. First, by observing that an extensive and important class of gestures, so-called emblems, far from being subordinate or epiphenomenal to speech, exceed it and are it seems incommensurable with it. Second, by showing that all speech has gesture folded into it: spoken words and their accompanying gestures, so-called gesticulation, arise from gestural and proto-gestural images. In addition, we shall remove speech's claim to any radical distinction from gesture by observing that speech is a variety of gesture, both in its perception no less than its production. This last will allow us to examine the gestural content of speech -- prosody -- as that which repeatedly overflows the resources of alphabetic writing, illustrating how that writing is both the triumph and horizon of the business of substituting words for absented bodies; a phenomenon that impinges on the endless deferral of meaning that figures so powerfully in Derrida's understanding of writing.

Language is more and other than speech (7)
First, by way of introducing gesture, let's observe that contrary to etymology (lingua = tongue) and general use, and against the assumptions of linguists until recently, one cannot identify languaging with speaking. The most deeply maintained form of anti-gesturalism occurs where speech is absent, in the history of the deaf and the suppression by their educators of Sign, the gesture-based system used by the deaf to communicate. A history and antagonism far from over: various universities in the United States currently are blocking the proposal to study ASL (American Sign Language) in fulfillment of foreign language requirements with three objections: ASL lacks a written form; ASL is not a 'real' language; ASL is not a 'foreign' language. Behind these objections lies the last remnants of oralism, a phonecentric educational philosophy which succeeded in 1880, at the international conference of deaf educators in Milan, in banning all use of Sign (from European and American schools) in favour of enforced voicing and lip-reading by the deaf. "Gesture", as the organizers put it, "is not the true language of man ... Gesture, instead of addressing the mind, addresses the imagination and the senses. Thus, for us, it is an absolute necessity to prohibit that language and to replace it with living speech, the only instrument of human thought." This phonocentric -- phonoimperialist -- eradication of gesture had gender support (imagination, senses, female) in favor of speech (mind, thought, man) and was aided implicitly by the pathologization (and even criminalization) of deafness. The oralists campaigned against the use of Sign on the grounds that it isolated the deaf from normal -- speaking -- society, threatening to promote what Alexander Graham Bell fearfully characterized as "a deaf variety of the human race". Thus, the recognition of the linguistic power of Sign, evidenced in Bell's fears (as well as in numerous accounts of the deaf since the 17th century acknowledging the educational efficacy of Sign), became a contributory reason for banishing it; which in turn erased this very recognition by allowing the prevailing caricature of Sign as pantomime to go unchallenged. In the 1960s the wall of hostility to gesture with its simplistic derogation of Sign exhibited its first crack. William Stokoe, a young professor at Gallaudet University, found it impossible to accede to the picture of Sign as exaggerated gestures, imitative movements lacking organization, regularity or internal structure. He observed that on the contrary, the gestures his students made were abstract not primarily mimetic or iconic, and were internally composite: made of distinct hand configurations, definite positions of hands in relation to the body, identifiable movements in space; they were also regular and predictable and appeared to form a language-like system in which gestures and meanings were sytematically paired. The result was the publication of a Dictionary of ASL in 1966 followed by a study of ASL in 1972 by Edward Klima and Ursula Bellugi that established beyond doubt that ASL had all the properties linguists expected in a language -- lexicon, grammar, syntax and morphological rules -- and appeared as expressive and capacious a medium as any spoken language. Since then, other varieties of Sign (British, Australian, Chinese, Danish, and many more) have likewise been shown to be fully-fledged languages. These varieties of Sign, though very different from each other, appear to exhibit certain similarities; they tend, for example, to avoid the accusative/nominative case distinction in favour of the ergative/absolute, a structure found in certain spoken languages such as Georgian. More widely, they are independent of the spoken language (but obviously not the effects of the culture, some of which are linguistic) surrounding them; American Sign Language, for example, is not signed English (ASL signs with English syntax and word order), but its own thing with a grammar and morphology quite different from that of English. More radically, all varieties of Sign exhibit a feature separating them from all spoken languages, namely their use of space and time compared to just the one dimension of time available to speech. This means not only a vastly richer capacity for depiction as opposed to verbal description (the world, for humans at least, is more visual and spatial than aural), but that at all levels -- grammar, syntax, lexicon, morphology -- what is necessarily one at a time, linear and sequential in speech can be simultaneous, parallel and multi-layered in Sign. The complex topological sequences of SignÕs gestural choreography, "language in four dimensions", as Stokoe has described ASL, thus exhibits possibilities of articulation and expression (and perhaps conceptualization) that are tantalizingly different from those of speech. The obverse side of Sign's spatial and dynamic complexity is that (as its detractors in the academy object) it has, at least so far, no accepted written form and hence no literature (literally: no body of art recorded in letters); a point I'll return to below. The challenge Sign poses to phonocentric linguistics has cultural and phenomenological correlates. Culturally (and politically), Sign enables the deaf to de-medicalize themselves, to de-signate themselves as deaf [small 'd'] and re-signate themselves as Deaf [capital 'd']; as, in other words, a linguistic group who use Sign (no differently from any other group defined/constituted by their language) and not as people who are speech/hearing deficient. Experientially, those born deaf can by definition have no voice-in-the-head so familiar to the hearing in its role as monitor, author, vehicle and evidence of consciousness; instead all these functions are discharged gesturally: those who Sign have internal kinesis, ghost movements not ghost voices, sights not sounds in the head. This raises the intriguing possibility of a parallelist self, an internally 'seen/felt' creation of a psyche different in certain phenomenological and cognitive directions, from the internally 'heard/voiced' linear self-consciousness of the hearing familiar to most of us. (A possibility that offers a new, not easily accommodated slant on the Lacanian thesis of the unconscious being structured like a language; insofar as that thesis assumes 'language' is speech). Finally, Sign -- a gestural alternative to speech whose structure, genesis and practices are independent of spoken languages and which unlike them such has no accepted written form -- poses an interesting roadblock to Derrida's totalization of 'writing', his conception of archewriting. For insofar as archewriting includes gesture, as much of his talk of spatializing would indicate, then it suffers the same metaphysical iniquities he attributes to speech, since Sign -- for the point in question-- is speech.Put differently, ASL separates (in fact opposes) the effect of presence-to-oneself from phonocentrism, whereas of course they are conjoined and theoretically inseparable in Derrida's oeuvre.

Emblems: symbolic gestures against speech
Like spoken words, ASL gestures are coded entirely by a linguistic system. Distinct from these, not captured by a code, forming only a "partial code" of gestures situated between the two linguistic systems, is the field of so-called emblems: self-contained gestures with fixed form and meanings such as kissing the fingertips, slapping the forehead, giving the finger, hailing, saluting, thumbs up, thumb down, winking, bowing the head, nodding the head, tossing the head, jerking the forearm, touching the lips, shrugging shoulders, sighing, tapping the foot, clasping hands, touching the nose, displaying a palm, bending an ear, pointing, holding up a fist, averting the face, wagging a finger, beckoning, genuflecting, kneeling, chopping the air, crossing the fingers, flicking the teeth, baring the breast, and other flexings, stretchings, knockings, clutchings, and strikings of poses it would be absurd to even consider typing out as combinations of alphabetic letters.

According to studies of gesture by Adam Kendon and David McNeill, (10) emblems are gestures whose principal function is to carry out certain social activities: "Emblems". McNeill writes, "are complete speech acts in themselves, but the speech acts they perform are restricted to a certain range of functions. They regulate and comment on behavior of others, reveal one's own emotional states, make promises, swear oaths, etc. They are used to salute, command, request, reply to some challenge, insult, threaten, seek protection, express contempt or fear." (64)

Emblems, then, are social, experiential and interpersonal; put there to make something happen, to impinge on the behaviour of the self and others; they're not really interested in making statements, asking questions, or conveying facts and propositions. They are in short performatives and, according to McNeill, discharge the functions carried out by speech acts. But unlike speech they do not combine via a syntax as part of a language. And they differ from words in that their meanings are neither explicitly defined nor (outside instruction in rhetoric or acting) are they intentionally learned or studied, but rather picked up, absorbed, and (perhaps for this reason) remain stable in form and import over long periods of time despite linguistic changes in the communities of their users. These features suggest that emblems might operate according to a different dynamic and logic, and might accomplish different ends from that of speech. Calling them 'speech' acts, assumes them to be individually interchangeable with or translatable into speech, as well as assuming that they operate, as a system, in the same ways and for the same purposes as speech. But is this so? Are emblems always translatable into spoken language? What, for example, is the speech equivalent of a wink? Or, for that matter, a shrug? a slap on the back? folding one's arms? hands clasped in prayer? And does their mode of operation resemble speech? If so, why do we bother with them?

The great range, robustness, and persistent use of emblems, their way of refusing and displacing speech, calls for an explanation. McNeill offers one in relation to the concept of 'word magic'. "Spoken words are special and carry with them the responsibility for being articulated. However, conveying the same meaning in gesture form avoids the articulatory act and, thanks to word magic, this lessened responsibility for speaking transfers to the speech act itself." (65) Doubtless, there is truth in the idea that gesturing rather than talking removes one from the net of justifications, arguments, questions, deceptions, and interpretive qualifications and recriminations that speech immediately introduces. But how many gestures admit of a translation into speech? What, absent any such reified abstraction does "having the 'same' meaning" mean? How convincingly can speech render an emblem? Giving the finger, for example, certainly carries a different charge from saying "up yours" or "fuck you" or the very different anatomical procedure invoked by "go screw/fuck yourself", and so on (the plurality of inequivalent verbalizations suggests emblems generate meanings by their very exclusion of speech). But, in any event, is the difference between gesturing and voicing the 'same' meanings one of lessened responsibility?

What of other deployments of emblems: their extensive, deeply embedded and seemingly indispensable use in ritual and religious practice, for example. Here something very different from lessened responsibility, almost the opposite seems to be operating; as if words are insufficiently responsible, not binding enough, too fleeting and precise at the same time, and only bodily action can fulfil the relevant devotional/liturgical purposes; as if gestures are able to create and stabilize belief, induce as well as express religious feelings, moods, and forms of consciousness more radically and with more appropriate affect than the specialized precision of speech. In this context, what Leroi-Gourhan says about speech's (and writing's) inferiority to art vis a vis religion, "that graphic expression restores to language the dimension of the inexpressible -- the possibility of multiplying the dimensions of a fact in instantly accessible visual symbols" (200), carries over from graphic symbols to visual gestures. The same point, but not tied to religious expressivity, threads through the word/image opposition insofar as this embraces the preference for visual emblems -- badges, logos, flags, insignia, etc -- over written words. Evidently emblems, as silent gesture acts able to say nothing through the displacement, substitution, suppression, or exclusion of speech, are a major corporeal semiotic whose relation to spoken language poses an essential question for any kind of gesturology.

Gesticulation and the uttering of words
A casual look at everyday conversation and story-telling shows verbal utterance to have two aspects: spoken words which are accompanied by gesticulation -- the fleeting, often barely discernible, idiosyncratic, and seemingly indefinite gestures of the fingers, hands, arms, shoulders, face that appear, in some way, to be connected or not unconnected to what is being said. These gesticulatory movements are far from consciously produced; they are involuntary, spontaneous, and mostly unnoticed. What is their function? Any role they play in spoken discourse must, it would seem, be limited, since their absence, though not without its effects, is not disabling in common practice: blind people understand speech, we converse on the telephone and listen to recorded messages, we hear speech on the radio, without apparently registering too much disturbance at the non presence of accompanying gestures. More than other kinds of gesture, gesticulation seems a superfluous addendum to utterance, an echo of some ancient pre-intellectual form of communication -- chimp discourse perhaps -- having little to do with the articulation of thought in words.

Empirical investigations of verbal narration within cognitive psychology over the past two decades suggest otherwise. If gesticulation is primitive, ancient or chimp-like, then so is human thought. Not only is gesticulation neither unconnected to thought nor a surface phenomenon, but on the contrary is a deep component of utterance having to do with the semantics, pragmatics and discursive aspects of speech. Specifically, gesticulation embraces at least four distinct kinds of gesture -- icons, metaphors, beats, deictics -- which appear designed to accomplish specific semiotic tasks at different levels of speech. Iconics illustrate the semantic content of speech (a twisting finger accompanying the words "spiral staircase"); metaphorics mark an abstraction introduced in speech (cupped hands -- the container metaphor for abstract entities in Western culture -- when narration jumps out of the story being told and refers abstractly to its 'genre'; beats, brief on/off gestures marking the word they accompany as significant "not for its own semantic content, but for its discourse-pragmatic content"(a hand flick when a new character or theme or a metalingual gloss is introduced into a story); deictics point to places or times spoken about (pointing down at the ground when asking an interlocutor "where did you come from?").

Plainly, gesticulations (notwithstanding their dispensability in various contexts) are linked to words at non-trivial levels of speech, contrary to any phonocentrism that would have them as marginal or epiphenomenal to speech. The possibility is nevertheless left open of a phonocentric interpretations of their function and genesis. One could, for example, attempt to explain them by arguing that gesticulation and speech share a covert verbal plan, or that gesticulation translates a prior sentence, or that gesticulatory movements are created to illustrate, amplify or gloss speech as the latter is produced. That such explanations (which give causal or conceptual priority to speech) are not feasible is a consequence of the tight temporal binding, accurate to fractions of a second discovered to operate between gesticulation and speech. Any gesture has a preparatory phase, a stroke phase in which the gesture proper occurs, and a withdrawal. In gesticulation, the preparation precedes the word it relates to whilst the gesture itself coincides exactly at the height of its stroke phase with the word in question, after which they disperse together; an anticipation, coincidence, and dispersal only possible if gesture and word are produced together from something preceding both of them; only if there were some earlier linkage between the two. For McNeill this consists of a dialectic of opposed modes of representation: gestural (imagistic, holistic and synthetic) and verbal (linear, segmental and analytic); the final utterance being the result of an interaction between a relatively unconstrained, individual gestural impulse and the socially constrained demands of a linguistic system. In short thinking, of the everyday kind that eventuates in speech, has its origins in pre-verbal visuo-kinetic images which then become gesticulated and verbalized to form an utterance. Shorter still, spoken thought starts life as a yet-to-be-realized gesture. One might note in this connection the work of Wallace Chafe whose analysis of verbal utterance finds it composed of 'idea units' which correspond to single 'thoughts'; the duration of each unit being about two seconds -- roughly the time for a complete gesture to take place. (11) More abstractly, support for McNeill's conclusion comes from cognitive science's contributions to the construction of the corporeal paradigm or era of the body mentioned earlier, according to which thought, not least abstract, rational thought such as mathematical and philosophical thinking, rests on metaphors derived from repeated and deeply layered patterns of body movement.(12)

Speech as gesture - a dual system
Speech is produced by sytematic and repeatable movements of the body -- the lips, tongue, cheeks, jaw, glottis, vocal chords, larynx, diaphragm -- suggesting that speech might be regarded as a species of gesture; auditory rather than visible gesture, but gestural nonetheless. Research in phonetics and artificial speech synthesis over the past two decades has indicated that it is precisely as a gestural system that the complex kinematics of speech are best comprehended. (In this connection, it is interesting that certain purely phonetic features of speech, such as co-articulation, also operate within a purely gestural context like fingerspelling.Specifically, task-dynamical models of the type developed originally for describing the skeleto-muscular organization of the body during walking have proved ideal for modeling the dynamics of the lips, tongue, etc during speech production.(13) Moreover, not only is the production of speech gestural, but so it turns out, less to be expected, is its perception: "Surprisingly", as evolutionary neurologist Terrence Deacon finds himself saying, "auditory processing of speech sounds does not appear to be based on extracting basic acoustic parameters of the signal, as a scientist might design a computer to do, before mapping them onto speech sounds. Speech analysis appears designed instead to predict which oral-vocal movements produced them and ignore the rest."(14) We listen, it seems, not to speech sounds as such but to what they signal about the movements of the body causing them; we focus on what happens between the sounds, to the dynamics of their preparatory phases, pauses, holds, accelerations, and completions; the very features of gestures we attend to when perceiving them. It's as if we listen to speech as symptoms of gestures, in much the same way as a physician listens to the sound of a patient's heart to detect its underlying patterns of movement.

It is certainly interesting that speech is a gestural system. More interesting, or more significant from the perspective of a nascent gesturology, is the division of speech into two distinct gestural systems: speech as discrete sequences of separate phonemic units, rapid gestures of the lips and tongue forming word-strings governed by syntax, and speech as much slower gestural waves made by the diaphragm and larynx patterned by the prosodic features of tone, emphasis, volume, rhythm, and pitch. The division into two systems and their unified co-presence in speech has deep anatomical and functional roots organized around a vertical axis of phylogenetic age and a horizontal axis of hemispheric separation. Roughly speaking, the face, tongue, lip, and jaw muscles responsible for the rapid production of individual phenemes and words are more directly controlled by the (evolutionary recent) cortex, as opposed to the lungs, diaphragm and larynx responsible for the slower and more extendxed prosodic dynamics influenced by the ancient limbic and midbrain structures. The two systems, deeply and multiply folded into each other. Their functions, as Deacon puts it, are "parallel and complementary", brought together in a manner "analogous to the way a sound wave can be superimposed on a much faster carrier wave in a radio transmission." (365) This joining of the phonemic and syntactical systems in the final production of speech necessitates, it seems, a horizontal division of labor within the brain. Like the background context of speech, the prosodic dynamics of words "compete" with the foreground processing of individual words "for recruitment of the same brain structures" (314). The brain avoids potential conflicts by dividing tasks hemispherically; handling the intra-sentential, speed-optimized tasks -- syntax -- on the left side and the inter-sentential, large time domain features -- prosody -- on the right.

In certain respects, prosody's relation to the phonemic echoes that of emblems and gesticulation to spoken language. Each type of gesture generates a mode of saying nothing, producing a linguistic silence inhabiting the speech they are associated with. Thus, discourse is comprised of spoken utterance that is heard plus emblems, contra-oral gestures, that say nothing through a performed alterneity/alterity to speech. Spoken utterance, in turn, splits into speech proper and the silent component of gesticulation, co-oral gestures of complicity or collusion, that say nothing by agreeing or never disagreeing. Whilst speech proper, as we see, splits into the purely phonemic-syntactical stream of words and prosodic gestures that say nothing by being a condition for the speakable. Gestures, one can say, manufacture non-speech, pre-speech, un-speech, counter-speech; holes inside and outside speech that are sites of emergence; from them come the antics, ecstasies and travails of the body without speech.

Of course, prosodic gestures though they say nothing are anything but silent; they pre-interpret speech by being part of the sound which constitutes it, determining thereby how it is to be taken, the spin or angle its words carry, the degree of belief, seriousness, irony, kind of intention, agenda or motivation they have, and so on. Descriptive linguistics parses the division between the syntactic-phonemic and the prosodic as the difference between what is said and how what is said is to be taken (15); a difference that, in terms of the contribution of prosody to speech's social, affective and interactional meanings, is crucial: in the absence of tone, for example, the differences between speech being gentle, withering, questioning, threatening, seductive, flattering, menacing, pleading, sardonic, gleeful, sad, encouraging, and countless other affects mediated by connections to our limbic brains, disappear. A disappearance that as we shall see is central to the metaphysical work accomplished by alphabetic writing.

Writing speech
Though alphabetic writing is not confined to the writing down of speech, it is in this that writing's horizon, its nemesis even, is to be found, since it is the body, through its gestures, that haunts writing. Confronted by the semiotic effects of the body -- the prosodic gestures lodged intimately inside speech, the gesticulatory ones tightly accompanying it, the emblem gestures that displace speech whose functions range from explicit signifying to the creation of implicit silences -- writing offers notations, inscriptions, traces, an organized system of marks constituting an algorithm; a code which the reader of these marks must decode in order to retrieve or reconstruct the original body effects.

Unlike writing in general, alphabetic writing is never free of speech, but is always within the domain of the readable, the sayable, the pronouncable. This is certainly not the case with non-alphabetic forms such as, for example, mathematical writing which, though it can be 'read' aloud if desired by the reading of names attached to its signs, is semiotically different and other than written speech. It was precisely the need for readability that punctuation -- everything from periods, commas, question marks to parentheses, dashes, ellipses, colons, quote marks, and other orthographic items -- was invented to satisfy. But all these items, though they make the more rudimentary speech meanings available to the reader, don't touch the dimensions of pitch, stress, volume, dynamics, and tone of voice through which the bulk of prosodic effects occur, and for which there are (outside musical notation) no accepted devices. This should be qualified: it is not true to say there are no devices for writing to convey prosodic effects. In the interests of brevity and directness, I am talking about writing insofar as it is 'printed', 'naked', and 'pure': printed, and hence omitting the ways handwriting can make certain prosodically mediated meanings and effects available; naked, in that the text has not been priorly processed and annotated up by one of the various mark-up languages or style sheets now being developed precisely in order to facilitate the reading of texts aloud by machines (though their ability to code prosodic effects is at present very limited); pure, in that the text is printed in a 'neutral' font, thus omitting the effects, most of which are prosodic, achievable through the use of different type-faces; a resource powerfully used in advertising and comic art. But these restrictions and exceptions do not disable the general point here, namely, that alphabetic writing deals with prosody -- insofar as it does -- through the addition of words, through description that leads to textual augmentation. Faced with prosodic effects, writing is obliged to deal with the information supplied by prosody by introducing more words -- descriptions, parses, glosses, amplifying locutions -- that function like stage directions.Thus, in addition to alphabetically notating (necessarily de-prosodized) words, writing has to supply a further coding of the missing gestures -- the manner of the words' saying -- by the addition of words about these words detailing how they are to be taken. Each time this happens, each time prosody is projected onto syntax, the text becomes longer, more wordy, more open to further augmenation, than the speech it is coding, since what was before a co-occurrence, a simultaneity of words-with-prosody, becomes sequential and linearized.(16) And so it goes on, text without end; always the edge of writing is occupied by an incomplete coding of prosody and always the recovery of some full -- prosodically complete -- meaning, affect, significance or purpose is deferred; a potential infinity of deferrence. In theory. In practice, one stops at some point of sufficient prosodic recuperation long before death by excessive length and unreadability intervene. Derrida's message of endless deferral of meaning (which, by his practice, is confined to alphabetic writing and which, by his insistence, is necessarily indifferent philosophically to any such extra-textual intervention or external source of closure) is thus an effect of alphabetic writing, built into the alphabet's asymptotic relation to the capture of prosody, an inevitable consequence of its inability to code the gestures inside de-gestured words except by the addition of more (de-gestured) words.

This failure, or rather the disjunction of words from their prosody that prompts it, which allowed a separation of 'speech' (i.e. speech virtualized as alphabetic inscription) from its embodied production, was of course writing's spectacular achievement. It allowed the alphabet to cut the word loose from the place, time, context, circumstances, voice, gestures, presence, and mortality of the one who utters it. But this separation has a neurological correlate, namely, that such virtualized speech issues from -- gives rise to, allows to come into being, gives autonomy to -- a dis-embodied cortex; for it is precisely the cortex's connection to the midbrain and so-called limbic regions that produces the words-plus-prosody amalgam; a loop which the autonomous presence of the de-prosodized alphabetically written word cuts. More sharply, one can locate the connection from the limbic to the cortical brain as routed through the prefrontal lobes and understand its severance as an orthographic equivalent of a pre-frontal lobotomy: certainly, descriptions of the speech of recipients of this procedure -- "In their words or attitudes, no traces of affection could be detected." -- resonate in an uncanny way with how we perceive words emptied of prosody. (17) Taken in the reverse direction, the correspondence allows this writing -- in its relation to speech -- to enact a transcendental escape from the body: the de-prosodized word issuing from a dis-embodied cortex forming one side of an equation the other side of which is a marginalized/occluded body whose horizon, its 'highest' achievement, is that of mere limbic gesturing. (18) Alphabetic writing thus contributes to the ontology of an abstract, invisible God, not by inventing it as an object of thought, but by operating as a vehicle or machine for the dissemination of an invisible, transcendental difference without which no such God can be understood to exist. The alphabet in other words furnishes the presence of a divine absence, not by representing or alluding to it as subject matter (though it does that endlessly) or invoking it within itself (though it does that too: "... The word was with God, the word was God"), but through direct performance: constantly transcending the midbrain which, in the terms of the metaphor here, functions as nothing less than the neo-cortex's body.(19)

Writing gestures
Alphabetic writing's relation to prosody is symptomatic of a general problem inherent in notating a performed action -- spoken word, musical phrase, dance step, body movement, gesture -- with a predetermined signifier of some system, since the system projects its own shape onto its subject-matter. In other words, the action notated never escapes from a mirroring of the separation into discrete, quasi-algebraic signifiers or 'notes' notating it. Certainly, within the algebra of alphabetic inscription, the effects of prosody -- geometric, topological, continuous, spread over a phrase or clause -- cannot, as we've seen, be represented as characteristics of separate words. The attempt to recoup these effects producing, instead, the familiar proliferation of ever-different interpretations, readings, and deferred meanings set in train by a recuperation with no pre-assignable boundary. Outside the notation of speech, the effects are more various. There is a plethora of different systems that have been invented (each with its own adherents, successful transcriptions and limitations) for notating dance -- a multiplicity that in itself suggests certain intrinsic difficulties at work -- but no single one works for all dances in the way that the alphabet does for all spoken languages. Likewise, efforts to notate gesture run up against limits to the feasability of taxonomizing and writing down the body's movements that seem to reflect an intrinsic limitation of 'writing', of the medium of paper itself, rather than any failure of method or imagination; and explanations for the absence of an accepted writing system for ASL do not challenge this.(20)

But writing's success was to sever (the words of) speech from its production, to create a form of virtual speech. And speech, as we've seen, is made from gestures; if writing could virtualize the products of these gestures, might not something similar be possible on the visual/kinetic products of gesture in general? What if the whole operation could in some way be repeated; if three millennia after spoken words were virtualized, it became possible to do something akin for other signifying productions of the body? What if gestures could be brought into consciousness, made into discrete objects of awareness, examined; made as identifiable, repeatable, portable, studiable, as free-standing, iterable and quotable as words? Would it possible, is my question, for there to be a medium that did for gestures what writing marks on paper did for the words of speech?

Notating speech rescues it from oblivion, captures it in a form that allows the original utterance to be (partially) reconstructed. Contemporary digital technology offers a technique -- appropriately named motion capture -- that promises to do precisely this for gesture and indeed any kind of physical movement, meaningful of otherwise. Before describing it, it as well to place it in the context of other and older meanings evoked by the idea of capturing gesture and movement; meanings corresponding to different interpretations of the term 'to capture', namely to mimic, to transduce, to record, to incorporate, and to sample.

Mimicry is the most primitive, least mediated form of gesture capture; as old as acting and dancing, mime uses the performer's body itself as the instrument for capturing the movement of another body. Transduction corresponds to the instrumental mode of gesture identified earlier, in which a human movement is captured within a machinic circuit where it is converted into other movements or regimes of action; winding up a clockwork mechanism or turning the handle of a pulley wheel are traditional examples. Recording movement is what film and video are normally credited with achieving; they capture movement (in relation to photography's freezing of it) not by way of a direct connection to its kinesis or production as a physical process, but as an appearence, in the highly mediated form of a two-dimensional visual representation. Incorporating movement occurs in calligraphy or various kinds of art production, for example, playing a violin or guitar, the hand gestures responsible for a line drawing, or the brushstokes that constitute a painting, or their digital versions. In such cases, what is captured enters directly and significantly into the created object. James Elkins talks of oil painting as making a cast of a painter's movements, of functioning to "preserve the memory of the tired bodies that made them, the quick jabs, the exhausted truces, the careful nourishing gestures." (x) What Painting Is, p.5. Finally, sampling movement is what takes place within the digital technology of motion capture. One attaches sensors (these can be responsive to visual, magnetic, aural, or inertial tracking technologies) to chosen points on the body (of an animal, machine, human) and takes periodic readings, i.e. digitized samples, of where in space these sensors are as the body moves. The resulting data-set contains the information needed to reproduce the original motion of the chosen aspects of the body in a potentially unlimited series of contexts. Unlike a film or video, the readings are raw data not a representation, they effect a de-territorialization of the original motion, cutting it free from the place, time, context, circumstances, physical form, and presence of its performance; in which form it can be re-territorialized able, for example, to drive an animation -- become the motion of an automaton, puppet, robot, cartoon figure, a virtual reality avatar. Captured gestures are already being used in art objects, computer games, virtual choreography, animated films, chat rooms and virtual places, different kinds of electronic installations, and various attempts at virtual theatre. Like any digitized object, captured gestures can be stored, instantly replicated, posted on the internet, and multiply processed. Capture technology, then, offers gestures the same kinds of mobility, dislocation and freedom from the contexts of their production as the notational system of alphabetic writing allowed speech.

Observe, from what has been said, that the question here is no longer one of marks on paper, of notation understood as inscription readable by the unaided eye. Digital technology has already changed the terms here, building into 'notation' the digital means by which what is notated can be read and produced; notation, in other words, embraces simulation. In light of this, one can juxtapose motion capture and notation. The alphabet notates sounds on paper, digital technology captures movement as stored data; notation is algorithmic (the letter sequence a-l-p-h-a-b-e-t is an instruction to a reader for reproducing a certain sound) and is limited by the pre-set identifications built into its symbols; capture is documentary and is limited by sensor resolution and sampling rate. Notation is metaphoric and requires interpretation, capture is metonymic and delivers fidelity. In capture the life of a movement, in our case the significant dynamic of a gesture, is sealed on the inside at creation; in notation the life is applied, with all the ambiguity, creative augmentation and loss this entails, to the skeleton by the interpreter.

The practical effects and theoretical consequences of these differences are yet to be worked out. In the meantime, one is left with the question: Could motion capture be about to effect a transformation as radical and far-reaching for the body's gestural semiotics as writing accomplished for speech? Could bringing (a newly digitized and objectified) gesture out from under the shadow of the spoken word install a new order of body signification? Without claiming (which would be absurd) that gesture could rival speech, it is undeniable that it is on the ascendancy, that in some sense (which I've tried to outline), certain kinds of silence and the saying of nothing, achievements of a newly valorized but once marginalized and despised body, are poised to come into prominence, have already arrived.

Or better, are in a constant state of arriving, since the saying of nothing, becoming mute, is a never-ending business of creating a wordless interior to speech. To achieve the body without organs of speech, it is necessary first to dumb the body, de-organize it, divest it of speech, silence it, so that, no longer governed by the sayable, it may become the field of other productions, other desires, be alive to other semiotics -- here the gestural -- that speech together (with its transcendentally troped alphabetically written form) is only too pleased to elide. Becoming mute is becoming infant, part of a willed accession to the state of human pre-speech, a return to, or renogotiation of, the past, except that what is involved in such a move is not a literal 'return' to the past or a regression in the sense of a move back along a linear ascent, but a reconfiguration of the present/future by altering its genesis, its supposedly necessary relation to that past. The result would be an alteration in the condition for the posibility of being human, a quasi- or neo-primitiveness in which humans partook of the characteristics of (presentday) children; a cultural neoteny whereby the adults of species come to resemble the young of their evolutionary forebears.

To say more about saying nothing, is to leap out of the semiotic mode governing the present account and consider gesture in its a-signifying instrumental and experiential modes. Particularly, the experiential, the mode of immersion, participation. Nowhere, within contemporary culture, is the experiential gesture more vital, uncompromising and powerful than in the planet-wide phenomenon of techno music.

Techno is an algorithm designed for generating gesture in the mode of experience: to dance to techno, to be immersed in its beat, is to both link and dissolve the relation of the individual self to the socially present other, to seek the plural, the simultaneous and the co-occurrent which drives the collective and its numerous realizations -- crowd, herd, pack, group, swarm, audience, mass, chorus, mob, throng -- articulated with such obsessive comprehension and brilliance by Elias Canetti.(21) Techno combines the a-signifying groove of music with dance, oceanic immersion and ecstatic participation with a silence in the face of words that is deafening, effacing logos, nullifying all attempts at reason, ideation, speech or language. Not to be understood as music for listening, techno is digital software for moving, dancing, gesturing, experiencing, for getting the body inside and outside the groove of the digital machine. The rave is the contemporary site of Dionysus. But that's another story.

" Considered jointly with speech, gestures open a 'window' onto the mind. ... taking gesture into account, we see patterns not revealed by speech alone and see more comprehensively how meanings are constructed. Gesture is not only a display of meaning but is part of the act of constructing meaning itself, adding a 'material carrier' that helps bring meaning into existence ... ." David McNeill

1. See chapter 2 of The Time of the Tribes: the Decline of Individualism in Mass Society (translated Don Smith). London: Sage Publications, 1996; and The Shadow of Dionysus: a Contribution to the Sociology of the Orgy (translated Cindy Linse and Mary Palmquist). Albany: State University of New York Press, 1993.

2. Roland Barthes Image-Music-Text (translated Stephen Heath). London: Fontana/Collins, 1977: p. 149

3. The claim that syntax of spoken languages derives -- in evolutionary, developmental and ultimately conceptual terms -- from the internal organization of gestures is the principal thesis of Gesture and the Nature of Language by David Armstrong, William Stokoe, Sherman Wilcox (Cambridge: Cambridge University Press, 1995)

4. Pathomyotomia, or a Dissection of the Significative Muscles of the Affections of the Minde (London,1649)

5. Nicholas Humphrey, A History of the Mind (New York: Simon and Shuster, 1992) and V.S. Ramachandran and Sandra Blakeslee Phantoms in the Brain (New York: Wm. Morrow and Co., 1998)

6. Grooming, Gossip, and the Evolution of Language (Cambridge, Mass.: Harvard University Press, 1996)

7. Of course, this is hardly a new proposition and has been asserted more than once by linguists and others this century. For a brief commentary on this, see Ernst von Glaserfeld ÒLingistic Communication: Theory and DefinitionÓ in Duane M. Rumbaugh (editor). Language Learning by a Chimpanzee . New York: Academic Press 1977

8. Quoted in Harlan Lane What the Mind Hears: A History of the Deaf New York: Random House, 1984), p. 391

9. Memoir Upon the Formation of a Deaf Variety of the Human Race (New Haven: National Academy of Science, 1883)

10. Adam Kendon Sign Languages of Aboriginal Australia: Cultural, Semiotic and Communicative Perspectives (Cambridge: Cambridge University Press, 1988) and many earlier papers cited there; David McNeill Hand and Mind: What Gestures Reveal about Thought (Chicago: Chicago University Press, 1992)

11. .Literacy, Language, and Learning, Cambridge: Cambridge University Press, 1985

12. For a recent work on this last, see George LakoffÕs and Mark JohnsonÕs Philosophy in the Flesh: the Embodied Mind and its Challenge to Western Thought (New York: Basic Books, 1998). For a list of earlier works, see note 4 of my ÒExuberant Materiality: De-Minding the StoreÓ, Configurations 2, 257-274, 1994

13. See the various works cited on pp. 8 - 11 of Gesture and the Nature of Language

14. The Symbolic Species: the Co-evolution of Language and the Brain (New York: Norton, 1997), 359.

15. I take this formulation from David Olson's The World on Paper (Cambridge: Cambridge University Press, 1994) where it is explicated in terms of the theory of speech acts put forward by J. L. Austin, according to which, what is communicated by prosody is the "illocutionary force" of an utterance. Much of this valuable, but I do not use that description here since I don't think the term 'communication' (understood as a process external to what is communicated ) underlying the theory of speech acts is ultimately a helpful way to think about the relation between speech and gesture

16. See Ad Infinitum for more on this in relation to the potential infinity built into the definition of a Turing machine as its specification of an infinite tape. Without this, Turing machines would not be universal; not able to reproduce the input/output workings of any purely mechanical device -- since, like the doomed attempt of alphabetic writing to incorporate prosodic effects, the machine needs unlimited (i.e. unbounded) sequential space to incorporate effects not given originally in sequential form.

17. Julio Rocha do Amaral and Jorge Martins de Oliveira Limbic System: The Center of Emotions.

18. Mathematical ideograms are a form of writing that constitute the most extreme development of cortical autonomy. The fact that mathematical languages exclude gesture, are devoid of any reference to an I, or to an other or to a listener, addressee, or signee, and are at the same time the ur-site of an abstract, transcendental ontology -- platonism -- should not, therefore, be surprising; at least not from a semiotic point of view.

19. For a different neurological take on the deleterious effects of alphabetic writing within Western culture in terms of its left-brained exclusion/repression of the right-brained, feminine-biased visual image, see Leonard Shlain The Alphabet v The Goddess (London: Allen Lane, 1999).

20. This is the stance adopted by Jerome Schein and David Stuart Language in Motion. Washington: Gallaudet University Press, 1995 who observe, for example, that ÒUnfortunately, the limits of print frustrate attempts to portray ASLÕs spatial-sequential beauty.Ó (42)

21. Crowds and Power (New York: Farrar, Straus and Giroux, 1984)