American Sign Language (ASL) is the principal means of communication of the deaf and hearing-impaired in the United States. Unlike signed English, which uses signs for English words and relies on English constructions, ASL is an independent language with its own lexicon and grammar. In general, each sign of ASL has four independent variables: handshape, location, movement, and orientation. However, among the signs of ASL is a manual alphabet consisting of single, one-handed shapes that do not vary with respect to location, orientation or movement. These manual letters occur in two different capacities: as constituents of a large number of ASL signs (for example, the handshape of the ASL signs for 'meet' and 'trespass' consists of the letter G), and as the means of fingerspelling. Fingerspelling is an essential resource of ASL, used to provide verbatim representation of English words, acronyms, phrases, and sentences or to communicate proper names of people, places, and events or technical terms or slang expressions or any item of English that lacks an established sign within ASL. Situated at the point where ASL and English come together, fingerspelling is a route for new signs to enter ASL: fingerspelled English words becoming new ASL signs in their own right. These features make fingerspelling a core element in the curriculum of all who learn ASL, whether deaf children whose first language is ASL or the many others who seek to acquire the language.

RESEARCH Linguistic study of ASL and other signed languages has shown them sharing many of the basic structural and dynamical characteristics of spoken languages. Thus, like words, individual signs are organized sequentially into segments and within the production of a sign a segment can interact with those that come before or after it; a phenomenon termed co-articulation by linguists. Even though the manual alphabet consists only of handshapes, evidence from video recordings and datagloves show that both these forms occur in fingerspelling. ASL also exhibits other dynamic characteristics of speech, such as acceleration and deceleration, damping, and degree of hold; these too are found in fingerspelling. The challenge facing the creation of an interactive fingerspelling program stems from the need to adequately reflect these structural and dynamical effects. Currently, there are various initiatives interested in creating fingerspelling programs and/or sign dictionaries, but it is not clear at this stage how and to what extent they will deal with these effects. The kind of dynamical problems involved are, however, familiar to computer graphics artists and animators: making figures realistically simulate the naturally occurring dynamics and transition movements of limbs and bodies is the most basic task facing any animator, and animators have assembled techniques for dealing with them. There are, to my knowledge, no fingerspelling initiatives from this direction. The project presented here proposes using the 3-dimensional modeling, figure animation and computer graphics resources at ACCAD as the basis for its approach; by combining animation techniques with knowledge of ASL and fingerspelling phonetics, it constitutes an original, interdisciplinary research initiative. Fingerspelling is only a small part of ASL. A considerably more ambitious undertaking would be a full-scale ASL simulation. The results of the present project could serve as a first phase of such an undertaking.

EDUCATIONAL VALUE A more immediate benefit to be derived from a fingerspelling program is in education. As already indicated, contrary to the natural and widespread belief, much more is involved in fingerspelling a word than making the appropriate manual letters in the correct order. As teachers of ASL never tire of pointing out, fingerspelling is a complex and demanding skill requiring many hundreds of hours practice to achieve a satisfactory level of attainment. Interestingly, the difficulties for adult learners whose first language is English and deaf children whose first language is ASL are inverses: the former need to fuse the letters into a single configuration, the latter to understand that the 'signs' presented to them are made of individual English letters. It is the former, however, for whom the problems are more severe. As linguist Sherman Wilcox observes "Although very little research exists on second language acquisition of fingerspelling, experience makes it clear that the task is extremely difficult. Students frequently express the opinion that learning to produce and comprehend fingerspelling is one of the most difficult tasks they face when learning a signed language." It is envisaged that the two principal features of the program projected here -- interactivity and virtual representation -- would together enable it to make a significant impact on the acquisition of fingerspelling within both categories of learners.
The educational advantages of interactivity in learning devices are widely acknowledged... the present project's design decision to replace video representation by virtual hand simulation offers a 3D viewpoint under the active control of the user, as well as obliging it to explicitly confront and solve the perceptual problems resulting from co-articulation.

SOCIAL VALUE Alongside educational benefits, the introduction of a computer program into the teaching of fingerspelling offers a valuable, less obvious social gain, related to the enculturation of deaf children within the larger, hearing community surrounding them. As is well known, the deaf suffer from various degrees of isolation and separation from mainstream culture. Partly this is the inevitable consequence of living inside a world long organized for the hearing, partly the result of the linguistic separation caused by ASL and not English being their first language (and the problems of literacy in English that accompany this), partly the residual suspicion of the deaf to mainstream culture which sought for many years to suppress signing in order to normalize them into the speaking community. Mainstream culture is now on many levels irrevocably scientific and technological, and this suspicion is no doubt a factor in a certain backwardness, even hostility, within parts of the deaf and ASL educational community towards science and technology. In addition, English is now recognized as the international language of technology, so that fingerspelling, as the place where English and ASL come together, can be seen as a cultural bridge between the deaf and hearing communities. In this context, promoting an attractive, user-friendly way of learning fingerspelling via a computer represents an interesting site at which to simultaneously improve literacy among deaf children and defuse antagonism to technology. Technology becomes the means not only of furthering its own values but those of the larger English speaking world within which it flourishes. Success would mean a generation of deaf children who, by being more computer literate in an area vital to them, will also become more literate in their second language of English.

Part of grant proposal submitted to the Battelle Endowment at the Ohio State University. The application was successful, and the project is now in its first phase. The current team consists of Brian Rotman [Principal Investigator], Matt Lewis [Technical Director],
and research assistants
Peter Gerstmann [VRML coding], Brandon Morse [Animation].

Project Description

AIM To construct a computer program for fingerspelling in American Sign Language (ASL) which would benefit:
educational practice -- giving ASL learners an interactive software tool designed to make the acquisition of fingerspelling easier and more attractive;
research -- contributing to the investigation of dynamical effects occurring within and between signs;
socialization -- facilitating the entry of deaf students into mainstream culture by simultaneously enhancing their literacy in English and appreciation of technology.
The program would show a virtual hand able to fingerspell any English word. The interface would allow the user to control the position in space from which the image was viewed, as well as its appearance (size, lighting, rendering) and dynamic presentation (speed, acceleration, degree of hold). The construction of the program would use the existing animation and 3-D modeling resources of Advanced Computer Center for Arts and Design (ACCAD), the home department of the co-ordinators.

There is much interest in gesture right now, especially, as in the project here, in relation to digital technology. One place to start is Waleed's useful compilation Machine Gesture and Sign Language Recognition

Another compilation in the form of an annotated gesture webography is given here by Ganesh Ramachandran