SIGN LANGUAGE

Presented by
Harash Pal
01211805309
Under the Guidance of
Dr. P. R. Gupta
 Facts about Sign Language
• Sign languages are NOT the same all over the world.
• Sign languages are NOT just gestures and pantomime, but do have
their own grammar.
• Sign languages are NOT dependent on spoken languages and do not
resemble spoken languages used in the same region.
• Sign languages are NOT “languages of the hands” only, but use non-
manual expressions as well.
• Sign languages have NOT been invented by hearing people.
• Western sign languages (e.g. American Sign Language) are NOT
better than Indian Sign Language.
• Signed codes for spoken languages (e.g. Signed English) are NOT
better than Indian Sign Language.
 Some Misconceptions
• All Deaf Users are Written-English Literate
• We Can Generate SL Text as Output
• SL is Just Manually Performed English where English and SL have
the same linguistic structure – that one language is merely a direct
encoding of the other.
• SL Can Easily Use Written-Language MT Technology
• It’s OK to Ignore Classifier Predicates
• Many software designers incorrectly assume that written English
text in a user-interface is always accessible to deaf users
• Many designers believe that if audio information is also presented
as written English text, then the needs of a deaf user are met
 Research Areas of Sign Language
• Sign Language Recognition
• Statistical Machine Translation
• English Text-to-Sign Language Animation(3D)
• Speech-to-Sign Language Animation(3D Avtar)
• Sign Language HCI & Computer Applications
• Sign Language into Speech
Sign language vs. English Language
• Sign Language has a grammar that is independent of and quite
distinct from English
• SL allows much freer word order compared to English
• English marks tense morphologically on verbs, whereas SL (like
many languages e.g. ASL) expresses tense lexically via temporal
adverbs
• ASL and English also differ in structure at the level of phonology
• Signed languages, like spoken languages, exhibit a level of
sublexical structure that involves segments and combinatorial rules,
but phonological features are manual rather than oral
• English and ASL differ quite dramatically with respect to how
spatial information is encoded
Sign language vs. English Language
• English, like many spoken languages, expresses locative
information with prepositions, such as “in,” “on,” or “under.”
• In contrast, SL encodes locative and motion information with
verbal classifier constructions where handshape morphemes
specify object type, and the position of the hands in signing
space schematically represents the spatial relation between
two objects
• Thus, English and ASL are quite distinct from each other
within phonological, morphological, and syntactic domains
Sign language vs. English Language
• A natural language with a linguistic structure distinct from
English
• Many deaf adults with English reading difficulty are fluent in
Sign Language
• A majority of deaf 18-year-olds in the United States have an
English reading level below that of a typical 10-year-old
student
• Studies have shown that the majority of deaf high school
graduates in the United States have only a fourth grade
English reading level (Holt, 1991).
 Example #1
For example, after saying “pipe,” a Signer
produced an ASL classifier construction
indicating a vertically oriented thin cylinder
without any accompanying speech. In this ASL
construction, two “F” handshapes (thumb and
index finger form a circle) overlapped with
contact, and the right hand moved upward. This
ASL expression does not have an exact English
translation and describes the spatial orientation
and general size of the pipe
 Example #2
English sentence and corresponding glosses:

(1) “So Sylvester who’s on the ledge [jumps into] the

apartment.” JUMP
(2) P7: “I [don’t] [think] he would [really] [live].”
NOT THINK REALLY LIVE

By convention, words in upper case represent English glosses

(the nearest equivalent translation) for ASL signs.
 Example #3
• “He goes over and sees the gutter going up the side of the
building. [Happen] [what]?
HAPPEN WHAT
• Right [next to] [the window]!”
NEXT-TO WINDOW
• In example (4), the phrase “Happen what?” is a word-for-
sign translation of an ASL conjunction phrase that could be
translated in this context as “And guess what?”. This
rhetorical question is followed by the answer “[it’s] right
next to the window!” . In the answer, the English expletive
“it” and copula “is” are deleted, which could also be
considered an example of ASL-influenced English.
 Example #4
“[An] [old] [woman] [seem] [her] [bird] [she] [protect].”
A-N OLD WOMAN SEEM POSS BIRD. PRO PROTECT
• Note:
– A-N = fingerspelled article;
– PRO = pronoun, a point to a location in signing space;
– POSS = possessive pronoun, a B handshape (fingers
extended and together) directed toward the same location
in space
• This example is another word-for-sign translation of
grammatical ASL. The ASL phrases could be translated as
“There’s this old woman, and it seems it’s her bird. She
protects it.”
 Elements of Sign Language
• Single handed lexical
• Double handed sign
• Facial Expression
• Head Tilt
• Signing Space
• Rotation
• Orientation
• Pace(rate of motion or change) and Timing
Common Sign Language Terms
Some signing communication systems
• Sign Languages, e.g.:
– American Sign Language(ASL)
– Indian Sign Language(ISL)
• Signed English(SE)
• Fingerspelling
 ASL: Widely Used Sign Language
• The primary means of communication for an estimated one-
half million to two million deaf people in the United States.
• American Sign Language (ASL), a full natural language with a
linguistic structure distinct from English (Lane et al., 1996,
Neidle et al., 2000, Liddell, 2003a; Mitchell, 2004).
 Signed English (SE)
• Signed English (SE) is another form of manual signing
communication that is distinct from ASL.
• It is not a full natural language.
• There are several different styles of SE communication, but all
of them encode an English sentence into a set of signs
performed by the signer’s hands.
• SE uses many of the same signs as ASL (and some additional
signs of its own).
• SE retains English sentence structure and word order, and it is
most commonly used in educational settings for deaf students
who are learning English.
 Fingerspelling
• Fingerspelling is a method of communicating the letters of the
English alphabet using special handshapes to spell words
during signing.
• It is usually reserved for titles, proper names, and other
specific situations.
• In an English-to-ASL interpretation/translation context,
fingerspelling can be more common
• An interpreter will often use fingerspelling to directly mimic
terminology or specialized vocabulary being used in the
original English language signal.
 Gloss
• Since there is no written form for ASL signs, transcriptions
frequently associate each ASL sign with some approximate
English translation, called a “gloss.”
• Since glosses are used as class labels, two signs are assigned
the same gloss if and only if they correspond to the same ASL
lexical item.
 Visual Space in Sign Language
• Signing Space
– This is the area of space in front of the signer’s torso where the
majority of classifier predicates and other SL signs are performed.
During an ASL conversation, signers can associate locations in
space around their body with people, objects, or concepts under
discussion.
• Depicting Space
– Sometimes the way in which objects are arranged in space is
meant to topologically indicate the 3D layout of those objects in
some scene under discussion. When the signing space is used in
this manner, it is referred to as Depicting Space (Liddell, 2003a).
• Token Space
– Sometimes the locations associated with objects in space is not
meant to indicate how the objects are arranged in a 3D scene. In
this case, we say that the space around the signer is being used as
a Token Space
 The Lexical Signing (LS)
• A signer constructs most SL sentences by syntactically combining
individual SL lexical items into complete sentences and add
grammatically meaningful facial expressions and other non-
manual signals (NMS) to the performance. The exact performance
of each lexical item (each sign) may also need to be modified to
accommodate syntactic, morphological, or phonological
requirements of particular SL.
• This form of signing, since it is fundamentally the generation of
sentences based on individual lexical signs, will be called Lexical
Signing (LS)
• During LS signing, the Signing Space is primarily used as a Token
Space. This means that the locations chosen for entities during LS
signing are not topologically meaningful; that is, one entity being
positioned to the left of another in the signing space does not
indicate the entity is to the left of the other in the real world
 Classifier Predicate (CP)
• CP, a phenomenon in which signers use special hand
movements to indicate the location and movement of
invisible objects (representing entities under discussion) in
space around their bodies.
• CPs are frequent in ASL and are necessary for conveying many
concepts.
• During a CP, the signers’ hands represent an entity in space in
front of them, and they position, move, trace, or re-orient this
imaginary object to indicate the location, movement, shape,
or other properties of some corresponding real world entity
under discussion.
(CP) Continued……..
• A CP consists of two simultaneous components:
– (1) the hand in a semantically meaningful hand shape and
– (2) a 3D movement path that the hand travels through
space in front of the signer.
• To produce a complete ASL sentence, a CP will typically follow
a noun phrase that indicates which object in the conversation
is being described by the CP.
 Creating CPs
• A classifier predicate is created by first selecting one of a closed
set of handshapes based on the characteristics of the entity in the
noun phrase (whether it be a vehicle, upright animate figure,
squat four-legged object, etc.) and what aspect of the entity the
signer wishes to discuss (its surface, size, position, motion, etc).
• The signer then produces a three-dimensional movement for the
hand which communicates a contour, a position in the space
around the signer, a motion through 3D-space, a
physical/abstract dimension, and/or some other property of the
object which needs to be communicated.
• Classifier predicates are therefore ideal for describing scenes,
articulation of tools, movements, sizes, and other information of
a visual/spatial or scene/process nature.
 Example: CPs
For example, to express “the car parked between the cat and the house,”
the signer could use a combination of three classifier predicates
•The non-dominant hand in a “Spread C” handshape would indicate a
location in space where a miniature invisible house could be envisioned.
The signer’s eye gaze would aim at the location assigned to the ‘house.’
•The dominant hand in a “Hooked V” handshape would indicate a location
in space where a miniature invisible cat could be envisioned. The signer’s
eye gaze would aim at the location assigned to the ‘cat.’
•The dominant hand in a “Number 3” handshape would trace a path in
space corresponding to a car driving and stopping in between the ‘house’
and ‘cat’ locations in space. As the car nears the end of its motion path, the
open palm ofthe non-dominant hand (in a “Flat B” handshape) would form
a platform on which the car parks. The signer’s eye gaze would follow the
motion of the car.
 Continued……

•Before each of these classifier predicates, the signer would perform

an ASL noun phrase which indicates which object is being referred
to: HOUSE, CAT, or CAR. During this noun phrase, the signer’s eye
gaze would aim at the audience.
 ASL Representation: Timing and Coordination

• In addition to under-specifying the behavior of some of the

articulators of the signing performance, the ASL representations
used by some previous MT systems have oversynchronized
• the timing relationships in the ASL performance. Fundamentally, all
four
• of these systems have treated the ASL performance as a string of
individual glosses
• (where a “gloss” is an English word that represents a particular ASL
sign). Within this
• linear one-dimensional representation, these systems have tried to
encode some of the
• other parallel/overlapping elements of the ASL performance.
Multi-Path SL MT Architecture
 Limitations while Translation
• Lack of SL writing system
• SL’s visual modality introduces unique
complexities into the translation problem. For
example, the space around the signer can be
used for many communicative purposes
 Indian Sign Language
India now has its own national association for sign
language interpreters, the
Association of Sign Language Interpreters (ASLI).
This organization was formed in 2007 by Arun C. Rao
and others. Interpreting in India is still growing,
although at this time it has not been recognized as a
profession by India's government.
Unlike other sign languages the Indian sign language
starts with emphasize to grammar rather than
vocabulary first.