Professional Documents
Culture Documents
A Paper Arabic Text To Arabic Sign Almasoud AlKhalifa
A Paper Arabic Text To Arabic Sign Almasoud AlKhalifa
BACKGROUND
INTRODUCTION
There are 17 million deaf people in the Arab world and 88,000
1. Sign language
deaf people in Saudi Arabia alone [1]. Arabic Sign Language
In the last century, Sign Language (SL) has gained increased
(ArSL) is the native language for many Arab deaf people. Also,
attention and universal recognition by many scientists in the field
of language and computer sciences. This is because SL is
Permission to make digital or hard copies of all or part of this work for considered the Native language of the deaf community and they
personal or classroom use is granted without fee provided that copies are can express their needs and the formation of concepts through it.
ArSL is considered a secondary language for their hearing
not made or distributed for profit or commercial advantage and that copies As spoken languages use throat, nose and mouth as articulators,
parents,
bear the and
this notice hearing
the fullchildren
citation onofthedeaf
first adults
page. Toascopy
well as to the
otherwise, or
also SL uses fingers, hands, arms and facial expressions. These
hearingtodeaf
republish, posteducators.
on servers or to redistribute to lists, requires prior specific
permission and/or a fee. articulators can be classified as phonemes articulators similar to
Second Kuwait Conf. on E-Services and E-Systems, April 5-7, 2011
Copyright 2010 ACM 1-58113-000-0/00/0010…$10.00. 79
those used in spoken languages that occur simultaneously, We will give a brief overview of the ways to write SL in notation
however they are linear and sequential in spoken languages [2]. system for the purpose of using it in machine translation.
Sign language differs in different regions and countries, but they Stokoe notation was developed in 1960 by William Stokoe, it is
all agree on several things regarding the sign basic parts such as written using symbols similar in the form to English alphabet
Manual Features (MF), Non-Manual Features (NMFs) and also symbols [6]. Figure 1 shows an example of Stokoe notation for
they agree in defining the Signing space. Manual Features are the word “story”: B means flat hand shape, a mean palm facing
signs performed by one or both hands using different shapes, up, z means side to side “means both of right and left hands are
locations, movement and orientations to represent meaning. side to side “ and ~ means up and down.
NMFs are those features that do not involve hands and are used to
give meaning and/or feeling or represent the morphological and
syntactic markers of a sentence [4]. Movements of body parts
such as head and shoulders, eye movements, eyebrows and facial
expression like puffed checks and mouth pattern, are kinds of Figure 1: Stokoe Notation example for: “Story” word [7]
NMF. Signing space is the space or the private area surrounding
the signer used to express signs in the sign language. This area HamNoSys was developed in 1989 by Hamburg University
extends from the top of the head to the waist level and extends by Research Group .The root of this system is Stokoe Notation in
the length of the signer arms [2]. addition to set of parameters set at the end of the word
representation such as: shape, location, orientation, extended
2. Arabic sign language finger orientation and movement of both dominant and the non-
dominant hands in addition to NMF representation [11]. Figure 2
Arabic Sign language is different in each Arab region or/and shows an example of HamNoSys notation for “Oh! Look! There!”
country with many dialects. This difference gives the difficulty of
communicating and dealing between deaf people in different Arab sentence: Means hand shape with extended Index finger,
countries. Means that hand orientation is away from the body, means
that palm of the hand is facing down, means hand location in
A need appeared to unify Arabic sign language in all Arab
the front of the signer neck.
countries. This derived the Council of Arab Ministers of Social
Affairs (CAMSA) to take a decision of developing a unified Arab
sign language dictionary and publish it to all countries, in an
attempt to help Arab deaf people to have a common language in
addition to their local language [3].This dictionary is mostly used
in education and in common communication such as sign Figure 2: HamNoSys notations example for: “Oh! Look! There!”
sentence [9]
language interpreters in television.
Gloss notation: is a textual representation of sign language used
Arabic sign language like other known sign languages depends on
for transcribing sign language video sequences [12].Sign Writing
three basic factors that are used to represent the manual features:
was developed by Valerie Sutton in 1974 [13]. Symbols used in
hand shape, hand location and orientation. In addition to the non-
this system are pictures that are similar to the real forms. Figure 3
manual features that are related to head, face, eyes, eyebrows,
shows an example of SignWriting notation for the word “girl”
shoulders and facial expression like puffed checks and mouth
pattern movements. ArSL is limited to represent nouns, adjectives where Means the head of the signer and the shaded parts
and verbs. Prepositions and adverbs are represented in the context means the hair of the signer, means hand with the index
of articulation by specifying locations, orientations and finger pointing out and the shaded square means that the back of
movement. Intensifiers represented by iteration [5]. Signs forming the hand is facing outward, means the points in the face that
and sequencing in the articulation, are done depending on the
the signer must touches by his index finger and means that the
Arabic sign language grammar and rules.
motion of the hand is downward.
80
TABLE I
SUMMARY OF NOTATIONS SYSTEMS
Stokoe Gloss HamNoSys SignWriting
As we can see from TABLE II, SignWriting is the best choice for Terminological ontology designed to represent terms that are used
our system, it is a language independent, contains large number of to represent knowledge in certain domain such as lexicons, (2)
basic symbols that can give a chance to build a large number of Information ontology designed to record and structure the
final symbols, it has a better support of NMF, it is understandable, database of a certain domain, and (3) Knowledge modelling
practical and it is usable from the deaf people in their daily life ontology designed to specify the conceptualizations of the
such as education, communication, reading. knowledge.
81
dictionaries that require translating a word to corresponding word words with the corresponding signs and file names of the sign
only without any detailed analysis of the syntactic structures of representation video. If the user enters a word that is available in
the inputted text or any relation to the meaning of the words or the database then the recorded clip will be shown, if the word is
relationship between them. Indirect path approach is the most not included then finger spelling is done. Similarly, Tawassol
sophisticated and widely used approach in machine translation. [40], is another Arabic system for translating Arabic text to
This approach is used to analyse the syntactic structure of the Arabic SL. The system is used as an educational tool. It contains a
inputted text and create an intermediate or abstract representation translator, a dictionary of Arabic words for a set of categories, in
of it and then generate a target language text from it, this means addition to a finger spelling editor. The system uses Buckwalter
that we need to specify the word structure, sentence structure and Arabic Morphological Analyzer to analyse the inputted text and
semantic structure in successive processes [2]. Vcommunicator Gesture Builder 2.0 with Sign Smith Studio
program to generate the animation output.
According to the nature of the intermediate representation,
indirect approach can be divided into Transfer-based and As we can see from the previous work that the final output is
Interlingua-based methodologies. Transfer-based is a language- either a video clip or an animated avatar, none have used
dependent, need to know the source and target languages. The SignWriting notation as an output, this does not mean that
analysis of the source language sentence is a shallow analysis and SignWriting is not usable. Actually, SignWriting is used in other
works on the syntactic level. Interlingual-based analysis is a applications either as a final stage of the translation or as an
deeper analysis of the source language sentence which creates intermediate stage. For instance, the JSPad system [41] is used to
structures of a more semantic nature. This structure can be write a Japanese sign language (JSL) using SignWriting. The
transferred into language independent semantic representation that system take a Japanese text then split it into signs, these signs are
we can use to produce any target language translation. In our mapped to SignWriting symbols referring to the JSL dictionary
system we will follow the Rule-based approach for two reasons: then it display them on the screen to permit the users to edit the
(1) we do not have an Arabic corpus to work on and (2) no generated signs then add them to the dictionary. Likewise, Ahmed
previous Arabic work followed this way. and Seong [43] developed a system for writing and reading text
messages in signs as an alternative to SMS on mobile phones. The
SignWriting notation system was used to convert text to sign
message and sign to text message in two-way communication.
LITERATURE REVIEW
Brito and Pereira [44] also proposed a model to support sign
Previous research in machine translation of written text to signed language content development and deployment in digital
language follow two approaches, as mentioned in the previous television scenarios by using SignWriting.
section: Rule based and Data driven approach.
To further extend the research in SignWriting and Arabic text to
In our literature review we will focus on previous research that ArSL translation, our proposed system will benefit from the two
used rule-based approach for translating text to SL. In fact, there domains as we will describe next.
are a number of successful Rule based systems that translate text
to sign language. We can divide these works into three groups,
International research, Arabic research and SignWriting research. PROPOSED SYSTEM
International research is any work carried out to convert from Given the previous work in the domains of text to SL translation
Non-Arabic text to sign language. TEAM [28] and eSIGN [18] and SignWriting notation, our proposed solution will enhance
(essential Sign Language Information on Government Networks) previous techniques used to translate Arabic text to ArSL by
are sample of two projects that translate English text to American considering ArSL translation rules and using a domain ontology
SL. ViSiCAST [32] is another project that translate English text to to produce SignWriting notation. The SignWriting will be used as
British SL. Also, Zijl [33] developed a system to translate English the final output of the system or as an intermediate level for future
text to South African SL. Baldassarri et al. [35] developed a avatar animation.
system to translate Spanish text to Spanish SL. Dasgupta et al.
[36] developed a system to translate English text to Indian sign Next we will describe in detail the components of our proposed
language. Sarkar et al. [37] developed a system to translate Bangla system.
text to Bangla SL. JEMNI and ELGHOUL [38] developed a
system to translate a given text to SL for multiple languages.
1. Domain Ontology component description
Arabic research developed to translate Arabic text to Arabic SL is
rare. For instance, Mohandes [39] developed a system to translate The domain of jurisprudence of prayer will consist of a set of
Arabic text into Arabic SL. This system is one stage in the process classes in taxonomic (subclass) hierarchy, as follows:
of developing a system to translate Arabic speech to Arabic sign
language. The system has a database to store Arabic dictionary “ ”دﯾﻦclass is a super class of “ ”دﯾﻦ إﺳﻼﻣﻲclass.
82
“ ”دﯾﻦ إﺳﻼﻣﻲclass has three sub classes:””ﻋﺒﺎدات,” ”ﻣﻌﺎﻣﻼتand Morphological analysis: This process takes Arabic text as an
“”أﺧﻼق. input and sends each sentence to the Morphological Analysis and
“ ”ﻋﺒﺎداتclass has three sub classes:””أرﻛﺎن,””واﺟﺒﺎت Disambiguation for Arabic (MADA) tool for Part of Speech
and””ﻣﺴﻨﻮﻧﺎت. (POS) tagging. MADA returns a feature line for each word in the
“”أرﻛﺎن class has five sub classes:” ﻧﻄﻖ inputted sentence, feature line consist of a set of
”اﻟﺸﮭﺎدﺗﯿﻦ,””اﻟﺼﻼة,””اﻟﺼﯿﺎم,””اﻟﺰﻛﺎةand””اﻟﺤﺞ.
<feature>:<value> pairs. Word features such as (Gender, Mood,
“ ”اﻟﺼﻼةclass has a set of sub classes:””ﻧﺎﻓﻠﺔ,” ”ﻓﺮضand” أھﻞ
”اﻷﻋﺬار, etc. Case … etc), POS (Nouns, Verbs, Adjectives, Pronouns … etc)
“ ”ﻧﺎﻓﻠﺔclass has a set of instances:””اﻟﺴﻨﻦ اﻟﺮواﺗﺐ,””اﻟﻮﺗﺮ and proclitic (the word, question, conjunction, preposition... etc),
,””اﻟﻜﺴﻮف,””اﻟﺨﺴﻮف,””اﻟﺠﻨﺎزة,””اﻟﻀﺤﻰ, ” ”اﻹﺳﺘﺨﺎرةand ” ﺗﺤﯿﺔ enclitics associated with (person, gender, number), the rest
”اﻟﻤﺴﺠﺪ. include the diacritic form (diac), the lexeme/lemma (lex), the
Buckwalter tag (bw) and the gloss (gloss) [12].
Also, there will be a set of properties for connecting classes and
instances with each other, this include: Grammatical transformation: The grammatical transformation
process takes the previous results as input and applies the Arabic
Has. Sign Language rules on each word depending on its feature.
Is-a.
Is a kind of. Semantic translation: This process takes the result of the
Is a synonym of. previous process and search for each word in the Domain
Ontology to get the word sign code. If the word does not have a
Figure 4 illustrates an example of an ontology component.
corresponding sign then replace this word by one of its synonyms
that have a sign in the SignWriting Database (DB). Then, replace
each sign code by the corresponding sign symbol stored in the
SignWriting DB. If the word does not have a corresponding sign
in the domain ontology, it will be finger spelled.
EVALUATION CRITERIA
System architecture
83
CONCLUSION Languages”, Institute of German Sign Language, Accessed in
7 December 2010, Available at:
This paper presents a proposed semantic translation system http://www.sign-lang.uni-hamburg.de/dgs-
for translating Arabic text to ArSL using ArSL rules for korpus/tl_files/inhalt_pdf/HamNoSys_06en.pdf.”
[15] R.F. de Brito and A.T.C. Pereira, “A model to support sign
grammatical transformation and domain ontologies for
language content development for digital television,” 2009
semantic translation.
IEEE International Workshop on Multimedia Signal
Processing, Rio De Janeiro, Brazil: 2009, pp. 1-6.
In order to check the feasibility of our proposed system, we
[16] G. van Heijst, A.T. Schreiber, and B.J. Wielinga, “Using
have limited the domain to the jurisprudence of prayer. We
explicit ontologies in KBS development,” International
plan to evaluate this system based on two evaluation
Journal of Human-Computer Studies, vol. 46, Feb. 1997, pp.
methods: (1) translate a set of sentences automatically then
183-292.
allow an ArSL expert to check its accuracy and (2) ask ArSL [17] L. Zhao, K. Kipper, W. Schuler, C. Vogler, N.I. Badler, and M.
experts to translate a set of sentences manually then compare Palmer, “A Machine Translation System from English to
their results to our system translation results. American Sign Language,” Proceedings of the 4th Conference
of the Association for Machine Translation in the Americas
The logical next step is to build and evaluate our system in on Envisioning Machine Translation in the Information
order to assure its consistency and correctness. Future, Springer-Verlag, 2000, pp. 54-67.
[18] “eSIGN at UEA,Accesses in 27 November 2010, Available at :
REFERENCES http://www.visicast.cmp.uea.ac.uk/eSIGN/index.html.”
[19] I. Marshall and É. Sáfár, “A prototype text to British Sign
[1] K. Eman, “ وﻣﺣدودﯾﺔ اﻟﻘﺎﻣوس ﺗﻣ ﻧﻊ اﻟﺻم.. ﺗﻌدد ﻟﻐﺎت اﻹﺷﺎرة:ﻣﺧﺗﺻون Language (BSL) translation system,” Proceedings of the 41st
ﻣن ﻓﮭم ﻣﺗرﺟﻣﻲ اﻟﺑراﻣﺞ اﻟﺗﻠﻔزﯾوﻧﯾﺔ,Accessed in 27 November 2010, Annual Meeting on Association for Computational Linguistics
Available at: - Volume 2, Sapporo, Japan: Association for Computational
http://www.signwriting.org/forums/linguistics/ling006.html, Linguistics, 2003, pp. 113-116.
” Asharq Al-Awsat. [20] L. Van Zijl and A. Combrink, “South African Sign Language
[2] S. Morrissey, “Data-Driven Machine Translation for Sign Machine Translation Project,” IN 8’TH INTERNATIONAL ACM
Languages,” PhD thesis, Dublin City University School of SIGACCESS CONFERANCE ON COMPUTERS AND
Computing, 2008. ACCESSIBILITY (ASSETS’06, 2006.
[3] P.V. Paul, Language and Deafness, Jones & Bartlett Learning, [21] S. Baldassarri, E. Cerezo, and F. Royo-Santas, “Automatic
2008. Translation System to Spanish Sign Language with a Virtual
[4] S. Samreen and M. Benali, , ﻗواﻋد ﻟﻐﺔ اﻹﺷﺎرة اﻟﻌرﺑﯾﺔ اﻟﻘطرﯾﺔ اﻟﻣوﺣدة Interpreter,” Proceedings of the 12th IFIP TC 13 International
2009. Conference on Human-Computer Interaction: Part I,
[5] M.A. Abdel-Fattah, “Arabic Sign Language: A Perspective,” Uppsala, Sweden: Springer-Verlag, 2009, pp. 196-199.
Journal of Deaf Studies and Deaf Education, vol. 10, Spring. [22] T. Dasgupta and A. Basu, “Prototype machine translation
2005, pp. 212 -221. system from text-to-Indian sign language,” Proceedings of
[6] S. abdulrahman, “””ﺗدوﯾن ﻟﻐﺔ اﻹﺷﺎرة, Accesses in 27 November the 13th international conference on Intelligent user
2010, Available at : interfaces - IUI '08, Gran Canaria, Spain: 2008, p. 313.
http://knol.google.com/k/#ﻟﻐﺔاﻹﺷﺎرة-ﺗدوﯾن.” [23] B. Sarkar, K. Datta, C.D. Datta, D. Sarkar, S.J. Dutta, I.D. Roy,
[7] J. Martin, “About The Stokoe System, Accessed in 27 A. Paul, J.U. Molla, and A. Paul, “A Translator for Bangla Text
November 2010, Available at: to Sign Language,” 2009 Annual IEEE India Conference,
http://www.signwriting.org/forums/linguistics/ling006.html. Ahmedabad, India: 2009, pp. 1-4.
” [24] M. JEMNI and O. ELGHOUL, “An avatar based approach for
[8] T. Dasgupta, S. Shukla, S. Kumar, S. Diwakar, and A. Basu, “A automatic interpretation of text to Sign language,” 9th
Multilingual Multimedia Indian Sign Language Dictionary European Conference for the Advancement of the Assistive
Tool,” The 6th Workshop on Asian Languae Resources, 2008. Technologies in Europe, San Sebastián , Spain: 2007.
[9] ““About The HamNoSys System, Accessed in 27 November [25] M. Mohandes, “Automatic Translation of Arabic Text to
2010, Available at: Arabic Sign Language,” AIML 06 International Conference,
http://www.signwriting.org/forums/linguistics/ling007.html Sharm El Sheikh, Egypt: 2006, pp. 71-76.
”.” [26] A. Alnafjan, “Tawassol,” Master project, King Saud
[10] A. Almohimeed, M. Wald, and R. Damper, “An Arabic Sign University, 2008.
Language Corpus for Instructional Language in School,” May. [27] T. Matsumoto, M. Kato, and T. Ikeda, “JSPad,” Proceedings
2010. of the 3rd International Universal Communication
[11] “About SignWriting, Accessed in 27 November 2010, Symposium on - IUCS '09, Tokyo, Japan: 2009, p. 363.
Available at: [28] A.S. Ahmed and D.S.K. Seong, “SignWriting on mobile
http://www.signwriting.org/forums/linguistics/ling004.html. phones for the deaf,” Proceedings of the 3rd international
” conference on Mobile technology, applications & systems -
[12] M. Abu Shaira, ,” ﻧظﺎم ﻣﺗﻛﺎﻣل ﻟﻛﺗﺎﺑﺔ ﻟﻐﺔ اﻹﺷﺎرة،ﻛﺗﺎﺑﺔ ﻟﻐﺔ اﻹﺷﺎرة Mobility '06, Bangkok, Thailand: 2006, p. 28.
ﺑرﻧﺎﻣﺞ ﺗدرﯾﺑﻲ، ”, Accesses in 27 November 2010, Available at : [29] R. de Brito and A. Pereira, “A model to support sign language
http://www.signwriting.org/archive/docs6/sw0504-Lessons- content development for digital television,” Multimedia
SignWriting-Arabic.pdf. Signal Processing, 2009. MMSP '09. IEEE International
[13] R. Studer, S. Decker, D. Fensel, and S. Staab, “Situation and Workshop on, 2009, pp. 1-6.
Perspective of Knowledge Engineering,” 2000.
[14] T. Hanke, ““ HamNoSys –Hamburg Notation System for Sign
84