A gesture is a form of non verbal or non vocal
communication in which visible bodily actions communicate
particular messages. Gestures include movement of the face,
hands or other parts of the body. Gestures allow persons to
speak a variety of feelings and thoughts, from condescension
and hostility to approval and affection, often together with body
language in addition to words when they speak. An emotion
plays a crucial role in person to person interaction. In recent
years, there has been a growing interest in improving all aspects
of interaction between humans and computers. Human
expression is used to interact with the computers. This paper
explores a ways of human-computer interaction that enable the
computer to be more aware of the user’s emotional expressions
we present an approach for the emotion recognition from a facial
expression, hand and body posture. Our model uses multimodal
emotion recognition system in which we use two different
models for facial expression recognition and for hand gesture
recognition and then combining the result of both classifiers
using a third classifier which give the resulting emotion.
Amit Pimpalkar : Gurunanak Institute of Engineering and Technology, RTM Nagpur University,
Dahegaon, Kalmeshwar Road, Nagpur, Maharashtra, 441501, India
Chaitali Nagalkar : Gurunanak Institute of Engineering and Technology, RTM Nagpur University,
Dahegaon, Kalmeshwar Road, Nagpur, Maharashtra, 441501, India
Sonal Waghmare : Gurunanak Institute of Engineering and Technology, RTM Nagpur University,
Dahegaon, Kalmeshwar Road, Nagpur, Maharashtra, 441501, India
Keshari Ingole : Gurunanak Institute of Engineering and Technology, RTM Nagpur University,
Dahegaon, Kalmeshwar Road, Nagpur, Maharashtra, 441501, India
Multimodal approach
Face Detection
Facial
expression recognition system
Hand gesture recognition
system
PCA
Cam Shift Algorithm
Gesture recognition can be seen as a way for computers to
begin to understand human body language, thus building
a richer bridge between machines and humans than
primitive text user interfaces, which still limit the majority of input to keyboard and mouse. We present
gesture recognition system by using webcam. We
proposed a multimodal approach in which we use two
different models one is Facial Expression Recognition
system (FERS) for recognizing facial expression and
second is Hand Gesture Recognition System (HGRS) for
recognize hand gesture as context. In our model we used
the database which was originally created “FABO
bimodal database [1]”consisting of combined face and
body expressions recorded simultaneously. We used
Principle Component Analysis algorithm (PCA) for
recognize face and Cam Shift algorithm for tracking the
hands and predicting their locations in image.
[1] Priya Metri, Jayshree Ghorpade and Ayesha Butalia
“Facial Emotion Recognition Using Context Based
Multimodal Approach”, Int. J. Emerg. Sci., 2(1), 171-
182, March 2012.
[2] Bajwa, I.S., Naveed, M.S., Asif, M.N., Hyder, I.S.
(2008), “Feature Based Image Classification by using
Principal Component Analysis”, in Journal of
Graphics, Vision and Image Processing, March 2008,
9(2) pp.11-17 .
[3] R. Highfield, R. Wiseman and R.Jenkins, “How your
looks betray your personality”, New Scientist, Feb
2012.
[4] J. Montagu, “The Expression of the Passions: The
Origin and Influence of Charles Le Brun’s
‘Conférence sur expression générale et
particulière,”Yale University Press, 2010.
[5] C. Darwin, edited by F. Darwin, “The Expression of
the Emotions in Man and Animals,”2ndedition, J.
Murray, London, 2009.
[6] C. Bell, “The Anatomy and Philosophy of Expression
as Connected with the Fine Arts,”5thedition, Henry G.
Bohn, London, 2009.
[7] A. Samal and P.A. Iyengar, “Automatic Recognition
and Analysis of Human Faces and Facial Expressions:
A Survey, Pattern Recognition”.
[8] Paul Ekman David Mastumoto and Wallance V.
Friesen University of California, “Facial
Expression In Affective Disorders”, New York
Oxford University Press 2008.
[9] Paul Ekman and Wallance V. Friesen University
of California, “Facial Signs Of Emotional
Experience Journal Of Personality And Social
Psychology”. 2007 Vol 19, No 6, 1123 1134.
