Page 1 of 7
European Journal of Business &
Social Sciences
Available at https://ejbss.org/
ISSN: 2235-767X
Volume 07 Issue 04
April 2019
Available online: https://ejbss.org/ P a g e | 2120
Expression Recognition with Appearance Based
Features of Facial Landmarks
B.PRIYANKA1
, P. SRINIDHI 2
, CH. SATHWIK3
1Assistant Professor: Dept. of ECE, Sreenidhi Institute of Science and Technology, Hyderabad
2UG Scholar: Dept. of ECE, Sreenidhi Institute of Science and Technology, Hyderabad
3UG Scholar: Dept. of ECE, Sreenidhi Institute of Science and Technology, Hyderabad
ABSTRACT:-
In this paper, Local Binary Patterns (LBP)
is used for Facial Expression Recognition (FER).
The concept of LBP is based on the information
that is present in color images of face. Multi-linear
image analysis can be done in different color spaces
using LBP and it can be seen that the color content
gives additional information about faces which can
lead to an efficient FER. Using LBP, the
components present in various color spaces such as
RGB, YCbCr and CIELuv or CIELab, are made
into two dimensional binary’s using multi-linear
algebra and concepts of binary’s and then Log- Gabor filters are used to extract these features. For
the selection of the features, mutual information
quotients method is used. Multiclass linear
discriminate analysis classifiers are used to classify
the features that were extracted.
Keywords: LBP, CIELab, CIELuv, FER, Log- Gabor filters
I. INTRODUCTION
A goal of the Human-Computer-Interaction
(HCI) systems is to enhance the communication
between the computer and user by making it user
friendly and user's needs. In [1] proposes the
important of the automatic facial expression
recognition (FER) plays an important role in the HCI
system and it has been studied extensively over the
past twenty years. Since the late 1960s use of the
facial expression for measuring people's emotions has
dominated psychology. Paul Ekman reawakened the
study of emotion by linking expressions to a group of
basic emotions (i.e., anger, disgust, fear, happiness,
sadness and surprise) [2]. The research study by
Megrabian [3] has indicated that 7% of the
communication information is transformed by
linguistic language, 55% by facial expression and
38% by paralanguage in human face-to-face
communication. It shows that facial expression
provides a large amount of information in human
communication. Many approaches have been
proposed for the FER in the past several decades
[1],[4]. Current state-of-art techniques mainly
focused on the gray-scale image features [1], rarely it
consider the color image feature [5]-[7].
Color feature mat provides more robust
classification results. Research reveals that the color
information enhances the face recognition and image
retrieval performance [8]-[11]. In [8] it was first
reported in that taking color information enhance the
reorganization rate as compared with the same
scheme using only the luminance information. Liu
and Liu in [10] proposed a new color space for face
recognition. In [11] Young, Man and Plataniotis
demonstrated that the facial color cues express the
improved face recognition performance using the
low-resolution face image. The RGB color binary has
enhanced the FER performance but it does not
Page 2 of 7
European Journal of Business &
Social Sciences
Available at https://ejbss.org/
ISSN: 2235-767X
Volume 07 Issue 04
April 2019
Available online: https://ejbss.org/ P a g e | 2121
consider the different illumination was reported in
[7]. Recent research shows the improved
performance by embedding the color components.
The capability of the color information in the RGB
color space in terms of the recognition performance
depends upon the type and angle of the light source,
often making recognition impossible. Thus the RGB
may not be always be the most desirable space for
processing color information. In [12] this issue can be
addresses using perceptually uniform color system. In
this paper a novel Local Binary Pattern (LBP) for
FER is introduced which provides the information
about the color facial images and investigates
performance contained in the color facial images and
investigates performance in perceptual color space
under slight variation in the illumination.
This paper is organized as follows Section II provides
the brief detail about the components of the FER
systems used for this investigation. Section III
defines and examines the binary-based representation
of color facial images in different color space and
explains the proposed LBP technique. Section IV
presents the experimental result and Section V
presents final conclusion.
II. CONSTRUCTION OF AN IMAGE-BASED
FER SYSTEM
The principal approaches (i.e., image-based
and model based) to FER using static images are
explained in [1]. Image-based extract feature form
the image without extensive knowledge about the
object of interest, which are fast and simple. The
model based methods attempt to recover the
volumetric geometry of the scene, which are slow
and complex [1]. Geometric features present the
shape and location of facial components (including
mouth, eyebrows, eyes and nose). The facial feature
points or facial components are obtained from the
feature vector that represents the face geometry.
Fig 1: System Level Diagram
The appearance feature can be taken from
either the whole face or specific regions in a face
image. This paper focused on the static color image
and a holistic technique of the image-based method is
used for feature extraction. Image based FER systems
consist of several components or modules, including
face detection and normalization, feature extraction,
classification and feature selection. The system level
diagram of FER system shown in Figure 1 The
following section will describe briefly about YCbCr,
CIELab, and CIELuv [13].
A. Face Detection and Normalization
In this module is to obtain face images,
which have normalized intensity, are uniform in
shape and size and depict only the face region. Face
area of an image is detected using the Viola-Jones
method based on the Haar-like features and the
AdaBoost learning algorithm [14]. The Viola and
Jones method is an object detection algorithm
provides competitive object detection in the real- time. Features used by Viola and Jones are derived
from pixels selected from rectangle area imposed
over the picture and exhibit high sensitivity to the
vertical and horizontal lines. After face detection the
image is scaled into some size (e.g.,64 × 64 pixels).
Color values in the face image are then normalized
with respect to RGB values of the image.
Color normalization is used to reduce the
lighting effect because the normalization process is
actually a brightness elimination process. Input image
of N1 × N2 pixels represented in the RGB color
space,
C:\Users\Dell\Downloads\Documents\1-s2.0-S1319157818303379-gr1.jpg
Page 3 of 7
European Journal of Business &
Social Sciences
Available at https://ejbss.org/
ISSN: 2235-767X
Volume 07 Issue 04
April 2019
Available online: https://ejbss.org/ P a g e | 2122
X = {X
n3[n1
, n2
]│1 ≤ n1 ≤ N1
, 1 ≤ n2 ≤ N2
, 1 ≤
n3 ≤ 3},
the normalized values, Xnorm
n3
[n1
, n2
], are defined by
Xnorm
n3
[n1
, n2
] =
X
n3[n1
, n2
]
∑ Xn3[n1
, n2
]
3
n3=1
----------(1)
where Xnorm
n3
[n1
, n2
] for n3 = 1,2,3 corresponding to
red, green, and blue (or R, G, and B) components of
the image X.
It is obvious that
∑ Xnorm
n3
[n1
, n2
] = 1
3
n3=1
----------(2)
R G B
(a)
(b)
Fig 2: Facial expression images: (a) the original color
components (b) the normalized color components.
B. Feature Extraction
Feature extraction have been studied and
compared in terms of their performance, including
principal components analysis, independent
components analysis, linear discriminates analysis
(LDA), the Gabor filter bank, etc. In [1] presents the
Gabor filter has better performance than the rest. The
Gabor filters model the receptive field profiles of
cortical simple cells quite good [1], [15]. Gabor filter
have two major drawbacks i.e., the maximum
bandwidth of Gabor filter the maximum bandwidth is
limited to approximately one octave, and the Gabor
filter are not optimal to achieve broad spectral
information with the maximum spatial localization
[16]. The Gabor filter are band pass filters, which
may suffers from loss of the low and the high- frequency information is reported in [17]. To
overcome the bandwidth limitation of the traditional
Gabor filter, Field proposed Log-Gabor filter [17].
Response of the Log-Gabor filter, is Gaussian when
viewed on a logarithmic frequency scale instead of a
linear. It allows more information to be capture in the
high-frequency area with desirable high pass
characteristics. A bank of 24 Log-Gabor filter is
employed to extract the facial features. Polar form of
2-D Log-Gabor filters in frequency domain is given
by
H(f, θ) = exp
{
− [ln(
f
f0
)]
2
2 [ln(
σf
fo
)]
2
}
exp {
−(θ − θ0)
2
2σθ
2
}
----------(3)
where H(f, θ) is frequency response function of the
2-D Log-Gabor filter, f and θ denotes the frequency
2-D Log-Gabor filters, f and θ denotes the frequency
and the phase/angle of the filter.fo is the filter center
frequency and θ0
the filter's direction. The constant
σf defines the radial bandwidth B in octaves and the
constant σθ angular bandwidth ∆Ω in radians.
B = √
2
ln2
× |ln (
σf
f0
)|
2
, ∆Ω = 2σθ√
2
ln2
----------(4)
In this paper describes here, the ratio σf/f0
is kept constant for varyingf0
, B is set to one octave
and the angular bandwidth is set to one octave and
the angular bandwidth is set to ΔΩ = π/4 radians. σf
is be determined for a varying value of f0
. Six scales
and four orientations are implemented to extract
features from face images. It leads to 24 filter transfer
functions representing different scales and
orientations. Image filtering is performed in the
frequency domain making the process faster
compared with the special domain convolution. After
2-D fast Fourier transform (FFT) into the frequency
