Major Pro On Sentiment Analysis of Mobile Reviews PDF
Major Pro On Sentiment Analysis of Mobile Reviews PDF
CERTIFICATE
This is to certify that the dissertation titled “Sentiment Analysis of Mobile Reviews
Using Supervised Learning Techniques” is submitted by Y.NIKHIL (11026A0524),
P. SNEHA (11026A0542), S.PRITHVI RAJ (11026A0529), I.AJAYRAM
(11026A0535) , E.RAJIV (11026A0555), students of B.Tech.(CSE - IIMDP) , in
partial fulfillment of the requirements for the award of the degree of Bachelor Of
Technology in Computer Science and Engineering is a record of bonafide work
carried out by them under my supervision.
Dr. L. Sumalatha
Internal Guide,
Head of the Department,
Professor,
Department of CSE,
University College of Engineering,
JNT University, Kakinada.
DECLARATION
This is to certify that the thesis titled “Sentiment Analysis of Mobile Reviews Using
Supervised Learning Techniques” is a bonafide work done by us, in partial
fulfillment of the requirements for the award of the degree B.Tech.(CSE-IIMDP) and
submitted to the Department of Computer Science & Engineering, University College
of Engineering, Jawaharlal Nehru Technological University, Kakinada.
I also declare that this project is a result of my own effort and that has not been copied
from anyone and I have taken only citations from the sources which are mentioned in
the references.
This work was not submitted earlier at any other University or Institute for the award
of any degree.
We express our deep gratitude and regards to Dr. L. Sumalatha, Internal Guide and
Professor, Head of Department of Computer Science & Engineering for her
encouragement and valuable guidance in bringing shape to this dissertation.
We thankful to all the Professors and Faculty Members in the department for their
teachings and academic support and thanks to Technical Staff and Non-teaching staff
in the department for their support.
Y NIKHIL (11026A0524)
P SNEHA (11026A0542)
S PRITHVI RAJ (11026A0529)
I AJAY RAM (11026A0535)
E RAJIV (11026A0555)
ii
ABSTRACT
Sentiment analysis or opinion mining is the computational study of people’s opinions,
sentiments, attitudes, and emotions expressed in written language. It is one of the most
active research areas in natural language processing and text mining in recent years. Its
popularity is mainly due to two reasons. First, it has a wide range of applications
because opinions are central to almost all human activities and are key influencers of
our behaviors. Whenever we need to make a decision, we want to hear other’s opinions.
Second, it presents many challenging research problems, which had never been
attempted before the year 2000. Part of the reason for the lack of study before was that
there was little opinionated text in digital forms. It is thus no surprise that the inception
and the rapid growth of the field coincide with those of the social media on the Web.
In fact, the research has also spread outside of computer science to management
sciences and social sciences due to its importance to business and society as a whole.
In this talk, I will start with the discussion of the mainstream sentiment analysis research
and then move on to describe some recent work on modeling comments, discussions,
and debates, which represents another kind of analysis of sentiments and opinions.
Sentiment classification is a way to analyze the subjective information in the text and
then mine the opinion. Sentiment analysis is the procedure by which information is
extracted from the opinions, appraisals and emotions of people in regards to entities,
events and their attributes. In decision making, the opinions of others have a significant
effect on customers ease, making choices with regards to online shopping, choosing
events, products, entities. The approaches of text sentiment analysis typically work at a
particular level like phrase, sentence or document level. This paper aims at analyzing a
solution for the sentiment classification at a fine-grained level, namely the sentence
level in which polarity of the sentence can be given by three categories as positive,
negative and neutral.
iii
TABLE OF CONTENTS
1 INTRODUCTION
1.1 Objective 4
1.2 Proposed Approach and Methods to be Employed 4
2 LITERATURE SURVEY
2.1 Models 6
2.1.1 Naïve Bayes 6
2.1.2 Bag Of Words 10
2.1.3 Support Vector Machine 14
2.1.4 Principal Component Analysis 21
4 IMPLEMENTATION
iv
5 TESTING
6 SCREEN SHOTS
CONCLUSION 63
REFERENCES 64
v
LIST OF FIGURES
Figure 1 : Example of Bag Of Words
Figure 8: Level 2 Data Flow Diagram for the process 1 (Naïve Bayes)
Figure 9 : Level 2 Data Flow Diagram for the process 2 (Bag of Words)
Figure 10 : Level 2 Data Flow Diagram for the process 3 (Support Vector Machine)
vi
LIST OF TABLES
vii
CHAPTER 1
INTRODUCTION
1
1.INTRODUCTION
Sentiment analysis refers to the use of natural language processing, text analysis and
computational linguistics to identify and extract subjective information in source materials.
Generally speaking, sentiment analysis aims to determine the attitude of a speaker or a writer
with respect to some topic or the overall contextual polarity of a document .The attitude may
be his or her judgment or evaluation affective state, or the intended emotional communication.
Sentiment analysis is the process of detecting a piece of writing for positive, negative, or neutral
feelings bound to it .Humans have the innate ability to determine sentiment; however, this
process is time consuming, inconsistent, and costly in a business context It’s just not realistic
to have people individually read tens of thousands of user customer reviews and score them for
sentiment .
For example if we consider Semantria’s cloud based sentiment analysis software .Semantria’s
cloud-based sentiment analysis software extracts the sentiment of a document and its
components through the following steps:
A document is broken in its basic parts of speech, called POS tags, which identify the
structural elements of a document, paragraph, or sentence (ie Nouns, adjectives, verbs,
and adverbs) .
Sentiment-bearing phrases, such as “terrible service”, are identified through the use of
specifically designed algorithms .
Each sentiment-bearing phrase in a document is given a score based on a logarithmic
scale that ranges between -10 and 10 .
Finally, the scores are combined to determine the overall sentiment of the document or
sentence Document scores range between -2 and 2 .
Semantria’s cloud-based sentiment analysis software is based on Natural Language Processing
and delivers you more consistent results than two humans. Using automated sentiment analysis,
Semantria analyzes each document and its components based on sophisticated algorithms
developed to extract sentiment from your content in a similar manner as a human – only 60,000
times faster.
Existing approaches to sentiment analysis can be grouped into three main categories:
Keyword spotting
Lexical affinity
Statistical methods
2
Keyword spotting is the most naive approach and probably also the most popular because of
its accessibility and economy .Text is classified into affect categories based on the presence of
fairly unambiguous affect words like ‘happy’, ‘sad’, ‘afraid’, and ‘bored’ .The weaknesses of
this approach lie in two areas: poor recognition of affect when negation is involved and reliance
on surface features .About its first weakness, while the approach can correctly classify the
sentence “today was a happy day” as being happy, it is likely to fail on a sentence like “today
wasn’t a happy day at all” About its second weakness, the approach relies on the presence of
obvious affect words that are only surface features of the prose .
In practice, a lot of sentences convey affect through underlying meaning rather than affect
adjectives For example, the text “My husband just filed for divorce and he wants to take
custody of my children away from me” certainly evokes strong emotions, but uses no affect
keywords, and therefore, cannot be classified using a keyword spotting approach .
Lexical affinity is slightly more sophisticated than keyword spotting as, rather than simply
detecting obvious affect words, it assigns arbitrary words a probabilistic ‘affinity’ for a
particular emotion For example, ‘accident’ might be assigned a 75% probability of being
indicating a negative affect, as in ‘car accident’ or ‘hurt by accident’ These probabilities are
usually trained from linguistic corpora .Though often outperforming pure keyword spotting,
there are two main problems with the approach First, lexical affinity, operating solely on the
word-level, can easily be tricked by sentences like “I avoided an accident” (negation) and “I
met my girlfriend by accident” (other word senses) Second, lexical affinity probabilities are
often biased toward text of a particular genre, dictated by the source of the linguistic corpora
This makes it difficult to develop a reusable, domain-independent model .
Statistical methods, such as Bayesian inference and support vector machines, have been
popular for affect classification of texts .By feeding a machine learning algorithm a large
training corpus of affectively annotated texts, it is possible for the system to not only learn the
affective valence of affect keywords (as in the keyword spotting approach), but also to take
into account the valence of other arbitrary keywords (like lexical affinity), punctuation, and
word co-occurrence frequencies. However, traditional statistical methods are generally
semantically weak, meaning that, with the exception of obvious affect keywords, other lexical
or co-occurrence elements in a statistical model have little predictive value individually .As a
result, statistical text classifiers only work with acceptable accuracy when given a sufficiently
large text input .So, while these methods may be able to affectively classify user’s text on the
3
page- or paragraph- level, they do not work well on smaller text units such as sentences or
clauses .
1.1 Objective
Sentiment classification is a way to analyze the subjective information in the text and then
mine the opinion .Sentiment analysis is the procedure by which information is extracted from
the opinions, appraisals and emotions of people in regards to entities, events and their attributes.
In decision making, the opinions of others have a significant effect on customers ease, making
choices with regards to online shopping, choosing events, products, entities .
Sentiment Analysis or Opinion Mining is a study that attempts to identify and analyze emotions
and subjective information from text Since early 2001, the advancement of internet technology
and machine learning techniques in information retrieval make Sentiment Analysis becomes
popular among researchers. Besides, the emergent of social networking and blogs as a
communication medium also contributes to the development of research in this area Sentiment
analysis or mining refers to the application of Natural Language Processing, Computational
Linguistics, and Text Analytics to identify and extract subjective information in source
materials .Sentiment mining extracts attitude of a writer in a document includes writer’s
judgement and evaluation towards the discussed issue.
Sentiment analysis allows us to identify the emotional state of the writer during writing, and
the intended emotional effect that the author wishes to give to the reader .In recent years,
sentiment analysis becomes a hotspot in numerous research fields, including natural language
processing (NLP), data mining (DM) and information retrieval (IR) This is due to the
increasing of subjective texts appearing on the internet .Machine Learning is commonly used
to classify sentiment from text .This technique involves with statistical model such ad Support
Vector Machine (SVM) , Bag of Words and Näive Bayes (NB) .The most commonly used in
sentiment mining were taken from blog, twitter and web review which focusing on sentences
that expressed sentiment directly .The main aim of this problem is to develop a sentiment
mining model that can process the text in the mobile reviews .
4
CHAPTER 2
LITERATURE SURVEY
5
2.LITERATURE SURVEY
2.1 Models
A naïve bayes classifier is a simple probability based algorithm. It uses the bayes theorem but
assumes that the instances are independent of each other which is an unrealistic assumption in
practical world naïve bayes classifier works well in complex real world situations .
The naïve bayes classifier algorithm can be trained very efficiently in supervised learning for
example an insurance company which intends to promote a new policy to reduce the promotion
costs the company wants to target the most likely prospects the company can collect the
historical data for its customers ,including income range ,number of current insurance policies
,number of vehicles owned ,money invested ,and information on whether a customer has
recently switched insurance companies .Using naïve bayes classifier the company can predict
how likely a customer is to respond positively to a policy offering. With this information,the
company can reduce its promotion costs by restricting the promotion to the most likely
customers .
The naïve bayes algorithm offers fast model building and scoring both binary and multiclass
situations for relatively low volumes of data this algorithm makes prediction using bayes
theorem which incorporates evidence or prior knowledge in its prediction bayes theorem
relates the conditional and marginal probabilities of stochastic events H and X which is
mathematically stated as
6
P stands for the probability of the variables within parenthesis .
P(H) is the prior probability of marginal probability of H it’s prior in the sense that it has not
yet accounted for the information available in X .
P(H/X) is the conditional probability of H, given X it is also called the posterior probability
because it has already incorporated the outcome of event X .
P(X/H) is the conditional probability of X given H .
P(X) is the prior or marginal probability of X, which is normally the evidence .
It can also represented as
Thus we find the class that maximizes P(Ci/X) The class Ci for which P(Ci /X) is maximized
is called the maximum posteriori hypothesis . By Baye’s theorem
As P(X) is the same for all classes, only P(X/Ci)* P(Ci) need be maximized If the class a priori
probabilities, P(Ci), are not known, then it is commonly assumed that the classes are equally
likely [P(C1) = P(C2) = = P(Ck )] and we would therefore maximize P(X/Ci) Otherwise we
maximize P(X/Ci) * P(Ci) .
Given data sets with many attributes, it would be computationally expensive to compute
P(X/Ci). In order to reduce computation in evaluating P(X/Ci) * P(Ci), the naive assumption
of class conditional independence is made This presumes that the values of the attributes are
conditionally independent of one another, given the class label of the sample .
7
Mathematically this means that
The probabilities [P(x1 /Ci), P(x2 /Ci) … P(xn /Ci)] can easily be estimated from the training
set .Recall that here xk refers to the value of attribute Ak for sample X .If Ak is categorical,
then P(xk /Ci) is the Ak number of samples of class Ci in T having the value xk for attribute ,
divided by freq(Ci, T), the number of sample of class Ci in T .
In order to predict the class label of X, P(X/Ci)* P(Ci) is evaluated for each class Ci .The
classifier predicts that the class label of X is Ci if and only if it is the class that maximizes
P(X/Ci) * P(Ci) .
Conditional probability
The conditional probability is the probability that a random variable will take on a particular
value given that the outcome for another random variable is known
8
P(I/positive)=5/10=0 5
The conditional probability for the word ‘LOVE’ in positive review is
P(Love/positive)=20/10=2
The conditional probability for the word ‘THIS’ in positive review is
P(This/positive)=5/10=0 5
The conditional probability for the word ‘FILM’ in positive review is
P(Film/positive)=4/10=0 4
Posterior probability
The posterior probabilities is the product of prior probability and conditional probabilities
P(positive)>P(negative)
The given test set “I Love This Film” is predicted by naïve bayes as a positive Sentiment
9
2.1.2 Bag of words
∝ ∏|v|
t=1 P(wt |C)
xit
|v|
We often won’t need the normalisation term ( ni !⁄∏t=1 xit ! ) because it does not depend on the
class, C .The numerator of the right hand side of this expression can be interpreted as the
product of word likelihoods for each word in the document, with repeated words taking part
for each repetition .
As for the Bernoulli model, the parameters of the likelihood are the probabilities of each word
given the document class P(wt |C), and the model parameters also include the prior probabilities
P(C). To estimate these parameters from a training set of documents labelled with class C = k,
let zik be an indicator variable which equals 1 when Di has class C=k, and equals 0 otherwise
If N is again the total number of documents, then we have:
|𝑣|
P(wt |C=k) = ∑𝑁 𝑁
𝑖=1 xit zik ⁄(∑𝑠=1 ∑𝑖=1 xis zik )
10
An estimate of the probability P(wt |C=k) as the relative frequency of wt in documents of class
C=k with respect to the total number of words in documents of that class .
Thus given a training set of documents (each labelled with a class) and a set of K classes, we
can estimate a multinomial text classification model as follows:
Define the vocabulary V the number of words in the vocabulary defines the
dimension of the feature vectors .
Count the following in the training set:
N the total number of documents .
Nk the number of documents labelled with class C=k, for each class k=1,...,K .
xit the frequency of word wt in document Di, computed for every word wt in V .
Estimate the priors P(C=k) .
Estimate the likelihoods P(wt | C=k) .
To classify an unlabelled document Dj , we estimate the posterior probability for each class in
terms of words u which occur in our document as
𝑙en(Di )
P(C|Di )∝ P(C) ∏h=1 p(uh /𝐶)
A drawback of relative frequency estimates for the multinomial model is that zero counts result
in estimates of zero probability .This is a bad thing because the Naive Bayes equation for the
likelihood involves taking a product of probabilities if any one of the terms of the product is
zero, then the whole product is zero .This means that the probability of the document belonging
to that particular class is zero which is impossible .
Just because a word does not occur in a document class in the training data does not mean that
it cannot occur in any document of that class .The problem is that equation of likelihood
underestimates the likelihoods of words that do not occur in the data .Even if word w is not
observed for class C=k in the training set, we would still like P(w | C=k) > 0 . Since probabilities
must sum to 1, if unobserved words have underestimated probabilities, then those words that
11
are observed must have overestimated probabilities .Therefore, one way to alleviate the
problem is to remove a small amount of probability allocated to observed events and distribute
this across the unobserved events .A simple way to do this, sometimes called Laplace’s law of
succession or add one smoothing, adds a count of one to each word type .If there are W word
types in total, then instead of previous likelihood formula replaced with:
The denominator was increased to take account of the |V| extra “observations” arising from the
“add 1” term, ensuring that the probabilities are still normalised .
3 Chinese Macao c
12
P = 𝑁𝑐 ⁄𝑁
P(c) = 3/4
The prior probability for the class j is
P(j) = 1/4
Conditional Probability:
ˆ ( w | c ) count( w, c ) 1
P
count(c ) | V |
The conditional probability for the word “Chinese” in c class is
P(Chinese | c) =(5+1)/(8+6)=6/14=3/7
The conditional probability for the word “Tokyo” in c class is
P(Tokyo | c) =(0+1)/(8+6)=1/14
The conditional probability for the word “Japan” in c class is
P(Japan | c) =(0+1)/(8+6)=1/14
The conditional probability for the word “Chinese” in c class is
P(Chinese | c) =(1+1)/(3+6)=2/9
Posterior probability
Support vector machines (SVMs) are a set of related supervised learning methods used for
classification and regression they belong to a family of generalized linear classifiers .In another
terms, Support Vector Machine (SVM) is a classification and regression prediction tool that
uses machine learning theory to maximize predictive accuracy while automatically avoiding
over-fit to the data .Support Vector machines can be defined as systems which use hypothesis
space of a linear functions in a high dimensional feature space, trained with a learning algorithm
from optimization theory that implements a learning bias derived from statistical learning
theory. Support vector machine was initially popular with the NIPS community and now is an
active part of the machine learning research around the world. SVM becomes famous when,
using pixel maps as input; it gives accuracy comparable to sophisticated neural networks with
elaborated features in a handwriting recognition task It is also being used for many applications,
such as hand writing analysis, face analysis and so forth, especially for pattern classification
and regression based applications .The foundations of Support Vector Machines (SVM) has
gained popularity due to many promising features such as better empirical performance .The
formulation uses the Structural Risk Minimization (SRM) principle, which has been shown to
be superior to traditional Empirical Risk Minimization (ERM) principle, used by conventional
neural networks.SRM minimizes an upper bound on the expected risk,where as ERM
minimizes the error on the training data .It is this difference which equips SVM with a greater
ability to generalize, which is the goal in statistical learning .SVMs were developed to solve
the classification problem, but recently they have been extended to solve regression problems.
A support vector machine (SVM) is preferred when data has exactly two classes .An SVM
classifies data by finding the best hyperplane that separates all data points of one class from
those of the other class.The best hyperplane for an SVM means the one with the
largest margin between the two classes .Margin means the maximal width of the slab parallel
to the hyperplane that has no interior data points .The support vectors are the data points that
are closest to the separating hyperplane; these points are on the boundary of the slab
14
Figure 2: Hyperplane separating two classes
Assume, there is a new company j which has to be classified as solvent or insolvent according
to the SVM score. In the case of a linear SVM the score looks like a DA or Logit score, which
is a linear combination of relevant financial ratios xj = (xj1 , xj2 , …xjd ), where xj is a vector with
d financial ratios and 𝑥𝑗𝑘 is the value of the financial ratio number k for company j ( k=1,…,d)
So 𝑧𝑗 the score of company j, can be expressed as:
𝑧𝑗 =𝑥𝑗𝑇 +b
Where w is a vector which contains the weights of the d financial ratios and b is a constant
The comparison of the score with a benchmark value (which is equal to zero for a balanced
sample) delivers the “forecast” of the class – solvent or insolvent – for company j .
To use this decision rule for the classification of company j, the SVM has to learn the values
of the score parameters w and b on a training sample. Assume this consists of a set of n
companies (i=1, 2, … , n) .From a geometric point of view, calculating the value of the
parameters w and b means looking for a hyperplane that best separates solvent from insolvent
companies according to some criterion .
The criterion used by SVMs is based on margin maximization between the two data classes of
solvent and insolvent companies .The margin is the distance between the hyper planes
bounding each class, where in the hypothetical perfectly separable case no observation may lie.
By maximizing the margin, we search for the classification function that can most safely
separate the classes of solvent and insolvent companies .The graph below represents a binary
space with two input variables .Here crosses represent the solvent companies of the training
sample and circles the insolvent ones .The threshold separating solvent and insolvent
companies is the line in the middle between the two margin boundaries, which are canonically
15
represented as x T w+b=1 and x T w+b=-1 .Then the margin is 2 / ||w|| where ||w|| is the norm
of the vector w .
In a non-perfectly separable case the margin is “soft” .This means that in-sample classification
errors occur and also have to be minimized .Let 𝜀𝑖 be a non-negative slack variable for in-
sample misclassifications .
In most cases 𝜀𝑖 =0, that means companies are being correctly classified .In the case of a
positive 𝜀𝑖 the company i of the training sample is being misclassified .A further criterion used
by SVMs for calculating w and b is that all misclassifications of the training sample have to be
minimized .
Let 𝑦𝑖 be an indicator of the state of the company, where in the case of solvency 𝑦𝑖 =-1 and in
the case of insolvency 𝑦𝑖 =1 . By imposing the constraint that no observation may lie within
the margin except some classification errors, SVMs require that either
xiT w + b ≥ 1- εi or
xiT w + b ≤ -1+ εi
The optimization problem for the calculation of w and b can thus be expressed by:
1
minw ||𝑤||2 +C∑𝑛𝑖=1 εi
2
xiT w + b ≥ -1+ εi
We maximize the margin 2 ⁄ ||𝑤|| by minimizing ||w||2/ 2, where the square in the form of
w comes from the second term, which originally is the sum of in-sample misclassification errors
2 ⁄ ||𝑤|| times the parameter C .Thus SVMs maximize the margin width while minimizing
16
errors .This problem is quadratic i e convex C = “capacity” is a tuning parameter, which
weights in-sample classification errors and thus controls the generalization ability of an SVM
The higher is C, the higher is the weight given to in-sample misclassifications , the lower is the
generalization of the machine . Low generalization means that the machine may work well on
the training set but would perform miserably on a new sample .Bad generalization may be a
result of overfitting on the training sample, for example, in the case that this sample shows
some untypical and non-repeating data structure .By choosing a low C, the risk of overfitting
an SVM on the training sample is reduced .It can be demonstrated that C is linked to the width
of the margin .The smaller is C, the wider is the margin, the more and larger in-sample
classification errors are permitted .
Solving the above mentioned constrained optimization problem of calibrating an SVM means
searching for the minimum Lagrange function and considering 𝛼𝑖 ≥ 0 are the Lagrange
multipliers for the inequality constraint and 𝑣𝑖 ≥0 are the Lagrange multipliers for the condition
𝜀𝑖 ≥0 .This is a convex optimization problem with inequality constraints, which is solved my
means of classical non-linear programming tools and the application of the Kuhn-Tucker
Sufficiency Theorem .The solution of this optimisation problem is given by the saddle-point of
the Lagrangian, minimized with respect to w, b, and 𝜀 and maximized with respect to α and ν.
The entire task can be reduced to a convex quadratic programming problem in 𝛼𝑖 .Thus, by
calculating 𝛼𝑖 we solve our classifier construction problem and are able to calculate the
parameters of the linear SVM model using the formulas
w = ∑𝑛𝑖=1 𝑦𝑖 𝛼𝑖 𝑥𝑖
𝑇 𝑇
b = (𝑥+1 , 𝑥−1 )/2
𝛼𝑖 must be non-negative, weighs different companies of the training sample
The companies, whose 𝛼𝑖 are not equal to zero, are called support vectors and are the relevant
ones for the calculation of w .Support vectors lie on the margin boundaries or, for non-perfectly
separable data, within the margin .By this way, the complexity of calculations does not depend
on the dimension of the input space but on the number of support vectors .Here x+1 and x-1
are any two support vectors belonging to different classes, which lie on the margin boundaries.
After simplifying the equations we obtain the score 𝑧𝑗 as a function of the scalar product of
the financial ratios of the company to be classified and the financial ratios of the support vectors
in the training sample, of 𝛼𝑖 and of 𝑦𝑖 .By comparing 𝑧𝑗 with a benchmark value, we are able
to estimate if a company has to be classified as solvent or insolvent
17
𝑧𝑗 = ∑𝑛𝑖=1 𝑦𝑖 𝛼𝑖 (𝑥𝑖 ,𝑥𝑗 )+b
the support vector machine returns one class as output which is our result .
Example :
Given 3 support vectors as
2 2 4
𝑆1= 𝑆2 = 𝑆3 =
1 −1 0
Given 3 support vectors as 𝑆1, 𝑆2 , 𝑆3 where 𝑆1 , 𝑆2 belongs to Negative class and 𝑆3 belongs to
Positive class.
2 2 4
𝑆1= 𝑆2 = 𝑆3 =
1 −1 0
A graph is plotted for these 3 support vectors
18
2
2
𝑆1= 𝑆̃1= 1
1
1
2
2 ̃
𝑆2 = 𝑆2 = −1
−1
1
4
4 ̃
𝑆3 = 𝑆3 =0
0
1
Substitute the values of 𝑆̃1 , 𝑆̃2 , 𝑆̃3 in the above 3 Linear equations
2 2 4
𝑆̃1= 1 𝑆̃2 = −1 𝑆̃3 =0
1 1 1
After substituting the values of 𝑆̃1, 𝑆̃2 , 𝑆̃3 in 3 linear equations and simplifying it reduces into
By solving the above 3 equations we will get the α1, α2, α3 values
19
α1= -3 25
α2= -3 25
α3= 3 50
The Hyperplane that seperates the positive class from negative class is given by formula
̃=∑𝑖 ∝𝑖 𝑆̃𝑖
𝑤
After simplifying
Our vectors are augmented with a bias Hence we can equate the entry in as the hyper plane
1
with an offset b The separating hyper plane equation 𝑦 = 𝑤𝑥 + 𝑏 with w= and offset 𝑏 = −3
0
Hence the positive class is seperated from the negative class at offset b=-3 .
b=3
20
6
Given S4 = for finding the class for this new support vector we use the formula
2
=wx in this case if w x > offset positive class
w x < offset negative class
1 6
we know w = and x= and offset=3
0 2
1 6
wx= * =6
0 2
w x > offset = 6 > 3
hence support vector machine classifies this newly added point belongs to the positive class.
The mean subtracted is the average across each dimension .This produces a data set whose
mean is zero
Mean 𝑥̅ =∑𝑛𝑖=1 𝑥𝑖 ⁄𝑛
21
Calculate the covariance matrix:
Variance and Covariance are a measure of the spread of a set of points around their center of
mass (mean).Variance is the measure of the deviation from the mean for points in one
dimension .Variance is calculated with the formula
𝜎 2 =∑(𝑥 − 𝑥̅ )2 ⁄𝑛 − 1
Covariance as a measure of how much each of the dimensions vary from the mean with respect
to each other .Covariance is measured between 2 dimensions to see if there is a relationship
between the 2 dimensions .The covariance between one dimension and itself is the variance .
For example a 3-dimensional data set (x,y,z), then you could measure the covariance between
the x and y dimensions, the y and z dimensions, and the x and z dimensions .Measuring the
covariance between x and x , or y and y , or z and z would give you the variance of the x , y
and z dimensions respectively
𝑣𝑎𝑟(𝑥, 𝑥) 𝑐𝑜𝑣(𝑥, 𝑦) 𝑐𝑜𝑣(𝑥, 𝑧)
C=[𝑐𝑜𝑣(𝑦, 𝑥) 𝑣𝑎𝑟(𝑦, 𝑦) 𝑐𝑜𝑣(𝑦, 𝑧)]
𝑐𝑜𝑣(𝑧, 𝑥) 𝑐𝑜𝑣(𝑧, 𝑦) 𝑣𝑎𝑟(𝑧, 𝑧)
Diagonal is the variances of x, y and z cov(x,y) = cov(y,x) hence matrix is symmetrical about
the diagonal For a N-dimensional data will result in NxN covariance matrix
Exact value is not as important as it’s sign A positive value of covariance indicates both
dimensions increase or decrease together example as the number of hours studied increases,
the marks in that subject increase.A negative value indicates while one increases the other
decreases, or vice-versa If covariance is zero: the two dimensions are independent of each
other.
The covariance matrix is a square, the calculation of eigenvectors and eigenvalues are possible
for this matrix. These are rather important, as they tell us useful information about our data
Eigen values are calculated with the formula |𝐶 − 𝜆𝐼| = 0
Where I is the identity matrix
Eigen vectors are calculated with the formula [𝐶 − 𝜆𝐼][𝑘] = 0
22
It is important to notice that these eigenvectors are both unit eigenvectors ie Their lengths are
both 1 which is very important for PCA .
The eigenvectors and eigenvalues obtained from the covariance matrix will have quite different
values .In fact, it turns out that the eigenvector with the highest eigenvalue is the principle
component of the data set .
In general, once eigenvectors are found from the covariance matrix, the next step is to order
them by eigenvalue, highest to lowest This gives you the components in order of significance
Now, if you like, you can decide to ignore the components of lesser significance .You do lose
some information, but if the eigenvalues are small, you don’t lose much If you leave out some
components, the final data set will have less n dimensions than the original .To be precise, if
you originally have dimensions in your data, and so you calculate n eigenvectors and
eigenvalues, and then you choose only the first p eigenvectors, then the final data set has only
p dimensions .
To form a feature vector, which is just a fancy name for a matrix of vectors . This is constructed
by taking the eigenvectors that you want to keep from the list of eigenvectors, and forming a
matrix with these eigenvectors in the columns
FeatureVector = (eigenvector1, eigenvector2, ,eigenvector𝑛 )
23
X1 X2
1 4000 1 6500
1 6000 1 9700
-1 4000 -1 7750
-2 0000 -2 5250
-3 0000 -3 9500
2 4000 3 0750
1 5000 2 0250
2 3000 2 7500
-3 2000 -4 0500
-4 1000 -4 8500
Given a two dimensional data as x1,x2 Dimensionality reduction using Principle component
analysis can be done in 5 steps:
To obtain covariance matrix .
To obtain eigen values .
To obtain eigen vectors .
To obtain coordinates of data point in the direction of eigen vectors .
Covariance matrix:
24
X1 X2 C= X1-X1bar D= X2-X2bar C*D
1 4000 1 6500 1 8500 2 2175 4 1024
1 6000 1 9750 2 0500 2 5425 5 3121
-1 4000 -1 7750 -0 9500 -1 2075 1 1471
-2 0000 -2 5250 -1 5500 -1 9575 3 0341
-3 0000 -3 9500 -2 5500 -3 3825 8 6254
2 4000 3 0750 2 8500 3 6425 10 3811
1 5000 2 0250 1 9500 2 5925 5 0554
2 3000 2 7500 2 7500 3 3175 9 1231
-3 2000 -4 0500 -2 7500 -3 4825 9 5769
-4 1000 -4 8500 -3 6500 -4 2825 15 6311
A=-0 4500 B= -0 5675
Cov(x1,x1)=6 4228
Cov(x1,x2)=7 9876
Cov(x2,x1)=7 9876
Cov(x2,x2)=9 9528
6 4228 7 9876
Covariance matrix =
7 9876 9 9528
Eigen values
25
After simplifying we get Eigen values as
λ1= 16 36809984
λ2= 0 007462657
Eigen vector
The eigen vectors for the given two dimension data set are obtained by using the formula
( A − λ I) x = 0
If we consider λ =16 36809984
6 4228 7 9876 1 0
( A − λ I) = - ( 16 36809984)
7 9876 9 9528 0 1
−9 9453 7 9876
=
7 9876 −6 4153
( A − λ I) x = 0
−9 9453 7 9876 𝑎
= * =0
7 9876 −6 4153 𝑏
By using row reduction and simplifying we get eigen vector as
0 6262
Eigen vector =
0 7797
If we consider λ = 0 007462657
6 4228 7 9876 1 0
( A − λ I) = - (0 007462657)
7 9876 9 9528 0 1
6 4153 7 9876
=
7 9876 8 9925
( A − λ I) x = 0
6 4153 7 9876 𝑎
= * =0
7 9876 8 9925 𝑏
By using row reduction and simplifying we get eigen vector as
0 7797
Eigen vector=
−0 6262
This is obtained by multiplying centered data matrix to the eigen vector matrix
0 6262 0 7797
Eigen vector matrix =
0 7797 −0 6262
26
X1-X1bar X2-X2bar
1 8500 2 2175
2 0500 2 5425
-0 9500 -1 2075
-1 5500 -1 9575
-2 5500 -3 3825
2 8500 3 6425
1 9500 2 5925
2 7500 3 3175
-2 7500 -3 4825
-3 6500 -4 2825
27
The variance of projections on the line of first principle component is 16 36809775 .
The variance of projections on the line of first principle component is 0 007462657 .
The variances of projections in the line of principle component is equal to the eigen values of
the principle components .First eigen vector is able to explain around 99% of total variance .
28
CHAPTER 3
29
3.SYSTEM ANALYSIS AND DESIGN
Software Requirements
Operating System : windows 7, windows vista, windows xp,windows 8
and higher versions
Language : MATLAB 2013a
Hardware requirements
Ram : 1 GB Ram and more
Processor : Any Intel Processor
HardDisk : 6 GB and more
Speed : 1GHZ and more
30
mathematics, engineering, and science .In industry, MATLAB is the tool of choice for high-
productivity research, development, and analysis.
MATLAB features a family of application-specific solutions called toolboxes .Very important
to most users of MATLAB, toolboxes allow you to learn and apply specialized technology.
Toolboxes are comprehensive collections of MATLAB functions (M-files) that extend the
MATLAB environment to solve particular classes of problems Areas in which toolboxes are
available include signal processing, control systems, neural networks, fuzzy logic, wavelets,
simulation, and many others .
The MATLAB system consists of five main parts:
This is a high-level matrix/array language with control flow statements, functions, data
structures,input/output, and object-oriented programming features.It allows both
"programming in the small" to rapidly create quick and dirty throw-away programs, and
"programming in the large" to create complete large and complex application programs .
This is the set of tools and facilities that you work with as the MATLAB user or programmer.
It includes facilities for managing the variables in your workspace and importing and exporting
data .It also includes tools for developing, managing, debugging, and profiling M-files,
MATLAB's applications.
Handle Graphics
This is the MATLAB graphics system .It includes high-level commands for two-dimensional
and three-dimensional data visualization, image processing, animation, and presentation
graphics .It also includes low-level commands that allow you to fully customize the appearance
of graphics as well as to build complete Graphical User Interfaces on your MATLAB
applications .
This is a vast collection of computational algorithms ranging from elementary functions like
sum, sine, cosine, and complex arithmetic, to more sophisticated functions like matrix inverse,
matrix eigenvalues, Bessel functions, and fast Fourier transforms.
31
The MATLAB Application Program Interface (API)
This is a library that allows you to write C and Fortran programs that interact with MATLAB.
It include facilities for calling routines from MATLAB (dynamic linking), calling MATLAB
as a computational engine, and for reading and writing MAT-files.
The main components in the MATLAB are
Command Window
Command History
Workspace
Current Directory
A data flow diagram (DFD) is a graphical representation of the "flow" of data through an
information system, modeling its process aspects .A DFD is often used as a preliminary step to
create an overview of the system, which can later be elaborated.
Process: A process takes data as input, execute some steps and produce data as output.
External Entity: Objects outside the system being modeled, and interact with
processes in system.
Data Store: Files or storage of data that store data input and output from process.
32
Data Flow: The flow of data from process to process.
33
Figure 8: Level 2 Data Flow Diagram for the process 1 (Naïve Bayes)
34
Figure 9: Level 2 Data Flow Diagram for the process 2 (Bag of Words)
35
Figure 11: Level 2 Data Flow Diagram for the process 3 (Support Vector Machine)
36
CHAPTER 4
IMPLEMENTATION
37
4.IMPLEMENTATION
We used MATLAB Technology for the implementation of “sentiment analysis of mobile
reviews using supervised learning techniques”. There are several stages involved during
implementation of our problem using different supervised learning techniques. Among them
training and testing are the two main phases that are involved .
The given Dataset of positive and negative reviews are passed through the elimination of
special Characters and Conversion to lower case part .This stage eliminates all the special
Characters from the reviews and converts all the upper case to lower case .
The implementation is
while ischar(line)
temp_line1=lower(line);
temp_line2=strrep(temp_line1,' ','');
temp_line3=strrep(temp_line2,',','');
temp_line4=strrep(temp_line3,';','');
temp_line5=strrep(temp_line4,':','');
temp_line6=strrep(temp_line5,'"','');
temp_line7=strrep(temp_line6,'\','');
temp_line8=strrep(temp_line7,')','');
temp_line9=strrep(temp_line8,'(','');
temp_line10=strrep(temp_line9,'?','');
temp_line11=strrep(temp_line10,'!','');
end
In this stage the number of occurences of each distinct word in the data set of both
positive and negative reviews is calculated .These word counts are considered as the values for
the attributes in Naïve Bayes and Bag of Words .
The implementation is
for i=1:Length(DataSet)
Value 1=DataSet(i,1);
38
Count=0;
for j=i+1:Length(DataSet)
Value 2= DataSet(j,1);
If Strcmp(Value 1, Value 2)
Count=count+1;
end
end
for k=1:Length(DataSetCount)
Value 3=DataSetCount(k,1);
if strcmp(Value 1, Value 3)
else
DataSetCount(k,1)=Value 1;
DataSetCount(k,1)=count;
K=k+1;
end
end
end
The Elimination of Special Characters and Conversion to Lower case and Word
Count stages are used in this method .The DataSet obtained from the WordCount is passed to
39
another stage where all the neutral words are eliminated by using Positive and Negative words.
Finally the obtained DataSet is given as a input to the Bag of Words method .
In Testing Phase, the Special characters and upper case letters are eliminated from the test data
. In this method to eliminate the Zero probability problem each word is considered as repeated
once .The prior,conditional and posterior probabilities are calculated using the input data .
Bag of Words
format short g ;
distinctwords={};
positivereviewwords={};
negativereviewwords={};
[~,~,raw]=xlsread('positivereviewwords.xlsx');
positivereviewwords=raw;
distinctwords=positivereviewwords;
po_size=size(positivereviewwords);
po_size1=po_size(1);
display('total no of positive words with no repetition');
display(po_size1);
40
cal_postivewordscount=0;
for lop=1:po_size1
po_value=positivereviewwords(lop,2);
po_value1=cell2mat(po_value);
cal_postivewordscount=cal_postivewordscount+po_value1;
end
display(' total no of positive words with repetition ');
display(cal_postivewordscount);
[~,~,raw1]=xlsread('negativereviewwords.xlsx');
negativereviewwords=raw1;
ne_size=size(negativereviewwords);
ne_size1=ne_size(1);
display('total no of negative words with no repetition');
display(ne_size1);
cal_negativewordscount=0;
for lon=1:ne_size1
ne_value=negativereviewwords(lon,2);
ne_value1=cell2mat(ne_value);
cal_negativewordscount=cal_negativewordscount+ne_value1;
end
display('total no of negative words with repetition ');
display(cal_negativewordscount);
lowt=po_size1+1;
for tyew=1:ne_size1
qwe1= negativereviewwords(tyew,1);
qwe2= negativereviewwords(tyew,2);
distinctwords(lowt,1)=qwe1;
distinctwords(lowt,2)=qwe2;
lowt=lowt+1;
end
distinct_size=size(distinctwords);
41
distinct_size1=distinct_size(1);
distinctwordset={};
gfd=1;
for utr=1:distinct_size1
distinct_word=distinctwords(utr,1);
if utr==1
distinctwordset(gfd,1)=distinct_word;
gfd=gfd+1;
else
distinctwrdset_size=size(distinctwordset);
distinctwrdset_size1=distinctwrdset_size(1);
jfd=0;
for hur=1:distinctwrdset_size1
distinct_word1=distinctwordset(hur,1);
if strcmp(distinct_word1,distinct_word)
jfd=1;
break;
end
end
if jfd==0
distinctwordset(gfd,1)=distinct_word;
gfd=gfd+1;
end
end
end
display('distinct words i e v is');
dwq=size(distinctwordset);
dwq1=dwq(1);
display(dwq1);
[positivereviews,~,~]=xlsread('positivereviewcount.xlsx');
[negativereviews,~,~]=xlsread('negativereviewcount.xlsx');
42
totalreviews=positivereviews+negativereviews;
piy1=positivereviews;
piy2=negativereviews;
positivepriorprobability=piy1/totalreviews;
negativepriorprobability=piy2/totalreviews;
display(totalreviews);
display(positivepriorprobability);
display(negativepriorprobability);
testdatawords={};
mi='';
ts='';
bt=1;
ky=0;
mt=0;
op1=fopen('testfile.txt','r');
file_pointer=fgets(op1);
while ischar(file_pointer)
file_pointer1=strtrim(file_pointer);
line_size=size(file_pointer1);
line_size1=line_size(2);
if line_size1==0
ts='';
mt=0;
end
for ta=1:line_size1
if ta==1
mi='';
ts='';
fed='';
ky=0;
mt=0;
end
fed=file_pointer1(ta);
43
if strcmp(fed,' ')
if strcmp(edr,' ')
ky=ky+1;
else
ts1=cellstr(ts);
testdatawords(bt,1)=ts1;
bt=bt+1;
edr=fed;
ky=ky+1;
mi='';
end
else
ts=strcat(mi,fed);
mi=ts;
edr=fed;
ky=ky+1;
if ky==line_size1
mt=1;
break;
end
end
end
if mt==1
ts1=cellstr(ts);
testdatawords(bt,1)=ts1;
bt=bt+1;
end
file_pointer=fgets(op1);
end
display('test data with repeated words');
display(testdatawords);
positivewordscountvalues={};
poqa=size(testdatawords);
44
poqa1=poqa(1);
hjo=1;
for poty=1:poqa1
testdata_wordvalue=testdatawords(poty,1);
loi=0;
for koty=1:po_size1
testdata_wordvalue1=positivereviewwords(koty,1);
if strcmp(testdata_wordvalue,testdata_wordvalue1)
testdata_word2value=positivereviewwords(koty,2);
testdata_word2value1=cell2mat(testdata_word2value);
povalue=testdata_word2value1+1;
povalue1=num2cell(povalue);
positivewordscountvalues(hjo,1)=testdata_wordvalue;
positivewordscountvalues(hjo,2)=povalue1;
hjo=hjo+1;
loi=1;
break;
end
end
if loi==0
positivewordscountvalues(hjo,1)=testdata_wordvalue;
povalue1=num2cell(1);
positivewordscountvalues(hjo,2)=povalue1;
hjo=hjo+1;
end
end
display('After eliminating zero probability the dataset will be);
display(positivewordscountvalues);
negativewordscountvalues={};
neqa=size(testdatawords);
neqa1=neqa(1);
hjo=1;
for nety=1:neqa1
45
testdata_wordvalue=testdatawords(nety,1);
loi=0;
for koty=1:ne_size1
testdata_wordvalue1=negativereviewwords(koty,1);
if strcmp(testdata_wordvalue,testdata_wordvalue1)
testdata_word2value=negativereviewwords(koty,2);
testdata_word2value1=cell2mat(testdata_word2value);
povalue=testdata_word2value1+1;
povalue1=num2cell(povalue);
negativewordscountvalues(hjo,1)=testdata_wordvalue;
negativewordscountvalues(hjo,2)=povalue1;
hjo=hjo+1;
loi=1;
break;
end
end
if loi==0
negativewordscountvalues(hjo,1)=testdata_wordvalue;
povalue1=num2cell(1);
negativewordscountvalues(hjo,2)=povalue1;
hjo=hjo+1;
end
end
display('after eliminating zero probability the word data set is’);
display(negativewordscountvalues);
negativeconditionalprobability=1;
negativewrdset_size=size(negativewordscountvalues);
divn=cal_negativewordscount+dwq1;
display(divn);
negativewrdset_size1=negativewrdset_size(1);
for ted=1:negativewrdset_size1
nbgc=negativewordscountvalues(ted,2);
nbgc1=cell2mat(nbgc);
46
negativeconditionalprobability=negativeconditionalprobability*(nbgc1/divn);
end
negativeposteriorprobability=negativepriorprobability*negativeconditionalprobability;
display(‘posterior probability for negative class is’);
display(negativeposteriorprobability);
positiveconditionalprobability=1;
positivewrdset_size=size(positivewordscountvalues);
divp=cal_postivewordscount+dwq1;
display(divp);
positivewrdset_size1=positivewrdset_size(1);
for ted1=1:positivewrdset_size1
pbgc=positivewordscountvalues(ted1,2);
pbgc1=cell2mat(pbgc);
positiveconditionalprobability=positiveconditionalprobability*(pbgc1/divp);
end
positiveposteriorprobability=positivepriorprobability*positiveconditionalprobability;
display(‘posterior probability for positive class is’);
display(positiveposteriorprobability);
if positiveposteriorprobability>negativeposteriorprobability
display('It is a positive review');
elseif negativeposteriorprobability>positiveposteriorprobability
display('It is a negative review');
else
display(' neutral ');
end
47
CHAPTER 5
TESTING
48
5.TESTING
Testing is the process of evaluating a system or its component’s with the intent to find that
whether it satisfies the specified requirements or not .This activity results in the actual,
expected and difference between their results i.e testing is executing a system in order to
identify any gaps, errors or missing requirements in contrary to the actual desire or
requirements.
5.1 Testing Strategies
In order to make sure that system does not have any errors, the different levels of testing
strategies that are applied at different phases of software development are
The goal of unit testing is to isolate each part of the program and show that individual parts
are correct in terms of requirements and functionality.
The testing of combined parts of an application to determine if they function correctly together
is Integration testing .This testing can be done by using two different methods
49
In Top-Down integration testing, the highest-level modules are tested first and then
progressively lower-level modules are tested.
Testing can be performed starting from smallest and lowest level modules and proceeding one
at a time .When bottom level modules are tested attention turns to those on the next level that
use the lower level ones they are tested individually and then linked with the previously
examined lower level modules.In a comprehensive software development environment,
bottom-up testing is usually done first, followed by top-down testing.
This is the next level in the testing and tests the system as a whole .Once all the components
are integrated, the application as a whole is tested rigorously to see that it meets Quality
Standards.
The main purpose of this Testing is to find whether application meets the intended
specifications and satisfies the client’s requirements .We will follow two different methods in
this testing.
This test is the first stage of testing and will be performed amongst the teams .Unit testing,
integration testing and system testing when combined are known as alpha testing. During this
phase, the following will be tested in the application:
Spelling Mistakes.
Broken Links.
The Application will be tested on machines with the lowest specification to test loading
times and any latency problems.
In beta testing, a sample of the intended audience tests the application and send their feedback
to the project team .Getting the feedback, the project team can fix the problems before releasing
the software to the actual users.
50
5.2 Testing Methods
White box testing is the detailed investigation of internal logic and structure of the Code. To
perform white box testing on an application, the tester needs to possess knowledge of the
internal working of the code .The tester needs to have a look inside the source code and find
out which unit/chunk of the code is behaving inappropriately.
The technique of testing without having any knowledge of the interior workings of the
application is Black Box testing .The tester is oblivious to the system architecture and does not
have access to the source code.Typically, when performing a black box test, a tester will
interact with the system’s user interface by providing inputs and examining outputs without
knowing how and where the inputs are worked upon.
5.3 Validation
All the levels in the testing (unit,integration,system) and methods (black box,white box)are
implemented on our application successfully and the results obtained as expected .
5.4 Limitations
The execution time for support vector machine is more so that the user may not receive the
result fast.
5 5 Test Results
The testing is done among the team members and by the end users. It satisfies the specified
requirements and finally we obtained the results as expected.
51
CHAPTER 6
SCREEN SHOTS
52
6.SCREEN SHOTS
53
Figure 13 : Naïve Bayes Screen with Test Data
54
Figure 14 : Naïve Bayes Screen with output
55
6.2 Bag of words
56
Figure 16 : Bag Of Words screen with Test Data
57
Figure 17 : Bag Of Words screen with output
58
6.3 Support Vector Machine using Principle Component Analysis
59
Figure 19 : Support Vector Machine screen with Test Data
60
Figure 20 : Support Vector Machine Screen with Output
61
CHAPTER 7
CONCLUSION
62
7.CONCLUSION
We considered three statistical models for solving our problem Support Vector Machine,Bag
Of Words and Naive Bayes.
The first method that we approached for our problem is Naïve Bayes. It is mainly based on the
independence assumption .Training is very easy and fast In this approach each attribute in each
class is considered separately.Testing is straightforward, calculating the conditional
probabilities from the data available . One of the major task is to find the sentiment polarities
which is very important in this approach to obtain desired output . In this naïve bayes approach
we only considered the words that are available in our dataset and calculated their conditional
probabilities . we have obtained successful results after applying this approach to our problem.
In supervised learning methods next we adopted bag of words .This approach assumes that
every single word in the test data is repeated atleast once, which eliminates the zero probability
problem . After applying this approach, the results are obtained correctly and their execution is
also very fast .The third method that we applied for our problem is support vector machine
along with principal component analysis .The main reason for using principal component
analysis is because of its dimensionality reduction .It reduces the large dimensions into smaller
without any loss of data . A window (comprised of five words on either side of the given word)
is used to count the number of appearances of each word in our data set ,to find the
combinations of words .Training is very long compared to naïve bayes, bag of words .The
training of support vector machine is done only with a small dataset. The outcomes of this
approach are obtained partially .
However, we were successful at predicting sentiment on topics in mobile reviews on a small
scale using three different approaches Naïve Bayes, Bag Of Words, Support Vector Machine
and also gained a lot of information in machine learning.
63
REFERENCES
[1] Chih-Wei Hsu, Chih-Chung Chang, Chih-Jen Lin,” A Practical Guide to Support Vector
Classification”, http://www csie ntu edu tw ,web, July 22 2014 .
[2] N Cristianini, J Shawe-Taylor, “An Introduction to Support Vector Machines and Other
Kernel-based Learning Methods”, Cambridge University Press, 2000 .
[3] Hiroshi Shimodaira,“Text classifying using Naïve Bayes ”, Document models, http://www
inf ed ac uk/teaching/courses/inf2b/learnnotes/inf2b-learn-note07-2up,11 Feb 2014,web, 15
August 2014 .
[4]Lindsay I Smith,“Principle Component Analysis”, http://www cs otago ac
nz/cosc453/student_tutorials/principal_components, 2002,web,August 5 2014 .
[5] Laura Auria , Rouslan A Moro , “Support Vector Machines ” , 2008,web,20 August 2014.
[6] H Kim, P Howland, and H Park “Dimension reduction in text classification with support
vector machines”, Journal of Machine Learning Research,2005 .
[7]Pang- Ning Tan ,Michael Steinbach,Vipin Kumar ,”Introduction to Data Mining”, pearson
publications.
[8] Dan Jurafsky “Text Classification and Naïve Bayes”, The Task Of Text Classification,
https://web stanford edu/class/cs124/lec/naivebayes ,web, July 28 2014 .
64