AI Unit-3 Notes
AI Unit-3 Notes
Knowledge Representation & Reasoning: Propositional logic, Theory of first order logic,
Inference in First order logic, Forward & Backward chaining, Resolution, Probabilistic
reasoning, Utility theory, Hidden Markov Models (HMM), Bayesian Networks.
Characteristics
Primitives. What is the underlying framework used to represent knowledge?
Semantic networks were one of the first knowledge representation primitives.
Meta-Representation. This is also known as the issue of reflection in computer
science. It refers to the capability of a formalism to have access to information
about its own state.
Incompleteness. Traditional logic requires additional axioms and constraints to
deal with the real world as opposed to the world of mathematics.
Definitions and Universals vs. facts and defaults. Universals are general
statements about the world such as "All humans are mortal". Facts are specific
examples of universals such as "Socrates is a human and therefore mortal". In
logical terms definitions and universals are about universal quantification while
facts and defaults are about existential quantifications.
Non-Monotonic reasoning. Non-monotonic reasoning allows various kinds of
hypothetical reasoning. The system associates facts asserted with the rules and
facts used to justify them and as those facts change updates the dependent
knowledge as well.
Expressive Adequacy. Researchers should be clear about how expressive (how
much of full FOL expressive power) they intend their representation to be.
Reasoning Efficiency. This refers to the run time efficiency of the system. The
ability of the knowledge base to be updated and the reasoner to develop new
inferences in a reasonable period of time.
Sometimes, a statement can contain one or more other statements as parts. Consider for example,
the following statement:
While the above compound sentence is itself a statement, because it is true, the two parts,
"Ganymede is a moon of Jupiter" and "Ganymede is a moon of Saturn", are themselves
statements, because the first is true and the second is false.
Propositional logic, also known as sentential logic, is that branch of logic that studies ways of
combining or altering statements or propositions to form more complicated statements or
propositions. Joining two simpler propositions with the word "and" is one common way of
combining statements. When two statements are joined together with "and", the complex
statement formed by them is true if and only if both the component statements are true. Because
of this, an argument of the following form is logically valid:
Paris is the capital of France and Paris has a population of over two million.
Therefore, Paris has a population of over two million.
Syntax
A proposition or statement is a sentence which is either true or false. The syntax of
propositional logic defines the allowable sentences. The atomic sentences consist of a single
proposition symbol. Each such symbol stands for a proposition that can be true or false. We use
symbols that start with an uppercase letter and may contain other letters or subscripts; for
example: P, Q, R and North. There are two proposition symbols with fixed meanings: True is the
always-true proposition and False is the always-false proposition. Complex sentences are
constructed from simpler sentences, using parentheses and logical connectives. There are five
connectives in common use:
Negation (Not):- A sentence such as P is called the negation of P.
Conjunction (AND):- P Q is the conjunction of P and Q
Disjunction (OR):- P Q is the conjunction of P and Q.
Implication (Conditional):- For propositions P and Q, the implication or
conditional statement PQ is false when P is true and Q is false, and is true
otherwise. P is called the premise or hypothesis, and Q is called the conclusion.
5. Biconditional (if and only if): For propositions P and Q, the Biconditional
statement PQ is true whenever both parts have the same truth value. P is called
the premise or hypothesis, and Q is called the conclusion.
1.
2.
3.
4.
F
F
T
F
T
T
T
F
F
T
T
F
PQ
F
F
F
T
PQ
F
T
T
T
PQ
T
T
F
T
PQ
T
F
F
T
p: A polygon is a triangle.
q: A polygon has exactly 3 sides.
q)
(q
p q p
p (p
TTT
TF F
F TT
F F T
q)
(q
p)
In the truth table above, when p and q have the same truth values, the compound statement
(p
q) (q
p) is true. When we combine two conditional statements this way, we have
a biconditional.
Definition: A biconditional statement is defined to be true whenever both parts have the
same truth value. The biconditional operator is denoted by a double-headed
arrow
. The biconditional p
q represents "p if and only if q," where p is
a hypothesis and q is a conclusion. The following is a truth table for
biconditional p
q.
Example 1:
Given:
p: A polygon is a triangle.
q: A polygon has exactly 3 sides.
q represent?
sides."
Note that in the biconditional above, the hypothesis is: "A polygon is a triangle" and the
conclusion is: "It has exactly 3 sides." It is helpful to think of the biconditional as a
conditional statement that is true in both directions.
and
and
Validity: - A sentence is valid if it is true in all models. For example, the sentence P V is valid.
Valid sentences are also known as tautologies.
P
F
T
P V
T
T
F
T
Satisfiability:- A sentence is satisfiable if it is true in some model. If a sentence is true in a
model m, then we say that m satisfies , or that m is a model of . Determining the
satisfiability of sentences in propositional logic was the first problem proved to be NP-complete.
Many problems in computer science are really satisfiability problems. For example, all the
constraint satisfaction problems are essentially asking whether the constraints are satisfiable by
some assignment.
Unsatifiabality (Contradiction):- A sentence is unsatisfiable if it is false in all models. For example,
the sentence P is unsatisfiable.
P
P
F
T
T
F
F
F
Exercise
1) Determine whether the following argument is valid.
If l work whole night on this problem, then l can solve it. If I solve the problem'
then I will understand the topic. Therefore, I will work whole night on this
problem, and then I will understand the topic
Let
P= l work whole night on this problem
Q= l can solve it
R= I will understand the topic
PQ
QR
(P Q) P R
((P Q) (Q R))
(Q R)
(P R)
topic
So result: ((P Q) (Q R)) ( R P)
PQ
QR
(P Q) ((P Q) (Q R))
(Q R)
( R P)
(P Q) P
((P Q)
( S
(RS)) (
(RS)
PS)
F
1. P
2. Q
3. Therefore, P and Q.
1. It is raining in New York.
2. It is raining in Boston
3. Therefore, it is raining in both New York and Boston
Simplification
1. P and Q.
2. Therefore, P.
1. It is raining in both New York and Boston.
2. Therefore, it is raining in New York.
Addition
1. P
2. Therefore, P or Q.
1. It is raining
2. Therefore, either either it is raining or the sun is shining.
Absorption
1. If P, then Q.
2. Therfore, If P then P and Q.
1. If it is raining, then I will get wet.
2. Therefore, if it is raining, then it is raining and I will get wet.
Modus Ponens
1. If P then Q.
2. P.
3. Therefore, Q.
1. If it is raining, then I will get wet.
2. It is raining.
3. Therefore, I will get wet.
Modus Tollens
1. If P then Q.
2. Not Q. (~Q).
3. Therefore, not P (~P).
1. If it had rained this morning, I would have gotten wet.
2. I did not get wet.
3. Therefore, it did not rain this morning.
Hypothetical Syllogism
1. If P then Q.
2. If Q then R.
3. Therefore, if P then R.
1. If it rains, then I will get wet.
2. If I get wet, then my shirt will be ruined.
3. If it rains, then my shirt will be ruined.
Disjunctive Syllogism
1. Either P or Q.
2. Not P (~P).
3. Therefore, Q.
1. Either it rained or I took a cab to the movies.
2. It did not rain.
3. Therefore, I took a cab to the movies.
Constructive Dilemma
1. (If P then Q) and (If R then S).
2. P or R.
3. Therefore, Q or S.
1. If it rains, then I will get wet and if it is sunny, then I will be dry.
2. Either it will rain or it will be sunny.
3. Therefore, either I will get wet or I will be dry.
The above rules of inference, when combined with the rules of replacement, mean that
propositional calculus is "complete." Propositional calculus is simply another name for formal
logic.
Resolution
Resolution is a rule of inference leading to a refutation theorem-proving technique for sentences
in propositional logic and first-order logic. In other words, iteratively applying the resolution rule
in a suitable way allows for telling whether a propositional formula is satisfiable and for proving
that a first-order formula is unsatisfiable; this method may prove the satisfiability of a first-order
satisfiable formula, but not always, as it is the case for all methods for first-order logic.
Resolution was introduced by John Alan Robinson in 1965.
Resolution in propositional logic
The resolution rule in propositional logic is a single valid inference rule that produces a new
clause implied by two clauses containing complementary literals. A literal is a propositional
variable or the negation of a propositional variable. Two literals are said to be complements if
one is the negation of the other (in the following, ai is taken to be the complement to bj). The
resulting clause contains all the literals that do not have complements. Formally:
where
all as and bs are literals,
ai is the complement to bj, and
the dividing line stands for entails
The clause produced by the resolution rule is called the resolvent of the two input clauses.
When the two clauses contain more than one pair of complementary literals, the resolution rule
can be applied (independently) for each such pair. However, only the pair of literals that are
resolved upon can be removed: all other pair of literals remain in the resolvent clause.
A resolution technique
When coupled with a complete search algorithm, the resolution rule yields a sound and complete
algorithm for deciding the satisfiability of a propositional formula, and, by extension, the validity
of a sentence under a set of axioms.
This resolution technique uses proof by contradiction and is based on the fact that any sentence
in propositional logic can be transformed into an equivalent sentence in conjunctive normal
form. The steps are as follows:
1).All sentences in the knowledge base and the negation of the sentence to be proved (the
conjecture) are conjunctively connected.
2).The resulting sentence is transformed into a conjunctive normal form with the conjuncts
viewed as elements in a set, S, of clauses.
For example
3).The resolution rule is applied to all possible pairs of clauses that contain complementary
literals. After each application of the resolution rule, the resulting sentence is simplified by
removing repeated literals. If the sentence contains complementary literals, it is discarded (as a
tautology). If not, and if it is not yet present in the clause set S, it is added to S, and is considered
for further resolution inferences.
4).If after applying a resolution rule the empty clause is derived, the complete formula is
unsatisfiable (or contradictory), and hence it can be concluded that the initial conjecture follows
from the axioms.
5).If, on the other hand, the empty clause cannot be derived, and the resolution rule cannot be
applied to derive any more new clauses, the conjecture is not a theorem of the original
knowledge base.
One instance of this algorithm is the original DavisPutnam algorithm that was later refined into
the DPLL algorithm that removed the need for explicit representation of the resolvents.
This description of the resolution technique uses a set S as the underlying data-structure to
represent resolution derivations. Lists, Trees and Directed Acyclic Graphs are other possible and
common alternatives. Tree representations are more faithful to the fact that the resolution rule is
binary. Together with a sequent notation for clauses, a tree representation also makes it clear to
see how the resolution rule is related to a special case of the cut-rule, restricted to atomic cutformulas. However, tree representations are not as compact as set or list representations, because
they explicitly show redundant subderivations of clauses that are used more than once in the
derivation of the empty clause. Graph representations can be as compact in the number of clauses
as list representations and they also store structural information regarding which clauses were
resolved to derive each resolvent.
Example
If a is true, then for the second premise to hold, c must be true. If a is false, then for the first
premise to hold, b must be true.
So regardless of a, if both premises hold, then b or c is true.
Bayesian network
Example
You have a new burglar alarm installed at home. It is fairly reliable at detecting a burglary, but
also responds on occasion to minor earthquakes. (This example is due to Judea Pearl, a resident
of Los Angeles-hence the acute interest in earthquakes.) You also have two neighbors, John and
Mary, who have promised to call you at work when they hear the alarm. John always calls when
he hears the alarm, but sometimes confuses the telephone ringing with the alarm and calls then,
too. Mary, on the other hand, likes rather loud music and sometimes misses the alarm altogether.
Given the evidence of who has or has not called, we would like to estimate the probability of a
burglary.
In the figure, each distribution is shown as a conditional probability table, or CPT. Each row in
a CPT contains the conditional probability of each node value for a conditioning case. A
conditioning case is just a possible combination of values for the parent nodes-a miniature
atomic event, if you like. Each row must sum to 1, because the entries represent an exhaustive set
of cases for the variable.
This identity holds true for any set of random variables and is called the chain rule. Comparing
it with Equation (14.1), we see that the specification of the joint distribution is equivalent to the
general assertion that, for every variable Xi in the network,
generated by an HMM gives some information about the sequence of states. Note that the
adjective 'hidden' refers to the state sequence through which the model passes, not to the
parameters of the model; the model is still referred to as a 'hidden' Markov model even if these
parameters are known exactly.
Hidden Markov models are especially known for their application in temporal pattern
recognition such as speech, handwriting, gesture recognition, part-of-speech tagging, musical
score following, partial discharges and bioinformatics.
Example
Coin Toss Models: Assume the following scenario. You are in a room with a barrier through
which you cannot see what is happening. On the other side of the barrier is another person who is
performing a coin (or multiple coin) tossing experiment. The other person will not tell you
anything about what he is doing exactly, he will only tell you the result of each coin flip. Thus a
sequence of hidden coin tossing experiments is performed, with the observation sequence
consisting of a series of heads and tails, for e.g., a typical observation sequence would be
O = O1 O2 O3 OT (Head Tail Head Tail)
to model this coin tossing event with HMM, the fundamental question is
What the state in model correspond to
How many state in the model should there be