Introduction to Automata

Theory
Reading: Chapter 1

1
 Material / Resources
 Text Books
1. John Hopcroft and Jeffrey Ullman, Introduction to
Automata Theory, Languages and Computation, 3Ed,
2000, Addison-Wesley.
2. Automata, Computability and Complexity: Theory and
Applications, Elaine A. Rich Prentice Hall; 1Ed, 2011.
3. Introduction to Computer Theory, by Daniel I. Cohen,
John Wiley and Sons, Inc., 2005, 2Ed.
4. Introduction to Languages and Theory of
Computation, by J. C. Martin, McGraw Hill Book Co.,
2003, 3Ed.
 Any other good book on Automata Theory
 Course Outlines
 Introduction to Formal Proof and Inductive Proofs. The Central Concepts of Automata Theory.
Overview of Formal Languages. Finite Automata. Transition Graphs. Deterministic & Nondeterministic
Finite Automata. Finite Automata With Epsilon Transitions. Conversion from NFA to DFA. Kleene’s
Theorem. Regular Expression and Languages. Finite Automata and Regular Expressions. Applications
of Regular Expressions. Algebraic Laws for Regular Expressions. Properties of Regular Languages.
Proving Languages Not to Be Regular (Pumping Lemma). Closure Properties of Regular Languages.
Decision Properties of Regular Languages. Finite State Machines with Output, Transducers (Moore and
Mealy machines). Context-Free Grammars and Languages. Parse Trees. Applications of Context-Free
Grammars. Ambiguity in Grammars and Languages. Properties of Context-Free Languages. Normal
Forms for Context-Free Grammars. Closure Properties of Context-Free Languages. Chomsky Normal
Form. Pushdown Automata (PDA). Definition of the Pushdown Automaton. The Languages of a PDA,
Equivalence of PDAs and CFGs. Introduction to Turing Machines. The Extension of Turing Machine.
Programming Techniques for Turing Machines Post Turing Machine. Variations on Turing Machine.
Turing Machine Encoding. Universal Turing Machine Defining Computers by Turing Machines.

3
 Mathematics of Theory of
Automata
 The mathematics employed in the theory of computation is rather
different from the mathematics used in other applied disciplines.
 It is generally discrete, in that the emphasis is not on real
numbers and continuous variables, but on finite sets and
sequences.
 It is based on very few and elementary concepts, and draws its
power and depth from the careful, extensive, layer-by-layer
manipulation of these concepts-just like the computer.

4
 What is Automata Theory?
 It is the plural of automation, and it means
“something that works automatically”.
 Automata theory is the study of abstract
computational devices and the computational
problems that can be solved using them.
 Abstract devices are (simplified) models of real
computations.
 In automata we will simulates parts of
computers.
5
 Helps in design and construction of different
software's and what we can expect from our
software's.
 Automata play a major role in theory of
computation, compiler design, artificial
intelligence.

6
Abstract Machine
 A procedure for executing a set of
instructions in some formal language
 It is not intended to be constructed as
hardware but are is used in thought
experiments about computability
 E.g. Finite State Machine, Turing
Machine

7
Components of Finite State
Machine

8
(A pioneer of automata theory)

Alan Turing (1912-1954)

 Father of Modern Computer
Science

 English mathematician

 Studied abstract machines called

Turing machines even before
computers existed

 Head of the Turing test?

9
Theory of Computation: A
Historical Perspective
1930s • Alan Turing studies Turing machines
• Decidability

• Halting problem

1940-1950s •“Finite automata” machines studied

• Noam Chomsky proposes the

“Chomsky Hierarchy” for formal

languages
1969 Cook introduces “intractable” problems
or “NP-Hard” problems
1970- Modern computer science: compilers,
computational & complexity theory evolve
10
Applications of the Theory
 Languages, enable both machine/machine and
person/machine communication. Without them, none of
today’s applications of computing could exist.
 Examples:
 Network communication protocols are languages.
 Most web pages are described using the Hypertext Markup
Language, HTML.
 The Semantic Web, whose goal is to support intelligent agents
working on the Web, exploits additional layers of languages, such
as RDF and OWL, that can be used to describe the content of the
Web.

11
Programming Languages
Dependency
 Both the design and the implementation of
modern programming languages rely heavily
on the theory of context-free languages.
 Context-free grammars are used to document
the languages‘ syntax and they form the
basis for the parsing techniques that all
compilers use.

12
 Languages & Grammars
 Languages: “A language is a
Or “words” collection of sentences of
finite length all constructed
from a finite alphabet of
symbols”
 Grammars: “A grammar can
be regarded as a device that
enumerates the sentences
of a language” - nothing
more, nothing less

 N. Chomsky, Information and

Control, Vol 2, 1959

Image source: Nowak et al. Nature, vol 417, 2002

13
The Chomsky Hierachy
• A containment hierarchy of classes of formal languages

Regular Context-
(DFA) Context- Recursively-
free
sensitive enumerable
(PDA)
(LBA) (TM)

14
Alphabet
An alphabet is a finite, non-empty set of symbols
 We use the symbol ∑ (sigma) to denote an

alphabet
 Examples:
 Binary: ∑ = {0,1}
 All lower case letters: ∑ = {a,b,c,..z}
 Alphanumeric: ∑ = {a-z, A-Z, 0-9}
 DNA molecule letters: ∑ = {a,c,g,t}
 …

15
Valid/In-valid alphabets
 While defining an alphabet, an alphabet may contain letters
consisting of group of symbols for example Σ1= {B, aB, bab,
d}.
 The BababB (string) can be tokenized in two different ways for
Σ2= {B, Ba, bab, d}:
 (Ba), (bab), (B)
 (B), (abab), (B)

 As when this string is scanned by the compiler (Lexical

Analyzer), first symbol B is identified as a letter belonging to Σ,
while for the second letter the lexical analyzer would not be able
to identify, so while defining an alphabet it should be kept in
mind that ambiguity should not be created.
Remarks:
 While defining an alphabet of letters
consisting of more than one symbols, no
letter should be started with the letter of the
same alphabet i.e. one letter should not be
the prefix of another. However, a letter may
be ended in the letter of same alphabet i.e.
one letter may be the suffix of another.
Conclusion
 Σ1= {B, aB, bab, d}
 Σ2= {B, Ba, bab, d}

 Σ1 is a valid alphabet while Σ2 is an in-

valid alphabet.
Strings
A string or word is a finite sequence of symbols
chosen from ∑
 Empty string is  (or “epsilon”) or (Small Greek
letter Lambda) λ or (Capital Greek letter Lambda)
Λ.
 Length of a string w, denoted by “|w|”, is equal
to the number of (non- ) characters in the string
 E.g., x = 010100 |x| = 6
 x = 01  0  1  00  |x| = ?
 xy = concatenation of two strings x and y

19
Powers of an alphabet
Let ∑ be an alphabet.

 ∑k = the set of all strings of length k

 ∑* = ∑0 U ∑1 U ∑2 U …

 ∑ + = ∑ 1 U ∑ 2 U ∑3 U …

20
 Defining Languages
 The languages can be defined in different ways ,
such as
1. Descriptive definition,
2. Recursive definition,
3. using Regular Expressions(RE) and
4. using Finite Automation(FA) etc.

 Descriptive definition of language:

The language is defined, describing the
conditions imposed on its words.
Languages
L is a said to be a language over alphabet ∑, only if L  ∑*
 this is because ∑* is the set of all strings (of all possible length
including 0) over the given alphabet ∑
Examples:
1. Let L be the language of all strings consisting of n 0’s followed by n
1’s:
L = {,01,0011,000111,…}
2. Let L be the language of all strings of with equal number of 0’s and
1’s:
L = {,01,10,0011,1100,0101,1010,1001,…}

Definition: Ø denotes the Empty language

 Let L = {}; Is L=Ø?

NO
22
The Membership Problem
Given a string w ∑*and a language L
over ∑, decide whether or not w L.

Example:
Let w = 100011
Q) Is w  the language of strings with
equal number of 0s and 1s?

23
 Finite State Automata in
Modeling
 Systems as diverse as
 Parity checkers,
 Vending machines,

 Communication protocols,

 Interactive video games,

 Building security devices

Can be straightforwardly described as finite state

machines.

24
 Finite Automata : Examples
action
 On/Off switch
H
state
ITC
SW

BATTERY

 Modeling recognition of the word “then”

Start state Transition Intermediate Final state

state
25
 Example
1 1
start off off
1
BATTERY
2 2 2 2
2 1
off on
1
inputs: switches 1 and 2
actions: 1 for “flip switch 1” bulb is on if and only if
actions: 2 for “flip switch 2” both switches were flipped
states: on, off an odd number of times
26
Structural expressions
 Grammars
 Regular expressions
 E.g., unix style to capture city names such
as “Palo Alto CA”:
 [A-Z][a-z]*([ ][A-Z][a-z]*)*[ ][A-Z][A-Z]

Start with a letter

A string of other
letters (possibly Should end w/ 2-letter state code
empty)

Other space delimited words

(part of city name) 27
Deductive Proofs
From the given statement(s) to a conclusion
statement (what we want to prove)
 Logical progression by direct implications

Example for parsing a statement:

 “If y≥4, then 2y≥y2.”

given conclusion
(there are other ways of writing this).

28
 Example: Deductive proof
Let Claim 1: If y≥4, then 2y≥y2.

Let x be any number which is obtained by adding the squares of

4 positive integers.

Given x and assuming that Claim 1 is true, prove that 2 x≥x2

 Proof:
1) Given: x = a2 + b2 + c2 + d2
2) Given: a≥1, b≥1, c≥1, d≥1
3) ⇒ a2≥1, b2≥1, c2≥1, d2≥1 (by 2)
4) ⇒ x ≥ 4 (x ≥ 12+12 +12 +12) (by 1 & 3)
5) ⇒ 2x ≥ x 2 (by 4 and Claim 1)
“implies” or “follows”
29
Quantifiers
“For all” or “For every”
 Universal proofs
 Notation = Ɐ
“There exists”
 Used in existential proofs

Notation = ⱻ
Implication is denoted by ⇒
 E.g., “IF A THEN B” can also be written as “A⇒B”

30
Proving techniques
 By contradiction
 Start with the statement contradictory to the given statement
 E.g., To prove (A => B), we start with:
 (A and ~B)
 … and then show that could never happen

What if you want to prove that “(A and B => C or D)”?

 By induction
 (3 steps) Basis, inductive hypothesis, inductive step
 By contrapositive statement
 If A then B ≡ If ~B then ~A

31
Proving techniques…
 By counter-example
 Show an example that disproves the claim

 Note: There is no such thing called a

“proof by example”!
 So when asked to prove a claim, an example that

satisfied that claim is not a proof

32
Different ways of saying the same
thing
 “If H then C”:
i. H implies C
ii. H => C
iii. C if H
iv. H only if C
v. Whenever H holds, C follows

33
“If-and-Only-If” statements
 “A if and only if B” (A <==> B)
 (if part) if B then A ( <= )
 (only if part) A only if B ( => )
(same as “if A then B”)
 “If and only if” is abbreviated as “iff”
 i.e., “A iff B”
 Example:
 Theorem: Let x be a real number. Then floor of x = ceiling
of x if and only if x is an integer.
 Proofs for iff have two parts
 One for the “if part” & another for the “only if part”

34
Summary
 Automata theory & a historical perspective
 Chomsky hierarchy
 Finite automata
 Alphabets, strings/words/sentences, languages
 Membership problem
 Proofs:
 Deductive, induction, contrapositive, contradiction,

counterexample
 If and only if

 Read chapter 1 for more examples and exercises

 Gradiance homework 1

35