Cs6402 DAA Notes (Unit-3)
Cs6402 DAA Notes (Unit-3)
Dept of CSE
RMD Engineering
College
Dynamic Programming
Computing a Binomial Coefficient
Warshalls algorithm
Floyd algorithm
Optimal Binary Search Trees
Knapsack Problem & Memory functions
Greedy Technique
Prims algorithm
Kruskal's Algorithm
Dijkstra's Algorithm
Huffman Trees
DYNAMIC PROGRAMMING
Disadvantage of divide and conquer technique:
One disadvantage of using Divide-and-Conquer is that the process of recursively solving separate subinstances can result in the same computations being performed repeatedly since identical sub-instances may arise.
What is dynamic programming?
Dynamic programming is a technique for solving problems with overlapping sub problems.
Main idea:
set up a recurrence relating a solution to a larger instance to solutions of some smaller instances
solve smaller instances once
record solutions in a table
extract solution to the initial instance from that table
Example:
The Fibonacci numbers are the elements of the sequence: 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, . . . ,
which can be defined by the simple recurrence
F(n) = F(n 1) + F(n 2) for n > 1
F(4)
F(3)
This F(2) is saved
F(2) F(1)
F(1)
F(2)
F(1)
F(0)
Page 2
The value of the binomial coefficient for nonnegative and is given explicitly by
Page 3
Analysis:
Example:
Value of C(5,3) is computed as follows
2.WARSHALLS ALGORITHM
Page 4
Transitive closure
Adjacencymatrix
the adjacency matrix A = {aij} of a directed graph is the boolean matrix that has 1 in its ith row and jth
column if and only if there is a directed edge from the ith vertex to the jth vertex.
Constructs transitive closure T as the last matrix in the sequence of n-by-n matrices R(0), ,R(k),
,R whereR(k)[i,j] = 1 iff there is nontrivial path from i to j with only first k vertices allowed as intermediate
Note that R(0) = A (adjacency matrix),R(n)= T (transitive closure)
(n)
Page 5
Example:
Page 6
3. FLOYD ALGORITHM
Computes the shortest path between all pair of vertices in weighted diagraph
Example:
Weighted matrix
the weighted matrix W = {wij} of a directed graph is that has +ve weight in its ith row and jth column if and
only if there is a directed edge from the ith vertex to the jth vertex else
Distancematrix
The Distance matrix D = {dij} ,wherethe element dij in the ith row and the jth column of this matrix
indicates the length of the shortest path from the ith vertex to the jth vertex.
Constructs Distancematrix D as the last matrix in the sequence of n-by-n matrices D0), ,D(k),
,D(n)whereD(k)[i,j] gives shortest path from i to j with only first k vertices allowed as intermediate
Note that D(0) = A (weighted matrix),D(n)= D(all pair shortest path matrix)
Recurrence
Page 7
Page 8
Page 9
The average number of comparisons in a successful search in the first of these trees is 0.1 . 1+ 0.2 . 2 + 0.4 .
3+ 0.3 . 4 = 2.9, and for the second one it is 0.1 . 2 + 0.2 . 1+ 0.4 . 2 + 0.3 . 3= 2.1. Neither of these two trees is, in
fact, optimal. Instead of trying all 14 possible BSTs, OBST algorithms computes the single optimal BST
Let C[i,j] be minimum average number of comparisons made in T[i,j], optimal BST for keys ai<
<aj,where 1 i j n. Consider optimal BST among all BSTs with some ak (i k j) as their root; T[i,j] is the best
among them.
Hence,
Page 10
B
0.1
C
0.2
D
0.4
0.3
The tables below are filled diagonal by diagonal: the left one is filled using the recurrence
C[i,j] = min {C[i,k-1] + C[k+1,j]} + ps ,
C[i,i] = pi ;
the right one, for trees roots, records ks values giving the minima
Page 11
Thus, the average number of key comparisons in the optimal tree is equal to1.7.
Page 12
5.KNAPSACK PROBLEM
Definition
Given n items of
integer weights: w1 w2 wn
values:
v1 v2 vn
a knapsack of integer capacity W
find most valuable subset of the items that fit into the knapsack
problem instance defined by first i items and capacity j (j W).
Let V[i,j] be optimal value of such instance. Then
Page 13
Example:
Solution
Page 14
V[4,5]=37
How to find the subset of items?
We can find the composition of an optimal subset by backtracing the computations of this entry in the table. Since
F(4, 5) > F(3, 5), item 4 has to be included in an optimal solution along with an optimal subset for filling 5 2 = 3
remaining units of the knapsack capacity. The value of the latter is F(3, 3). Since F(3, 3) = F(2, 3), item 3 need not
be in an optimal subset. Since F(2, 3) > F(1, 3), item 2 is a part of an optimal selection, which leaves element F(1, 3
1) to specify its remaining composition. Similarly, since F(1, 2) > F(0, 2), item 1 is the final part of the optimal
solution {item 1, item 2, item 4}.
Complexity:
The time efficiency and space efficiency of this algorithm are both in (nW).
The time needed to find the composition of an optimal solution is inO(n).
Memory Functions
The direct top-down approach to finding a solution to such a recurrence leads to an algorithm that solves common
subproblems more than once and hence is very inefficient (typically, exponential or worse). The classic dynamic
programming approach, on the other hand, works bottom up: it fills a table with solutions to all smaller
subproblems, but each of them is solved only once. An unsatisfying aspect of this approach is that solutions to some
of these smaller subproblems are often not necessary for getting a solution to the problem given. Since this
drawback is not present in the top-down approach, it is natural to try to combine the strengths of the top-down and
bottom-up approaches. The goal is to get a method that solves only subproblems that are necessary and does so only
once. Such a method exists; it is based on using memory functions.
This method solves a given problem in the top-down manner but, in addition, maintains a table of the kind that
would have been used by a bottom-up dynamic programming algorithm. Initially, all the tables entries are
initialized with a special null symbol to indicate that they have not yet been calculated. Thereafter, whenever a
Page 15
Let us apply the memory function method to the instance considered in previous Example. The table in Figure
gives the results. Only 11 out of 20 nontrivial values (i.e., not those in row 0 or in column 0) have been computed
Just one nontrivial entry, V (1, 2), is retrieved rather than being recomputed. For larger instances, the proportion of
such entries can be significantly larger.
In general, we cannot expect more than a constant-factor gain in using the memory function method for the
knapsack problem, because its time efficiency class is the same as that of the bottom-up algorithm.
Page 16
GREEDY TECHNIQUES
What is greedy technique?
Constructs a solution to an optimization problem piece by piece through a sequence of choices that are
feasible, i.e., it has to satisfy the problems constraints
locally optimal, i.e., it has to be the best local choice among all feasible choices available on that step
irrevocable, i.e., once made, it cannot be changed on subsequent steps of the algorithm
For some problems, yields an optimal solution for every instance. For most, does not but can be useful for
fast approximations.
Optimal solutions:
change making for normal coin denominations
minimum spanning tree (MST)
single-source shortest paths
Huffman codes
Approximations:
traveling salesman problem (TSP)
knapsack problem
other combinatorial optimization problems
1.PRIMS ALGORITHM
This algorithm is used to find a minimum spanning tree for a given weighted connected graph.
Basic idea of the algorithm
Start with tree T1 consisting of one (any) vertex and grow tree one vertex at a time to produce MST through a
series of expanding subtrees T1, T2, , Tn
On each iteration, construct Ti+1 from Ti by adding vertex not in Ti that is closest to those already in Ti (this is a
greedy step!)-ties can be broken arbitrarily.
Stop when all vertices are included
Page 17
Complexity:
O(n2) for weight matrix representation of graph and array implementation of priority queue
O(m log n) for adjacency list representation of graph with n vertices and m edges and min-heap implementation of
priority queue
Example:
Page 18
2.KRUSHKALS ALGORITHM
This algorithm is also used to find a minimum spanning tree for a given weighted connected graph.
Basic idea of the algorithm
Sort the edges in non-decreasing order of lengths
Grow tree one edge at a time to produce MST through a series of expanding forests F1, F2, , Fn-1
On each iteration, add the next edge on the sorted list unless this would create a cycle. (If it would, skip the edge.)
Page 19
Page 20
Page 21
Example :
Page 22
Codewords:
Page 23
Compression ratio:
compression ratio: (3-2.25)/3*100% = 25%
Thus, Huffmans encoding of the text will use 25% less memory than its fixed-length encoding.
Advantage and disadvantage (fixed/static Huffman code)
Huffmans encoding is one of the most important file-compression methods. In addition to its simplicity and
versatility, it yields an optimal, i.e., minimal-length, encoding (provided the frequencies of symbol occurrences are independent
and known in advance).
The simplest version of Huffman compression calls, in fact, for a preliminary scanning of a given text to count the
frequencies of symbol occurrences in it. Then these frequencies are used to construct a Huffman coding tree and encode the text
as described above.
This scheme makes it necessary, however, to include the coding table into the encoded text to make its decoding
possible. This drawback can be overcome by using dynamic Huffman encoding, in which the coding tree is updated each time a
new symbol is read from the source text.
Page 25