Professional Documents
Culture Documents
Comparison of Algorithms: Modified From Clifford A. Shaffer and George Bebis
Comparison of Algorithms: Modified From Clifford A. Shaffer and George Bebis
6
Comparison of Algorithms
2
Algorithm Efficiency
4
Estimation Techniques
5
Estimation Example
• Estimate:
• Pages/inch
• Feet/shelf
• Shelves/bookcase
6
Analysis of Algorithms
7
Input Size
• polynomial degree
• # of elements in a matrix
8
Best, Worst, Average Cases
9
Time Analysis
10
Worst Case
11
Best Case
12
Average Case
13
Which Analysis to Use?
14
How to Measure Efficiency?
• Critical resources:
• Time, memory, programmer effort, user effort
15
How do we analyze an algorithm?
• Need to define objective measures.
16
How do we analyze an
algorithm? (cont.)
(3) Express running time t as a function
of problem size n (i.e., t=f(n) )
Asymptotic Algorithm Analysis
- Given two algorithms having running times
f(n) and g(n),
- find which functions grows faster
18
Understanding Rate of Growth
Approximation:
• Cost ~ cost_of_elephants
19
Understanding Rate of Growth
(cont’d)
• The low order terms of a function are
relatively insignificant for large n
n4 + 100n2 + 10n + 50
Approximation:
n4
20
Rate of Growth ≡ Asymptotic Analysis
22
Asymptotic Notation
23
Asymptotic notations
• O-notation
24
Asymptotic Notation
25
Asymptotic notations (cont.)
• - notation
26
Asymptotic Notation
27
Asymptotic notations (cont.)
• -notation
28
Relations Between Different Sets
• Subset relations between order-of-growth
sets.
RR
O( f ) ( f )
•f
( f )
29
Visualizing Orders of Growth
• On a graph, as
you go to the
right, a faster
Value of function
fA(n)=30n+8
growing
function
eventually fB(n)=n2+1
becomes
larger...
Increasing n
30
Common
orders of
magnitude
31
Complexity
• Let us assume two algorithms A and B that solve
the same class of problems.
• The time complexity of A is 5,000n,
the one for B is 1.1n for an input with n elements
• For n = 10,
• A requires 50,000 steps,
• but B only 3,
• so B seems to be superior to A.
32
N log2N N*log2N N2 2N
1 0 0 1 2
2 1 2 4 4
4 2 8 16 16
8 3 24 64 256
16 4 64 256 65,536
34
Algorithm speed vs function growth
• An O(n2) algorithm will be slower than an O(n)
algorithm (for large n).
• But an O(n2) function will grow faster than an O(n)
function.
Value of function
fA(n)=30n+8
fB(n)=n2+1
Increasing n
35
Running Time Examples
Algorithm 1 Cost
arr[0] = 0; c1
arr[1] = 0; c1
arr[2] = 0; c1
...
arr[N-1] = 0; c1
-----------
c1+c1+...+c1 = c1 x N
Algorithm 2 Cost
for(i=0; i<N; i++) c2
arr[i] = 0; c1
-----------
(N+1) x c2 + N x c1 = (c2 + c1) x N + c2
36
Running Time Examples (cont.’d)
Cost
sum = 0; c1
for(i=0; i<N; i++) c2
for(j=0; j<N; j++) c2
sum += arr[i][j]; c3
------------
c1 + c2 x (N+1) + c2 x N x (N+1) + c3 x N x N
37
How do we find f(n)?
(1) Associate a "cost" with each statement
38
Big-O Visualization
O(g(n)) is the set of
functions with
smaller or same
order of growth as
g(n)
39
More Running Time Examples
i = 0;
while (i<N) {
X=X+Y; // O(1)
result = mystery(X); // O(N)
i++; // O(1)
}
• The body of the while loop: O(N)
• Loop is executed: N times
N x O(N) = O(N2)
40
More Running Time Examples
(cont.’d)
if (i<j)
for ( i=0; i<N; i++ ) O(N)
X = X+i;
else
O(1)
X=0;
41
Complexity Examples
What does the following algorithm compute?
int who_knows(int a[], int n) {
int m = 0;
for (int i = 0; i<n; i++)
for (int j = i+1; j<n; j++)
if ( abs(a[i] – a[j]) > m )
m = abs(a[i] – a[j]);
return m;
}
returns the maximum difference between any two
numbers in the input array
Comparisons: n-1 + n-2 + n-3 + … + 1 = (n-1)n/2 = 0.5n2 - 0.5n
Time complexity is O(n2)
42
Complexity Examples
Another algorithm solving the same problem:
int max_diff(int a[n]) {
int min = a[0];
int max = a[0];
for (int i = 1; i<n; i++)
if ( a[i] < min )
min = a[i];
else if ( a[i] > max )
max = a[i];
return max-min;
}
Comparisons: 2n - 2
Time complexity is O(n). 43
Names of Orders of Magnitude
“Popular” functions g(n) are n log n, log n, 1, 2n, n2, n!, n, n3
Listed from slowest to fastest growth:
O(1) bounded (by a constant) time
45
Examples of Growth Rate
/* @return Position of largest value in "A“ */
static int largest(int[] A) {
int currlarge = 0; // Position of largest
for (int i=1; i<A.length; i++)
if (A[currlarge] < A[i])
currlarge = i; // Remember position
return currlarge; // Return largest
position
}
46
Examples (cont)
sum = 0;
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
sum++;
47
Time Complexity Examples (1)
a = b;
sum = 0;
for (i=1; i<=n; i++)
sum += n;
48
Time Complexity Examples (2)
sum = 0;
for (j=1; j<=n; j++)
for (i=1; i<=j; i++)
sum++;
for (k=0; k<n; k++)
A[k] = k;
49
Time Complexity Examples (3)
sum1 = 0;
for (i=1; i<=n; i++)
for (j=1; j<=n; j++)
sum1++;
sum2 = 0;
for (i=1; i<=n; i++)
for (j=1; j<=i; j++)
sum2++;
50
Examples (cont.’d)
51
Analyze the complexity of the
following code segments
52