SGDE3
SGDE3
YiTung Huang a1678019
ADSA SGDE 4
01 Knapsack Problem
This report contains an comparative analysis of two different methods used to solve the
01 Knapsack Problem. The two methods will be named Brute Force and Greedy. Our
particular focus will be a comparison of the different run times and behaviour of the two
algorithms presented.
Description
Suppose we have a knapsack with a weight capacity W and N items. For items for
i=1..N, the ith item has weight w and a value v
i . We would like to find the optimal
i
combination of items that will fit in the knapsack such that the sum of all the item values
is maximised.
Greedy
[1]
The greedy algorithm presented here greedily selects which items to be placed into
the knapsack based on the ratio value / weight. Items with a higher ratio will be selected
preferentially over items with lower ratios. The run time of this algorithm is O(N). The
steps of the algorithm are:
1. Sort items by v / w
i in descending order.
i
2. Set i = 1.
3. Try to add the i’th item to the Knapsack. It can fit if adding its weight to the
knapsack does not cause overflow.
4. Set i = i + 1 and repeat step 3 until no more items will fit.
[1]
P. Kilpelainen showed that the error ε, of the solution found by this algorithm is
bounded by 50% of of W. ie ε ≤ 12 W
Brute Force
with DP
The brute force algorithm is combined with the use of dynamic programming where
every integer knapsack of capacity size Z (in which Z < W) are solved. The final optimal
solution of the knapsack with capacity W is then built on the optimal solution out of the
solved Z. Given enough time, this approach produces the optimal solution for any
problem instance. It’s run time is O(N*W) for all problem instances.
Method
Randomly generated problem instances of varying sizes were created. Our
implementations of the algorithms described above were run on each problem instance
and had each run time recorded. To overcome inconsistencies with CPU resources over
Emil Bigaj a1194398
YiTung Huang a1678019
ADSA SGDE 4
the set of simulations, each instance was solved 10 times and the average run time
results are presented here.
Two executables were written for this analysis.The file GenerateProblem.cpp generated
test cases and output the data to a text file. The file Knapsack.cpp parsed the problem
instances then ran the two algorithms on the data. The output of this is a text file
displaying the runtime and value achieved by the two algorithms. These files are
present in the appendix.
Results
The table below presents the mean run time of the two algorithms with various problem
instance sizes.
Greedy Brute Force
N Value Mean Run Value Mean Run
Time(MS) Time(MS)
100000 23053 67 23067 3442
120000 25884 89 25898 4006
140000 27399 99 27414 4729
160000 28526 113 28542 5968
180000 30363 141 30380 7390
200000 32913 162 32931 102204
Using the above table, the runtime of the two algorithms are plotted below.
Emil Bigaj a1194398
YiTung Huang a1678019
ADSA SGDE 4
Discussion
Inspection of the graphs above show that as the problem instance size increases the
runtime of the Brute Force DP increases in polynomial time (or pseudopolynomial time
if W is also increased), while the greedy algorithm runs in linear time. The data here is
inline with the theoretical run times.
Our second observation is that the greedy approach never produced a solution as good
as the brute force solution. However the greedy algorithm provided a very near
approximation, within %99.9 of the optimal solution for the set of instances. The
accuracy of the greedy approach in our test cases is likely due to the nature of the
Emil Bigaj a1194398
YiTung Huang a1678019
ADSA SGDE 4
problem instances. i.e. If W is constant, as N increases the number of occurrences of
items with very low weight but high value increases. This results in a very good
accuracy of the algorithm. However, if we were to increase W and hold N constant we
would expect the accuracy to approach the theoretical bound of 12 W .
References
[1] Pekka Kilpelainen “On the approximation ratio of Greedy Knapsack”
http://www.cs.uef.fi/~kilpelai/ASA11/greedyKnapsack_0.5_approx.pdf
Appendix
The following files have been included:
GenerateProblem.cpp creates and outputs a problem instances
instance(09).txt problem instances created by GenerateProblem
KnapsackSolver.hpp holds and solves a knapsack problem by brute force or greedy
algorithms.
KnapsackSolver.cpp reads the problem instances, solves them using brute force and
greedy solvers and outputs the runtimes and values.