Data Structures&Algorithm Analysis in C++(Second Edition)

Table of Contents

Sites

FTP

ftp.awl.com

source codes

  • codes review

    Global Utilities

    StartConv.h EndConv.h getline.cpp Chapter 1: Arrays, Pointers, Structures

    ArrayDemo.cpp GetInts.cpp TestString.cpp TestSwap.cpp Chapter 2: Classes

    IntCell.h IntCell.cpp TestIntCell.cpp BuggyIntCell.cpp DeepIntCell.cpp Rational.h Rational.cpp RatMain.cpp mystring.h string.cpp Chapter 3: Templates

    FuncTemplates.cpp InsSort.cpp MemoryCell.h MemoryCell.cpp TestMemoryCell.cpp Matrix.h vector.h vector.cpp Chapter 4: Inheritance

    Except.h Shape.cpp Hiding.cpp StaticBinding.cpp Chapter 5: Patterns

    Rectangle.cpp Wrapper.h Ambiguity.cpp AutoPtr.cpp StorageCell.h TestStorageCell.cpp Iterator1.cpp Iterator2.cpp Iterator3.cpp pair.h Observer.cpp Chapter 6: Running Times

    MaxSum.cpp BinarySearch.cpp Chapter 7: STL

    vector.h vector.cpp functional.h TestFunctional.cpp algorithm.h SimpleSetDemo.cpp TestPQ.cpp Chapter 8: Recursion

    RecSum.cpp PrintInt.cpp BinarySearchRec.cpp Ruler.java FractalStar.java Math.cpp MaxSum.cpp MkChnge.cpp TicTacSlow.cpp Chapter 9: Sorting

    Duplicate.cpp Sort.h TestSort.cpp Chapter 10: Randomization

    Random.h Random.cpp RandTest.cpp Permute.cpp Math.cpp Chapter 11: Fun and Games

    WordSrch.cpp TicTac.cpp Chapter 12: Applications of Stacks – Compilers and Parsing

    Tokenizer.h Tokenizer.cpp Balance.cpp Infix.cpp Chapter 13: Utilities

    Hzip.cpp Tokenizer.h Tokenizer.cpp Xref.cpp Chapter 14: Simulation

    Josephus.cpp Modems.cpp Chapter 15: Shortest Path Algorithms

    Paths.cpp Note for DotNet users: on line 40 of PairingHeap.cpp, insert the word typename in front of the line so it appears as typename PairingHeap::Position Chapter 16: Stacks

    StackAr.h StackAr.cpp TestStackAr.cpp StackLi.h StackLi.cpp TestStackLi.cpp QueueAr.h QueueAr.cpp TestQueueAr.cpp QueueLi.h QueueLi.cpp TestQueueLi.cpp Chapter 17: Linked Lists

    LinkedList.h LinkedList.cpp SortLinkedList.h SortLinkedList.cpp TestLinkedList.cpp list.h list.cpp TestList.cpp Chapter 18: Trees

    BinaryTree.h BinaryTree.cpp Iterate.h Iterate.cpp TestBinaryTree.cpp Chapter 19: Search Trees

    BinarySearchTree.h BinarySearchTree.cpp TestBinarySearchTree.cpp RedBlackTree.h RedBlackTree.cpp TestRedBlackTree.cpp AATree.h AATree.cpp TestAATree.cpp set.h set.cpp TestSet.cpp map.h map.cpp TestMap.cpp Chapter 20: Hash Tables

    QuadraticProbing.h QuadraticProbing.cpp TestQuadraticProbing.cpp Chapter 21: Heaps

    BinaryHeap.h BinaryHeap.cpp TestBinaryHeap.cpp queue.h queue.cpp TestQueue.cpp Chapter 22: Splay Trees

    SplayTree.h SplayTree.cpp TestSplayTree.cpp Chapter 23: Pairing Heaps

    PairingHeap.h PairingHeap.cpp TestPairingHeap.cpp Note for DotNet users: on line 40 of PairingHeap.cpp, insert the word typename in front of the line so it appears as typename PairingHeap::Position Chapter 24: Disjoint Sets

    DisjSets.h DisjSets.cpp TestDisjSets.cpp Appendix D: Primitive Arrays

    PointerHopping.cpp

References

Chapter 1

There are many good textbooks covering the mathematics reviewed in this chapter. A small subset is [1], [2], [3], [11], [13], and [14]. Reference [11] is specifically geared toward the analysis of algorithms. It is the first volume of a three-volume series that will be cited throughout this text. More advanced material is covered in [6]. Throughout this book we will assume a knowledge of C [10]. Occasionally, we add a feature where necessary for clarity. We also assume familiarity with pointers and recursion (the recursion summary in this chapter is meant to be a quick review). We will attempt to provide hints on their use where appropriate throughout the textbook. Readers not familiar with these should consult [4], [8], [12], or any good intermediate programming textbook.

General programming style is discussed in several books. Some of the classics are [5], [7], and [9].

  1. M. O. Albertson and J. P. Hutchinson, Discrete Mathematics with Algorithms, John Wiley & Sons, New York, 1988.
  2. Z. Bavel, Math Companion for Computer Science, Reston Publishing Company, Reston, Va., 1982.
  3. R. A. Brualdi, Introductory Combinatorics, North-Holland, New York, 1977.
  4. W. H. Burge, Recursive Programming Techniques, Addison-Wesley, Reading, Mass., 1975.
  5. E. W. Dijkstra, A Discipline of Programming, Prentice Hall, Englewood Cliffs, N.J., 1976.
  6. R. L. Graham, D. E. Knuth, and O. Patashnik, Concrete Mathematics, Addison-Wesley, Reading, Mass., 1989.
  7. D. Gries, The Science of Programming, Springer-Verlag, New York, 1981.
  8. P. Helman and R. Veroff, Walls and Mirrors: Intermediate Problem Solving and Data Structures, 2d ed., Benjamin Cummings Publishing, Menlo Park, Calif., 1988.
  9. B. W. Kernighan and P. J. Plauger, The Elements of Programming Style, 2d ed., McGraw- Hill, New York, 1978.
  10. B. W. Kernighan and D. M. Ritchie, The C Programming Language, 2d ed., Prentice Hall, Englewood Cliffs, N.J., 1988.
  11. D. E. Knuth, The Art of Computer Programming, Vol. 1: Fundamental Algorithms, 2d ed., Addison-Wesley, Reading, Mass., 1973.
  12. E. Roberts, Thinking Recursively, John Wiley & Sons, New York, 1986.
  13. F. S. Roberts, Applied Combinatorics, Prentice Hall, Englewood Cliffs, N.J., 1984.
  14. A. Tucker, Applied Combinatorics, 2d ed., John Wiley & Sons, New York, 1984.

Chapter 2

  1. A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, Mass., 1974.
  2. J. L. Bentley, Writing Efficient Programs, Prentice Hall, Englewood Cliffs, N.J., 1982.
  3. J. L. Bentley, Programming Pearls, Addison-Wesley, Reading, Mass., 1986.
  4. J. L. Bentley, More Programming Pearls, Addison-Wesley, Reading, Mass., 1988.
  5. D. E. Knuth, The Art of Computer Programming, Vol 1: Fundamental Algorithms, 2d ed., Addison-Wesley, Reading, Mass., 1973.
  6. D. E. Knuth, The Art of Computer Programming, Vol 2: Seminumerical Algorithms, 2d ed., Addison-Wesley, Reading, Mass., 1981.
  7. D. E. Knuth, The Art of Computer Programming, Vol 3: Sorting and Searching, Addison-Wesley, Reading, Mass., 1975.
  8. D. E. Knuth, "Big Omicron and Big Omega and Big Theta," ACM SIGACT News, 8 (1976), 18-23.

Chapter 4: Trees

More information on binary search trees, and in particular the mathematical properties of trees can be found in the two books by Knuth [23] and [24]. Several papers deal with the lack of balance caused by biased deletion algorithms in binary search trees. Hibbard's paper [20] proposed the original deletion algorithm and established that one deletion preserves the randomness of the trees. A complete analysis has been performed only for trees with three [21] and four nodes[5]. Eppinger's paper [15] provided early empirical evidence of nonrandomness, and the papers by Culberson and Munro, [11], [12], provide some analytical evidence (but not a complete proof for the general case of intermixed insertions and deletions).
AVL trees were proposed by Adelson-Velskii and Landis [1]. Simulation results for AVL trees, and variants in which the height imbalance is allowed to be at most k for various values of k, are presented in [22]. A deletion algorithm for AVL trees can be found in [24]. Analysis of the averaged depth of AVL trees is incomplete, but some results are contained in [25]. [3] and [9] considered self-adjusting trees like the type in Section 4.5.1. Splay trees are described in [29].
B-trees first appeared in [6]. The implementation described in the original paper allows data to be stored in internal nodes as well as leaves. The data structure we have described is sometimes known as a B+ tree. A survey of the different types of B-trees is presented in [10]. Empirical results of the various schemes is reported in [18]. Analysis of 2-3 trees and B-trees can be found in [4], [14], and [33].
Exercise 4.14 is deceptively difficult. A solution can be found in [16]. Exercise 4.26 is from [32]. Information on B*-trees, described in Exercise 4.38, can be found in [13]. Exercise 4.42 is from [2]. A solution to Exercise 4.43 using 2n -6 rotations is given in [30]. Using threads, a la Exercise 4.45, was first proposed in [28]. k-d trees were first proposed in [7]. Their major drawback is that both deletion and balancing are difficult. [8] discusses k-d trees and other methods used for multidimensional searching.
Other popular balanced search trees are red-black trees [19] and weight-balanced trees [27]. More balanced tree schemes can be found in the books [17], [26], and [31].
  1. G. M. Adelson-Velskii and E. M. Landis, "An Algorithm for the Organization of Information," Soviet Math. Doklady 3 (1962), 1259-1263.
  2. A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, MA, 1974.
  3. B. Allen and J. I. Munro, "Self Organizing Search Trees," Journal of the ACM, 25 (1978), 526-535.
  4. R. A. Baeza-Yates, "Expected Behaviour of B+- trees under Random Insertions," Acta Informatica 26 (1989), 439-471.
  5. R. A. Baeza-Yates, "A Trivial Algorithm Whose Analysis Isn't: A Continuation," BIT 29 (1989), 88-113.
  6. R. Bayer and E. M. McGreight, "Organization and Maintenance of Large Ordered Indices," Acta Informatica 1 (1972), 173-189.
  7. J. L. Bentley, "Multidimensional Binary Search Trees Used for Associative Searching," Communications of the ACM 18 (1975), 509-517.
  8. J. L. Bentley and J. H. Friedman, "Data Structures for Range Searching," Computing Surveys 11 (1979), 397-409.
  9. J. R. Bitner, "Heuristics that Dynamically Organize Data Structures," SIAM Journal on Computing 8 (1979), 82-110.
  10. D. Comer, "The Ubiquitous B-tree," Computing Surveys 11 (1979), 121-137.
  11. J. Culberson and J. I. Munro, "Explaining the Behavior of Binary Search Trees under Prolonged Updates: A Model and Simulations," Computer Journal 32 (1989), 68-75.
  12. J. Culberson and J. I. Munro, "Analysis of the Standard Deletion Algorithms' in Exact Fit Domain Binary Search Trees," Algorithmica 5 (1990) 295-311.
  13. K. Culik, T. Ottman, and D. Wood, "Dense Multiway Trees," ACM Transactions on Database Systems 6 (1981), 486-512.
  14. B. Eisenbath, N. Ziviana, G. H. Gonnet, K. Melhorn, and D. Wood, "The Theory of Fringe Analysis and its Application to 2-3 Trees and B-trees," Information and Control 55 (1982), 125-174.
  15. J. L. Eppinger, "An Empirical Study of Insertion and Deletion in Binary Search Trees," Communications of the ACM 26 (1983), 663-669.
  16. P. Flajolet and A. Odlyzko, "The Average Height of Binary Trees and Other Simple Trees," Journal of Computer and System Sciences 25 (1982), 171-213.
  17. G. H. Gonnet and R. Baeza-Yates, Handbook of Algorithms and Data Structures, second edition, Addison-Wesley, Reading, MA, 1991.
  18. E. Gudes and S. Tsur, "Experiments with B-tree Reorganization," Proceedings of ACM SIGMOD Symposium on Management of Data (1980), 200-206.
  19. L. J. Guibas and R. Sedgewick, "A Dichromatic Framework for Balanced Trees," Proceedings of the Nineteenth Annual IEEE Symposium on Foundations of Computer Science (1978), 8-21.
  20. T. H. Hibbard, "Some Combinatorial Properties of Certain Trees with Applications to Searching and Sorting," Journal of the ACM 9 (1962), 13-28.
  21. A. T. Jonassen and D. E. Knuth, "A Trivial Algorithm Whose Analysis Isn't," Journal of Computer and System Sciences 16 (1978), 301-322.
  22. P. L. Karlton, S. H. Fuller, R. E. Scroggs, and E. B. Kaehler, "Performance of Height Balanced Trees," Communications of the ACM 19 (1976), 23-28.
  23. D. E. Knuth, The Art of Computer Programming: Volume 1: Fundamental Algorithms, second edition, Addison-Wesley, Reading, MA, 1973.
  24. D. E. Knuth, The Art of Computer Programming: Volume 3: Sorting and Searching, second printing, Addison-Wesley, Reading, MA, 1975.
  25. K. Melhorn, "A Partial Analysis of Height-Balanced Trees under Random Insertions and Deletions," SIAM Journal of Computing 11 (1982), 748-760.
  26. K. Melhorn, Data Structures and Algorithms 1: Sorting and Searching, Springer-Verlag, Berlin, 1984.
  27. J. Nievergelt and E. M. Reingold, "Binary Search Trees of Bounded Balance," SIAM Journal on Computing 2 (1973), 33-43.
  28. A. J. Perlis and C. Thornton, "Symbol Manipulation in Threaded Lists," Communications of the ACM 3 (1960), 195-204.
  29. D. D. Sleator and R. E. Tarjan, "Self-adjusting Binary Search Trees," Journal of ACM 32 (1985), 652-686.
  30. D. D. Sleator, R. E. Tarjan, and W. P. Thurston, "Rotation Distance, Triangulations, and Hyperbolic Geometry," Journal of AMS (1988), 647-682.
  31. H. F. Smith, Data Structures-Form and Function, Harcourt Brace Jovanovich, 1987.
  32. R. E. Tarjan, "Sequential Access in Splay Trees Takes Linear Time," Combinatorica 5 (1985), 367-378.
  33. A. C. Yao, "On Random 2-3 trees," Acta Informatica 9 (1978), 159-170.

CHAPTER 5: HASHING

Despite the apparent simplicity of hashing, much of the analysis is quite difficult and there are still many unresolved questions. There are also many interesting theoretical issues, which generally attempt to make it unlikely that the worst-case possibilities of hashing arise.
An early paper on hashing is [17]. A wealth of information on the subject, including an analysis of closed hashing with linear probing can be found in [11]. An excellent survey on the subject is [14]; [15] contains suggestions, and pitfalls, for choosing hash functions. Precise analytic and simulation results for all of the methods described in this chapter can be found in [8].
An analysis of double hashing can be found in [9] and [13]. Yet another collision resolution scheme is coalesced hashing, as described in [18]. Yao [20] has shown that uniform hashing, in which no clustering exists, is optimal with respect to cost of a successful search.
If the input keys are known in advance, then perfect hash functions, which do not allow collisions, exist [2], [7]. Some more complicated hashing schemes, for which the worst case depends not on the particular input but on random numbers chosen by the algorithm, appear in [3] and [4].
Extendible hashing appears in [5], with analysis in [6] and [19]. One method of implementing Exercise 5.5 is described in [16]. Exercise 5.11 (a-d) is from [10]. Part (e) is from [12], and part (f) is from [1].
  1. R. S. Boyer and J. S. Moore, "A Fast String Searching Algorithm," Communications of the ACM 20 (1977), 762-772.
  2. J. L. Carter and M. N. Wegman, "Universal Classes of Hash Functions," Journal of Computer and System Sciences 18 (1979), 143-154.
  3. M. Dietzfelbinger, A. R. Karlin, K. Melhorn, F. Meyer auf der Heide, H. Rohnert, and R. E. Tarjan, Dynamic Perfect Hashing: Upper and Lower Bounds," Proceedings of the Twenty-ninth IEEE Symposium on Foundations of Computer Science (1988), 524-531.
  4. R. J. Enbody and H. C. Du, "Dynamic Hashing Schemes," Computing Surveys 20 (1988), 85-113.
  5. R. Fagin, J. Nievergelt, N. Pippenger, and H. R. Strong, "Extendible Hashing-A Fast Access Method for Dynamic Files," ACM Transactions on Database Systems 4 (1979), 315-344.
  6. P. Flajolet, "On the Performance Evaluation of Extendible Hashing and Trie Searching," Acta Informatica 20 (1983), 345-369.
  7. M. L. Fredman, J. Komlos, and E. Szemeredi, "Storing a Sparse Table with O(1) Worst Case Access Time," Journal of the ACM 31 (1984), 538-544.
  8. G. H. Gonnet and R. Baeza-Yates, Handbook of Algorithms and Data Structures, second edition, Addison-Wesley, Reading, MA, 1991.
  9. L. J. Guibas and E. Szemeredi, "The Analysis of Double Hashing," Journal of Computer and System Sciences 16 (1978), 226-274.
  10. R. M. Karp and M. O. Rabin, "Efficient Randomized Pattern-Matching Algorithms," Aiken Computer Laboratory Report TR-31-81, Harvard University, Cambridge, MA, 1981.
  11. D. E. Knuth, The Art of Computer Programming, Vol 3: Sorting and Searching, second printing, Addison-Wesley, Reading, MA, 1975.
  12. D. E. Knuth, J. H. Morris, V. R. Pratt, "Fast Pattern Matching in Strings," SIAM Journal on Computing 6 (1977), 323-350.
  13. G. Lueker and M. Molodowitch, "More Analysis of Double Hashing," Proceedings of the Twentieth ACM Symposium on Theory of Computing (1988), 354-359.
  14. W. D. Maurer and T. G. Lewis, "Hash Table Methods," Computing Surveys 7 (1975), 5-20.
  15. B. J. McKenzie, R. Harries, and T. Bell, "Selecting a Hashing Algorithm," Software–Practice and Experience 20 (1990), 209-224.
  16. R. Morris, "Scatter Storage Techniques," Communications of the ACM 11 (1968), 38-44.
  17. W. W. Peterson, "Addressing for Random Access Storage," IBM Journal of Research and Development 1 (1957), 130-146.
  18. J. S. Vitter, "Implementations for Coalesced Hashing," Communications of the ACM 25 (1982), 911-926.
  19. A. C. Yao, "A Note on The Analysis of Extendible Hashing," Information Processing Letters 11 (1980), 84-86.
  20. A. C. Yao, "Uniform Hashing is Optimal," Journal of the ACM 32 (1985), 687-693.

CHAPTER 6: PRIORITY QUEUES (HEAPS)

The binary heap was first described in [21]. The linear-time algorithm for its construction is from [9].
The first description of d -heaps was in [14]. Leftist heaps were invented by Crane [7] and described in Knuth [15]. Skew heaps were developed by Sleator and Tarjan [17]. Binomial queues were invented by Vuillemin [20]; Brown provided a detailed analysis and empirical study showing that they perform well in practice [2], if carefully implemented.
Exercise 6.7 (b-c) is taken from [12]. A method for constructing binary heaps that uses about 1.52n comparisons on average is described in [16]. Lazy deletion in leftist heaps (Exercise 6.21) is from [6]. A solution to Exercise 6.33 can be found in [5].
Min-max heaps (Exercise 6.15) were originally described in [1]. More efficient implementation of the operations is given in [13] and [18]. An alternate representation for double ended priority queues is the deap. Details can be found in [3] and [4].
A theoretically interesting priority queue representation is the Fibonacci heap [11], which we will describe in Chapter 11. The Fibonacci heap allows all operations to be performed in O(1) amortized time, except for deletions, which are O(log n). Relaxed heaps [8] achieve identical bounds in the worst case. Another interesting implementation is the pairing heap [10]. Finally, a priority queue that works when the data consists of small integers is described in [19].
  1. M. D. Atkinson, J. R. Sack, N. Santoro, and T. Strothotte, "Min-Max Heaps and Generalized Priority Queues," Communications of the ACM 29 (1986), 996-1000.
  2. M. R. Brown, "Implementation and Analysis of Binomial Queue Algorithms," SIAM Journal on Computing 7 (1978), 298-319.
  3. S. Carlsson, "The Deap–A Double-ended Heap to Implement Double-ended Priority Queues," Information Processing Letters 26 (1987), 33-36.
  4. S. Carlsson, J. Chen, and T. Strothotte, "A Note on the Construction of the Data Structure 'Deap'," Information Processing Letters 31 (1989), 315-317.
  5. S. Carlsson, J. I. Munro, and P. V. Poblete, "An Implicit Binomial Queue with Constant Insertion Time," Proceedings of First Scandinavian Workshop on Algorithm Theory, 1988, 1-13.
  6. D. Cheriton and R. E. Tarjan, "Finding Minimum Spanning Trees," SIAM Journal on Computing 5 (1976), 724-742.
  7. C. A. Crane, "Linear Lists and Priority Queues as Balanced Binary Trees," Technical Report STAN-CS-72-259, Computer Science Department, Stanford University, Stanford, CA, 1972.
  8. J. R. Driscoll, H. N. Gabow, R. Shrairman, and R. E. Tarjan, "Relaxed Heaps: An Alternative to Fibonacci Heaps with Applications to Parallel Computation," Communications of the ACM 31 (1988), 1343-1354.
  9. R. W. Floyd, "Algorithm 245: Treesort 3:", Communications of the ACM 7 (1964), 701.
  10. M. L. Fredman, R. Sedgewick, D. D. Sleator, and R. E. Tarjan, "The Pairing Heap: A New Form of Self-adjusting Heap," Algorithmica 1 (1986), 111-129.
  11. M. L. Fredman and R. E. Tarjan, "Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms," Journal of the ACM 34 (1987), 596-615.
  12. G. H. Gonnet and J. I. Munro, "Heaps on Heaps," SIAM Journal on Computing 15 (1986), 964-971.
  13. A. Hasham and J. R. Sack, "Bounds for Min-max Heaps," BIT 27 (1987), 315-323.
  14. D. B. Johnson, "Priority Queues with Update and Finding Minimum Spanning Trees," Information Processing Letters 4 (1975), 53-57.
  15. D. E. Knuth, The Art of Computer Programming, Vol 3: Sorting and Searching, second printing, Addison-Wesley, Reading, MA, 1975.
  16. C. J. H. McDiarmid and B. A. Reed, "Building Heaps Fast," Journal of Algorithms 10 (1989), 352-365.
  17. D. D. Sleator and R. E. Tarjan, "Self-adjusting Heaps," SIAM Journal on Computing 15 (1986), 52-69.
  18. T. Strothotte, P. Eriksson, and S. Vallner, "A Note on Constructing Min-max Heaps," BIT 29 (1989), 251-256.
  19. P. van Emde Boas, R. Kaas, E. Zijlstra, "Design and Implementation of an Efficient Priority Queue," Mathematical Systems Theory 10 (1977), 99-127.
  20. J. Vuillemin, "A Data Structure for Manipulating Priority Queues," Communications of the ACM 21 (1978), 309-314.
  21. J. W. J. Williams, "Algorithm 232: Heapsort," Communications of the ACM 7 (1964), 347-348.

CHAPTER 7: SORTING

Knuth's book [10] is a comprehensive, though somewhat dated, reference for sorting. Gonnet and Baeza-Yates [4] has some more recent results, as well as a huge bibliography.
The original paper detailing Shellsort is [21]. The paper by Hibbard [5] suggested the use of the increments 2k - 1 and tightened the code by avoiding swaps. Theorem 7.4 is from [12]. Pratt's lower bound, which uses a more complex method than that suggested in the text, can be found in [14]. Improved increment sequences and upper bounds appear in [9], [20], and [23]; matching lower bounds have been shown in [24]. A recent unpublished result by Poonen shows that no increment sequence gives an O(n log n) worst-case running time. An identical result was obtained independently and appears in [13]. The average-case running time for Shellsort is still unresolved. Yao [26] has performed an extremely complex analysis for the three-increment case. The result has yet to be extended to more increments. Experiments with various increment sequences appear in [22].
Heapsort was invented by Williams [25]; Floyd [1] provided the linear-time algorithm for heap construction. The analysis of its average case has only recently been obtained [15].
An exact average-case analysis of mergesort has been claimed in [3]; the paper detailing the results is forthcoming. An algorithm to perform merging in linear time without extra space is described in [8].
Quicksort is from Hoare [6]. This paper analyzes the basic algorithm, describes most of the improvements, and includes the selection algorithm. A detailed analysis and empirical study was the subject of Sedgewick's dissertation [19]. Many of the important results appear in the three papers [16], [17], and [18].
Decision trees and sorting optimality are discussed in Ford and Johnson [2]. This paper also provides an algorithm that almost meets the lower bound in terms of number of comparisons (but not other operations). This algorithm was eventually shown to be slightly suboptimal by Manacher [11].
External sorting is covered in detail in [10]. Stable sorting, described in Exercise 7.24, has been addressed by Horvath [7].
  1. R. W. Floyd, "Algorithm 245: Treesort 3," Communications of the ACM 7 (1964), 701.
  2. L. R. Ford and S. M. Johnson, "A Tournament Problem," American Mathematics Monthly 66 (1959),387-389.
  3. M. Golin and R. Sedgewick, "Exact Analysis of Mergesort," Fourth SIAM Conference on Discrete Mathematics, 1988.
  4. G. H. Gonnet and R. Baeza-Yates, Handbook of Algorithms and Data Structures, second edition, Addison-Wesley, Reading, MA, 1991.
  5. T. H. Hibbard, "An Empirical Study of Minimal Storage Sorting," Communications of the ACM 6
  6. C. A. R. Hoare, "Quicksort," Computer Journal 5 (1962), 10-15.
  7. E. C. Horvath, "Stable Sorting in Asymptotically Optimal Time and Extra Space," Journal of the ACM 25 (1978), 177-199.
  8. B. Huang and M. Langston, "Practical In-place Merging," Communications of the ACM 31 (1988), 348-352.
  9. J. Incerpi and R. Sedgewick, "Improved Upper Bounds on Shellsort," Journal of Computer and System Sciences 31 (1985), 210-224.
  10. D. E. Knuth, The Art of Computer Programming. Volume 3: Sorting and Searching, second printing, Addison-Wesley, Reading, MA, 1975.
  11. G. K. Manacher, "The Ford-Johnson Sorting Algorithm Is Not Optimal," Journal of the ACM 26 (1979), 441-456.
  12. A. A. Papernov and G. V. Stasevich, "A Method of Information Sorting in Computer Memories," Problems of Information Transmission 1 (1965), 63-75.
  13. C. G. Plaxton and T. Suel, "Improved Lower Bounds for Shellsort," Proceedings of the Thirtythird Annual IEEE Symposium on the Foundations of Computer Science, (1992).
  14. V. R. Pratt, Shellsort and Sorting Networks, Garland Publishing, New York, 1979. (Originally presented as the author's Ph.D. thesis, Stanford University, 1971.)
  15. R. Schaffer and R. Sedgewick, "The Analysis of Heapsort," Journal of Algorithms, to appear.
  16. R. Sedgewick, "Quicksort with Equal Keys," SIAM Journal on Computing 6 (1977), 240-267.
  17. R. Sedgewick, "The Analysis of Quicksort Programs," Acta Informatica 7 (1977), 327-355.
  18. R. Sedgewick, "Implementing Quicksort Programs," Communications of the ACM 21 (1978), 847- 857.
  19. R. Sedgewick, Quicksort, Garland Publishing, New York, 1978. (Originally presented as the author's Ph.D. thesis, Stanford University, 1975.)
  20. R. Sedgewick, "A New Upper Bound for Shellsort," Journal of Algorithms 2 (1986), 159-173.
  21. D. L. Shell, "A High-Speed Sorting Procedure," Communications of the ACM 2 (1959), 30-32.
  22. M. A. Weiss, "Empirical Results on the Running Time of Shellsort," Computer Journal 34 (1991), 88-91.
  23. M. A. Weiss and R. Sedgewick, "More On Shellsort Increment Sequences," Information Processing Letters 34 (1990), 267-270.
  24. M. A. Weiss and R. Sedgewick, "Tight Lower Bounds for Shellsort," Journal of Algorithms 11 (1990), 242-251.
  25. J. W. J. Williams, "Algorithm 232: Heapsort," Communications of the ACM 7 (1964), 347-348.
  26. A. C. Yao, "An Analysis of (h, k, 1) Shellsort," Journal of Algorithms 1 (1980), 14-50.

CHAPTER 8: THE DISJOINT SET ADT

Various solutions to the union/find problem can be found in [5], [8], and [10]. Hopcroft and Ullman showed the O(m log* n) bound of Section 8.6. Tarjan [14] obtained the bound O(m (m,n)). A more precise (but asymptotically identical) bound for m < n appears in [2] and [17]. Various other strategies for path compression and unions also achieve the same bound; see [17] for details.
A lower bound showing that under certain restrictions (m (m,n)) time is required to process m union/find operations was given by Tarjan [15]. Identical bounds under less restrictive conditions have been recently shown in [6] and [13].
Applications of the union/find data structure appear in [1] and [9]. Certain special cases of the union/find problem can be solved in O(m) time [7]. This reduces the running time of several algorithms, such as [1], graph dominance, and reducibility (see references in Chapter 9) by a factor of (m,n). Others, such as [9] and the graph connectivity problem in this chapter, are unaffected. The paper lists 10 examples. Tarjan has used path compression to obtain efficient algorithms for several graph problems [16].
Average-case results for the union/find problem appear in [4], [11], and [19]. Results bounding the running time of any single operation (as opposed to the entire sequence) appear in [3] and [12].
Exercise 8.8 is solved in [18].
  1. A. V. Aho, J. E. Hopcroft, J. D. Ullman, "On Finding Lowest Common Ancestors in Trees," SIAM Journal on Computing 5 (1976), 115-132.
  2. L. Banachowski, "A Complement to Tarjan's Result about the Lower Bound on the Complexity of the Set Union Problem," Information Processing Letters 11 (1980), 59-65.
  3. N. Blum, "On the Single-operation Worst-case Time Complexity of the Disjoint Set Union Problem," SIAM Journal on Computing 15 (1986), 1021-1024.
  4. J. Doyle and R. L. Rivest, "Linear Expected Time of a Simple Union Find Algorithm," Information Processing Letters 5 (1976), 146-148.
  5. M. J. Fischer, "Efficiency of Equivalence Algorithms," Complexity of Computer Computation (eds. R. E. Miller and J. W. Thatcher), Plenum Press, 1972, 153-168.
  6. M. L. Fredman and M. E. Saks, "The Cell Probe Complexity of Dynamic Data Structures," Proceedings of the Twenty-first Annual Symposium on Theory of Computing (1989), 345-354.
  7. H. N. Gabow and R. E. Tarjan, "A Linear-time Algorithm for a Special Case of Disjoint Set Union,"Journal of Computer and System Sciences 30 (1985), 209-221.
  8. B. A. Galler and M. J. Fischer, "An Improved Equivalence Algorithm," Communications of the ACM

7 (1964), 301-303.

  1. J. E. Hopcroft and R. M. Karp, "An Algorithm for Testing the Equivalence of Finite Automata," Technical Report TR-71-114, Department of Computer Science, Cornell University, Ithaca, NY, 1971.
  2. J. E. Hopcroft and J. D. Ullman, "Set Merging Algorithms," SIAM Journal on Computing 2 (1973), 294-303.
  3. D. E. Knuth and A. Schonhage, "The Expected Linearity of a Simple Equivalence Algorithm," Theoretical Computer Science 6 (1978), 281-315.
  4. J. A. LaPoutre, "New Techniques for the Union-Find Problem," Proceedings of the First Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 54-63.
  5. J. A. LaPoutre, "Lower Bounds for the Union-Find and the Split-Find Problem on Pointer Machines," Proceedings of the Twenty Second Annual ACM Symposium on Theory of Computing (1990), 34-44.
  6. R. E. Tarjan, "Efficiency of a Good but Not Linear Set Union Algorithm," Journal of the ACM 22 (1975), 215-225.
  7. R. E. Tarjan, "A Class of Algorithms Which Require Nonlinear Time to Maintain Disjoint Sets,"Journal of Computer and System Sciences 18 (1979), 110-127.
  8. R. E. Tarjan, "Applications of Path Compression on Balanced Trees," Journal of the ACM 26 (1979), 690-715.
  9. R. E. Tarjan and J. van Leeuwen, "Worst Case Analysis of Set Union Algorithms," Journal of the ACM 31 (1984), 245-281.
  10. J. Westbrook and R. E. Tarjan, "Amortized Analysis of Algorithms for Set Union with Backtracking," SIAM Journal on Computing 18 (1989), 1-11.
  11. A. C. Yao, "On the Average Behavior of Set Merging Algorithms," Proceedings of Eighth Annual ACM Symposium on the Theory of Computation (1976), 192-195.

CHAPTER 9: GRAPH ALGORITHMS

Good graph theory textbooks include [7], [12], [21], and [34]. More advanced topics, including the more careful attention to running times, are covered in [36], [38], and [45].
Use of adjacency lists was advocated in [23]. The topological sort algorithm is from [28], as described in [31]. Dijkstra's algorithm appeared in [8]. The improvements using d-heaps and Fibonacci heaps are described in [27] and [14], respectively. The shortest-path algorithm with negative edge weights is due to Bellman [3]; Tarjan [45] describes a more efficient way to guarantee termination.
Ford and Fulkerson's seminal work on network flow is [13]. The idea of augmenting along shortest paths or on paths admitting the largest flow increase is from [11]. Other approaches to the problem can be found in [9], [30], and [20]. An algorithm for the min-cost flow problem can be found in [18].
An early minimum spanning tree algorithm can be found in [4]. Prim's algorithm is from [39]; Kruskal's algorithm appears in [32]. Two O(|E| log log |V|) algorithms are [5] and [46]. The theoretically best-known algorithms appear in [14] and [16]. An empirical study of these algorithms suggests that Prim's algorithm, implemented with decrease_key, is best in practice on most graphs [37].
The algorithm for biconnectivity is from [41]. The first linear-time strong components algorithm (Exercise 9.28) appears in the same paper. The algorithm presented in the text is due to Kosaraju (unpublished) and Sharir [40]. Other applications of depth-first search appear in [24], [25], [42], and [43] (as mentioned in Chapter 8, the results in [42] and [43] have been improved, but the basic algorithm is unchanged).
The classic reference work for the theory of NP-complete problems is [19]. Additional material can be found in [1]. The NP-completeness of satisfiability is shown in [6]. The other seminal paper is [29], which showed the NP-completeness of 21 problems. An excellent survey of complexity theory is [44]. An approximation algorithm for the traveling salesman problem, which generally gives nearly optimal results, can be found in [35].
A solution to Exercise 9.8 can be found in [2]. Solutions to the bipartite matching problem in Exercise 9.13 can be found in [22] and [33]. The problem can be generalized by adding weights to the edges and removing the restriction that the graph is bipartite. Efficient solutions for the unweighted matching problem for general graphs are quite complex. Details can be found in [10], [15], and [17].
Exercise 9.35 deals with planar graphs, which commonly arise in practice. Planar graphs are very sparse, and many difficult problems are easier on planar graphs. An example is the graph isomorphism problem, which is solvable in linear time for planar graphs [26]. No polynomial time algorithm is known for general graphs.
  1. A. V. Aho, J. E. Hopcroft, and J. D. Ullman, The Design and Analysis of Computer Algorithms, Addison-Wesley, Reading, MA, 1974.
  2. R. K. Ahuja, K. Melhorn, J. B. Orlin, and R. E. Tarjan, "Faster Algorithms for the Shortest Path Problem," Journal of the ACM 37 (1990), 213-223.
  3. R. E. Bellman, "On a Routing Problem," Quarterly of Applied Mathematics 16 (1958), 87-90.
  4. O. Boruvka, "Ojistém problému minimálním (On a Minimal Problem)," Práca Moravské 3 (1926), 37-58.
  5. D. Cheriton and R. E. Tarjan, "Finding Minimum Spanning Trees," SIAM Journal on Computing 5 (1976), 724-742.
  6. S. Cook, "The Complexity of Theorem Proving Procedures," Proceedings of the Third Annual ACM Symposium on Theory of Computing (1971), 151-158.
  7. N. Deo, Graph Theory wtth Applications to Engineering and Computer Science, Prentice Hall, Englewood Cliffs, NJ, 1974.
  8. E. W. Dijkstra, "A Note on Two Problems in Connexion with Graphs," Numerische Mathematik 1 (1959), 269-271.
  9. E. A. Dinic, "Algorithm for Solution of a Problem of Maximum Flow in Networks with Power Estimation," Soviet Mathematics Doklady 11 (1970), 1277-1280.
  10. J. Edmonds, "Paths, Trees, and Flowers," Canadian Journal of Mathematics 17 (1965) 449-467.
  11. J. Edmonds and R. M. Karp, "Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems," Journal of the ACM 19 (1972), 248-264.
  12. S. Even, Graph Algorithms, Computer Science Press, Potomac, MD, 1979.
  13. L. R. Ford, Jr. and D. R. Fulkerson, Flows in Networks, Princeton University Press, Princeton, NJ, 1962.
  14. M. L. Fredman and R. E. Tarjan, "Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms," Journal of the ACM 34 (1987), 596-615.
  15. H. N. Gabow, "Data Structures for Weighted Matching and Nearest Common Ancestors with Linking," Proceedings of First Annual ACM-SIAM Symposium on Discrete Algorithms (1990), 434-443.
  16. H. N. Gabow, Z. Galil, T. H. Spencer, and R. E. Tarjan, "Efficient Algorithms for Finding Minimum Spanning Trees on Directed and Undirected Graphs," Combinatorica 6 (1986), 109-122.
  17. Z. Galil, "Efficient Algorithms for Finding Maximum Matchings in Graphs," ACM Computing Surveys 18 (1986), 23-38.
  18. Z. Galil and E. Tardos,"An O(n2(m + n log n)log n) Min-Cost Flow Algorithm," Journal of the ACM 35 (1988), 374-386.
  19. M. R. Garey and D. S. Johnson, Computers and Intractability: A Guide to the Theory of NPCompleteness, Freeman, San Francisco, 1979.
  20. A. V. Goldberg and R. E. Tarjan, "A New Approach to the Maximum-Flow Problem," Journal of the ACM 35 (1988), 921-940.
  21. F. Harary, Graph Theory, Addison-Wesley, Reading, MA, 1969.
  22. J. E. Hopcroft and R. M. Karp, "An n5/2 Algorithm for Maximum Matchings in Bipartite Graphs," SIAM Journal on Computing 2 (1973), 225-231.
  23. J. E. Hopcroft and R. E. Tarjan, "Algorithm 447: Efficient Algorithms for Graph Manipulation," Communications of the ACM 16 (1973), 372-378.
  24. J. E. Hopcroft and R. E. Tarjan, "Dividing a Graph into Triconnected Components," SIAM Journal on Computing 2 (1973), 135-158.
  25. J. E. Hopcroft and R. E. Tarjan, "Efficient Planarity Testing," Journal of the ACM 21 (1974),
  26. J. E. Hopcroft and J. K. Wong, "Linear Time Algorithm for Isomorphism of Planar Graphs," Proceedings of the Sixth Annual ACM Symposium on Theory of Computing (1974), 172-184.
  27. D. B. Johnson, "Efficient Algorithms for Shortest Paths in Sparse Networks," Journal of the ACM 24 (1977), 1-13.
  28. A. B. Kahn, "Topological Sorting of Large Networks," Communications of the ACM 5 (1962), 558- 562.
  29. R. M. Karp, "Reducibility among Combinatorial Problems," Complexity of Computer Computations (eds. R. E. Miller and J. W. Thatcher), Plenum Press, New York, 1972, 85-103.
  30. A. V. Karzanov, "Determining the Maximal Flow in a Network by the Method of Preflows," Soviet Mathematics Doklady 15 (1974), 434-437.
  31. D. E. Knuth, The Art of Computer Programming, Vol. 1: Fundamental Algorithms, second edition, Addison-Wesley, Reading, MA, 1973.
  32. J. B. Kruskal, Jr. "On the Shortest Spanning Subtree of a Graph and the Traveling Salesman Problem," Proceedings of the American Mathematical Society 7 (1956), 48-50.
  33. H. W. Kuhn, "The Hungarian Method for the Assignment Problem," Naval Research Logistics Quarterly 2 (1955), 83-97.
  34. E. L. Lawler, Combinatorial Optimization: Networks and Matroids, Holt, Reinhart, and Winston, New York, NY, 1976.
  35. S. Lin and B. W. Kernighan, "An Effective Heuristic Algorithm for the Traveling Salesman Problem," Operations Research 21 (1973), 498-516.
  36. K. Melhorn, Data Structures and Algorithms 2: Graph Algorithms and NP-completeness, Springer- Verlag, Berlin, 1984.
  37. B. M. E. Moret and H. D. Shapiro, "An Empirical Analysis of Algorithms for Constructing a Minimum Spanning Tree," Proceedings of the Second Workshop on Algorithms and Data Structures (1991), 400-411.
  38. C. H. Papadimitriou and K. Steiglitz, Combinatorial Optimization: Algorithms and Complexity, Prentice Hall, Englewood Cliffs, NJ, 1982.
  39. R. C. Prim, "Shortest Connection Networks and Some Generalizations," Bell System Technical Journal 36 (1957), 1389-1401.
  40. M. Sharir, "A Strong-Connectivity Algorithm and Its Application in Data Flow Analysis," Computers and Mathematics with Applications 7 (1981), 67-72.
  41. R. E. Tarjan, "Depth First Search and Linear Graph Algorithms," SIAM Journal on Computing 1 (1972), 146-160.
  42. R. E. Tarjan, "Testing Flow Graph Reducibility," Journal of Computer and System Sciences 9 (1974), 355-365.
  43. R. E. Tarjan, "Finding Dominators in Directed Graphs," SIAM Journal on Computing 3 (1974), 62-89.
  44. R. E. Tarjan, "Complexity of Combinatorial Algorithms," SIAM Review 20 (1978), 457-491.
  45. R. E. Tarjan, Data Structures and Network Algorithms, Society for Industrial and Applied Mathematics, Philadelphia, PA, 1983.
  46. A. C. Yao, "An O( |E | log log |V | ) Algorithm for Finding Minimum Spanning Trees," Information Processing Letters 4 (1975), 21-23.

CHAPTER 10: ALGORITHM DESIGN TECHNIQUES

#beginhtml The original paper on Huffman codes is 1. Variations on the algorithm are discussed in 2, 3, and 4. Another popular compression scheme is Ziv-Lempel encoding 5, 6. Here the codes have a fixed length but represent strings instead of characters. 7 and 8 are good surveys of the common compression schemes.<br>

The analysis of bin-packing heuristics first appeared in Johnson's Ph.D. thesis and was published in 9. The improved lower bound for on-line bin packing given in Exercise 10.8 is from 10; this result has been improved further in 11. 12 describes another approach to on-line bin packing.<br>

Theorem 10.7 is from 13. The closest points algorithm appeared in 14. 15 describes the turnpike reconstruction problem and its applications. Two books on the relatively new field of computational geometry are 16 and 17. 18 contains the lecture notes for a computational geometry course taught at MIT; it includes an extensive bibliography.<br>

The linear-time selection algorithm appeared in 19. 20 discusses the sampling approach that finds the median in 1.5n expected comparisons. The O(n1.59) multiplication is from 21. Generalizations are discussed in 22 and 23. Strassen's algorithm appears in the short paper

The discussion of random number generators is based on 24. Park and Miller attribute the portable implementation to Schrage 25. Skip lists are discussed by Pugh in 26. The randomized primality-testing algorithm is due to Miller 27 and Rabin 28. The theorem that at most (n - 9)/4 values of a fool the algorithm is from Monier 29. Other randomized algorithms are discussed in 30.<br>

More information on - pruning can be found in 31, 32, and 33. The top programs that play chess, checkers, Othello, and backgammon have all achieved world class status. 34 describes an Othello program. The paper appears in a special issue on computer games (mostly chess); this issue is a gold mine of ideas. One of the papers describes the use of dynamic programming to solve chess endgames completely when only a few pieces are left on the board. Related research has resulted in the change of the 50-move rule in certain cases. Exercise 10.41 is solved in 35. It is the only known case of a homometric point set with no duplicate distances. Determining whether any others exist for n > 6 is open. Christofides 36 gives a solution to Exercise 10.47, and also an algorithm which generates a tour at most optimal. Exercise 10.52 is discussed in 37. Exercise 10.55 is solved in 38. An O(kn) algorithm is given in 39. Exercise 10.57 is discussed in 40, but do not be misled by the title of the paper. #+endhtml

  1. B. Abramson, "Control Strategies for Two-Player Games," ACM Computing Surveys, 21 (1989), 137- 161.
  2. A. Aggarwal and J. Wein, Computational Geometry: Lecture Notes for 18.409, MIT Laboratory for Computer Science, 1988.
  3. T. Bell, I. H. Witten, and J. G. Cleary, "Modeling for Text Compression," ACM Computing Surveys, 21 (1989), 557-591.
  4. R. E. Bellman, Dynamic Programming, Princeton University Press, Princeton, NJ, 1957.
  5. R. E. Bellman and S. E. Dreyfus, Applied Dynamic Programming, Princeton University Press, Princeton, NJ, 1962.
  6. J. L. Bentley, D. Haken, and J. B. Saxe, "A General Method for Solving Divide-and-Conquer Recurrences," SIGACT News, 12 (1980), 36-44.
  7. G. S. Bloom, "A Counterexample to the Theorem of Piccard," Journal of Combinatorial Theory A (1977), 378-379.
  8. M. Blum, R. W. Floyd, V. R. Pratt, R. L. Rivest, and R. E. Tarjan, "Time Bounds for Selection," Journal of Computer and System Sciences 7 (1973), 448-461.
  9. A. Borodin and J. I. Munro, The Computational Complexity of Algebraic and Numerical Problems, American Elsevier, New York, 1975.
  10. L. Chang and J. Korsh, "Canonical Coin Changing and Greedy Solutions," Journal of the ACM 23 (1976), 418-422.
  11. N. Christofides, "Worst-case Analysis of a New Heuristic for the Traveling Salesman Problem," Management Science Research Report #388, Carnegie-Mellon University, Pittsburgh, PA, 1976.
  12. D. Coppersmith and S. Winograd, "Matrix Multiplication via Arithmetic Progressions," Proceedings of the Nineteenth Annual ACM Symposium of the Theory of Computing (1987), 1-6.
  13. H. Edelsbrunner, Algorithms in Combinatorial Geometry, Springer-Verlag, Berlin, 1987.
  14. D. Eppstein, Z. Galil, R. Giancarlo, "Speeding up Dynamic Programming," Proceedings of the Twenty-ninth Annual IEEE Symposium on the Foundations of Computer Science, (1988), 488-495.
  15. R. W. Floyd, "Algorithm 97: Shortest Path," Communications of the ACM 5 (1962), 345.
  16. R. W. Floyd and R. L. Rivest, "Expected Time Bounds for Selection," Communications of the ACM 18 (1975), 165-172.
  17. M. L. Fredman, "New Bounds on the Complexity of the Shortest Path Problem," SIAM Journal on Computing 5 (1976), 83-89.
  18. S. Godbole, "On Efficient Computation of Matrix Chain Products," IEEE Transactions on Computers 9 (1973), 864-866.
  19. T. C. Hu and M. R. Shing, "Computations of Matrix Chain Products, Part I," SIAM Journal on Computing 11 (1982), 362-373.
  20. D. A. Huffman, "A Method for the Construction of Minimum Redundancy Codes," Proceedings of the IRE 40 (1952), 1098-1101.
  21. D. S. Johnson, A. Demers, J. D. Ullman, M. R. Garey, and R. L. Graham, "Worst-case Performance Bounds for Simple One-Dimensional Packing Algorithms," SIAM Journal on Computing, 3 (1974), 299-325.
  22. A. Karatsuba and Y. Ofman, "Multiplication of Multi-digit Numbers on Automata," Doklady Akademii Nauk SSSR 145 (1962), 293-294.
  23. D. E. Knuth, The Art of Computer Programming, Vol 2: Seminumerical Algorithms, second edition, Addison-Wesley, Reading, MA, 1981.
  24. D. E. Knuth, "Optimum Binary Search Trees," Acta Informatica 1 (1971), 14-25.
  25. D. E. Knuth and R. W. Moore, "Estimating the Efficiency of Backtrack Programs," Mathematics of Computation 29, (1975) 121-136.
  26. D. E. Knuth, "An Analysis of Alpha-Beta Cutoffs," Artificial Intelligence 6 (1975), 293-326.
  27. D. E. Knuth, TEX and Metafont, New Directions in Typesetting, Digital Press, Bedford, MA, 1981.
  28. D. E. Knuth, "Dynamic Huffman Coding,"Journal of Algorithms 6 (1985), 163-180.
  29. G. M. Landau and U. Vishkin, "Introducing Efficient Parallelism into Approximate String Matching and a New Serial Algorithm," Proceedings of the Eighteenth Annual ACM Symposium on Theory of Computing (1986), 220-230.
  30. L. L. Larmore, "Height-Restricted Optimal Binary Trees," SlAM Journal on Computing 16 (1987), 1115-1123.
  31. L. L. Larmore and D. S. Hirschberg, "A Fast Algorithm for Optimal Length-Limited Huffman Codes," Journal of the ACM 37 (1990), 464-473.
  32. K. Lee and S. Mahajan, "The Development of a World Class Othello Program," Artificial Intelligence 43 (1990), 21-36.
  33. D. A. Lelewer and D. S. Hirschberg, "Data Compression," ACM Computing Surveys 19 (1987), 261- 296.
  34. F. M. Liang, "A Lower Bound for On-line Bin Packing," Information Processing Letters 10 (1980), 76-79.
  35. G. L. Miller, "Riemann's Hypothesis and Tests for Primality," Journal of Computer and System Sciences 13 (1976), 300-317.
  36. L. Monier, "Evaluation and Comparison of Two Efficient Probabilistic Primality Testing Algorithms," Theoretical Computer Science 12 (1980), 97-108.
  37. V. Pan, "Strassen's Algorithm is Not Optimal," Proceedings of the Nineteenth Annual IEEE Symposium on the Foundations of Computer Science (1978), 166-176.
  38. S. K. Park and K. W. Miller, "Random Number Generators: Good Ones are Hard To Find," Communications of the ACM 31 (1988), 1192-1201.
  39. F. P. Preparata and M. I. Shamos, Computational Geometry: An Introduction, Springer-Verlag, New York, NY, 1985.
  40. W. Pugh, "Skip Lists: A Probabilistic Alternative to Balanced Trees," Communications of the ACM 33 (1990), 668-676.
  41. M. O. Rabin, "Probabilistic Algorithms," in Algorithms and Complexity, Recent Results and New Directions (J. F. Traub, ed.), Academic Press, New York, 1976, 21-39.
  42. M. O. Rabin, "Probabilistic Algorithms for Testing Primality," Journal of Number Theory, 12 (1980), 128-138.
  43. P. Ramanan, D. J. Brown, C. C. Lee, and D. T. Lee, "On-line Bin Packing in Linear Time," Journal of Algorithms 10 (1989), 305-326.
  44. M. I. Shamos and D. Hoey, "Closest-Point Problems," Proceedings of the Sixteenth Annual IEEE Symposium on the Foundations of Computer Science (1975), 151-162.
  45. L. Schrage, "A More Portable FORTRAN Random Number Generator," ACM Transactions on Mathematics Software 5 (1979), 132-138.
  46. S. S. Skiena, W. D. Smith, and P. Lemke, "Reconstructing Sets From Interpoint Distances," Proceedings of the Sixth Annual ACM Symposium on Computational Geometry (1990), 332-339.
  47. V. Strassen, "Gaussian Elimination is Not Optimal," Numerische Mathematik 13 (1969), 354-356.
  48. R. A. Wagner and M. J. Fischer, "The String-to-String Correction Problem," Journal of the ACM 21 (1974), 168-173.
  49. A. C. Yao, "New Algorithms for Bin Packing," Journal of the ACM 27 (1980), 207-227.
  50. F. F. Yao, "Efficient Dynamic Programming Using Quadrangle Inequalities," Proceedings of the Twelfth Annual ACM Symposium on the Theory of Computing (1980), 429-435.
  51. J. Ziv and A. Lempel, "A Universal Algorithm for Sequential Data Compression," IEEE Transactions on Information Theory IT23 (1977), 337-343.
  52. J. Ziv and A. Lempel, "Compression of Individual Sequences via Variable-rate Coding," IEEE Transactions on Information Theory IT24 (1978), 530-536.

CHAPTER 11: AMORTIZED ANALYSIS

An excellent survey of amortized analysis is provided in [9].
Most of the references below duplicate citations in earlier chapters. We cite them again for convenience and completeness. Binomial queues were first described in [10] and analyzed in [1]. Solutions to 11.3 and 11.4 appear in [8]. Fibonacci heaps are described in [3]. Exercise 11.9a shows that splay trees are optimal, to within a constant factor of the the best static search trees. 11.9b shows that splay trees are optimal, to within a constant factor of the best optimal search trees. These, as well as two other strong results, are proved in the original splay tree paper [6].
The merge operation for splay trees is described in [5]. Exercise 11.12 is solved, with an implicit use of amortization, in [2]. The paper also shows how to merge 2-3 trees efficiently. A solution to 11.13 can be found in [4].
Amortized analysis is used in [7] to design an on-line algorithm that processes a series of queries in time only a constant factor larger than any off-line algorithm in its class.
  1. M. R. Brown, "Implementation and Analysis of Binomial Queue Algorithms," SIAM Journal on Computing 7 (1978), 298-319.
  2. M. R. Brown and R. E. Tarjan, "Design and Analysis of a Data Structure for Representing Sorted Lists," SIAM Journal on Computing 9 (1980), 594-614.
  3. M. L. Fredman and R. E. Tarjan, "Fibonacci Heaps and Their Uses in Improved Network Optimization Algorithms," Journal of the ACM 34 (1987), 596-615.
  4. H. Gajewska and R. E. Tarjan, "Deques with Heap Order," Information Processing Letters 22 (1986), 197-200.
  5. G. Port and A. Moffat, "A Fast Algorithm for Melding Splay Trees," Proceedings of the First Workshop on Algorithms and Data Structures, 1989, 450-459.
  6. D. D. Sleator and R. E. Tarjan, "Self-adjusting Binary Search Trees," Journal of the ACM 32 (1985), 652-686.
  7. D. D. Sleator and R. E. Tarjan, "Amortized Efficiency of List Update and Paging Rules," Communications of the ACM 28 (1985), 202-208.
  8. D. D. Sleator and R. E. Tarjan, "Self-adjusting Heaps," SIAM Journal on Computing 15 (1986), 52-69.
  9. R. E. Tarjan, "Amortized Computational Complexity," SIAM Journal on Algebraic and Discrete Methods 6 (1985), 306-318.
  10. J. Vuillemin, "A Data Structure for Manipulating Priority Queues," Communications of the ACM 21 (1978), 309-314.

Notes

Chapter 1: INTRODUCTION

  • 1.2.3

    http://orgmode.org/org.html#LaTeX-fragments

    ∑i=1N ik ≈ \frac{Nk+1}{\abs{k+1}} k ≠ -1

  • 1.3

    When writhing recursive routines, it is crucial to keep in mind the four basic rules of recursion:

    1. Base cases. You must always have some base cases, which can be solved without recursion.
    2. Making progress. For the cases that are to be solved recursively, the recursive call must always be to a case that makes progress toward a base class.
    3. Design rule. Assume that all the recursive calls work.
    4. Compound interest rule. Never duplicate work by solving the same instance of a problem in separate recursive calls.
  • 1.5.2

    To summarize the parameter-passing options:

    • Call by value is appropriate for small objects that should not be altered by the function.
    • Call by constant reference is appropriate for large objects that should not be altered by the function.
    • Call by reference is appropriate for all objects that may be altered by the function.
  • 1.5.5

    This is a so-called shallow copy. Typically, we would expect a deep copy, in which a clone of the entire object is made.

CHAPTER 2: ALGORITHM ANALYSIS

  • 2.1

    logk N = O(N) for any constant k.

    compute limN→∞ f(N)/g(N), using L'Hopital's rule if necessary.

    The rule is if limN→∞ f(N) = ∞ and limN→∞ g(N) = ∞, then limN→∞ f(N)/g(N) = limN→∞f'(N)/g'(N).

    The limit can have four possible values:

    • The limit is 0: This means that f(N) = o(g(N)).
    • The limit is c ≠ 0: This means that f(N) = Θ(g(N)).
    • The limit is ∞: This means that g(N) = o(f(N)).
    • The limit is oscillates: There is no relation.
  • 2.4.2
    • RULE 1-FOR LOOPS:

    The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations.

    • RULE 2-NESTED FOR LOOPS:

    Analyze these inside out. The total running time of a statement inside a group of nested for loops is the running time of the statement multiplied by the product of the sizes of all the for loops.

    • RULE 3-CONSECUTIVE STATEMENTS:

    These just add (which means that the maximum is the one that counts – see 1(a) on page 16).

    • RULE 4-lF/ELSE:

    For the fragment 

    if( cond )
S1
else
S2

    the running time of an if/else statement is never more than the running time of the test plus the larger of the running times of S1 and S2.

  • 2.4.3
    • Algorithm 1 (O(N3)) 
    int maxSubSum1(const vector<int> &a)
{
  int maxSum = 0;
  for(int i = 0; i < a.size(); i++)
    for(int j = i; j < a.size(); j++){
      int thisSum = 0;
      for(int k = i; k <= j; k++){
        thisSum += a[k];
      }
      if(maxSum < thisSum)
        maxSum = thisSum;
    }
  return maxSum;
}
    • Algorthm 2 (O(N2)) 
    int maxSubSum2(const vector<int> &a)
{
  int maxSum = 0;
  for(int i = 0; i < a.size(); i++){
    int thisSum = 0;
    for(int j = i; j < a.size(); j++){
      thisSum += a[j];
      if(maxSum < thisSum)
        maxSum = thisSum;
    }
  }
  return maxSum;
}
    • Algorithm 3 (O(Nlog(N))) 
    int max3(int a, int b, int c)
{
  int max;
  if(a > b)
    max = a;
  else
    max = b;
  if(max < c)
    max = c;
  return max;
}

int maxSubRec(const vector<int> &a, int left, int right)
{
  if(left == right)
    if(a[left] > 0)
      return a[left];
    else
      return 0;

  int center = (left + right) / 2;

  int maxLeftSum = maxSubRec(a, left, center);
  int maxRightSum = maxSubRec(a, center + 1, right);

  int maxLeftBorderSum = 0, leftBorderSum = 0;
  for(int i = center; i >= left; i--){
    leftBorderSum += a[i];
    if(maxLeftBorderSum < leftBorderSum)
      maxLeftBorderSum = leftBorderSum;
  }

  int maxRightBorderSum = 0, rightBorderSum = 0;
  for(int i = center+1; i <= right; i++){
    rightBorderSum += a[i];
    if(maxRightBorderSum < rightBorderSum)
      maxRightBorderSum = rightBorderSum;
  }

  return max3(maxLeftSum, maxRightSum, maxLeftBorderSum + maxRightBorderSum);
}


int maxSubSum3(const vector<int> &a)
{
  return maxSubRec(a, 0, a.size() - 1);
}
    • Algorithm 4 (O(log(N))) 
    int maxSubSum4(const vector<int> &a)
{
  int maxSum = 0;
  int thisSum = 0;
  for(int i = 0; i < a.size(); i++){
    thisSum += a[i];
    if(thisSum < 0)
      thisSum = 0;
    if(thisSum > maxSum)
      maxSum = thisSum;

  }
  return maxSum;
}
    • testmain.cpp 
    int main(int  argc, char *argv[])
{
  int array[] = { 4, -3, 5, -2, -1, 2, 6, -1};
  vector<int> a(array, array + sizeof(array) / sizeof(int));

  int maxSum;
  maxSum = maxSubSum1(a);
  cout << "maxSum1 = " << maxSum << endl;
  maxSum = maxSubSum2(a);
  cout << "maxSum2 = " << maxSum << endl;
  maxSum = maxSubSum3(a);
  cout << "maxSum3 = " << maxSum << endl;
  maxSum = maxSubSum4(a);
  cout << "maxSum4 = " << maxSum << endl;
  return 0;
}
  • 2.4.4
    • Binary Search(O(logN)) 
    template <class Comparable>
int binarySearch(const vecotr<Comparable> &a, const Comparable &x)
{
  int left = 0, right = a.size() - 1;

  while(left < right){
    int mid = (left + right) / 2;
    if(a[mid] == x)
      return mid;
    else if(a[mid] > x)
      right = mid - 1;
    else
      left = mid + 1;
  }
  return NOT_FOUND; // NOT_FOUND s defined as -1
}
    • Euclid's Algorithm (O(logN)) 
    long gcd(long m, long n) //assume m > n
{
  long rem;
  while(n =! 0){
    rem = m % n;
    m = n;
    n = rem;}
  return m;
}
    • Exponentiation 
    long pow(ling x, int n)
{
  if(n = 0)
    return 1;

  if(isEven(n))
    return pow(x * x, n / 2);
  else
    return  pow(x * x, n / 2) * x;
}

Chapter 3: Lists, Stacks, and Queues

Chapter 4: Trees

CHAPTER 5: HASHING

CHAPTER 6: PRIORITY QUEUES (HEAPS)

CHAPTER 7: SORTING

CHAPTER 8: THE DISJOINT SET ADT

CHAPTER 9: GRAPH ALGORITHMS

CHAPTER 10: ALGORITHM DESIGN TECHNIQUES

CHAPTER 11: AMORTIZED ANALYSIS

Footnotes:

1

DEFINITION NOT FOUND.

2

DEFINITION NOT FOUND.

3

DEFINITION NOT FOUND.

4

DEFINITION NOT FOUND.

5

DEFINITION NOT FOUND.

6

DEFINITION NOT FOUND.

7

DEFINITION NOT FOUND.

8

DEFINITION NOT FOUND.

9

DEFINITION NOT FOUND.

10

DEFINITION NOT FOUND.

11

DEFINITION NOT FOUND.

12

DEFINITION NOT FOUND.

13

DEFINITION NOT FOUND.

14

DEFINITION NOT FOUND.

15

DEFINITION NOT FOUND.

16

DEFINITION NOT FOUND.

17

DEFINITION NOT FOUND.

18

DEFINITION NOT FOUND.

19

DEFINITION NOT FOUND.

20

DEFINITION NOT FOUND.

21

DEFINITION NOT FOUND.

22

DEFINITION NOT FOUND.

23

DEFINITION NOT FOUND.

24

DEFINITION NOT FOUND.

25

DEFINITION NOT FOUND.

26

DEFINITION NOT FOUND.

27

DEFINITION NOT FOUND.

28

DEFINITION NOT FOUND.

29

DEFINITION NOT FOUND.

30

DEFINITION NOT FOUND.

31

DEFINITION NOT FOUND.

32

DEFINITION NOT FOUND.

33

DEFINITION NOT FOUND.

34

DEFINITION NOT FOUND.

35

DEFINITION NOT FOUND.

36

DEFINITION NOT FOUND.

37

DEFINITION NOT FOUND.

38

DEFINITION NOT FOUND.

39

DEFINITION NOT FOUND.

40

DEFINITION NOT FOUND.

Author: Shi Shougang

Created: 2017-01-26 Thu 21:10

Emacs 24.3.1 (Org mode 8.2.10)

Validate