RecurrenceRelations

Introduction

Recurrence relations are a tool for analyzing recursive algorithms.

MergeSort?

An example you may be familiar with is MergeSort? . A recursion diagram for mergesort looks like this:

---

---

Define T(n) to be the time taken to mergesort n elements (neglecting constant factors). Then T(n) satisfies the following:

T(1) = 1

T(n) = 2 T(n/2) + n (for n > 1)

The above relation between T(n) and T(n/2) is called a recurrence relation.

The relation is true because the time to mergesort an array with n elements equals the time to mergesort an array with n/2 elements, plus the time to mergesort another array with n/2 elements, plus n.

Binary Search

Similarly, a recurrence relation for the time to do a binary search on an array of N elements is:

T(1) = 1

T(N) = 1 + T(N/2).

That is, a binary search in an array of size N does constant work, then recurses on one of the two halves of the array (a problem of size N/2).

The recursion diagram in this case just looks like this:

      
     N
     |
    N/2
     |
    N/4
     |
    ...
     |
     4
     |
     2
     |
     1

In this case, the work associated with each node of the tree is 1, and the depth of the tree is log(N), so the total work done is O(log N).

Towers of Hanoi

The number of single-disc moves to transfer N discs in the TowersOfHanoi? problem satisfies:

T(N) = 2 T(N-1) + 1

T(1) = 1

The corresponding recursion diagram is a complete binary tree of depth N, and for each node, 1 disc is moved. Thus, the total number of moves made to transfer N discs is 2^N-1.

---

Solving recurrence relations

Consider these:

1. T(n) = 2 T(n/3) + n
2. T(n) = 3 T(n/3) + n
3. T(n) = 4 T(n/3) + n

1. recursion diagram is a complete binary tree of depth log₃ n. The Ith level from the top has 2ⁱ nodes, each with work n/3ⁱ.

Total work is

n[ 1 + (2/3) + (2/3)² + ... + (2/3)^{log₃ n}]

which is O(n).

2. same as above but each node has 3 children, so the Ith level from the top has 3ⁱ nodes.

Total work is

n[ 1 + (3/3) + (3/3)² + ... + (3/3)^{log₃ n}]

which is O(n log n).

3. now each node has 4 children, so total work is

n[ 1 + (4/3) + (4/3)² + ... + (4/3)^{log₃ n}]

which is O(n (4/3)^{log₃ n}) = O(n^{1+log₃ 4/3}) = O(n^1.26...) .

More

1. T(n) = 8 T(n/3) + n²
2. T(n) = 9 T(n/3) + n²
3. T(n) = 10 T(n/3) + n²

1. recursion diagram is of depth log₃ n. Each node has 8 children. The Ith level from the top has 8ⁱ nodes, each with work (n/3ⁱ)² = n² / 9ⁱ .

Total work is

n²[ 1 + (8/9) + (8/9)² + ... + (8/9)^{log₃ n}]

which is O(n²).

2. Total work is

n²[ 1 + (9/9) + (9/9)² + ... + (9/9)^{log₃ n}]

which is O(n² log n).

3. Total work is

n²[ 1 + (10/9) + (10/9)² + ... + (10/9)^{log₃ n}]

which is O(n² (10/9)^{log₃ n}) = O(n^2.095...).

Repeated Substitution

T(n) = 1.5 T(n/3.1) + n^0.5.

No recursion diagram!?

Try repeated substitution. Use the recurrence relation to replace T(n/3.1) with (1.5 T(n/3.1²) + (n/3.1)^0.5 :

T(n) = 1.5 T(n/3.1) + n^0.5

= 1.5 [ (1.5 T(n/3.1²) + (n/3.1)^0.5 ] + n^0.5 .

Then use the recurrence relation to expand T(n/3.1²), and so on, simplifying as you go.

After i substitutions, get T(n) = (1.5)ⁱ T(n / 3.1ⁱ) + (1.5)^i-1 [n/3.1^i-1]^0.5 + ... .

Taking i = \log_3.1 n, after i substitutions the term left is T(1) (or at least T(O(1))), which we substitute using the base case of the recurrence relation. This gives us

T(n) = n^0.5[ 1 + (1.5/3.1^0.5) + (1.5/3.1^0.5)² + ... + (1.5/3.1^0.5)^{log_1.5 n}]

Since 1.5 < 3.1^0.5, this is O(n^0.5). (See GeometricSums .)

Proving bounds using induction

1. T(n) = T(n/3) + T(n/2) + n, T(1) = 1.

Can make a recursion diagram for this. Every node has two children, but depth is uneven, and work is not the same for each node on a level. Try guessing, then prove by induction.

Guess that T(n) ≤ 10 n. Attempt proof by induction:

Base case: T(1) ≤ 10 . (true)

Inductive step: T(n) = T(n/3) + T(n/2) + n ≤ 10(n/3) + 10(n/2) + n = (10/3+10/2 + 1) n = 9.33.. n ≤ 10 n. (verified)

What is the best constant that would work?

Guess that T(n) ≤ c n. Attempt proof by induction:

Base case: T(1) ≤ c . (need c ≥ 1 )

Inductive step: T(n) = T(n/3) + T(n/2) + n ≤ c(n/3) + c(n/2) + n = (c(1/3+1/2) + 1) n ≤ c n ?

Need c(1/3+1/2)+1 ≤ c . True iff c ≥ 6 .

Conclude T(n) ≤ 6 n .

---

References

• Section 5.2 in GoodrichAndTomassia
• ClassS04CS141/ChallengeProblem2