logarithms, polynomials, exponentials, Moore's law, big-O notation, a trick for bounding sums, geometric sums
(under construction)
Review of chapter 3.
polynomials:
For any polynomial p(n), multiplying n by a constant factor changes p(n) by at most a constant factor.
If your algorithm has worst-case running time bounded by a polynomial in the size of the input, that is good!
exponentials:
Exponentials with base > 1 (e.g. 2^n) grow large very fast. Exponentials with base < 1 (e.g. 2^{-n}) grow small very fast.
For any exponential function f(n) (such as 2n), if you increase n by a constant factor (say a factor of 2) how much can the function increase by?
If your algorithm takes time exponential in the size of the input, it won't be useful for solving very large problems!
logarithms:
log(n) grows quite slowly. e.g. log10(10100) is only 100.
If you increase n by a constant factor (say a factor of 2) how much does log(n) increase by?
summations: ∑i=ab f(i)
1. We try to bound the running time from above by a function of the size of the input.
For example, "to bubblesort an n-item list takes at most time proportional to n2". Here we take the number of items in the list as a measure of the input size.
"Euclid's algorithm takes at most time proportional to log(min(i,j))"? What is the size of the input? (Trick question. Generally, by the size of an input instance, we mean the number of characters it takes to write down the instance. So, the size of an integer n, in this sense is proportional to log(n).) Euclid's algorithm runs in polynomial time.
2. We usually neglect constant factors in the running time. ("the time is at most proportional to..." something.)
3. Running time is proportional to the number of basic operations made by the algorithm. What is a basic operation? Need to think about what machine has to do to support the operation. Basic arithmetic, array access, expression evaluation, etc.
Moore's law: Every few years, computers get faster by a constant factor.
If you get a computer that is faster by a constant factor than your old one, the new one can solve bigger problems than your old one. How much bigger? Depends on the algorithm:
We've been using the phrase proportional to. To be more precise about what we mean, we will start using the big-O notation. Here we define it.
Or, we might say that f(n) is at most proportional to g(n).
llustration: 3 (sin(x/10)+2) is O(x) (take n0 = 100 and c=4. Also, x is O(3 (sin(x/10)+2))
For example, let's apply the definition to try to show 3n2+10 is O(n2).
We need to find constants n0 and c such that (∀ n ≥ n0) 3n2+10 ≤ c n2. Let's work backwards to find the constants that will work.
We want 3n2+10 ≤ c n2 for n ≥ n0.
Simplifying and solving for c, we want c ≥ 3+10/n2 for n ≥ n0.
Let's try n0 = 10. Since 10/n2 gets smaller as n gets larger, we will have want we want provided c ≥ 3+10/100 for n ≥ 10. This is true for c = 4.
So, we've worked out that 3n2 + 10 is O(n2). If we wanted to make a clean proof out of our reasoning, it might look like this:
Or, if we are confident of our algebra and confident that the intended reader of the proof can work it out without help, we could just say:
exercise: Using the definition, prove that 1000 n2 is O(n10).
exercise: Using the definition (and contradiction), prove that n2 is <i>not</i> O(n).
exercise: Define f(n) = 100. Prove that f(n) is O(1).
Rules of thumb:
We also discussed:
Recall that ∑i=1..n i2 is 1+22+32+ ... + n2.
This looks at first like a quadratic polynomial , so you might be tempted to say it is O(n2). But that's wrong. If you try to apply the definition of big-O, you will see why.
It is possible to use clever tricks to get "closed-form" expressions for sums such as this. For example, you may know that 1+2+3+...+n = n(n+1)/2.
But in this class, we generally are happy to get upper and lower bounds on the sum that are within constant factors of each other. Here's a useful trick for doing that:
Consider ∑i=1..n i2 is 1+22+32+ ... + n2. To get the upper bound, note that there are n terms, and each term is at most n2. Thus,
This trick will give tight bounds (up to constant factors) for sums where the largest n/2 terms are proportional to the maximum term.
Geometric sums do not have this property. For example, consider the sum
The upper bound trick gives us that this sum is at most n2n, the lower bound trick gives us that the sum is at least n 2n/2/2.
Note that 2n/2 is MUCH SMALLER than 2n. To see this consider their ratio: 2n / 2n/2 = 2n-n/2 = 2n/2. So, the lower bound is off. A better lower bound is simply to take the largest term:
The punch line: a geometric sum has value proportional to its largest term.