It was a question on my final I took earlier and I had no idea how to answer it.
Well it was
What is Merge sort's worst case runtime but MORE IMPORTANTLY, why?
The divide-and-conquer contributes a log(n) factor. You divide the array in half log(n) times, and each time you do, for each segment, you have to do a merge on two sorted array. Merging two sorted arrays is O(n). The algorithm is just to walk up the two arrays, and walk up the one that's lagging.
The recursion you get is r(n) = O(n) + r(roundup(n/2))+r(rounddown(n/2).
The problem is that you cant use the Masters Theorem for solving this due to the rounding. Hence you can ether do the math or use a little hack-like solution. If ur input isn't a power of two number just "blow it up". Then u can use the masters theorem on r(n) = O(n) + 2r(n/2). Obviously this leads to O(nlogn). The function merge() itself is in O(n), because in the worst case you need n-1 compares.
Related
The Wikipedia page: wikipedia page states that
If m operations, either Union or Find, are applied to n elements, the total run time is O(m log*n).
The detailed analysis arrived at this result is :
My questions are:
Shouldn't the sum be (m+n)log*n instead of mlog*n?
Are the average time complexity for 1000 Find operations the same as the time complexity of each individual Find?
Disclaimer: I'm still trying to understand these proofs myself, so make no claim to being an expert! I think I may have some insights though.
1) I think they have assumed that m = O(n), thereby making O((m + n)lg*(n)) into O(mlg*(n)). In Tarjan's original proof (of the inverse Ackermann function bound) (found here: https://dl.acm.org/doi/pdf/10.1145/321879.321884?download=true) he assumes that the number m of FIND operations exceeds n. In Introduction to Algorithms (CLRS - ch.21), the bound they prove is for m operations of which n are MAKE-SET. It seems like people are assuming that m will be asymptotically greater than or equal to n.
2) What they have proved is an amortized cost for each operation. This is an analysis technique which bounds to within a constant factor the time taken for a series of operations, from which you can trivially compute the average time per operation. There are several different ways to go about it (I believe this is an example of aggregate analysis?). It's worth looking into!
For clarification, first(where:) method keeps iterating through the sequence until it finds the satisfied element and returns it.
Based on that I would assume that it is not O(n) (linear time) because at some point it doesn't has to iterate through the whole sequence until its end.
You could check: What is the difference between filter(_:).first and first(where:)?
I'm not sure if it could be something relates to O(log n), AFAIK it has something to do with splitting into halves...
It would be great if someone could describe how we can determine the time complexity for such a process.
We are usually interested in the worst case running time of a program. Based on that, it should be O(n) as the worst case is when it iterates through all the elements.
On average you'll only have to check 1/2 of the values, so you'd think first(where:) would be O(1/2 N). But O() notation ignores constants. O(N) means it grows linearly as the number of elements grows. For 10 items, you'd check 5 on average, for 100, you'd check 50, for 1000, you'd check 500 on average. Connect the points (10,5), (100,50), (1000, 500). That's a straight line.
How is O(n)/n=1 in aggregate method of amortized analysis as given in the coursera course on data structures in lesson 5-Amortized analysis:Aggregate method?
Short answer
O(n)/n = cn/n = c = O(1)
Long answer
We use amortized analysis in order to analyze the cost of a sequence of operations rather that the cost of a single operation. In the last case we use asymptotic analysis (some of the asymptotic notations are: Theta, Big O, Big Omega, Little O and Little Omega), but it doesn't work that well when we come across a sequence of operations and want to understand the cost of that sequence.
The reason is that if we apply "regular" asymptotic analysis, our, for example, asymptotical upper bound in the worst case analysis might be too pessimistic. Classical example is inserting into a dynamic array. You insert elements into a dynamically allocated array and when it's full, you define new array (twice as big, for example) and copy all the elements. The thing is most of the insertions will work in constant time (or in O(1)), but when you need to redefine your array, it will take linear time (O(n)), because you need to copy all the elements.
So imagine that you insert n elements and you need to redefine your array only once, then you have n operations, each operation is O(n) in the worst case, hence the cost of the sequence of operations in the worst case is O(n^2), which seems too pessimistic considering the fact that most of your operations are O(1) in the worst case and only one of them is O(n).
We define the amortized cost of a sequence of operations as (cost of n operations) / n. In your case the cost of n operations is O(n) which is equal to cn (where c is some constant) just by the definition of the Big O notation, divide it by n and you get just c, which is equal to O(1) because, once again, c is just some constant.
Is it O(n) or O(n logn)? I have n elements that I need to setup in a hash table, what is the worst-case and average runtime?
Worst case is unlimited. You need to calculate hash codes and may have to compare elements, and the time for that is not limited.
Assuming that calculating hashes and comparing elements is constant time, for insertion the worst case is O (n^2). What saves you is the fact that the worst case would be exceedingly rare, assuming a halfway decent has function. Average time for a decent implementation is O (n).
Mainly, why is it so fast (for big numbers)? The documentation only tells me how to use it. For example, it needs at most one second to find the largest prime factor of 1234567890987654, which, to me, seems insane.
>>max(factor(1234567890987654))
ans =
69444443
The largest factor to be tried is sqrt(N), or 35136418 in this case. Also even the most elementary optimizations would skip all even numbers > 2, leaving only 17568209 candidates to be tested. Once the candidate 17777778 (and it's cofactor 69444443) is found, the algorithm would be wise enough to stop.
This can be somewhat easily improved further by a modified sieve to skip multiples of small primes 2,3,5[,7].
Basically even the sqrt(N) optimization is enough for the noted performance, unless you are working on an exceptionally old CPU (8086).
It's interesting to look at the source code of the factor and primes functions.
factor(N) essentiallty calls primes to find out all primes up to sqrt(N). Once they have been identified, it tests them one by one to see if they divide N.
primes(n) uses Eratosthenes' sieve: for each identified prime, remove all its multiples, exploiting sqrt again to reduce complexity.