Expected bounds on universal hashing - hash

Question

I am sure this is a simple problem to work out, but I don't see an obvious solution... If I have a hash table with m bins and hash into this n < m keys, what is the probability that no bin receives more than k hash-keys. I'm trying to figure out how many rehash operations I should expect if I fill up a table to load n / m and then rehash until I see no more than k collisions in any bin (obviously with k > n / m).

Answer 1

With uniform distribution, this is the same as throwing balls into bins, which has been studied in "Balls into Bins - A Simple and Tight Analysis" from M. Raab and A. Steger.
This is a bit related to cuckoo hashing, but here you just use one hash function.
As this is stackoverflow.com, I give you a simulation program that can be used to verify your formula. According to this, it also depends on the number of balls / buckets, and not just on the the average number of balls per bucket.
public static void main(String... args) throws InterruptedException {
for (int k = 1; k < 4; k++) {
test(10, 30, k);
test(100, 300, k);
}
}
public static void test(int ballCount, int binCount, int k) {
int rehashCount = 0;
Random r = new Random(1);
int testCount = 100000000 / ballCount;
for(int test = 0; test < testCount; test++) {
long[] balls = new long[ballCount];
int[] bins = new int[binCount];
for (int i = 0; i < ballCount; i++) {
balls[i] = r.nextLong();
}
// it's very unlikely to get duplicates, but test
Arrays.sort(balls);
for (int i = 1; i < ballCount; i++) {
if (balls[i - 1] == balls[i]) {
throw new AssertionError();
}
}
int universalHashId = 0;
boolean rehashNeeded = false;
for (int i = 0; i < ballCount; i++) {
long x = balls[i];
// might as well do y = x
long y = supplementalHashWeyl(x, universalHashId);
int binId = reduce((int) y, binCount);
if (++bins[binId] > k) {
rehashNeeded = true;
break;
}
}
if (rehashNeeded) {
rehashCount++;
}
}
System.out.println("balls: " + ballCount + " bins: " + binCount +
" k: " + k + " rehash probability: " + (double) rehashCount / testCount);
}
public static int reduce(int hash, int n) {
// http://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/
return (int) (((hash & 0xffffffffL) * n) >>> 32);
}
public static int supplementalHashWeyl(long hash, long index) {
long x = hash + (index * 0xbf58476d1ce4e5b9L);
x = (x ^ (x >>> 32)) * 0xbf58476d1ce4e5b9L;
x = ((x >>> 32) ^ x);
return (int) x;
}
Outputs:
balls: 10 bins: 30 k: 1 rehash probability: 0.8153816
balls: 100 bins: 300 k: 1 rehash probability: 1.0
balls: 10 bins: 30 k: 2 rehash probability: 0.1098305
balls: 100 bins: 300 k: 2 rehash probability: 0.777381
balls: 10 bins: 30 k: 3 rehash probability: 0.0066018
balls: 100 bins: 300 k: 3 rehash probability: 0.107309