for example 20986 and 96208 should generate the same key (but not 09862 or 9862 as leading zero means it not even a 5 digit number so we igore those).
One option is to get the least/max sorted permutation and then the sorted number is the hashkey, but sorting is too costly for my case. I need to generate key in O(1) time.
Other idea I have is to traverse the number and get frequency of each digits and the then get a hash function out of it. Now whats the best function to combine the frequencies given that 0<= Summation(f[i]) <= no_of_digits.
To create an order-insensitive hash simply hash each value (in your case the digits of the number) and then combine them using a commutative function (e.g. addition/multiplication/XOR). XOR is probably the most appropriate as it retains a constant hash output size and is very fast.
Also, you will want to strip away any leading 0's before hashing the number.
I would like to partition a number into an almost equal number of values in each partition. The only criteria is that each partition must be in between 60 to 80.
For example, if I have a value = 300, this means that 75 * 4 = 300.
I would like to know a method to get this 4 and 75 in the above example. In some cases, all partitions don't need to be of equal value, but they should be in between 60 and 80. Any constraints can be used (addition, subtraction, etc..). However, the outputs must not be floating point.
Also it's not that the total must be exactly 300 as in this case, but they can be up to a maximum of +40 of the total, and so for the case of 300, the numbers can sum up to 340 if required.
Assuming only addition, you can formulate this problem into a linear programming problem. You would choose an objective function that would maximize the sum of all of the factors chosen to generate that number for you. Therefore, your objective function would be:
(source: codecogs.com)
.
In this case, n would be the number of factors you are using to try and decompose your number into. Each x_i is a particular factor in the overall sum of the value you want to decompose. I'm also going to assume that none of the factors can be floating point, and can only be integer. As such, you need to use a special case of linear programming called integer programming where the constraints and the actual solution to your problem are all in integers. In general, the integer programming problem is formulated thusly:
You are actually trying to minimize this objective function, such that you produce a parameter vector of x that are subject to all of these constraints. In our case, x would be a vector of numbers where each element forms part of the sum to the value you are trying to decompose (300 in your case).
You have inequalities, equalities and also boundaries of x that each parameter in your solution must respect. You also need to make sure that each parameter of x is an integer. As such, MATLAB has a function called intlinprog that will perform this for you. However, this function assumes that you are minimizing the objective function, and so if you want to maximize, simply minimize on the negative. f is a vector of weights to be applied to each value in your parameter vector, and with our objective function, you just need to set all of these to -1.
Therefore, to formulate your problem in an integer programming framework, you are actually doing:
(source: codecogs.com)
V would be the value you are trying to decompose (so 300 in your example).
The standard way to call intlinprog is in the following way:
x = intlinprog(f,intcon,A,b,Aeq,beq,lb,ub);
f is the vector that weights each parameter of the solution you want to solve, intcon denotes which of your parameters need to be integer. In this case, you want all of them to be integer so you would have to supply an increasing vector from 1 to n, where n is the number of factors you want to decompose the number V into (same as before). A and b are matrices and vectors that define your inequality constraints. Because you want equality, you'd set this to empty ([]). Aeq and beq are the same as A and b, but for equality. Because you only have one constraint here, you would simply create a matrix of 1 row, where each value is set to 1. beq would be a single value which denotes the number you are trying to factorize. lb and ub are the lower and upper bounds for each value in the parameter set that you are bounding with, so this would be 60 and 80 respectively, and you'd have to specify a vector to ensure that each value of the parameters are bounded between these two ranges.
Now, because you don't know how many factors will evenly decompose your value, you'll have to loop over a given set of factors (like between 1 to 10, or 1 to 20, etc.), place your results in a cell array, then you have to manually examine yourself whether or not an integer decomposition was successful.
num_factors = 20; %// Number of factors to try and decompose your value
V = 300;
results = cell(1, num_factors);
%// Try to solve the problem for a number of different factors
for n = 1 : num_factors
x = intlinprog(-ones(n,1),1:n,[],[],ones(1,n),V,60*ones(n,1),80*ones(n,1));
results{n} = x;
end
You can then go through results and see which value of n was successful in decomposing your number into that said number of factors.
One small problem here is that we also don't know how many factors we should check up to. That unfortunately I don't have an answer to, and so you'll have to play with this value until you get good results. This is also an unconstrained parameter, and I'll talk about this more later in this post.
However, intlinprog was only released in recent versions of MATLAB. If you want to do the same thing without it, you can use linprog, which is the floating point version of integer programming... actually, it's just the core linear programming framework itself. You would call linprog this way:
x = linprog(f,A,b,Aeq,beq,lb,ub);
All of the variables are the same, except that intcon is not used here... which makes sense as linprog may generate floating point numbers as part of its solution. Due to the fact that linprog can generate floating point solutions, what you can do is if you want to ensure that for a given value of n, you could loop over your results, take the floor of the result and subtract with the final result, and sum over the result. If you get a value of 0, this means that you had a completely integer result. Therefore, you'd have to do something like:
num_factors = 20; %// Number of factors to try and decompose your value
V = 300;
results = cell(1, num_factors);
%// Try to solve the problem for a number of different factors
for n = 1 : num_factors
x = linprog(-ones(n,1),[],[],ones(1,n),V,60*ones(n,1),80*ones(n,1));
results{n} = x;
end
%// Loop through and determine which decompositions were successful integer ones
out = cellfun(#(x) sum(abs(floor(x) - x)), results);
%// Determine which values of n were successful in the integer composition.
final_factors = find(~out);
final_factors will contain which number of factors you specified that was successful in an integer decomposition. Now, if final_factors is empty, this means that it wasn't successful in finding anything that would be able to decompose the value into integer factors. Noting your problem description, you said you can allow for tolerances, so perhaps scan through results and determine which overall sum best matches the value, then choose whatever number of factors that gave you that result as the final answer.
Now, noting from my comments, you'll see that this problem is very unconstrained. You don't know how many factors are required to get an integer decomposition of your value, which is why we had to semi-brute-force it. In fact, this is a more general case of the subset sum problem. This problem is NP-complete. Basically, what this means is that it is not known whether there is a polynomial-time algorithm that can be used to solve this kind of problem and that the only way to get a valid solution is to brute-force each possible solution and check if it works with the specified problem. Usually, brute-forcing solutions requires exponential time, which is very intractable for large problems. Another interesting fact is that modern cryptography algorithms use NP-Complete intractability as part of their ciphertext and encrypting. Basically, they're banking on the fact that the only way for you to determine the right key that was used to encrypt your plain text is to check all possible keys, which is an intractable problem... especially if you use 128-bit encryption! This means you would have to check 2^128 possibilities, and assuming a moderately fast computer, the worst-case time to find the right key will take more than the current age of the universe. Check out this cool Wikipedia post for more details in intractability with regards to key breaking in cryptography.
In fact, NP-complete problems are very popular and there have been many attempts to determine whether there is or there isn't a polynomial-time algorithm to solve such problems. An interesting property is that if you can find a polynomial-time algorithm that will solve one problem, you will have found an algorithm to solve them all.
The Clay Mathematics Institute has what are known as Millennium Problems where if you solve any problem listed on their website, you get a million dollars.
Also, that's for each problem, so one problem solved == 1 million dollars!
(source: quickmeme.com)
The NP problem is amongst one of the seven problems up for solving. If I recall correctly, only one problem has been solved so far, and these problems were first released to the public in the year 2000 (hence millennium...). So... it has been about 14 years and only one problem has been solved. Don't let that discourage you though! If you want to invest some time and try to solve one of the problems, please do!
Hopefully this will be enough to get you started. Good luck!
I'm looking for a hashfunction which exploits the following requirements:
N distinct integer values will be stored in the hashtable
At any given point in time there will be no more than M values present in the hashtable
Hashtable stays static for several queries (i.e. at some point the whole hashtable will be initialized and the following calls only read from the hash table)
largest possible key value K is known at the initialization of the hashtable (K >> N)
Every queried key-value pair is present in the hashtable
So far I'm using a hash-function like:
h(k) = 7 * k % M
with M = PRIME_CLOSE_TO(7*N)
7 is somewhat arbitrary.
Do you have any suggestions on how to improve this?
This is a starting point: http://en.wikipedia.org/wiki/Perfect_hash_function
In practice, any ordinary hash function would be fine. But if you want a minimal perfect hash for some reason, you may look into a library that does perfect hashing, such as: CMPH - C Minimal Perfect Hashing Library
I have a function that is looking at a number of elements. Each element is of the form of an 8x1 column vector. Each entry in the vector is an integer less than 1000. Every time I see such a vector, I'd like to add it to a list of "already seen" vectors, after checking to see that the vector is not already on this list. The function will examine on the order of ~100,000 such vectors.
Originally I tried using ismember(v', M, 'rows'), but found this to be very slow. Next I tried:
found = containers.Map('KeyType', 'double', 'ValueType', 'any');
Then each time I examine a new vector v, compute:
key = dot(v, [1000000000000000000000 1000000000000000000 1000000000000000 ...
1000000000000 1000000000 1000000 1000 1]);
Then check isKey(found, key). If the key is not in the container, then found(key) = 1.
This seems like a pretty lousy solution, even though it does run considerably faster than ismember. Any help/suggestions would be greatly appreciated.
EDIT: Perhaps it would be better to use mat2str to generate the key, rather than this silly dot product?
The easiest way to generate a key/hash in your case would be to just convert the vector of integer values to a character array using char. Since your integer values never go above 1000, and char can accept numeric values from 0 to 65535 (corresponding to Unicode characters), this will give you a unique 8-character key for every unique 8-by-1 vector. Here's an example:
found = containers.Map('KeyType', 'char', 'ValueType', 'any');
v = randi(1000, [8 1]); % Sample vector
key = char(v);
if ~isKey(found, key)
found(key) = 1;
end
your idea is good. but you need to find a better hash function. use some standard hash function.
There is an implementation of 'sha' algorithms you's like to see:
http://www.se.mathworks.com/matlabcentral/fileexchange/31795-sha-algorithms-160224256384-512
If you find the sha algorithm slow then you can probably resort to some tricks. One that i can think of now is following:
take a seed number > 1000 e.g. 1024
divide each number in the vector by the seed and store the remainder in a string.
concatenate all the remainders which will serve as your 'code' for your vector element. which can be used for comparison when a you see a new element.
this should probably work but you'll have to check.
Not really into hashing, but still believe to have found the simplest way to solve your problem.
This runs about 10x faster than ismember.
any(v(1)==M(1)&v(2)==M(2)&v(3)==M(3)&v(4)==M(4)&v(5)==M(5)&v(6)==M(6)&v(7)==M(7)&v(8)==M(8));
Please help interpret the Birthday effect as described in Wikipedia:
A birthday attack works as follows:
Pick any message m and compute h(m).
Update list L. Check if h(m) is in the list L.
if (h(m),m) is already in L, a colliding message pair has been found.
else save the pair (h(m),m) in the
list L and go back to step 1.
From the birthday paradox we know that we can expect to find a
matching entry, after performing about
2^(n/2) hash evaluations.
Does the above mean 2^(n/2) iterations through the above entire loop (i.e. 2^(n/2) returns to step 1), OR does it mean 2^(n/2) comparisons to individual items already in L?
It means 2^(n/2) iterations through the loop. But note that L would not be a normal list here, but a hash table mapping h(m) to m. So each iteration would only need a constant number (O(1)) of comparisons in average, and there would be O(2^(n/2)) comparisons in total.
If L had been a normal array or a linked list, then the number of comparisons would be much larger since you would need to search through the whole list each iteration. This would be a bad way to implement this algorithm though.