I need to generate unique random numbers in Postgresql with a fixed length of 13 digits.
I've found a similar thread where was used a sequence encrypted using "pseudo_encrypt", but the returned number was not with a fixed length.
So, what i need is: get an encrypted random sequence with a fixed length of 13 digits where the min value is 0000000000001 and a max value is 9999999999999.
Is it possible? If start with the zeros in front is not possible is not a big problem (i think), i can set them programmatically during the reading from the db, but would be great if Postgresql can do it by itself.
-- EDIT --
After have realized some useful things i must change the question in order to explain better what i need:
I need to generate unique random numbers (bigint) in Postgresql with a fixed max length of 13 digits. Actually i'm trying to use pseudo_encrypt function (64 bit), but the returned number obviously is not with a fixed max length of 13, in the 32 bit case the max length is 10 digits (int), and for the 64 bit is 19 (bigint).
So, how to get an encrypted random sequence with a fixed max length of 13 digits, where the min value is 1 and the max value is 9999999999999 ?
Is it possible to modify the 64 bit pseudo_ecrypt function in order to get this result? Or if is not possible, there are other methods to get an unique sequence with this requirements?
Pseudo Encrypt function (64bit)
CREATE OR REPLACE FUNCTION pseudo_encrypt(VALUE bigint) returns bigint AS $$
DECLARE
l1 bigint;
l2 bigint;
r1 bigint;
r2 bigint;
i int:=0;
BEGIN
l1:= (VALUE >> 32) & 4294967295::bigint;
r1:= VALUE & 4294967295;
WHILE i < 3 LOOP
l2 := r1;
r2 := l1 # ((((1366.0 * r1 + 150889) % 714025) / 714025.0) * 32767*32767)::int;
l1 := l2;
r1 := r2;
i := i + 1;
END LOOP;
RETURN ((l1::bigint << 32) + r1);
END;
$$ LANGUAGE plpgsql strict immutable;
Tweaking the existing function for N < 64 bits values
It's relatively simple to tweak the bigint variant to reduce the output to 2^N values, where N is even, and less than 64.
To get 13 decimal digits, consider the maximum N for which 2^N has 13 digits. That's N=42, with 2^42=4398046511104.
The algorithm works by breaking the input value into two halves with an equal number of bits, and make them flow through the Feistel network, essentially XOR'ing with the result of the round function and swapping halves at each iteration.
If at every stage of the process, each half is limited to 21 bits then the result combining both halves is guaranteed not to exceed 42 bits.
So here's my proposed variant:
CREATE OR REPLACE FUNCTION pseudo_encrypt42(VALUE bigint) returns bigint
AS $$
DECLARE
l1 bigint;
l2 bigint;
r1 bigint;
r2 bigint;
i int:=0;
b21 int:=(1<<21)-1; -- 21 bits mask for a half-number => 42 bits total
BEGIN
l1:= VALUE >> 21;
r1:= VALUE & b21;
WHILE i < 3 LOOP
l2 := r1;
r2 := l1 # (((((1366*r1+150889)%714025)/714025.0)*32767*32767)::int & b21);
l1 := l2;
r1 := r2;
i := i + 1;
END LOOP;
RETURN ((l1::bigint << 21) + r1);
END;
$$ LANGUAGE plpgsql strict immutable;
The input must be less than (2^42)-1, otherwise the outputs will collide , as pseudo_encrypt42(x) = pseudo_encrypt42(x mod 2^42).
What can be done about the missing numbers between 2^42 and 10^13 ?
2^42 - 10^13 = 5601953488896 so that's quite a lot of missing numbers.
I don't know how to help with that in a single pass with the Feistel network. One workaround that might be acceptable, though, is to generate another set of unique values in 0..M and add 2^42 to them, so there's no risk of collision.
This another set could be obtained by the same function, just with the offset added. 4398046511104 + pseudo_encrypt42(x) is guaranteed to be between 4398046511104 and 2*4398046511104 = 8796093022208 unique values so that's closer to the goal. The same technique could be applied with several other ranges, not even necessarily of the same size.
However this workaround degrades the random-looking behavior , as instead of having a single output range where every number can be between 0 and X, you'd get N distinct output ranges of X/N numbers. With several distinct partitions like that, it's easy to guess in what partition the output will be, just not what value inside the partition.
Related
I want to make subprogram for adding array's elements with Ada.
subprogram "Add_Data" have 3 parameters-
first parameter = generic type array (array of INTEGER or array of REAL)
second parameter = INTEGER (size of array)
third parameter = generic type sum (array of INTEGER -> sum will be INTEGER, array of REAL -> sum will be REAL)
I programmed it from ideone.com.
(I want to see just result by array of INTEGER. After that, I will test by array of REAL)
With Ada.Text_IO; Use Ada.Text_IO;
With Ada.Integer_Text_IO; Use Ada.Integer_Text_IO;
procedure test is
generic
type T is private;
type Tarr is array (INTEGER range <>) of T;
--function "+" (A,B : T) return T;
--function "+" (A, B : T) return T is
--begin
-- return (A+B);
--end "+";
procedure Add_Data(X : in Tarr; Y : in INTEGER; Z : in out T);
procedure Add_Data(X : in Tarr; Y : in INTEGER; Z : in out T) is
temp : T;
count : INTEGER;
begin
count := 1;
loop
temp :=temp+ X(count); //<-This is problem.
count := count + 1;
if count > Y then
exit;
end if;
end loop;
Z:=temp;
end Add_Data;
type intArray is array (INTEGER range <>) of INTEGER;
intArr : intArray := (1=>2, 2=>10, 3=>20, 4=>30, 5=>8);
sum : INTEGER;
procedure intAdd is new Add_Data(Tarr=>intArray, T=>INTEGER);
begin
sum := 0;
intAdd(intArr, 5, sum);
put (sum);
end test;
when I don't overload operator "+", It makes error.
"There is no applicable operator "+" for private type "T" defined."
What can I do for this?
If a generic’s formal type is private, then nothing in the generic can assume anything about the type except that it can be assigned (:=) and that it can be compared for equality (=) and inequality (/=). In particular, no other operators (e.g. +) are available in the generic unless you provide them.
The way to do that is
generic
type T is private;
with function "+" (L, R : T) return T is <>;
This tells the compiler that (a) there is a function "+" which takes two T’s and returns a T; and (b) if the actual T has an operator "+" which matches that profile, to allow it as the default.
So, you could say
procedure intAdd is new Add_Data (T => Integer, ...
or, if you didn’t feel like using the default,
procedure intAdd is new Add_Data (T => Integer, "+" => "+", ...
In addition to not knowing how to declare a generic formal subprogram (Wright has shown how to do this for functions), your code has a number of other issues that, if addressed, might help you move from someone who thinks in another language and translates it into Ada into someone who actually uses Ada. Presuming that you want to become such a person, I will point some of these out.
You declare your array types using Integer range <>. It's more common in Ada to use Positive range <>, because people usually refer to positions starting from 1: 1st, 2nd, 3rd, ...
Generics are used for code reuse, and in real life, such code is often used by people other than the original author. It is good practice not to make unstated assumptions about the values clients will pass to your operations. You assume that, for Y > 0, for all I in 1 .. Y => I in X'range and for Y < 1, 1 in X'range. While this is true for the values you use, it's unlikely to be true for all uses of the procedure. For example, when an array is used as a sequence, as it is here, the indices are immaterial, so it's more natural to write your array aggreate as (2, 10, 20, 30, 8). If I do that, Intarr'First = Integer'First and Intarr'Last = Integer'First + 4, both of which are negative. Attempting to index this with 1 will raise Constraint_Error.
Y is declared as Integer, which means that zero and negative values are acceptable. What does it mean to pass -12 to Y? Ada's subtypes help here; if you declare Y as Positive, trying to pass non-positive values to it will fail.
Z is declared mode in out, but the input value is not referenced. This would be better as mode out.
Y is not needed. Ada has real arrays; they carry their bounds around with them as X'First, X'Last, and X'Length. Trying to index an array outside its bounds is an error (no buffer overflow vulnerabilities are possible). The usual way to iterate over an array is with the 'range attribute:
for I in X'range loop
This ensures that I is always a valid index into X.
Temp is not initialized, so it will normally be initialized to "stack junk". You should expect to get different results for different calls with the same inputs.
Instead of
if count > Y then
exit;
end if;
it's more usual to write exit when Count > Y;
Since your procedure produces a single, scalar output, it would be more natural for it to be a function:
generic -- Sum
type T is private;
Zero : T;
type T_List is array (Positive range <>) of T;
with function "+" (Left : T; Right : T) return T is <>;
function Sum (X : T_List) return T;
function Sum (X : T_List) return T is
Result : T := Zero;
begin -- Sum
Add_All : for I in X'range loop
Result := Result + X (I);
end loop Add_All;
return Result;
end Sum;
HTH
ERROR at line 8: PL/SQL: Statement ignored
CREATE OR REPLACE PROCEDURE POS(A IN NUMBER,M IN NUMBER,TOTAL OUT NUMBER)
AS
BEGIN
TOTAL:=0;
WHILE A>0 LOOP
M:=MOD(A,10);
TOTAL:=TOTAL+M;
A:=(A/10);//statement ignored error
END LOOP;
DBMS_OUTPUT.PUT_LINE(TOTAL);
END;
DECLARE
X NUMBER;
Y NUMBER:=5;
Z NUMBER;
BEGIN
POA(X,Y,Z);
END;
OK, I've had a look at your procedure and tried to resolve the issues you are having with it.
You haven't explained much (or indeed anything) about what you are trying to achieve which makes it really difficult to get you an answer.
People on here really want to help but you have to at least give us the tools with witch to provide that help.
Anyhow, with a host of assumptions, here is my version of your procedure with the following assumptions:
Your procedure is names POS (you name it POS in the procedure definition but then try to execute it as POA).
Your main issue was trying to assign a new value to input parameter "a" within the loop. As it is an input parameter it is immutable and you cannot assign new values to it. I have got round this by declaring a local variable "v_iter" and assigning that the value of "a" and then using it to control the loop.
I have added an "exception" section to handle any unexpected errors and output that error via DBMS_OUTPUT. You might want to make this more robust.
You do not test to check if the input parameter "a" is null or a valid number (i.e. not negative), you might want to do this to make your procedure more robust.
Here is the changed code:
CREATE OR REPLACE
PROCEDURE POS (
a IN NUMBER,
m IN NUMBER,
total OUT NUMBER
)
AS
— Declare variables
v_iter NUMBER := a;
BEGIN
— Initialise total
total := 0;
— Loop through “v_iter”
WHILE v_iter > 0
LOOP
m := MOD(v_iter,10);
total := total + m;
v_iter := (v_iter/10);
END LOOP;
DBMS_OUTPUT.put_line(total);
EXCEPTION
WHEN others
THEN
— Output and raise an error;
DBMS_OUTPUT.put_line(sqlerrm);
RAISE;
END POS;
/
To call it:
DECLARE
X NUMBER;
Y NUMBER:=5;
Z NUMBER;
BEGIN
POS(X,Y,Z);
END;
/
Hope it helps.
I would like to use the pseudo_encrypt function mentioned a few times on StackOverflow to make my IDs look more random: https://wiki.postgresql.org/wiki/Pseudo_encrypt
How can I customize this to output unique "random" numbers for just me. I read somewhere that you can just change the 1366.0 constant, but I don't want to take any risks with my IDs as any potential ID duplicates would cause major issues.
I really have no idea what each constant actually does, so I don't want to mess around with it unless I get some direction. Does anyone know which constants I can safely change?
Here it is:
CREATE OR REPLACE FUNCTION "pseudo_encrypt"("VALUE" int) RETURNS int IMMUTABLE STRICT AS $function_pseudo_encrypt$
DECLARE
l1 int;
l2 int;
r1 int;
r2 int;
i int:=0;
BEGIN
l1:= ("VALUE" >> 16) & 65535;
r1:= "VALUE" & 65535;
WHILE i < 3 LOOP
l2 := r1;
r2 := l1 # ((((1366.0 * r1 + 150889) % 714025) / 714025.0) * 32767)::int;
r1 := l2;
l1 := r2;
i := i + 1;
END LOOP;
RETURN ((l1::int << 16) + r1);
END;
$function_pseudo_encrypt$ LANGUAGE plpgsql;
for bigint's
CREATE OR REPLACE FUNCTION "pseudo_encrypt"("VALUE" bigint) RETURNS bigint IMMUTABLE STRICT AS $function_pseudo_encrypt$
DECLARE
l1 bigint;
l2 bigint;
r1 bigint;
r2 bigint;
i int:=0;
BEGIN
l1:= ("VALUE" >> 32) & 4294967295::bigint;
r1:= "VALUE" & 4294967295;
WHILE i < 3 LOOP
l2 := r1;
r2 := l1 # ((((1366.0 * r1 + 150889) % 714025) / 714025.0) * 32767*32767)::bigint;
r1 := l2;
l1 := r2;
i := i + 1;
END LOOP;
RETURN ((l1::bigint << 32) + r1);
END;
$function_pseudo_encrypt$ LANGUAGE plpgsql;
Alternative solution: use different ciphers
Other cipher functions are now available on postgres wiki. They're going to be significantly slower, but aside from that, they're better candidates for generating customized random-looking series of unique numbers.
For 32 bit outputs, Skip32 in plpgsql will encrypt its input with a 10 bytes wide key, so you just have to choose your own secret key to have your own specific permutation (the particular order in which the 2^32 unique values will come out).
For 64 bit outputs, XTEA in plpgsql will do similarly, but using a 16 bytes wide key.
Otherwise, to just customize pseudo_encrypt, see below:
Explanations about pseudo_encrypt's implementation:
This function has 3 properties
global unicity of the output values
reversability
pseudo-random effect
The first and second property come from the Feistel Network, and as already explained in #CodesInChaos's answer, they don't depend on the choice of these constants: 1366 and also 150889 and 714025.
Make sure when changing f(r1) that it stays a function in the mathematical sense, that is x=y implies f(x)=f(y), or in other words the same input must always produce the same output. Breaking this would break the unicity.
The purpose of these constants and this formula for f(r1) is to produce a reasonably good pseudo-random effect. Using postgres built-in random() or similar method is not possible because it's not a mathematical function as described above.
Why these arbitrary constants? In this part of the function:
r2 := l1 # ((((1366.0 * r1 + 150889) % 714025) / 714025.0) * 32767)::int;
The formula and the values 1366, 150889 and 714025 come from Numerical recipes in C (1992, by William H.Press, 2nd ed.), chapter 7: random numbers, specifically p.284 and 285.
The book is not directly indexable on the web but readable through an interface here: http://apps.nrbook.com/c/index.html .It's also cited as a reference in various source code implementing PRNGs.
Among the algorithms discussed in this chapter, the one used above is very simple and relatively effective. The formula to get a new random number from a previous one (jran) is:
jran = (jran * ia + ic) % im;
ran = (float) jran / (float) im; /* normalize into the 0..1 range */
where jran is the current random integer.
This generator will necessarily loop over itself after a certain number of values (the "period"), so the constants ia, ic and im have to be chosen carefully for that period to be as large as possible. The book provides a table p.285 where constants are suggested for various lengths of the period.
ia=1366, ic=150889 and im=714025 is one of the entries for a period of
229 bits, which is way more than needed.
Finally the multiplication by 32767 or 215-1 is not part of the PRNG but meant to produce a positive half-integer from the 0..1 pseudo-random float value. Don't change that part, unless to widen the blocksize of the algorithm.
This function looks like a blockcipher based on a Feistel network - but it's lacking a key.
The Feistel construction is bijective, i.e. it guarantees that there are no collisions. The interesting part is: r2 := l1 # f(r1). As long as f(r1) only depends on r1 the pseudo_encrypt will be bijective, no matter what the function does.
The lack of key means that anybody who knows the source code can recover the sequential ID. So you're relying on security-though-obscurity.
The alternative is using a block cipher which takes a key. For 32 bit blocks there are relatively few choices, I know of Skip32 and ipcrypt. For 64 bit blocks there are many ciphers to choose from, including 3DES, Blowfish and XTEA.
I want to make a program which I will check if here are any Armstron numbers (numbers which are equal to the cubes of it's figures, for example 153).
Which are also perfect numbers (numbers equal to the sum of it's divisors, not including itself, for example 28 (1+2+4+7+14))
below 1000. So I made a program to see if a number is Armstrong number and to see if it's perfect number.
program Armstrong;
var i,n,j,d,s,p:integer;
begin
for i:=1 to 1000 do
begin
j:=i mod 10;
d:=i div 10 mod 10;
s:=i div 100;
n:=j*j*j+d*d*d+s*s*s;
if n=i then
writeln(i);
end;
end.
And for perfect numbers it's
program Perfect;
var n,s,i:integer;
begin
for n:=1 to 1000 do
begin
s:=0;
for i:=1 to n do
begin
if n mod i = 0 then
s:=s+i;
end;
if s=n then
writeln(n);
end;
end.
So I don't know how to merge them and see if any number I get from the first program appears in the second as well.
I did Pascal long time back. So assume I have used pseudo code for this answer.
Remove your loop from Armstrong and put the code in this function.
function Armstrong(i: integer): integer;
(* code goes here. And return 0 or 1 based on Armstrong *)
Remove your loop from Perfect, and put this code in this function
function Perfect(i: integer): integer;
(* code goes here. And return 0 or 1 based on Perfect *)
Then call it from a loop like this way:
for i:=1 to 1000 do
x = Armstrong(i);
y = Perfect(i);
if(x == 1 and y == 1) then
(* do something *)
end;
Setting the 32nd and 64th bits is tricky.
32-bit Solution:
I got it to work for 32-bit fields. The trick is to cast the return value of the POWER function to binary(4) before casting it to int. If you try to cast directly to int, without first casting to binary(4), you will get an arithmetic overflow exception when operating on the 32nd bit (index 31). Also, you must ensure the expression passed to POWER is of a sufficiently large type (e.g. bigint) to store the maximum return value (2^31), or the POWER function will throw an arithmetic overflow exception.
CREATE FUNCTION [dbo].[SetIntBit]
(
#bitfieldvalue int,
#bitindex int, --(0 to 31)
#bit bit --(0 or 1)
)
RETURNS int
AS
BEGIN
DECLARE #bitmask int = CAST(CAST(POWER(CAST(2 as bigint),#bitindex) as binary(4)) as int);
RETURN
CASE
WHEN #bit = 1 THEN (#bitfieldvalue | #bitmask)
WHEN #bit = 0 THEN (#bitfieldvalue & ~#bitmask)
ELSE #bitfieldvalue --NO CHANGE
END
END
64-bit Problem:
I was going to use a similar approach for 64-bit fields, however I'm finding that the POWER function is returning inaccurate values, despite using the decimal(38) type for the expression/return value.
For example: "select POWER(CAST(2 as decimal(38)), 64)" returns 18446744073709552000 (only the first 16 digits are accurate) rather than the correct value of 18446744073709551616. And even though I'd only raise 2 to the 63rd power, that result is still inaccurate.
The documentation of the POWER function indicates that "Internal conversion to float can cause loss of precision if either the money or numeric data types are used." (note that numeric type is functionally equivalent to decimal type).
I think the only way to handle 64-bit fields properly is to operate on their 32-bit halves, but that involves an extra check on the #bitindex property to see which half I need to operate on. Are there any built-in function or better ways to explicitly set those final bits in 32-bit and 64-bit bitmasked fields in TSQL?
64-bit Solution:
So far, the simplest solution I can come up with to my own question is to add an exceptional case for problematic computation of the bitmask for the 64th bit (i.e. 2^63), where the bitmask value is hardcoded so that it does not have to be computed by POWER. POWER computes 2^62 and smaller values accurately as far as I can see.
CREATE FUNCTION [dbo].[SetBigIntBit]
(
#bitfieldvalue bigint,
#bitindex int, --(0 to 63)
#bit bit --(0 or 1)
)
RETURNS bigint
AS
BEGIN
DECLARE #bitmask bigint = case WHEN #bitindex = 63 THEN CAST(0x8000000000000000 as bigint)
ELSE CAST(CAST(POWER(CAST(2 as bigint),#bitindex) as binary(8)) as bigint)
RETURN
CASE
WHEN #bit = 1 THEN (#bitfieldvalue | #bitmask)
WHEN #bit = 0 THEN (#bitfieldvalue & ~#bitmask)
ELSE #bitfieldvalue --NO CHANGE
END
END
EDIT: Here's some code to test the above function...
declare #bitfield bigint = 0;
print #bitfield;
declare #bitindex int;
set #bitindex = 0;
while #bitindex < 64
begin
set #bitfield = tutor.dbo.SetBigIntBit(#bitfield,#bitindex,1);
print #bitfield;
set #bitindex = #bitindex + 1;
end
set #bitindex = 0;
while #bitindex < 64
begin
set #bitfield = tutor.dbo.SetBigIntBit(#bitfield,#bitindex,0);
print #bitfield;
set #bitindex = #bitindex + 1;
end