I have a huge Firebird database with a table that counts 41 millions of rows. Recently I have added a new float column and would like to fill it with incremental data. Each next value should be a previous incremented by RAND(). The very first value is also RAND().
How to do this?
The query
SELECT ID FROM MY_TABLE WHERE MY_COLUMN IS NULL ROWS 1;
takes up to 15 seconds so I wouldn't count on this query executed in a loop.
The table has an indexed ID column which is a part of composite primary key.
something like
update MyTable set MyColumn = Gen_ID( TempGen,
round( rand() * 100000) ) / 100000.0
Create a temporary Generator - https://www.firebirdsql.org/manual/generatorguide.html
use the integer generator as your float value scaled by some coefficient, like 100 000 would stand for 1.0 and 10 000 for 0.1, etc
use the GEN_ID function to forward a generator for a specified number of integer units
drop the generator
alternatively use Stored Procedure or EXECUTE BLOCK
https://www.firebirdsql.org/refdocs/langrefupd20-execblock.html
http://firebirdsql.su/doku.php?id=execute_block
something like
execute block
as
declare f double precision = 0;
declare i int;
begin
for select ID FROM MY_TABLE WHERE MY_COLUMN IS NULL order by id into :I
do begin
f = f + rand();
update MY_TABLE SET MY_COLUMN = :f where ID = :i;
end;
end
Or you may try using cursors, but I did not try so I do not know for sure how it would work.
https://www.firebirdsql.org/refdocs/langrefupd25-psql-forselect.html
execute block
as
declare f double precision = 0;
begin
for select ID FROM MY_TABLE WHERE MY_COLUMN IS NULL order by id
as cursor C do begin
f = f + rand();
update MY_TABLE SET MY_COLUMN = :f where current of C;
end;
end
Related
I want to generate random numbers in PostgreSQL just like I have done in MySQL like below. I want to do so in a Postgres function.
MySQL:
DROP PROCEDURE IF EXISTS Generate_random;
DELIMITER $$
CREATE PROCEDURE Generate_random()
BEGIN
Drop table if exists aa_dev.`Agents`;
CREATE TABLE aa_dev.`Agents`(AgentID int PRIMARY KEY);
SET #first = 1;
SET #last = 1000;
WHILE(#first <= #last) Do
INSERT INTO aa_dev.`Agents` VALUES(FLOOR(RAND()*(2900000-2800000+1)+2800000))
ON DUPLICATE KEY UPDATE AgentID = FLOOR(RAND()*(2900000-2800000+1)+2800000);
IF ROW_COUNT() = 1 THEN
SET #first = #first + 1;
END IF;
END WHILE;
END$$
DELIMITER ;
CALL Generate_random();
I have so far generated random numbers in Postgres but they are getting repeated in the column. Please tell me how can I achieve the above MySQL code in PostgreSQL.
drop function if exists aa_dev.rand_cust(low INT, high INT, total INT);
CREATE OR REPLACE FUNCTION aa_dev.rand_cust(low INT ,high INT, total INT)
RETURNS TABLE (Cust_id int) AS
$$
declare
counter int := 0;
rand int := 0;
begin
------------------- Creating a customer table with Cust_id----------------------------
DROP TABLE IF EXISTS aa_dev.Customer;
CREATE TABLE IF NOT EXISTS aa_dev.Customer (
Cust_id INT
);
--------------------- Loop to insert random -----------------------
while counter < total loop
rand = floor(random()* (high-low + 1) + low);
Insert into aa_dev.Customer (Cust_id) values(rand);
counter := counter + 1;
end loop;
return query
select *
from aa_dev.customer;
end
$$
LANGUAGE plpgsql;
select * from aa_dev.rand_cust(1, 50, 100);
For Postgres you've asked for 100 numbers between 1 and 50 - there will naturally be duplicates!
The MySQL code has a much wider range of possible values (100000) and only 1000 of them are sampled. Also the MySQL code generates random numbers until there is no key error, i.e. there are no duplicates in the column.
So, for Postgres, you could try checking for duplicates and retrying if found. Making the column unique will prevent duplicate insertion, but you have to handle it.
Also, a sample size that is larger than the number of values is required. Be careful with the retries, don't replicate the MySQL example. If the sample size is smaller than the required count, the loop will never terminate.
Update
Here is a function that will generate unique random numbers within a range and populate a table with them:
DROP FUNCTION IF EXISTS rand_cust (low INT, high INT, total INT);
CREATE OR REPLACE FUNCTION rand_cust (low INT, high INT, total INT)
RETURNS TABLE (Cust_id INT)
AS
$$
BEGIN
------------------- Creating a customer table with Cust_id----------------------------
DROP TABLE IF EXISTS Customer;
CREATE TABLE IF NOT EXISTS Customer(Cust_id INT);
RETURN query
INSERT INTO Customer(Cust_id)
SELECT *
FROM generate_series(low, high)
ORDER BY random() LIMIT total
RETURNING -- returns the id's you generated
Customer.Cust_id;
END $$
LANGUAGE plpgsql;
SELECT *
FROM rand_cust(1000, 2000, 100); -- 100 unique numbers between 1000 and 2000 inclusive
Note that this will not be able to generate more numbers than the sample size, e.g. you can't generate 100 numbers between 1 and 50, only a maximum of 50. That's a consequence of the uniqueness requirement. The LIMIT clause will not cause errors, but you could add code to check that (hi - low) >= total before attempting the query.
If you'd prefer a simple function to generate n random unique numbers:
DROP FUNCTION IF EXISTS sample(low INT, high INT, total INT);
CREATE OR REPLACE FUNCTION sample(low INT, high INT, total INT)
RETURNS TABLE (Cust_id INT)
AS
$$
BEGIN
RETURN query
SELECT *
FROM generate_series(low, high)
ORDER BY random() LIMIT total;
END $$
LANGUAGE plpgsql;
-- create a table of unique random values
SELECT INTO Customer FROM sample(100, 200, 10);
As said before, you have a range between 1 and 50 and you want to create 100 records. That will never be unique. And your query doesn't ask for unique values anyway, so even with a million records you can have duplicates.
But, your code can be much simpler as well, without a loop and just a single query:
DROP FUNCTION IF EXISTS aa_dev.rand_cust ( low INT, high INT, total INT );
CREATE OR REPLACE FUNCTION aa_dev.rand_cust ( low INT, high INT, total INT )
RETURNS TABLE ( Cust_id INT )
AS
$$
BEGIN
------------------- Creating a customer table with Cust_id----------------------------
DROP TABLE IF EXISTS aa_dev.Customer;
CREATE TABLE IF NOT EXISTS aa_dev.Customer ( Cust_id INT );
--------------------- No Loop to insert random -----------------------
RETURN query
INSERT INTO aa_dev.Customer ( Cust_id )
SELECT FLOOR ( random( ) * ( high - low + 1 ) + low ) -- no uniqueness!
FROM generate_series(1, total) -- no loop needed
RETURNING -- returns the id's you generated
Customer.Cust_id;
END $$
LANGUAGE plpgsql;
SELECT
*
FROM
aa_dev.rand_cust ( 1, 50, 100 );
I'm converting a BDE query (Paradox) to a Firebird (2.5, not 3.x) and I have a very convenient conversion in it:
select TRIM(' 1') as order1, CAST(' 1' AS INTEGER) AS order2 --> 1
select TRIM(' 1 bis') as order1, CAST(' 1 bis' AS INTEGER) AS order2 --> 1
Then ordering by the cast value then the trimmed value (ORDER order2, order1) provide me the result I need:
1
1 bis
2 ter
100
101 bis
However, in Firebird casting an incorrect integer will raise an exception and I did not find any way around to provide same result. I think I can tell if a number is present with something like below, but I couldn't find a way to extract it.
TRIM(' 1 bis') similar to '[ [:ALPHA:]]*[[:DIGIT:]]+[ [:ALPHA:]]*'
[EDIT]
I had to handle cases where text were before the number, so using #Arioch'The's trigger, I got this running great:
SET TERM ^ ;
CREATE TRIGGER SET_MYTABLE_INTVALUE FOR MYTABLE ACTIVE
BEFORE UPDATE OR INSERT POSITION 0
AS
DECLARE I INTEGER;
DECLARE S VARCHAR(13);
DECLARE C VARCHAR(1);
DECLARE R VARCHAR(13);
BEGIN
IF (NEW.INTVALUE is not null) THEN EXIT;
S = TRIM( NEW.VALUE );
R = NULL;
I = 1;
WHILE (I <= CHAR_LENGTH(S)) DO
BEGIN
C = SUBSTRING( S FROM I FOR 1 );
IF ((C >= '0') AND (C <= '9')) THEN LEAVE;
I = I + 1;
END
WHILE (I <= CHAR_LENGTH(S)) DO
BEGIN
C = SUBSTRING( S FROM I FOR 1 );
IF (C < '0') THEN LEAVE;
IF (C > '9') THEN LEAVE;
IF (C IS NULL) THEN LEAVE;
IF (R IS NULL) THEN R=C; ELSE R = R || C;
I = I + 1;
END
NEW.INTVALUE = CAST(R AS INTEGER);
END^
SET TERM ; ^
Converting such a table, you have to add a special indexed integer column for keeping the extracted integer data.
Note, this query while using "very convenient conversion" is actually rather bad: you should use indexed columns to sort (order) large amounts of data, otherwise you are going into slow execution and waste a lot of memory/disk for temporary sorting tables.
So you have to add an extra integer indexed column and to use it in the query.
Next question is how to populate that column.
Better would be to do it once, when you move your entire database and application from BDE to Firebird. And from that point make your application when entering new data rows fill BOTH varchar and integer columns properly.
One time conversion can be done by your convertor application, then.
Or you can use selectable Stored Procedure that would repeat the table with such and added column. Or you can make Execute Block that would iterate through the table and update its rows calculating the said integer value.
How to SELECT a PROCEDURE in Firebird 2.5
If you would need to keep legacy applications, that only insert text column but not integer column, then I think you would have to use BEFORE UPDATE OR INSERT triggers in Firebird, that would parse the text column value letter by letter and extract integer from it. And then make sure your application never changes that integer column directly.
See a trigger example at Trigger on Update Firebird
PSQL language documentation: https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-psql.html
Whether you would write procedure or trigger to populate the said added integer indexed column, you would have to make simple loop over characters, copying string from first digit until first non-digit.
https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-functions-scalarfuncs.html#fblangref25-functions-string
https://www.firebirdsql.org/file/documentation/reference_manuals/fblangref25-en/html/fblangref25-psql-coding.html#fblangref25-psql-declare-variable
Something like that
CREATE TRIGGER my_trigger FOR my_table
BEFORE UPDATE OR INSERT
AS
DECLARE I integer;
DECLARE S VARCHAR(100);
DECLARE C VARCHAR(100);
DECLARE R VARCHAR(100);
BEGIN
S = TRIM( NEW.MY_TXT_COLUMN );
R = NULL;
I = 1;
WHILE (i <= CHAR_LENGTH(S)) DO
BEGIN
C = SUBSTRING( s FROM i FOR 1 );
IF (C < '0') THEN LEAVE;
IF (C > '9') THEN LEAVE;
IF (C IS NULL) THEN LEAVE;
IF (R IS NULL) THEN R=C; ELSE R = R || C;
I = I + 1;
END
NEW.MY_INT_COLUMN = CAST(R AS INTEGER);
END;
In this example your ORDER order2, order1 would become
SELECT ..... FROM my_table ORDER BY MY_INT_COLUMN, MY_TXT_COLUMN
Additionally, it seems your column actually contains a compound data: an integer index and an optional textual postfix. If so, then the data you have is not normalized and the table better be restructured.
CREATE TABLE my_table (
ORDER_Int INTEGER NOT NULL,
ORDER_PostFix VARCHAR(24) CHECK( ORDER_PostFix = TRIM(ORDER_PostFix) ),
......
ORDER_TXT COMPUTED BY (ORDER_INT || COALESCE( ' ' || ORDER_PostFix, '' )),
PRIMARY KEY (ORDER_Int, ORDER_PostFix )
);
When you would move your data from Paradox to Firebird - make your convertor application check and split those values like "1 bis" into two new columns.
And your query then would be like
SELECT ORDER_TXT, ... FROM my_table ORDER BY ORDER_Int, ORDER_PostFix
if you're using fb2.5 you can use the following:
execute block (txt varchar(100) = :txt )
returns (res integer)
as
declare i integer;
begin
i=1;
while (i<=char_length(:txt)) do begin
if (substring(:txt from i for 1) not similar to '[[:DIGIT:]]')
then txt =replace(:txt,substring(:txt from i for 1),'');
else i=i+1;
end
res = :txt;
suspend;
end
in fb3.0 you have more convenient way to do the same
select
cast(substring(:txt||'#' similar '%#"[[:DIGIT:]]+#"%' escape '#') as integer)
from rdb$database
--assuming that the field is varchar(15))
select cast(field as integer) from table;
Worked in firebird version 2.5.
How to use a recursive query and then using cursor to update multiple rows in postgresql. I try to return data but no data is found. Any alternative to using recursive query and cursor, or maybe better code please help me.
drop function proses_stock_invoice(varchar, varchar, character varying);
create or replace function proses_stock_invoice
(p_medical_cd varchar,p_post_cd varchar, p_pstruserid character varying)
returns void
language plpgsql
as $function$
declare
cursor_data refcursor;
cursor_proses refcursor;
v_medicalCd varchar(20);
v_itemCd varchar(20);
v_quantity numeric(10);
begin
open cursor_data for
with recursive hasil(idnya, level, pasien_cd, id_root) as (
select medical_cd, 1, pasien_cd, medical_root_cd
from trx_medical
where medical_cd = p_pstruserid
union all
select A.medical_cd, level + 1, A.pasien_cd, A.medical_root_cd
from trx_medical A, hasil B
where A.medical_root_cd = B.idnya
)
select idnya from hasil where level >=1;
fetch next from cursor_data into v_medicalCd;
return v_medicalCd;
while (found)
loop
open cursor_proses for
select B.item_cd, B.quantity from trx_medical_resep A
join trx_resep_data B on A.medical_resep_seqno = B.medical_resep_seqno
where A.medical_cd = v_medicalCd and B.resep_tp = 'RESEP_TP_1';
fetch next from cursor_proses into v_itemCd, v_quantity;
while (found)
loop
update inv_pos_item
set quantity = quantity - v_quantity, modi_id = p_pstruserid, modi_id = now()
where item_cd = v_itemCd and pos_cd = p_post_cd;
end loop;
close cursor_proses;
end loop;
close cursor_data;
end
$function$;
but nothing data found?
You have a function with return void so it will never return any data to you. Still you have the statement return v_medicalCd after fetching the first record from the first cursor, so the function will return from that point and never reach the lines below.
When analyzing your function you have (1) a cursor that yields a number of idnya values from table trx_medical, which is input for (2) a cursor that yields a number of v_itemCd, v_quantity from tables trx_medical_resep, trx_resep_data for each idnya, which is then used to (3) update some rows in table inv_pos_item. You do not need cursors to do that and it is, in fact, extremely inefficient. Instead, turn the whole thing into a single update statement.
I am assuming here that you want to update an inventory of medicines by subtracting the medicines prescribed to patients from the stock in the inventory. This means that you will have to sum up prescribed amounts by type of medicine. That should look like this (note the comments):
CREATE FUNCTION proses_stock_invoice
-- VVV parameter not used
(p_medical_cd varchar, p_post_cd varchar, p_pstruserid varchar)
RETURNS void AS $function$
UPDATE inv_pos_item -- VVV column repeated VVV
SET quantity = quantity - prescribed.quantity, modi_id = p_pstruserid, modi_id = now()
FROM (
WITH RECURSIVE hasil(idnya, level, pasien_cd, id_root) AS (
SELECT medical_cd, 1, pasien_cd, medical_root_cd
FROM trx_medical
WHERE medical_cd = p_pstruserid
UNION ALL
SELECT A.medical_cd, level + 1, A.pasien_cd, A.medical_root_cd
FROM trx_medical A, hasil B
WHERE A.medical_root_cd = B.idnya
)
SELECT B.item_cd, sum(B.quantity) AS quantity
FROM trx_medical_resep A
JOIN trx_resep_data B USING (medical_resep_seqno)
JOIN hasil ON A.medical_cd = hasil.idnya
WHERE B.resep_tp = 'RESEP_TP_1'
--AND hacil.level >= 1 Useless because level is always >= 1
GROUP BY 1
) prescribed
WHERE item_cd = prescribed.item_cd
AND pos_cd = p_post_cd;
$function$ LANGUAGE sql STRICT;
Important
As with all UPDATE statements, test this code before you run the function. You can do that by running the prescribed sub-query separately as a stand-alone query to ensure that it does the right thing.
The table is:
F TIME1 END_TIME
- --------------------------------------------------------------------------- ----------------------
C 16-NOV-16 09.45.32.000000 AM 17-NOV-16 09.45.32.000000 AM
A 16-NOV-16 10.14.54.000000 AM 16-NOV-16 11.14.54.000000 AM
A 16-NOV-16 10.14.56.000000 AM 16-NOV-16 11.14.56.000000 AM
I have created a function..
CREATE OR REPLACE FUNCTION datediff
(
time1 TIMESTAMP
, time2 TIMESTAMP
)
RETURN number
as
tot number;
BEGIN
SELECT(extract(DAY FROM time2-time1)*24*60*60)+
(extract(HOUR FROM time2-time1)*60*60)
into tot from tt ;
RETURN tot;
END;
I am then calling the function in procedure...
CREATE OR REPLACE PROCEDURE P1
IS
CURSOR c1
IS
select count(*) as cnt,time1,end_time
from tt group by time1,end_time ;
a number;
BEGIN
FOR i IN c1
LOOP
declare
a number;
BEGIN
insert into y1 values(i.cnt,datediff(i.time1,i.end_time)) ;
--display(i.cnt||' '||a);
/* EXCEPTION
WHEN OTHERS THEN
dbms_output.put_line('Error updating record ' || SUBSTR (SQLERRM, 1, 250));*/
END;
END LOOP;
END P1;
The error I am getting is ...
SQL> exec p1
BEGIN p1; END;
*
ERROR at line 1:
ORA-01422: exact fetch returns more than requested number of rows
ORA-06512: at "ANU.DATEDIFF", line 11
ORA-06512: at "ANU.P1", line 14
ORA-06512: at line 1
This is working for single record in the table ,but not for multiple records..? please guide..
The problem is in your datediff function.
Whenever you have a SELECT ... INTO ... statement in PL/SQL, the SELECT query must return a single row. If it returns no rows at all, you get an ORA-01403 no data found error, and if it returns more than one row, you get the ORA-01422 exact fetch returns more than requested number of rows error you see above.
So, how can you get your query to return one row? Well, you can start by noticing that you aren't selecting any values from of the table tt. All your query is returning is the same value for each row in tt. If tt has one row, your query returns the value once. If tt has three rows, your query returns the same value three times. All this is unnecessary as you only want the one value, regardless of how many rows there are in tt.
So instead of SELECTing from tt, use the built-in table dual instead. The dual table only ever has one row in it:
SELECT(extract(DAY FROM time2-time1)*24*60*60)+
(extract(HOUR FROM time2-time1)*60*60)
into tot from dual;
However, in your case you don't even need a query: your datediff function could be rewritten to just perform the calculation and return the value:
CREATE OR REPLACE FUNCTION datediff
(
time1 TIMESTAMP
, time2 TIMESTAMP
)
RETURN number
as
BEGIN
RETURN (extract(DAY FROM time2-time1)*24*60*60)+
(extract(HOUR FROM time2-time1)*60*60);
END;
/
I have a table which doesn't have an unique ID. I want to make a stored procedure which is adding to each row the number of the row as ID, but I don't know how to get the current row number. This is what I have done until now
CREATE OR ALTER PROCEDURE INSERTID_MYTABLE
returns (
cnt integer)
as
declare variable rnaml_count integer;
begin
/* Procedure Text */
Cnt = 1;
for select count(*) from MYTABLE r into:rnaml_count do
while (cnt <= rnaml_count) do
begin
update MYTABLE set id=:cnt
where :cnt = /*how should I get the rownumber here from select??*/
Cnt = Cnt + 1;
suspend;
end
end
I think better way will be:
Add new nullable column (let's call it ID).
Create a generator/sequence (let's call it GEN_ID).
Create a before update/insert trigger that fetches new value from sequence whenever the NEW.ID is null. Example.
Do update table set ID = ID. (This will populate the keys.)
Change the ID column to not null.
A bonus. The trigger can be left there, because it will generate the value in new inserted rows.