How can I measure the amount of space taken by blobs on a Firebird 2.1 database? - firebird

I have a production database, using Firebird 2.1, where I need to find out how much space is used by each table, including the blobs. The blob-part is the tricky one, because it is not covered using the standard statistical report.
I do not have easy access to the server's desktop, so installing UDFs etc. is not a good solution.
How can I do this easily?

You can count total size of all BLOB fields in a database with following statement:
EXECUTE BLOCK RETURNS (BLOB_SIZE BIGINT)
AS
DECLARE VARIABLE RN CHAR(31) CHARACTER SET UNICODE_FSS;
DECLARE VARIABLE FN CHAR(31) CHARACTER SET UNICODE_FSS;
DECLARE VARIABLE S BIGINT;
BEGIN
BLOB_SIZE = 0;
FOR
SELECT r.rdb$relation_name, r.rdb$field_name
FROM rdb$relation_fields r JOIN rdb$fields f
ON r.rdb$field_source = f.rdb$field_name
WHERE f.rdb$field_type = 261
INTO :RN, :FN
DO BEGIN
EXECUTE STATEMENT
'SELECT SUM(OCTET_LENGTH(' || :FN || ')) FROM ' || :RN ||
' WHERE NOT ' || :FN || ' IS NULL'
INTO :S;
BLOB_SIZE = :BLOB_SIZE + COALESCE(:S, 0);
END
SUSPEND;
END

I modified the code example of Andrej to show the size of each blob field, not only the sum of all blobs.
And used SET TERM so you can copy&paste this snippet directly to tools like FlameRobin.
SET TERM #;
EXECUTE BLOCK
RETURNS (BLOB_SIZE BIGINT, TABLENAME CHAR(31), FIELDNAME CHAR(31) )
AS
DECLARE VARIABLE RN CHAR(31) CHARACTER SET UNICODE_FSS;
DECLARE VARIABLE FN CHAR(31) CHARACTER SET UNICODE_FSS;
DECLARE VARIABLE S BIGINT;
BEGIN
BLOB_SIZE = 0;
FOR
SELECT r.rdb$relation_name, r.rdb$field_name
FROM rdb$relation_fields r JOIN rdb$fields f
ON r.rdb$field_source = f.rdb$field_name
WHERE f.rdb$field_type = 261
INTO :RN, :FN
DO BEGIN
EXECUTE STATEMENT
'SELECT SUM(OCTET_LENGTH(' || :FN || ')) AS BLOB_SIZE, ''' || :RN || ''', ''' || :FN || '''
FROM ' || :RN ||
' WHERE NOT ' || :FN || ' IS NULL'
INTO :BLOB_SIZE, :TABLENAME, :FIELDNAME;
SUSPEND;
END
END
#
SET TERM ;#
This example doesn't work with ORDER BY, maybe a more elegant solution without EXECUTE BLOCK exists.

Related

How to concate join condition based on input flag value inside cursor in DB2

I have a below SP which has sql query which need to refactor db2 query,in db2 i dont know how to concatenate the flag condition remaining query to main query.
CREATE PROCEDURE EMPLOYEE
(IN EMPID varchar(1000),
IN BFLAG char(3))
RESULT SETS 1
LANGUAGE SQL
P1: BEGIN
SET v_sql = 'select c.id,c.name from emp c'
IF BFLAG <> 'T' THEN
SET v_sql = v_sql ||
' left outer join dept U
where c.empid in (' || EMPID || ') ';
ELSE
SET v_sql = v_sql ||
' where c.empid in (' || EMPID || ') ';
END IF;
how to concatenate query in db2 based on flag value specified above condition.
DECLARE c_id CURSOR WITH RETURN FOR
select c.id,c.name from emp c;
Too many errors.
You should study how the compound-statement must be constructed.
Every variable must be declared.
The order of statements inside is significant: variable declarations first, then statements, cursors. SQL-procedure-statements are afterwards only.
You get erroneous dynamic statement if BFLAG <> 'T' is true: left join without conditions.
If the questions is how to use cursor with dynamic statements, then here is an example:
CREATE PROCEDURE TEST_DYNAMIC (IN P_REST varchar(1000))
DYNAMIC RESULT SETS 1
LANGUAGE SQL
P1: BEGIN
DECLARE v_sql VARCHAR (1000);
DECLARE v_s STATEMENT;
DECLARE v_c1 CURSOR WITH RETURN FOR v_s;
SET v_sql = 'select ' || P_REST;
PREPARE v_s FROM v_sql;
OPEN v_c1;
END
#
CALL TEST_DYNAMIC (' * FROM EMPLOYEE WHERE EMPNO IN (''000010'', ''000020'')')#
If not, then try to compile your routine without errors at least and show the runtime error / unexpected result if any...

Update Null columns to Zero dynamically in Redshift

Here is the code in SAS, It finds the numeric columns with blank and replace with 0's
DATA dummy_table;
SET dummy_table;
ARRAY DUMMY _NUMERIC_;
DO OVER DUMMY;
IF DUMMY=. THEN DUMMY=0;
END;
RUN;
I am trying to replicate this in Redshift, here is what I tried
create or replace procedure sp_replace_null_to_zero(IN tbl_nm varchar) as $$
Begin
Execute 'declare ' ||
'tot_cnt int := (select count(*) from information_schema.columns where table_name = ' || tbl_nm || ');' ||
'init_loop int := 0; ' ||
'cn_nm varchar; '
Begin
While init_loop <= tot_cnt
Loop
Raise info 'init_loop = %', Init_loop;
Raise info 'tot_cnt = %', tot_cnt;
Execute 'Select column_name into cn_nm from information_schema.columns ' ||
'where table_name ='|| tbl_nm || ' and ordinal_position = init_loop ' ||
'and data_type not in (''character varying'',''date'',''text''); '
Raise info 'cn_nm = %', cn_nm;
if cn_nm is not null then
Execute 'Update ' || tbl_nm ||
'Set ' || cn_nm = 0 ||
'Where ' || cn_nm is null or cn_nm =' ';
end if;
init_loop = init_loop + 1;
end loop;
End;
End;
$$ language plpgsql;
Issues I am facing
When I pass the Input parameter here, I am getting 0 count
tot_cnt int := (select count(*) from information_schema.columns where table_name = ' || tbl_nm || ');'
For testing purpose I tried hardcode the table name inside proc, I am getting the error amazon invalid operation: value for domain information_schema.cardinal_number violates check constraint "cardinal_number_domain_check"
Is this even possible in redshift, How can I do this logic or any other workaround.
Need Expertise advise here!!
You can simply run an UPDATE over the table(s) using the NVL(cn_nm,0) function
UPDATE tbl_raw
SET col2 = NVL(col2,0);
However UPDATE is a fairly expensive operation. Consider just using a view over your table that wraps the columns in NVL(cn_nm,0)
CREATE VIEW tbl_clean
AS
SELECT col1
, NVL(col2,0) col2
FROM tbl_raw;

Concat text variables in postgresql function

I have this code and I want to concatenate the variables but don't work.
This is my DDL code for the view:
CREATE OR REPLACE function acd.add_credito2()
RETURNS void
SET SCHEMA 'acd'
SET search_path = acd
AS $$
DECLARE
auxsigla text;
auxnome text;
_sql text := 'CREATE OR REPLACE VIEW acd.teste AS SELECT md.matriz_disciplina_id AS id, dcp.nome, mc.curso, mc.versao AS matriz';
_join text := ' FROM matriz_disciplina as md LEFT JOIN disciplina as dcp ON md.disciplina_id = dcp.disciplina_id LEFT JOIN matriz_curricular as mc ON md.matriz_curricular_id = mc.matriz_curricular_id';
BEGIN
select into auxsigla, auxnome from ( select sigla, nome from acd.categoria_credito where categoria_credito_id = 9) as foo;
_join := _join || ' LEFT JOIN (SELECT creditos, matriz_disciplina_id FROM acd.disciplina_credito WHERE categoria_credito_id = ' || x || ') AS ' || "auxsigla" ' ON ' || "auxsigla" || '.matriz_disciplina_id = md.matriz_disciplina_id';
_sql := _sql || ', ' || "auxsigla" || '.' || auxnome || ' AS ' || auxnome;
_sql := _sql || _join;
EXECUTE _sql;
END;
$$ LANGUAGE plpgsql
So, when I execute the function
database-1=# select acd.add_credito2();
This error appears:
ERROR: type "auxsigla" does not exist
LINE 1: ...WHERE categoria_credito_id = ' || x || ') AS ' || "auxsigla"...
^
QUERY: SELECT _join || ' LEFT JOIN (SELECT creditos, matriz_disciplina_id FROM acd.disciplina_credito WHERE categoria_credito_id = ' || x || ') AS ' || "auxsigla" ' ON ' || "auxsigla" || '.matriz_disciplina_id = md.matriz_disciplina_id'
CONTEXT: PL/pgSQL function add_credito2() line 13 at assignment
Can anyone help me? I don't know what to do now.
(I know, this study view don't have a purpose but this is the idea that I want to use in the real view)
The error comes from this construct:
"auxsigla" ' ON '
You forgot a concatenation operator || between these two tokens, and now the SQL parser interprets it as
data_type string_constant
which is a way to specify constants of a certain data type.
Working examples would be DATE '2018-09-20' or INTEGER '-20'.
Your function has numerous other problems, two of which I could spot:
select into auxsigla, auxnome from will always set the variables to NULL because you forgot to specify which columns you want to select.
you do not properly escape single quotes while composing your dynamic query string. What if auxsigla has the value with'quote?
Use format() or quote_literal() and quote_ident() for that.

Postgresql create a log schema

So my problem is simple. I have a schema prod with many tables, and another one log with the exact same tables and structure (primary keys change that's it).
When I do UPDATE or DELETE in the schema prod, I want to record old data in the log schema.
I have the following function called after a update or delete:
CREATE FUNCTION prod.log_data() RETURNS trigger
LANGUAGE plpgsql AS $$
DECLARE
v RECORD;
column_names text;
value_names text;
BEGIN
-- get column names of current table and store the list in a text var
column_names = '';
value_names = '';
FOR v IN SELECT * FROM information_schema.columns WHERE table_name = quote_ident(TG_TABLE_NAME) AND table_schema = quote_ident(TG_TABLE_SCHEMA) LOOP
column_names = column_names || ',' || v.column_name;
value_names = value_names || ',$1.' || v.column_name;
END LOOP;
-- remove first char ','
column_names = substring( column_names FROM 2);
value_names = substring( value_names FROM 2);
-- execute the insert into log schema
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' ( ' || column_names || ' ) VALUES ( ' || value_names || ' )' USING OLD;
RETURN NULL; -- no need to return, it is executed after update
END;$$;
The annoying part is that I have to get column names from information_schema for each row.
I would rather use this:
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || OLD;
But some values can be NULL so this will execute:
INSERT INTO log.user SELECT 2,,,"2015-10-28 13:52:44.785947"
instead of
INSERT INTO log.user SELECT 2,NULL,NULL,"2015-10-28 13:52:44.785947"
Any idea to convert ",," to ",NULL,"?
Thanks
-Quentin
First of all I must say that in my opinion using PostgreSQL system tables (like information_schema) is the proper way for such a usecase. Especially that you must write it once: you create the function prod.log_data() and your done. Moreover it may be dangerous to use OLD in that context (just like *) as always because of not specified elements order.
But,
to answer your exact question the only way I know is to do some operations on OLD. Just observe that you cast OLD to text by doing concatenation ... ' SELECT ' || OLD. The default casting create that ugly double-commas. So, next you can play with that text. In the end I propose:
DECLARE
tmp TEXT
...
BEGIN
...
/*to make OLD -> text like (2,,3,4,,)*/
SELECT '' || OLD INTO tmp; /*step 1*/
/*take care of commas at the begining and end: '(,' ',)'*/
tmp := replace(replace(tmp, '(,', '(NULL,'), ',)', ',NULL)'); /*step 2*/
/* replace rest of commas to commas with NULL between them */
SELECT array_to_string(string_to_array(tmp, ',', ''), ',', 'NULL') INTO tmp; /*step 3*/
/* Now we can do EXECUTE*/
EXECUTE 'INSERT INTO log.' || TG_TABLE_NAME || ' SELECT ' || tmp;
Of course you can do steps 1-3 in one big step
SELECT array_to_string(string_to_array(replace(replace('' || NEW, '(,', '(NULL,'), ',)', ',NULL)'), ',', ''), ',', 'NULL') INTO tmp;
In my opinion this approach isn't any better from using information_schema, but it's your call.

PostgreSQL ERROR: EXECUTE of SELECT ... INTO is not implemented

When I run the following command from a function I defined, I get the error "EXECUTE of SELECT ... INTO is not implemented". Does this mean the specific command is not allowed (i.e. "SELECT ...INTO")? Or does it just mean I'm doing something wrong? The actual code causing the error is below. I apologize if the answer is already out here, however I looked and could not find this specific error. Thanks in advance... For whatever it's worth I'm running 8.4.7
vCommand = 'select ' || stmt.column_name || ' as id ' ||
', count(*) as nCount
INTO tmpResults
from ' || stmt.table_name || '
WHERE ' || stmt.column_name || ' IN (select distinct primary_id from anyTable
WHERE primary_id = ' || stmt.column_name || ')
group by ' || stmt.column_name || ';';
EXECUTE vCommand;
INTO is ambiguous in this use case and then is prohibited there.
You can use a CREATE TABLE AS SELECT instead.
CREATE OR REPLACE FUNCTION public.f1(tablename character varying)
RETURNS integer
LANGUAGE plpgsql
AS $function$
begin
execute 'create temp table xx on commit drop as select * from '
|| quote_ident(tablename);
return (select count(*) from xx);
end;
$function$
postgres=# select f1('omega');
f1
────
2
(1 row)