I have few functions containing select queries and dbms_output.put_line statements. Further, I am trying to execute them via a script (master.sql) and need to send their corresponding dbms_output results to their respective text files.
Sample function:
CREATE OR REPLACE FUNCTION myschema.my_func1() RETURNS VOID AS $func$
DECLARE
cur_fetch_my_data CURSOR IS
select col1,col2 from myschema.table1;
BEGIN
DBMS_OUTPUT.put_line('column 1'||CHR(9)||'column 2');
for data_cur in cur_fetch_my_data LOOP
DBMS_OUTPUT.put_line(data_cur.col1||CHR(9)||data_cur.col2);
END LOOP;
END;
$func$ LANGUAGE sql;
Script for executing functions:
--master.sql
begin
dbms_output.serveroutput(true);
end;
\o DUMP_OF_func1.txt
SELECT myschema.my_func1();
\o DUMP_OF_func2.txt
SELECT myschema.my_func2();
After above script execution, all dbms_output results are printed on console but not in text file. Only thing which is dumped into text file is
my_func1
------------
(1 row)
Further, i tried changing the master script. I changed
dbms_output.serveroutput(true)
to
dbms_output.serveroutput(false)
After doing this, console output did not print. But dump into text file remained same.
Can anyone please tell me how I can divert dbms_output results to their text files?
Related
I'm writing my own regression test, and I'm using \o command to output my query result to a temporary file. For example
create or replace function aaaa()
returns integer as
$$
BEGIN
RAISE NOTICE 'ssss';
RETURN 1;
END;
$$ language plpgsql;
\o a.tmp
select aaaa();
With those commands, I'm able to output the return value (1 in this case) to file a.tmp. However, the message NOTICE: ssss is print directly to my psql client.
I'm wondering if there is a way to print the NOTICE: ssss also to a.tmp.
I am brand new to PostgreSQL and I need someone to point me in
the right direction on how to write the results of a function to a text
file. Is it possible to do this within the PostgreSQL PL/pgSQL
language? I have done this before in Oracle using the UTL_FILE commands
and I was hoping that PostgreSQL had similar functionality. Thanks in advance for any help that you can give.
Kindly provide some examples of file operations.
CREATE OR REPLACE FUNCTION NP_AC015_FETCH.proc_log (P_MSG text, P_MODE integer default 1) RETURNS VOID AS $body$
DECLARE
V_F_IS_OPEN boolean; --IF LOG FILE IS ALREADY OPEN THIS IS SET TO TRUE
V_LOG_MSG varchar(32767); --LOG FILE NAME
V_LOG_DIR varchar(30) := 'ND_GANJIS_LOG_DIR'; --LOG DIRECOTY
vTemp UTL_FILE.FILE_TYPE;
BEGIN
select get_var('GM_LOG_FILE') INTO vTemp;
V_F_IS_OPEN := utl_file.is_open(vTemp);
if not V_F_IS_OPEN then
-- Log File Open
-- 32767 IS THE MAXIMUM NUMBER OF CHARACTERS PER LINE, INCLUDING THE NEWLINE CHARACTER, FOR THIS FILE.
vTemp := UTL_FILE.FOPEN(V_LOG_DIR, 'NIA_PLSQL_'||to_char(clock_timestamp(), 'yyyymmdd')||'.log', 'A', 32767);
end if;
-- LOG MSG TO BE WRITTEN TO THE LOG FILE
V_LOG_MSG := TO_CHAR(CURRENT_TIMESTAMP, 'yyyy/mm/dd hh24:mi:ss:ff3') ||' '|| P_MSG;
--Output messages to a file
UTL_FILE.PUT_LINE(vTemp, V_LOG_MSG);
--Closing Log File
if P_MODE = current_setting('NP_AC015_FETCH.PV_LOG_CLOSE_MODE')::pls_integer and utl_file.is_open(vTemp) then
utl_file.fclose(vTemp);
end if;
--HERE THE EXCEPTION PART IS NOT INCLUDED,
--Reason： PROGRAM WILL GO ON INFINITE LOOP IF SOME ERROR OCCURS HERE, BECAUSE, EACH EXCEPTION WRITES INTO
--LOG FILE, USING THIS PROCEDURE.
exception
when others then
RAISE EXCEPTION '%', dbms_utility.format_error_backtrace||chr(10)||dbms_utility.format_error_stack||chr(10)||dbms_utility.format_call_stack, true;
END;
$body$
LANGUAGE PLPGSQL;
You can install the adminpack contrib module and use the function pg_file_write(filename text, data text, append boolean).
Note that this function is restricted to superusers, but you can create a SECURITY DEFINER function owned by a superuser that provides the necessary functionality to the users you choose.
I may be contorting postgres here, but fundamentally, what I would like to do is take a string variable and pass it to an sql command (in this case COPY) using only psql.
So this is what I came up with. The commands are separated into 2 files because I want to be able to use the mydb_functions in other situations:
file one: mydb_functions--1.0.sql (in share/extension and with mydb_functions.control also setup as described in the manual. Given a filename, returns a full filepath. this is done solely to make the COPY statements in add_data.sql below, neater.
CREATE OR REPLACE FUNCTION fpn(filename text) RETURNS TEXT as '
DECLARE
mypath text := ''/path/to/my/directory/'';
BEGIN
RETURN mypath || filename;
END
' LANGUAGE plpgsql;
file two: add_data.sql . This exists solely to copy data into existing postgres tables using psql at the command line. Note: running psql with superuser privileges is required because of the CREATE EXTENSION command.
CREATE EXTENSION IF NOT EXISTS mydb_functions;
-- haven't figured out how to create a function without arguments yet.
CREATE OR REPLACE FUNCTION test (dummyvar text) RETURNS text as '
DECLARE
filepath RECORD;
BEGIN
SELECT * INTO filepath from fpn(''mydatafile.data'');
COPY tablename (columnname) FROM filepath;
END
' LANGUAGE plpgsql;
The part I am stuck on is how to extract a text from the filepath record to use in the COPY command. Any tips on an easier way to achieve this are also welcome. I think creating a table to store the variable is far easier than this. But I would like to finish the last step.
If your issue is running COPY with a dynamic target path, use EXECUTE to run a formatted SQL query.
CREATE OR REPLACE FUNCTION test () RETURNS text as $$
DECLARE
filepath RECORD;
BEGIN
SELECT * INTO filepath from fpn('mydatafile.data');
EXECUTE format('COPY tablename (columnname) FROM %L', filepath);
END
$$ LANGUAGE plpgsql;
See, this fails:
DO
$$
DECLARE
somevar text;
BEGIN
somevar := '/tmp/somepath.csv';
COPY tablename(columnname) FROM somevar;
END;
$$;
but this works:
DO
$$
DECLARE
somevar text;
BEGIN
somevar := '/tmp/somepath.csv';
EXECUTE format('COPY tablename(columnname) FROM %L', somevar);
END;
$$;
See:
EXECUTE
format()
format()'s %L specifier auto-quotes literals. %I does the same for identifiers.
If as I originally thought, you're talking about getting data into psql from outside, you may find psql variables and variable interpolation useful:
$ psql -v filepath=/path/to/my/directory/mydatafile.data regress
regress=> SELECT :'filepath';
?column?
---------------------------------------
/path/to/my/directory/mydatafile.data
(1 row)
Note that the colon is unquoted, then the variable name is quoted. Odd syntax, I know. This only works in psql; it won't work in (say) PgAdmin-III.
Alternately, you can use a here-document (unix-like shell specific, won't work in Windows' cmd.exe) to do quoted text interpolation:
$ FILEPATH=/path/to/my/directory/mydatafile.data
$ psql regress <<__END__
SELECT '$FILEPATH';
__END__
?column?
---------------------------------------
/path/to/my/directory/mydatafile.data
(1 row)
Both of these show how to insert a variable from the shell level into psql. From there, it's a matter of using it in your function. Like this, say, with -v filename=myfilename.csv:
CREATE OR REPLACE FUNCTION test() RETURNS text as $$
DECLARE
filepath RECORD;
BEGIN
SELECT * INTO filepath from fpn(:'filename');
COPY tablename (columnname) FROM filepath;
END
$$ LANGUAGE plpgsql;
or just this with -v filepath=/full/path/to/myfilename.csv:
$ psql -v filepath=/path/to/my/directory/mydatafile.data regress
regress=> COPY tablename (columnname) FROM :'filepath';
I'm trying to get a PSQL script running using variables in an example like the one below without declaring functions and having to call them.
DECLARE
result TEXT;
BEGIN
SELECT INTO result name
FROM test;
RAISE NOTICE result;
END;
Where table test only has 1 row and column. Is this possible without having to wrap this script inside a function. This will allow me to call the script via say command line easier.
Thanks guys.
You can use DO to create and execute an anonymous function:
DO executes an anonymous code block, or in other words a transient anonymous function in a procedural language.
Something like this:
do $$
declare result text;
begin
select name into result from test;
raise notice '%', result;
end;
$$;
I also fixed your raise notice.
If you just want to dump the single value from the table to the standard output in a minimal format (i.e. easy to parse), then perhaps --tuples-only will help:
-t
--tuples-only
Turn off printing of column names and result row count footers, etc. This is equivalent to the \t command.
So you could say things like this from the shell:
result=$(echo 'select name from test;' | psql -t ...)
This PostgreSQL COPY command works:
copy tablename from E'c:\\abc\\a.txt';
but I want the tablename to be dynamically generated. How can I do this?
You need to build a string, concatenating in the dynamic table name, and then use execute. Note that you escape the ' by ''. This also includes a dynamic name to save the file too. You need to replace savedir with the actual directory you are using.
CREATE OR REPLACE FUNCTION dynamicCopy(tablename text, outname text) RETURNS VOID AS $$
DECLARE STATEMENT TEXT;
BEGIN
STATEMENT := 'COPY (select * from ' || quote_ident(tablename) || ') to ''savedir' || outname ||'.txt''';
EXECUTE STATEMENT;
END;
$$ LANGUAGE 'plpgsql';
EDIT:
Since I first wrote this, I have discovered the format function, which I think is generally easier to read than SQL generated with the concatenation operator || and more flexible.
CREATE OR REPLACE FUNCTION dynamicCopy(tablename text, outname text) RETURNS VOID AS
$BODY$
BEGIN
EXECUTE FORMAT('COPY (SELECT * FROM %s) TO ''savedir%s.csv''',
tablename,
outname);
END
$BODY$
LANGUAGE plpgsql;
See the official docs for a full discussion: https://www.postgresql.org/docs/current/static/plpgsql-statements.html#PLPGSQL-STATEMENTS-EXECUTING-DYN
You can copy from multi csv files to an table via shell script.
Make an script file: vim csvtotable
write csvtotable script. Here is my example:
#!/bin/sh DBNAME=postgres files=$1 for file in ${files}; do psql -d ${DBNAME} -c "\copy parent_tree(parent_id, some_text) FROM '${file}' delimiters ',' csv header" done
Execute the script.
./csv2table "$(ls *.out.csv)"
Obviously, Local CSV file should be match with table. Then It will import from csv to database tables if csv file name ending with .out.csv.
I am not sure csv name match is global or just at present directory match.
Reference link:
-c command reference https://www.postgresql.org/docs/current/app-psql.html#APP-PSQL-PATTERNS
chomd command: https://linuxize.com/post/chmod-command-in-linux/
bash shanebang: https://linuxize.com/post/bash-shebang/