i'm learning psql in Postgres.
My basic question is when i create a function like this one :
CREATE OR REPLACE FUNCTION totalRecords ()
RETURNS integer AS $total$
declare
total integer;
BEGIN
SELECT count(*) into total FROM COMPANY;
RETURN total;
END;
$total$ LANGUAGE plpgsql;
i must write all of the code in the prompt command line.
How can i save this code in a script and call it from the command line ? the extentsion of the scirpt must be a .sql ? how can i call this script.
Save your script to a file. Then execute like this:
psql -p portnumber -d database -U user -f mysqlscrpt.sql
The extension of the script does not matter.
#jkdba's answer is sufficient but, alternatively, if you are already logged in the psql console, you can use:
\i <sqlscript>.sql
The \i being for input. \o is output, and it would write to the file instead of bringing it in.
I mention this because this is what made me remember the command :)
Related
I want to run some sql scripts if a specific schema exists. This is my attempt:
do $$
begin
if exists (select 1 from information_schema.schemata where schema_name = 'public')
then
\ir ./script.sql
end if;
end
$$;
But I get this error:
$ psql my_database -f test.sql
psql:test.sql:8: ERROR: syntax error at or near "\"
LINE 5: \ir ./script.sql
The same script without the if statement works and is able to trigger ./script.sql.
Adding the code from script.sql directly in the if statement also works, there doesn't seem to be anything wrong with the if syntax.
How can you conditionally trigger SQL scripts?
You cannot do that in PL/pgSQL.
You could write a function in PL/PerlU or PL/Python that does that, or you can install the PL/sh language.
I write PL/SQL scripts using Oracle and I'm trying my first PL/pgSQL script with PostgreSQL. For example, I created this test.sql file.
DECLARE
v_loc_nbr INTEGER;
BEGIN
v_loc_nbr := 0;
END;
Which I try to execute using the command line:
\postgresql\9.3\bin\psql.exe -d postgres -f test.sql
but I get syntax errors like:
psql:test.sql:4: ERROR: syntax error at or near "v_loc_nbr"
I think the problem is it trying to execute SQL when I want it to run PL/pgSQL. What should the command be?
Like with other procedural languages, you cannot run PL/pgSQL directly.
You can only run SQL. Use plpgsql inside a function body or wrapped in a DO command, where the range of DO commands is limited since they cannot return data.
Check out the plpgsql tag for examples.
I don't want to explain more about this because Erwin explained well.You need to wrap your sql inside a DO, so your test.sql should write like this
DO
$$
DECLARE
v_loc_nbr INTEGER;
BEGIN
v_loc_nbr := 0;
END;
$$
and try to execute it \postgresql\9.3\bin\psql.exe -d postgres -f test.sql
mohpc04pp1: /h/u544835 % psql arco
Welcome to psql 8.1.17, the PostgreSQL interactive terminal.
Type: \copyright for distribution terms
\h for help with SQL commands
\? for help with psql commands
\g or terminate with semicolon to execute query
\q to quit
WARNING: You are connected to a server with major version 8.3,
but your psql client is major version 8.2. Some backslash commands,
such as \d, might not work properly.
dbname=> \o /h/u544835/data25000.csv
dbname=> select url from urltable where scoreid=1 limit 25000;
dbname=> \q
This is took from a link online of basically what I have been doing, but what I need to do is make a script that I can use to produce csv files daily
So my aim of the script is to while in the script connect to the db, run the \o etc commands then close it
but I'm having trouble scripting it to say go into the psql arco database then run those queries.
command line to connect to db = psql arco then once the scrits recognised I'm in that databse perform those commands to automate a query to a csv file.
if anyone can get me started or point me towards reading material for me to get past that bit, it will be duely appreciated.
i'm running all this off a standard windows xp, ssh'ing to a SLES set-up web server that holds my postgresql database running psql version 8.1.17
First of all you should fix your setup. As it turns out, we are dealing with PostgreSQL 8.1 here. This version has reached end of live in 2010. You need to seriously think about upgrading - or at least remind the guys running the server. Current version is 9.1.
The command you are looking for:
psql arco -c "\copy (select url from urltable where scoreid=1 limit 25000) to '/h/u544835/data25000.csv'"
Assuming your db is named "arco". Adjusted for changed question (including changed port).
I now see version 8.1 popping up in your question, but it's all contradictory. You need Postgres 8.2 or later to use a query (instead of a table) with the \copy meta-command.
Details about psql in the fine manual.
Alternative approach that should work with obsolete PostgreSQL 8.1:
psql arco -o /h/u544835/data25000.csv -t -A -c 'SELECT url FROM urltable WHERE scoreid = 1 LIMIT 25000'
Find some more info about these command line options under this related question on dba.SE.
With function (syntax compatible with 8.1)
Another way would be to create a server side function (if you can!) that executes COPY from a temp table (old syntax - works with pg 8.1):
CREATE OR REPLACE FUNCTION f_copy_file()
RETURNS void AS
$BODY$
BEGIN
CREATE TEMP TABLE u_tmp AS (
SELECT url FROM urltable WHERE scoreid = 1 LIMIT 25000
);
COPY u_tmp TO '/h/u544835/data25000.csv';
DROP TABLE u_tmp;
END;
$BODY$
LANGUAGE plpgsql;
And then from the shell:
psql arco -c 'SELECT f_copy_file()'
Change the separator
\f sets the field separator. I quote the manual again:
-F separator
--field-separator=separator
Use separator as the field separator for unaligned output.
This is equivalent to \pset fieldsep or \f.
Or you can change the column separator in Excel, here are the instructions from MS.
Thanks to Erwin's help and a link I read up on he posted for me I managed to combine the two to get
#!/bin/sh
dbname='arco'
username='' # If you actually supply a username, you need to add the -U switch!
psql $dbname $username << EOF
\f ,
\o /h/u544835/showme.csv
SELECT * FROM storage;
EOF
which will write my queries to a csv file etc for me.
From what there is above, it is not separating the sql query so if I load it straight into excel, they all stay in the same column too which means I'm having a problem with the delimiter
I've tried tabbed delimiters, also tried , ; etc but none are letting me separate it
I need for it
is there an option I can click to see which delimiter is being used with my psql? or a different way of dumping the data from a query into a file that can be read by excel, so a different column for each row etc
i am working wit h PostgreSQL 9 for an application, i have a database with a table 'species'
where i store fish species details along with the image of the species.
the table is
CREATE TABLE fishes
(
fishes character varying(7) NOT NULL,
speciesimages oid,
CONSTRAINT species_pkey PRIMARY KEY (species)
)
WITH (
OIDS=TRUE
);
i use
INSERT INTO species(fishes,fishesimages VALUES('01',lo_import('C://01.jpg'));
To store the images in the database.
to retrieve the images i use
SELECT lo_export(fishes.fishesimages,'c://outimage.jpg')
FROM fishes
WHERE fishes= '01';
This works fine when the host is Localhost but when it is the server i cannot use the path c:// as this may not exist on the server system and i dont have permissions anyways.
so i set out to use
\COPY
like this
"C:\Program Files\PostgreSQL\9.0\bin\psql.exe" -h 192.168.1.101 -p 5432 -d myDB -U DB_admin -c "\COPY (SELECT lo_export(fishes.fishesimages,'01.jpg') FROM fishes WHERE species = '01') TO 'C://leeImage.jpeg' WITH BINARY";
but this create a image file but when i open it its invalid image
can anyone tell me how to use lo_export function from the server machine and create the image on client machine?
Basically:
lo_export will instruct the server to write a file locally (always)
\copy will be transformed by psql to a COPY ... TO STDOUT command and the output written to the specified file. (So what is written to that file is the result of the select statment you were doing before)
So, you can not use lo_export in this way, it will always write a file onto the server's filesystem.
Of course, you can solve this simply by having the server write to a shared drive, and then read the file from that drive. Ugly, but effective IMHO.
For some recent versions of psql (not sure when this was introduced) there is a \lo_export psql command which takes an OID and filename, e.g.:
\lo_export 28914648 testfile
However you need to get the OID of the file into the script somehow...
You can write a PL/PGsql function like this to dump the file as a bytea:
CREATE OR REPLACE FUNCTION getfiledata(lobjid oid) RETURNS bytea
STRICT STABLE LANGUAGE plpgsql AS $$
DECLARE
fd int4;
imgsize int4;
INV_READ int4 := 262144;
SEEK_SET int4 := 0;
SEEK_END int4 := 2;
BEGIN
SELECT lo_open(lobjid, INV_READ) INTO fd;
PERFORM lo_lseek(fd, 0, SEEK_END);
SELECT lo_tell(fd) INTO imgsize;
PERFORM lo_lseek(fd, 0, SEEK_SET);
RETURN loread(fd, imgsize);
END;
$$;
Now calling this function with the OID of the large object will return its content as a bytea value. You can thus call this function in a COPY command and it will return the file data... and by using \copy it will be sent to the client.
It's generally recommended these days to use bytea columns directly rather than this large object interface (bytea was introduced a lot later). PostgreSQL will automatically move large values into out-of-line storage ("TOAST tables") and will also compress them (unless the storage mode is set to "external" to suppress this, which is probably the right thing to do for JPEG images etc)
EDIT :
\lo_export
Try this from commandprompt.com
`"C:\Program Files\PostgreSQL\9.0\bin\psql.exe" -h 192.168.1.101 -p 5432 -d myDB -U DB_admin -c "\lo_export 19135 'C://leeImage.jpeg' ";`
where the number 19135 is the OID of the species whose image i want on the client system.. the OID you can get from the fishes table fishesimages OID
use the OID in the above code and you can use the OID get the images.
I am not a programmer, I am struggling a bit with this.
I have a batch file connecting to my PostgreSQL server, and then open a sql script. Everything works as expected. My question is how to pass a variable (if possible) from one to the other.
Here is my batch file:
set PGPASSWORD=xxxx
cls
#echo off
C:\Progra~1\PostgreSQL\8.3\bin\psql -d Total -h localhost -p 5432 -U postgres -f C:\TotalProteinImport.sql
And here's the script:
copy totalprotein from 'c:/TP.csv' DELIMITERS ',' CSV HEADER;
update anagrafica
set pt=(select totalprotein.resultvalue from totalprotein where totalprotein.accessionnbr=anagrafica.id)
where data_analisi = '12/23/2011';
delete from totalprotein;
This is working great, now the question is how could I pass a variable that would carry the date for data_analisi?
Like in the batch file, "Please enter date", and then the value is passed to the sql script.
You could create a function out of your your SQL script like this:
CREATE OR REPLACE FUNCTION f_myfunc(date)
RETURNS void AS
$BODY$
CREATE TEMP TABLE t_tmp ON COMMIT DROP AS
SELECT * FROM totalprotein LIMIT 0; -- copy table-structure from table
COPY t_tmp FROM 'c:/TP.csv' DELIMITERS ',' CSV HEADER;
UPDATE anagrafica a
SET pt = t.resultvalue
FROM t_tmp t
WHERE a.data_analisi = $1
AND t.accessionnbr = a.id;
-- Temp table is dropped automatically at end of session
-- In this case (ON COMMIT DROP) after the transaction
$BODY$
LANGUAGE sql;
You can use language SQL for this kind of simple SQL batch.
As you can see I have made a couple of modifications to your script that should make it faster, cleaner and safer.
Major points
For reading data into an empty table temporarily, use a temporary table. Saves a lot of disc writes and is much faster.
To simplify the process I use your existing table totalprotein as template for the creation of the (empty) temp table.
If you want to delete all rows of a table use TRUNCATE instead of DELETE FROM. Much faster. In this particular case, you need neither. The temporary table is dropped automatically. See comments in function.
The way you updated anagrafica.pt you would set the column to NULL, if anything goes wrong in the process (date not found, wrong date, id not found ...). The way I rewrote the UPDATE, it only happens if matching data are found. I assume that is what you actually want.
Then ask for user input in your shell script and call the function with the date as parameter. That's how it could work in a Linux shell (as user postgres, with password-less access (using IDENT method in pg_haba.conf):
#! /bin/sh
# Ask for date. 'YYYY-MM-DD' = ISO date-format, valid with any postgres locale.
echo -n "Enter date in the form YYYY-MM-DD and press [ENTER]: "
read date
# check validity of $date ...
psql db -p5432 -c "SELECT f_myfunc('$date')"
-c makes psql execute a singe SQL command and then exits. I wrote a lot more on psql and its command line options yesterday in a somewhat related answer.
The creation of the according Windows batch file remains as exercise for you.
Call under Windows
The error message tells you:
Function tpimport(unknown) does not exist
Note the lower case letters: tpimport. I suspect you used mixe case letters to create the function. So now you have to enclose the function name in double quotes every time you use it.
Try this one (edited quotes!):
C:\Progra~1\PostgreSQL\8.3\bin\psql -d Total -h localhost -p 5432 -U postgres
-c "SELECT ""TPImport""('%dateimport%')"
Note how I use singe and double quotes here. I guess this could work under windows. See here.
You made it hard for yourself when you chose to use mixed case identifiers in PostgreSQL - a folly which I never tire of warning against. Now you have to double quote the function name "TPImport" every time you use it. While perfectly legit, I would never do that. I use lower case letters for identifiers. Always. This way I never mix up lower / upper case and I never have to use double quotes.
The ultimate fix would be to recreate the function with a lower case name (just leave away the double quotes and it will be folded to lower case automatically). Then the function name will just work without any quoting.
Read the basics about identifiers here.
Also, consider upgrading to a more recent version of PostgreSQL 8.3 is a bit rusty by now.
psql supports textual replacement variables. Within psql they can be set using \set and used using :varname.
\set xyz 'abcdef'
select :'xyz';
?column?
----------
abcdef
These variables can be set using command line arguments also:
psql -v xyz=value
The only problem is that these textual replacements always need some fiddling with quoting as shown by the first \set and select.
After creating the function in Postgres, you must create a .bat file in the bin directory of your Postgres version, for example C:\Program Files\PostgreSQL\9.3\bin. Here you write:
#echo off
cd C:\Program Files\PostgreSQL\9.3\bin
psql -p 5432 -h localhost -d myDataBase -U postgres -c "select * from myFunction()"