postgresql copy from csv file into table in windows through procedure - postgresql

below is my procedure executed to upload file into table and do joins etc.
CREATE OR REPLACE FUNCTION sp_product_price()
RETURNS void
LANGUAGE 'plpgsql'
COST 100
AS $BODY$
BEGIN
truncate table prd_product_data;
truncate table price_import;
COPY price_import FROM 'C:\Users\Ram\Documents\prices.csv' CSV HEADER;
truncate table active_product_price;
insert into active_product_price(key,name,price)
SELECT prd.key,prd.name,prd.price FROM prd_product_data prd JOIN price_import import ON prd.name = import.name;
raise notice 'success';
END
$BODY$;
Above procedure giving error could not open file "C:\Users\Ram\Documents\prices.csv" for reading: No such file or directory HINT: COPY FROM instructs the PostgreSQL server process to read a file. You may want a client-side facility such as psql's \copy.
I have given access to file for everyone in file properties.
I tried \copy in procedure but this gives error syntax error at or near "\" .
\copy is working when I ran cmd in psql but not in above procedure.
Is there a way to import file into table in above procedure/functions ?

The procedure and the COPY statement are running on the database server, so the file C:\Users\Ram\Documents\prices.csv must be on the database server as well (and your database user must either be a superuser or a member of pg_read_server_files).
The only way you can use COPY to load data from the client is COPY ... FROM STDIN, and you won't be able to use that in a procedure.
\copy is a psql command, not an SQL statement, so it cannot be used in a procedure either.
My suggestion is to use \copy, but to forget about the procedure.

Related

How to deal with error:SQL Error [58P01]?

I'm learning PostgreSQL, and i run into problem:
I have created table and trying to export it, but I have all the time same error
SQL Error [58P01]: ERROR: could not open file "D:\Janis\Mācības\SQL\Test\typestest.txt" for writing: No such file or directory
Hint: COPY TO instructs the PostgreSQL server process to write a file. You may want a client-side facility such as psql's \copy.
Script that I'm running:
CREATE TABLE char_data_types (
varchar_column varchar(10),
char_column char(10),
text_column text
);
INSERT INTO char_data_types
VALUES
('abc', 'abc', 'abc'),
('defghi', 'defghi', 'defghi');
COPY char_data_types TO 'D:\Janis\Mācības\SQL\Test\typestest.txt'
WITH (FORMAT CSV, HEADER, DELIMITER '|');
I have taken off all possible security restriction from file typestest.txt
There is no directory D:\Janis\Mācības\SQL\Test on the PostgreSQL server. Note that COPY writes to the database server, no the client (the statement runs on the server, and the server cannot access the client machine).
To export the data to the client machine, you have to use psql so that you can use \copy:
\copy char_data_types TO 'D:\Janis\Mācības\SQL\Test\typestest.txt' WITH (FORMAT CSV, HEADER, DELIMITER '|')
The command has to be in a single line.

Creating Batch Files with PostgreSQL \copy Command in Jetbrains Datagrip

I'm familiarizing myself with the standalone version of Datagrip and having a bit of trouble understanding the different approaches to composing SQL via console, external files, scratch files, etc.
I'm managing, referencing the documentation, and am happy to figure things out as such.
However, I'm trying to ingest CSV data into tables via batch files using the Postgres \copy command. Datagrip will execute this command without error but no data is being populated.
This is my syntax, composed and ran in the console view:
\copy tablename from 'C:\Users\username\data_file.txt' WITH DELIMITER E'\t' csv;
Note that the data is tab-separated and stored in a .txt file.
I'm able to use the import functions of Datagrip (via context menu) just fine but I'd like to understand how to issue commands to do similarly.
\copy is a command of the command-line PostgreSQL client psql.
I doubt that Datagrip invokes psql, so it won't be able to use \copy or any other “backslash command”.
You probably have to use Datagrip's import facilities. Or you start using psql.
Ok, but what about the SQL COPY command https://www.postgresql.org/docs/12/sql-copy.html ?
How can I run something like that with datagrip ?
BEGIN;
CREATE TEMPORARY TABLE temp_json(values text) ON COMMIT DROP;
COPY temp_json FROM 'MY_FILE.JSON';
SELECT values->>'aJsonField' as f
FROM (select values::json AS values FROM temp_json) AS a;
COMMIT;
I try to replace 'MY_FILE.JSON' with full path, parameter (?), I put it in sql directory etc.
The data grip answer is :
[2021-05-05 10:30:45] [58P01] ERROR: could not open file '...' for reading : No such file or directory
EDIT :
I know why. RTFM! -_-
COPY with a file name instructs the PostgreSQL server to directly read from or write to a file. The file must be accessible by the PostgreSQL user (the user ID the server runs as) and the name must be specified from the viewpoint of the server.
Sorry.....

PostgreSQL PL/PerlU trigger issue

I'm trying to create a PostgreSQL trigger on Linux written in Perl which should execute code based on external libraries. The SQL script containing the trigger looks like this:
CREATE OR REPLACE FUNCTION notify_mytable_update() RETURNS trigger AS $$
use lib "full_path_to_lib_dir";
use MyModule;
return;
$$ LANGUAGE plperlu
SECURITY DEFINER
SET search_path = myschema, public, pg_temp;
DROP TRIGGER IF EXISTS notify_mytable_update ON mytable;
CREATE TRIGGER notify_mytable_update AFTER UPDATE ON mytable
FOR EACH ROW
EXECUTE PROCEDURE notify_mytable_update();
The issue with this is that whenever I try to this script with psql I get a permission denied error in the Perl code for accessing MyModule. Giving full access to my home directory to postgres didn't help.
Thank you in advance!
Don't forget that to have access to a file, you not only need permissions to the file and the directory where it resides, but also to all directories in the path.
So if your module is /home/george/MyModule.pm, you need access to / and /home in addition to /home/george and the file itself.
You'll have to give these permissions to the operating system user running the PostgreSQL server process, commonly postgres.

How do you drop a PostgreSQL database with a carriage return in the name?

Somehow, via a restore done through code I ended up with a database with the following name (simulated output from \l command)
Name |
------------------|
\r +|
DATABASE_NAME |
I would like to be able to drop it, but I have no idea how to construct the name properly to include the carriage return when specified via DROPDB or DROP DATABASE commands.
If I can't drop it - can I change the owner to hide it - doesn't that also involve specifying the name?
PostgreSQL 9.2.4 on Ubuntu
Use unicode for line feed while dropping the database like below
drop database U&"\000ADATABASE_NAME";
You can as well ALTER the name like
alter database U&"\000ADATABASE_NAME" rename to "DATABASE_NAME_NEW";
do $$
begin
execute format('drop database %I', E'\rDATABASE_NAME');
end;
$$
Rahul's answer should be
drop database U&"\000ADATABASE_NAME";

Mysql to Posgresql query conversion

Please help to create postgresql query equal to mysql query
LOAD DATA LOCAL INFILE 'file.txt' REPLACE INTO TABLE newtable TERMINATED BY ',' IGNORE 1 LINES;
There is no equivalent feature in PostgreSQL - at least in the current 9.3 or any prior version.
You must do this in a few steps:
CREATE TEMPORARY TABLE ...
COPY into the temp table
Do an UPDATE ... FROM followed by an INSERT INTO ... WHERE NOT EXISTS (...) to merge data
DROP the temp table
Search for "postgresql bulk upsert" or "postgresql copy upsert".
you might be looking for COPY
COPY will be run by the PostgreSQL backend (user "postgres"). The backend user requires permissions to read & write to the data file in order to copy from/to it. You need to use an absolute pathname with COPY. \COPY on the other hand, runs under the current $USER, and with that users environment. And \COPY can handle relative pathnames. The psql \COPY is accordingly much easier to use if it handles what you need.