I'm trying to pass a sql file to psql. After reading the docs, tried:
psql_args=(
"password='$INPUT_PASSWORD'"
dbname=analytics
"host='$INPUT_HOST'"
user=analytics
port=32648
file=query.sql
)
psql "${psql_args[*]}"
psql: error: invalid connection option "file"
root#380773cb4e26:/#
If I remove the file=query.sql arg this results in a connection to psql. I just don't know how to pass it a query file?
On the docs, two arguments look like ones of interest here:
-f filename
--file=filename
Read commands from the file filename, rather than standard input
and also:
-c command
--command=command
Specifies that psql is to execute the given command string, command
I tried the file=query.sql one but that failed with the error message above.The command one wants a string whereas I want to pass a .sql file. I tried anyway:
psql_args=(
"password='$INPUT_PASSWORD'"
dbname=analytics
"host='$INPUT_HOST'"
user=analytics
port=32648
command=query.sql
)
psql "${psql_args[*]}"
psql: error: invalid connection option "command"
Is there a way that I can pass query.sql to psql in order to run a query?
You seem to be packaging options up into a connection string. But --file must be given directly as an option to psql, not as part of a connection string.
psql "${psql_args[*]}" --file=query.sql
Since other answer seem to overlook this.
Here is how to store dynamic options into an array, and pass it as arguments to the command:
#!/usr/bin/env bash
psql_args=(
"--dbname=analytics"
"--host=$INPUT_HOST"
"--user=analytics"
"--port=32648"
"--file=query.sql"
)
psql "${psql_args[#]}"
Related
/COPY MondayLotto FROM 'https://thelottoproject.blob.core.windows.net/data/MondayLotto.csv' DELIMITER ',' CSV HEADER
The command returns these error messages
(In Azure Cloud Shell terminal)
https:/thelottoproject.blob.core.windows.net/data/MondayLotto.csv: No such file or directory
(In SQL Shell (psql) on Windows10)
https:/thelottoproject.blob.core.windows.net/data/MondayLotto.csv: Invalid argument
I guess the // caused the error because the error message shows only / after https:
PostgreSQL server in Azure
The CSV file in Azure blob storage is accessible.
Is there any solution for this problem?
psql special commands starting by \ so the syntax should be
\copy MondayLotto FROM ...
Probably there will be second issue too. I don't know how the table MondayLotto was created. This looks like case sensitive identifier, and if it is really case sensitive identifier, then it should be used inside parentheses like "MondayLotto".
To load data from a web site, you could run something like
wget -O - https://thelottoproject.blob.core.windows.net/data/MondayLotto.csv | psql -c "COPY mondaylotto FROM STDIN (FORMAT 'csv', HEADER)"
I'm pretty new to PostgreSQL, but I have to query some results using psql in an interactive command line session. I am connecting through a cluster and I would like to extract the output of the query into a file for further analysis.
The command I use to connect is psql -h hostname.with.dbs -p 5432 -U my-username and inside I do the query. But it is not clear to me how to pipe that into a file in my user folder in the machine used to connect to Presto.
If I have to add more details, let me know, as I am not an expert and might forgot to add important information. Thank you all!
I found a solution to that. Basically appending \g file_name; at the end of the query. It saves the file in the directory where I launched the command to connect to the database. I didn't try to add full path to the file name, but I assume it would work as well.
I am writing a shell script that fetches data (.csv file) form AWS S3, downloads it locally onto an EC2 Linux AMI Instance, and then copies the data to an RDS PostGresql database.
My Shell code is the following:
FILE="$(ls DB)"
PARAMETERFORDB= "'\\COPY table(x,y) FROM ''$FILE'' CSV HEADER'"
$(psql --host=XXXXX --port=XXXXX --username=XXXXX --password --dbname=XXXXX -c ${PARAMETERFORDB})
So when the data from S3 is downloaded, I store the files' name inside the FILE variable (it is the only file in the folder, the folder will be deleted after the Database query).
I get following error message:
./shellTest.sh: line 21: '\COPY table(x,y) FROM ''14.9.2016.csv'' CSV HEADER': command not found
psql: option requires an argument -- 'c'
Try "psql --help" for more information.
What am I doing wrong?
In the line
PARAMETERFORDB= "'\\COPY table(x,y) FROM ''$FILE'' CSV HEADER'"
remove the space after the = and remove one level of single quotes:
PARAMETERFORDB="\\COPY table(x,y) FROM '$FILE' CSV HEADER"
In the line where psql is invoked, enclose ${PARAMETERFORDB} in double quotes since it contains spaces.
I am trying to log a complete session in psql into a .txt file. The command given to me was initially this:
psql db_name| tee file_name.txt
However, my SSH client does nothing until I quit it. That means, it does not recognize any command. More like a document, no action happens no matter what I write. So far, only '\q' is recognised which lets me get out of it. Any ideas what is happening? How am I to write the query if shell will not read anything. Also, I tried the following (this is before connecting to database) :
script filename.txt
It does show the message : script started, file is filename.txt, but I dont know where this file is stored and how to retrieve it.
Any help with the above will be welcome and really appreciated! Thanks a lot :)
There is option to psql for log query and results:
-L filename
--log-file filename
Write all query output into file filename, in addition to the normal output destination.
Try this:
psql db_name -L file_name.txt
I am not a programmer, I am struggling a bit with this.
I have a batch file connecting to my PostgreSQL server, and then open a sql script. Everything works as expected. My question is how to pass a variable (if possible) from one to the other.
Here is my batch file:
set PGPASSWORD=xxxx
cls
#echo off
C:\Progra~1\PostgreSQL\8.3\bin\psql -d Total -h localhost -p 5432 -U postgres -f C:\TotalProteinImport.sql
And here's the script:
copy totalprotein from 'c:/TP.csv' DELIMITERS ',' CSV HEADER;
update anagrafica
set pt=(select totalprotein.resultvalue from totalprotein where totalprotein.accessionnbr=anagrafica.id)
where data_analisi = '12/23/2011';
delete from totalprotein;
This is working great, now the question is how could I pass a variable that would carry the date for data_analisi?
Like in the batch file, "Please enter date", and then the value is passed to the sql script.
You could create a function out of your your SQL script like this:
CREATE OR REPLACE FUNCTION f_myfunc(date)
RETURNS void AS
$BODY$
CREATE TEMP TABLE t_tmp ON COMMIT DROP AS
SELECT * FROM totalprotein LIMIT 0; -- copy table-structure from table
COPY t_tmp FROM 'c:/TP.csv' DELIMITERS ',' CSV HEADER;
UPDATE anagrafica a
SET pt = t.resultvalue
FROM t_tmp t
WHERE a.data_analisi = $1
AND t.accessionnbr = a.id;
-- Temp table is dropped automatically at end of session
-- In this case (ON COMMIT DROP) after the transaction
$BODY$
LANGUAGE sql;
You can use language SQL for this kind of simple SQL batch.
As you can see I have made a couple of modifications to your script that should make it faster, cleaner and safer.
Major points
For reading data into an empty table temporarily, use a temporary table. Saves a lot of disc writes and is much faster.
To simplify the process I use your existing table totalprotein as template for the creation of the (empty) temp table.
If you want to delete all rows of a table use TRUNCATE instead of DELETE FROM. Much faster. In this particular case, you need neither. The temporary table is dropped automatically. See comments in function.
The way you updated anagrafica.pt you would set the column to NULL, if anything goes wrong in the process (date not found, wrong date, id not found ...). The way I rewrote the UPDATE, it only happens if matching data are found. I assume that is what you actually want.
Then ask for user input in your shell script and call the function with the date as parameter. That's how it could work in a Linux shell (as user postgres, with password-less access (using IDENT method in pg_haba.conf):
#! /bin/sh
# Ask for date. 'YYYY-MM-DD' = ISO date-format, valid with any postgres locale.
echo -n "Enter date in the form YYYY-MM-DD and press [ENTER]: "
read date
# check validity of $date ...
psql db -p5432 -c "SELECT f_myfunc('$date')"
-c makes psql execute a singe SQL command and then exits. I wrote a lot more on psql and its command line options yesterday in a somewhat related answer.
The creation of the according Windows batch file remains as exercise for you.
Call under Windows
The error message tells you:
Function tpimport(unknown) does not exist
Note the lower case letters: tpimport. I suspect you used mixe case letters to create the function. So now you have to enclose the function name in double quotes every time you use it.
Try this one (edited quotes!):
C:\Progra~1\PostgreSQL\8.3\bin\psql -d Total -h localhost -p 5432 -U postgres
-c "SELECT ""TPImport""('%dateimport%')"
Note how I use singe and double quotes here. I guess this could work under windows. See here.
You made it hard for yourself when you chose to use mixed case identifiers in PostgreSQL - a folly which I never tire of warning against. Now you have to double quote the function name "TPImport" every time you use it. While perfectly legit, I would never do that. I use lower case letters for identifiers. Always. This way I never mix up lower / upper case and I never have to use double quotes.
The ultimate fix would be to recreate the function with a lower case name (just leave away the double quotes and it will be folded to lower case automatically). Then the function name will just work without any quoting.
Read the basics about identifiers here.
Also, consider upgrading to a more recent version of PostgreSQL 8.3 is a bit rusty by now.
psql supports textual replacement variables. Within psql they can be set using \set and used using :varname.
\set xyz 'abcdef'
select :'xyz';
?column?
----------
abcdef
These variables can be set using command line arguments also:
psql -v xyz=value
The only problem is that these textual replacements always need some fiddling with quoting as shown by the first \set and select.
After creating the function in Postgres, you must create a .bat file in the bin directory of your Postgres version, for example C:\Program Files\PostgreSQL\9.3\bin. Here you write:
#echo off
cd C:\Program Files\PostgreSQL\9.3\bin
psql -p 5432 -h localhost -d myDataBase -U postgres -c "select * from myFunction()"