Heroku PSQL pass dynamic parameter to SQL file - postgresql

I am trying to execute a .sql file in Heroku PSQL and want to pass dynamic parameter values in the .sql file.
Below is the script which I am using
heroku pg:psql --app application_name <./somepath/file_to_execute.sql --param1="'$File_name'" --param2="'$Tag_id'" --param3="'$job_name'" --param4="$id"
The sql file contains insert script:
INSERT INTO version_table (col1, col2, col3, col4)
VALUES (:param1,:param2,:param3,:param4);
I get below error message from Heroku:
Error: Unexpected arguments: --param2='1.1.1', --param3='test-name', --param4=12
How to execute this sql file with dynamic value in Heroku PSQL
I also tried below query:
heroku pg:psql --app application_name <./somepath/file_to_execute.sql --v param1="'$File_name'" --v param2="'$Tag_id'" --v param3="'$job_name'" --v param4="$id"
Got below error message:
Error: Unexpected arguments: param1='file_name.sql', --v, param2='1.1.1', --v, param3='test-name', --v, param4=12

You can get the URL to the database with the following heroku command:
$ heroku pg:credentials:url --app application_name
This will print something like:
Connection information for default credential.
Connection info string:
"dbname=xyz host=something.compute.amazonaws.com port=1234 user=foobar password=p422w0rd sslmode=require"
Connection URL:
postgres://foobar:p422w0rd#something.compute.amazonaws.com:1234/xyz
The URL (last line) can be used directly with psql. With grep we can get that line and pass it to psql:
$ psql $(heroku pg:credentials:url --app application_name | grep 'postgres://') \
< ./somepath/file_to_execute.sql \
-v param1="$File_name" \
-v param2="$Tag_id" \
-v param3="$job_name" \
-v param4="$id"
Note that rather than putting single quotes in the parameter on the command line, you should access the parameter in the SQL as :'param1'.

Related

psql faild to restor file from network drive

i have a dump file on drive z (network drive)
im opening the psql from PgAdmin4
this is the command that im writeing:
psql -U postgres -d postgres -f Z:\DB_BU\md_20220729.sql
and this is the error that im getting:
Invalid command \DB_BU. Try \? for help.
when im doing this:
psql -U postgres -d postgres -f i\ Z:\DB_BU\md_20220729.sql
Invalid command \DB_BU. Try ? for help.
and when im doing this:
psql -U postgres -d postgres -f "Z:\DB_BU\md_20220729.sql"
im not getting any error but also its not restoring the file. how can i restor the file?
You're trying to call psql from within psql or PGAdmin. Since
psql is a standalone program, not an SQL command you can run in PGAdmin SQL window or psql's own, internal meta-command you're getting the error
Invalid command \DB_BU. Try \? for help
indicating that there was an attempt to interpret your entire command as a SQL query or an internal command and that this attempt failed.
You can open "psql tool" from within PGAdmin but your command won't work there either because it's trying to call psql itself, with some command-line options, which you cannot do when you're already inside an interactive psql session. The command
psql -U postgres -d postgres -f Z:\DB_BU\md_20220729.sql
can be used outside psql and PGAdmin, in your terminal, like zsh on Mac, sh/bash on Linux, cmd or PowerShell on Windows, where psql is installed and visible, along with your network path.
If you're able to open the psql tool window in PGAdmin, you can instead try and use an internal psql \i meta-command which is basically the same thing as the -f command-line option, but meant for use inside the psql session:
\i "Z:\DB_BU\md_20220729.sql"

Copying local postgresql to AWS RDS

I have a Postgesql database that I want to copy/replicate from my local machine to an AWS RDS instance.
I've created a dump as follows:
pg_dump -v -U postgres mydb > mydb.dump
I then tried to copy to my RDS as follows:
psql -f mydb.dump -h XXXXXXXXXXXX.us-east-1.rds.amazonaws.com -p 5432 -U postgres -d mydb
However, I get the following error:
psql:mydb.dump:1: error: invalid command \
psql:mydb.dump:6: ERROR: syntax error at or near "ÿ_"
LINE 1: ÿ_-"\o""edrp\nou"tnsctme e
^
I then tried to rule out any issues with RDS by copying the archive to another local database as follows:
pg_restore -U postgres -d mydbcopy mydb.dump
I received an error:
pg_restore: error: input file does not appear to be a valid archive
I also tried the preceding as follows:
psql -f my.dump -U postgres -d mydbcopy
And got the same error as before:
psql:mydb.dump:1: error: invalid command \
psql:mydb.dump:6: ERROR: syntax error at or near "ÿ_"
LINE 1: ÿ_-"\o""edrp\nou"tnsctme e
^
I am using Windows Powershell and Postgresql 13.2.
Am I missing something ?
It seems to be an issue with PSQL tools in view that I'm getting the error even when trying to copy locally.
I managed to solve this problem by using
pg_dump -f dump.sql --inserts mydb
It turns out using the redirect ">" means PowerShell does the file writing and it uses UTF16 which doesn't play nice with psql. The -f flag makes pg_dump do the writing itself.

How to restore a postgres database from a custom-format dump?

I'm having an issue getting a database restore using pg_restore to work. I've got Postgres 12.2 installed on my Mac running High Sierra (10.13.6). It was installed with Brew.
I created a simple database "foo" with table "foo" to test this out:
Nates-MacBook-Pro:k8s natereed$ psql -d foo
psql (12.2, server 12.1)
Type "help" for help.
foo=# SELECT table_schema,table_name
FROM information_schema.tables
WHERE table_schema='public' ORDER BY table_schema,table_name;
table_schema | table_name
--------------+------------
public | foo
(1 row)
Generate dump:
pg_dump -Fc foo -f foo.backup
When I try to restore, nothing happens. pg_restore just hangs:
pg_restore -h localhost -p 5432 -U postgres -v -Fc -f foo.backup
I've tried various permutations of this command, but every time it just hangs. In Activity Monitor, I see the process but it is not using any CPU.
Online help says:
pg_restore restores a PostgreSQL database from an archive created by pg_dump.
Usage:
pg_restore [OPTION]... [FILE]
General options:
-d, --dbname=NAME connect to database name
-f, --file=FILENAME output file name
-F, --format=c|d|t backup file format (should be automatic)
-l, --list print summarized TOC of the archive
-v, --verbose verbose mode
-V, --version output version information, then exit
-?, --help show this help, then exit
For pg_restore -f is parameter for the output file: it is not the input file. Remove -f in your example:
pg_restore -h localhost -p 5432 -U postgres -v -Fc foo.backup

executing query from docker to psql db shows psql: warning: extra command-line argument "" ignored

Hi I am in my docker console and trying to run shell script to connect to PSQL DB and execute queries.
#!/bin/sh
query = "select * from default$default."LightZone"";
#psql -U postgres -p postgres psql -e "$query";
This is my tablecreation.sh file and when I do ./tablecreation.sh
./tablecreation.sh: 5: ./tablecreation.sh: query: not found
psql: warning: extra command-line argument "" ignored
Password for user prisma:
psql (10.3 (Debian 10.3-1.pgdg90+1))
Type "help" for help.
The above message comes . But the same query is running from the psql command prompt.
can someone help me if i am missing something . I want to connect to postgres->prisma db >default$default schema > LightZone table and run some queries.
Escape the double quotes with \
query = "select * from default$default.\"LightZone\"";

Postgres restore from .dump file: Invalid byte sequence for encoding "UTF8"

I've downloaded the latest copy of my heroku database with
$ curl -o latest.dump `heroku pg:backups public-url --app myapp`
And this gives a file with the following type on my local machine:
$ file db/latest.dump
db/latest.dump: PostgreSQL custom database dump - v1.12-0
I'm trying to import this into my local Postgres DB (my_db) using psql. However, I'm getting lots of errors:
$ psql my_db < db/latest.dump
...
psql:db/latest.dump:119425: invalid command \?A~??ܣb?{#?+????LS??
psql:db/latest.dump:119426: invalid command \D%!ѡ/F??g????A???dC????|2?M?$?8GTJ??c?E?;??֛Sh??S?[NP?f?2?+H?W????k
... [thousands of lines]
psql:db/latest.dump:261719: ERROR: invalid byte sequence for encoding "UTF8": 0xba
The command psql -f db/latest.dump my_db fails the same way.
What needs to be done in order to import this file locally into a new database with the same schema, etc?
I was able to use pg_restore to solve the problem:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d my_db db/latest.dump