In MS SQL Server, I create my scripts to use customizable variables:
DECLARE #somevariable int
SELECT #somevariable = -1
INSERT INTO foo VALUES ( #somevariable )
I'll then change the value of #somevariable at runtime, depending on the value that I want in the particular situation. Since it's at the top of the script it's easy to see and remember.
How do I do the same with the PostgreSQL client psql?
Postgres variables are created through the \set command, for example ...
\set myvariable value
... and can then be substituted, for example, as ...
SELECT * FROM :myvariable.table1;
... or ...
SELECT * FROM table1 WHERE :myvariable IS NULL;
edit: As of psql 9.1, variables can be expanded in quotes as in:
\set myvariable value
SELECT * FROM table1 WHERE column1 = :'myvariable';
In older versions of the psql client:
... If you want to use the variable as the value in a conditional string query, such as ...
SELECT * FROM table1 WHERE column1 = ':myvariable';
... then you need to include the quotes in the variable itself as the above will not work. Instead define your variable as such ...
\set myvariable 'value'
However, if, like me, you ran into a situation in which you wanted to make a string from an existing variable, I found the trick to be this ...
\set quoted_myvariable '\'' :myvariable '\''
Now you have both a quoted and unquoted variable of the same string! And you can do something like this ....
INSERT INTO :myvariable.table1 SELECT * FROM table2 WHERE column1 = :quoted_myvariable;
One final word on PSQL variables:
They don't expand if you enclose them in single quotes in the SQL statement.
Thus this doesn't work:
SELECT * FROM foo WHERE bar = ':myvariable'
To expand to a string literal in a SQL statement, you have to include the quotes in the variable set. However, the variable value already has to be enclosed in quotes, which means that you need a second set of quotes, and the inner set has to be escaped. Thus you need:
\set myvariable '\'somestring\''
SELECT * FROM foo WHERE bar = :myvariable
EDIT: starting with PostgreSQL 9.1, you may write instead:
\set myvariable somestring
SELECT * FROM foo WHERE bar = :'myvariable'
You can try to use a WITH clause.
WITH vars AS (SELECT 42 AS answer, 3.14 AS appr_pi)
SELECT t.*, vars.answer, t.radius*vars.appr_pi
FROM table AS t, vars;
Specifically for psql, you can pass psql variables from the command line too; you can pass them with -v. Here's a usage example:
$ psql -v filepath=/path/to/my/directory/mydatafile.data regress
regress=> SELECT :'filepath';
?column?
---------------------------------------
/path/to/my/directory/mydatafile.data
(1 row)
Note that the colon is unquoted, then the variable name its self is quoted. Odd syntax, I know. This only works in psql; it won't work in (say) PgAdmin-III.
This substitution happens during input processing in psql, so you can't (say) define a function that uses :'filepath' and expect the value of :'filepath' to change from session to session. It'll be substituted once, when the function is defined, and then will be a constant after that. It's useful for scripting but not runtime use.
FWIW, the real problem was that I had included a semicolon at the end of my \set command:
\set owner_password 'thepassword';
The semicolon was interpreted as an actual character in the variable:
\echo :owner_password
thepassword;
So when I tried to use it:
CREATE ROLE myrole LOGIN UNENCRYPTED PASSWORD :owner_password NOINHERIT CREATEDB CREATEROLE VALID UNTIL 'infinity';
...I got this:
CREATE ROLE myrole LOGIN UNENCRYPTED PASSWORD thepassword; NOINHERIT CREATEDB CREATEROLE VALID UNTIL 'infinity';
That not only failed to set the quotes around the literal, but split the command into 2 parts (the second of which was invalid as it started with "NOINHERIT").
The moral of this story: PostgreSQL "variables" are really macros used in text expansion, not true values. I'm sure that comes in handy, but it's tricky at first.
postgres (since version 9.0) allows anonymous blocks in any of the supported server-side scripting languages
DO '
DECLARE somevariable int = -1;
BEGIN
INSERT INTO foo VALUES ( somevariable );
END
' ;
http://www.postgresql.org/docs/current/static/sql-do.html
As everything is inside a string, external string variables being substituted in will need to be escaped and quoted twice. Using dollar quoting instead will not give full protection against SQL injection.
You need to use one of the procedural languages such as PL/pgSQL not the SQL proc language.
In PL/pgSQL you can use vars right in SQL statements.
For single quotes you can use the quote literal function.
I solved it with a temp table.
CREATE TEMP TABLE temp_session_variables (
"sessionSalt" TEXT
);
INSERT INTO temp_session_variables ("sessionSalt") VALUES (current_timestamp || RANDOM()::TEXT);
This way, I had a "variable" I could use over multiple queries, that is unique for the session. I needed it to generate unique "usernames" while still not having collisions if importing users with the same user name.
Another approach is to (ab)use the PostgreSQL GUC mechanism to create variables. See this prior answer for details and examples.
You declare the GUC in postgresql.conf, then change its value at runtime with SET commands and get its value with current_setting(...).
I don't recommend this for general use, but it could be useful in narrow cases like the one mentioned in the linked question, where the poster wanted a way to provide the application-level username to triggers and functions.
I've found this question and the answers extremely useful, but also confusing. I had lots of trouble getting quoted variables to work, so here is the way I got it working:
\set deployment_user username -- username
\set deployment_pass '\'string_password\''
ALTER USER :deployment_user WITH PASSWORD :deployment_pass;
This way you can define the variable in one statement. When you use it, single quotes will be embedded into the variable.
NOTE! When I put a comment after the quoted variable it got sucked in as part of the variable when I tried some of the methods in other answers. That was really screwing me up for a while. With this method comments appear to be treated as you'd expect.
I really miss that feature. Only way to achieve something similar is to use functions.
I have used it in two ways:
perl functions that use $_SHARED variable
store your variables in table
Perl version:
CREATE FUNCTION var(name text, val text) RETURNS void AS $$
$_SHARED{$_[0]} = $_[1];
$$ LANGUAGE plperl;
CREATE FUNCTION var(name text) RETURNS text AS $$
return $_SHARED{$_[0]};
$$ LANGUAGE plperl;
Table version:
CREATE TABLE var (
sess bigint NOT NULL,
key varchar NOT NULL,
val varchar,
CONSTRAINT var_pkey PRIMARY KEY (sess, key)
);
CREATE FUNCTION var(key varchar, val anyelement) RETURNS void AS $$
DELETE FROM var WHERE sess = pg_backend_pid() AND key = $1;
INSERT INTO var (sess, key, val) VALUES (sessid(), $1, $2::varchar);
$$ LANGUAGE 'sql';
CREATE FUNCTION var(varname varchar) RETURNS varchar AS $$
SELECT val FROM var WHERE sess = pg_backend_pid() AND key = $1;
$$ LANGUAGE 'sql';
Notes:
plperlu is faster than perl
pg_backend_pid is not best session identification, consider using pid combined with backend_start from pg_stat_activity
this table version is also bad because you have to clear this is up occasionally (and not delete currently working session variables)
Variables in psql suck. If you want to declare an integer, you have to enter the integer, then do a carriage return, then end the statement in a semicolon. Observe:
Let's say I want to declare an integer variable my_var and insert it into a table test:
Example table test:
thedatabase=# \d test;
Table "public.test"
Column | Type | Modifiers
--------+---------+---------------------------------------------------
id | integer | not null default nextval('test_id_seq'::regclass)
Indexes:
"test_pkey" PRIMARY KEY, btree (id)
Clearly, nothing in this table yet:
thedatabase=# select * from test;
id
----
(0 rows)
We declare a variable. Notice how the semicolon is on the next line!
thedatabase=# \set my_var 999
thedatabase=# ;
Now we can insert. We have to use this weird ":''" looking syntax:
thedatabase=# insert into test(id) values (:'my_var');
INSERT 0 1
It worked!
thedatabase=# select * from test;
id
-----
999
(1 row)
Explanation:
So... what happens if we don't have the semicolon on the next line? The variable? Have a look:
We declare my_var without the new line.
thedatabase=# \set my_var 999;
Let's select my_var.
thedatabase=# select :'my_var';
?column?
----------
999;
(1 row)
WTF is that? It's not an integer, it's a string 999;!
thedatabase=# select 999;
?column?
----------
999
(1 row)
I've posted a new solution for this on another thread.
It uses a table to store variables, and can be updated at any time. A static immutable getter function is dynamically created (by another function), triggered by update to your table. You get nice table storage, plus the blazing fast speeds of an immutable getter.
I may have an odd request. I'm not finding any help via Google.
I am using the DbVisualizer Pro 10.0.15 gui tool connected to a PostgreSQL db.
I need to create a csv file from a database table. I select the records I need in a query then export the results to a .csv file. I can do that easy.
select note from notes;
highlight all results records >> right-click >> select export >> choose csv
Some of the records have both single and/or double-quotes in the content.
The person receiving this file needs to upload the csv file into another system. They are stating that these single and double-quotes in the content will not work in their upload. I've been asked to escape these quotes. They want to keep them in the content, but have them appear in the field with the backslash escape character, i.e: it is John's ball would show in the csv file as: it is John\'s ball. The same for dbl-quotes.
I could probably do this with a search-and-replace function in a text editor after creating the csv file, but I'd like to think this can be done via sql.
I've tried playing with the regexp_replace() function.
select regexp_replace(note, '"', '\"') as notes from notes works on the dbl-quotes, but I'm not having any luck on the single quotes.
Help? Is there a way to do this?
You can escape double quotes by doing:
postgres=# SELECT REGEXP_REPLACE('this "is" a string', '"', '\"', 'g');
regexp_replace
----------------------
this \"is\" a string
(1 row)
For single quotes, the approach is similar, but you have to escape them using another single quote. So instead of having something like /', it should be ''. The query is:
postgres=# SELECT REGEXP_REPLACE('this ''is'' a string', '''', '\''', 'g');
regexp_replace
----------------------
this \'is\' a string
(1 row)
Note the 'g' flag in the end, this forces it to replace all occurrences and not just the first one found.
You can also replace both single and double quotes in a single statement, although they are replaced with the same string (\" in this case).
postgres=# SELECT REGEXP_REPLACE('this "is" a ''normal'' string', '["'']', '\"', 'g');
regexp_replace
---------------------------------
this \"is\" a \"normal\" string
(1 row)
Long story...
I am trying to geenrate a crosstab query dynamically and run it as a psql script..
To achieve this, I want the last line of the sql to generated and appended to the top portion of the sql.
The last line of the sql is like this.... "as final_result(symbol character varying,"431" numeric,"432" numeric,"433" numeric);"
Of which, the "431", "432" etc are to be generated dynamically as these are the pivot columns and they change from time to time...
So I wrote a query to output the text as follows....
psql -c "select distinct '"'||runweek||'" numeric ,' from calendar where runweek between current_runweek()-2 and current_runweek() order by 1;" -U USER -d DBNAME > /tmp/gengen.lst
While the sql provides the output, when I run it as a script, because of the special characters (', "", ) it fails.
How should I get it working? My plan was then loop through the "lst" file and build the pivot string, and append that to the top portion of the sql and execute the script... (New to postgres, does not know , dynamic sql generation and execution etc.. but I am comfortable with UNIX scripting..)
If I could somehow get the output as
("431" numeric, "432" numeric....etc) in a single step, if there is a recommendation to achieve this, it will be greatly appreciated.....
Since you're using double quotes around the argument, double quotes inside the argument must be escaped with a backslash:
psql -c "select distinct '\"'||runweek||'\" numeric ,' from calendar where runweek between current_runweek()-2 and current_runweek() order by 1;"
Heredoc can also be used instead of -c. It accepts multi-line formatting so that makes the whole thing more readable.
(psql [arguments] <<EOF
select distinct '"'||runweek||'" numeric ,'
from calendar
where runweek between current_runweek()-2 and current_runweek()
order by 1;
EOF
) > output
By using quote_ident which is specifically meant to produce a quoted identifier from a text value, you don't even need to add the double quotes. The query could be like:
select string_agg( quote_ident(runweek::text), ',' order by runweek)
from calendar
where runweek between current_runweek()-2 and current_runweek();
which also solves the problem that your original query has a stray ',' at the end, whereas this form does not.
I need to do an UPDATE script using the Replace() function of Postgres but I don't know the exact string that I have to replace and I'd like to know if there is some way that I can do this similary the LIKEoperator, using Wildcards.
My problem is that I got a table that contains some scripts and at the end of each one there is a tag <signature> like this:
'SELECT SCRIPT WHATEVER.... < signature>782798e2a92c72b270t920b< signature>'
What I need to do is:
UPDATE table SET script = REPLACE(script,'<signature>%<signature>','<signature>1234ABCDEF567890<signature>')
Whatever the signature is, I need to replace with a new one defined by me. I know using the '%' doesn't work, it was just to ilustrate the effect i want to perform. Is there any way to do this in Postgres 9.5?
with expr
as
(select 'Hello <signature>thisismysig</signature>'::text as
full_text, '<signature>'::text as open,
'</signature>'::text as close
)
select
substring(full_text from
position(open in full_text)+char_length(open)
for
position(close in full_text)- char_length(open)-position(open in full_text)
)
note: with part added for ease of understanding (hopefully).
Use POSIX regex to do the same thing as other answer (but shorter)
select
substring('a bunch of other stuff <signature>mysig</signature>'
from '<signature>(.*?)</signature>')
Today i was executing DML statements from sql file in sqlplus, oracle 10.2, i found something weird which is actually a question "Enter value for hamburger:". See attached
and ending of question "Enter value for XXX" kept change with intervals. i mean after few seconds i found statement "Enter value for mcdonald"....
I want to know is it normal with sqlplus ????
i have tried to execute DML statement by giving location of sql file in sqlplus.
SQL>#/tmp/myDir/dmls.sql;
An Exemplary statement from sql file is
Insert into MYSCHEMA.MYTABLE (ID,DIX_DIR_ID,DIX_AGL,DIX_XML_INLINE) values (23202,1100080319000620471,'directory','<values>
<dir_class_title>Person</dir_class_title>
</values>
');
You most certainly have an ampersand in your statement.
When SQL*Plus encounters an ampersand, it tries to replace it with what it is defined to. If it is undefined, it prompts for a value:
This is a statment without ampersand:
SQL> select 'foo' from dual;
'FO
---
foo
Now, another statement with an ampersand. The value for foo is not (yet) defined, so it prompts for one:
SQL> select '&foo' from dual;
Enter value for foo: bla
old 1: select '&foo' from dual
new 1: select 'bla' from dual
'BL
---
bla
Now, foo is defined to be bar:
SQL> define foo=bar
The select statement (with &foo) returns bar:
SQL> select '&foo' from dual;
old 1: select '&foo' from dual
new 1: select 'bar' from dual
'BA
---
bar
See also DEFINE and Defining Substitution Variables
If you want to turn that behavior off, do as
SQL> set define off
Edit: changed set define= to set define off as per Adam Musch's suggestion.