psql variable substitution for postgres script with functions - postgresql

Question

I have a Postgres script that I have that I intend to run against multiple databases. The sample SQL has functions and tables like
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom:=ST_SetSRID(geom, :CRS) ;
RETURN NEW;
END
$$;
CREATE TABLE admin (
gid integer NOT NULL,
geom geometry(Polygon,:CRS)
);
I have put a variable inside so that I can substitute it on runtime as
psql -d db -p 5432 -U username -h localhost -f test.sql --variable=CRS=3857
Why does the variable only get properly substituted in table definitions and not function definition

Answer 1

https://www.postgresql.org/docs/current/static/app-psql.html#APP-PSQL-INTERPOLATION
Therefore, a construction such as ':foo' doesn't work to produce a
quoted literal from a variable's value (and it would be unsafe if it
did work, since it wouldn't correctly handle quotes embedded in the
value).
a function definition is put between quotes - in your case $$ instead of ', so it fails. You can change ST_SetSRID(geom, :CRS) to ST_SetSRID(geom, $$:CRS) to see it start being handled again (of course ruining the function body)
As a workaround you can try using bash variable instead:
vao#vao-VirtualBox:~$ export CRS=33
vao#vao-VirtualBox:~$ psql t << EOF
> CREATE OR REPLACE FUNCTION point() RETURNS trigger
> LANGUAGE plpgsql
> AS \$\$
> BEGIN
> NEW.the_geom:=ST_SetSRID(geom, $CRS) ;
> RETURN NEW;
> END
> \$\$;
> \sf point()
> EOF
CREATE FUNCTION
CREATE OR REPLACE FUNCTION public.point()
RETURNS trigger
LANGUAGE plpgsql
AS $function$
BEGIN
NEW.the_geom:=ST_SetSRID(geom, 33) ;
RETURN NEW;
END
$function$
Of course it can be scripted to a file that you can call as bash with argument

Answer 2

Shell substitution, as Vao Tsun suggests, might work fine for small psql scripts, but it's still not a general solution. For instance, shells are not aware of psql and pgSQL syntaxes and may substitute sub-strings that are not meant to be substituted in the psql script:
-- If CA and AU are undefined environment variable (likely),
-- the following SQL, after shell substitution, will result in:
-- ... VALUES ('Australian Dollar', ''), (Canadian Dollar, '');
INSERT (name, symbol) INTO currencies VALUES
('Australian Dollar', '$AU'),
('Canadian Dollar', '$CA');
This will become unmanageable with large scripts without additional measures. A better approach is to use string concatenation. Something like this would be ideal:
-- This won't work; syntax error
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom := ST_SetSRID(geom, $$ || :'CRS' || $$);
RETURN NEW;
END
$$;
Unfortunately, pgSQL syntax expects literal strings as function bodies, not arbitrary string expressions. Fortunately, however, PostgreSQL implements as part of its SQL syntax implicit concatenation between single-quoted strings separated by only whitespaces and at least one line break:
-- This gets parsed as "SELECT 'mystring';"
SELECT 'my'
'string';
The catch though is that it only works with single-quoted strings, not dollar-quoted strings. The following will work fine, but it's an eye-sore:
-- This will work, but it's ugly
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS
-- Mind the trailing space below
'BEGIN '
'NEW.the_geom := ST_SetSRID(geom, $$'
:'CRS'
'$$);'
'RETURN NEW;'
'END';
This is perhaps a more eye-pleasing variant:
-- This will work
-- (assign appropriate ownership and permissions if CRS contains sentitive data)
CREATE FUNCTION _get_crs() RETURNS text LANGUAGE sql AS 'SELECT $$'
:'crs'
'$$';
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom := ST_SetSRID(geom, _get_crs());
RETURN NEW;
END
$$;
And here's another variant:
-- This will also work
-- (may expose sensitive data in the catalog)
SET "app.crs" = :'crs';
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom := ST_SetSRID(geom, current_setting('app.crs'));
RETURN NEW;
END
$$;

Answer 3

The function body is a sting literal. psql has hardly any string modification functionality. However, PostgreSQL has plenty of it. With psql variables and \gset (since PostgreSQL 9.3) you can utilize this to first build your command and then execute it:
--for demo purpose set CRS directly. The OP did pass it from outside to psql
\set CRS 1234
--here we wrap the whole CREATE FUNCTION call in a literal separated by $_$
--this improves the readabilty and covers the inner $$. The result will be
--stored in psql variable func. This is workaround for a multi-line \set
SELECT $_$
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom:=ST_SetSRID(geom, :CRS) ;
RETURN NEW;
END
$$;
$_$ AS func \gset
--replace the substring ':CRS' with content of variable CRS
--and store the result in func again
SELECT replace(:'func', ':CRS', :'CRS') AS func \gset
--we can now have a look to the resulting DDL
\echo :func
--and we can execute it:
:func
--clean up psql variables
\unset func
\unset CRS
with the help of \gexec (since PostgreSQL 9.6) we can also combine the last two steps:
--replace the substring ':CRS' with content of variable CRS
--and execute the result
SELECT replace(:'func', ':CRS', :'CRS') \gexec
or even do all in one command:
SELECT replace($_$
CREATE FUNCTION point() RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
NEW.the_geom:=ST_SetSRID(geom, :CRS) ;
RETURN NEW;
END
$$;
$_$, ':CRS', :'CRS') \gexec