Default schema for native SQL queries (spring-boot + hibernate + postgresql + postgis) - postgresql

I am introducing spring to the existing application (hibernate has already been there) and encountered a problem with native SQL queries.
A sample query:
SELECT ST_MAKEPOINT(cast(longitude as float), cast(latitude as float)) FROM
OUR_TABLE;
OUR_TABLE is in OUR_SCHEMA.
When we connect to the db to OUR_SCHEMA:
spring.datasource.url: jdbc:postgresql://host:port/db_name?currentSchema=OUR_SCHEMA
the query fails because function ST_MAKEPOINT is not found - the function is located in schema: PUBLIC.
When we connect to the db without specifying the schema, ST_MAKEPOINT is found and runs correctly, though schema name needs to be added to the table name in the query.
As we are talking about thousands of such queries and all the tables are located in OUR_SCHEMA, is there a chance to anyhow specify the default schema, so still functions from PUBLIC schema were visible?
So far, I have tried the following springboot properties - with no success:
spring.jpa.properties.hibernate.default_schema: OUR_SCHEMA
spring.datasource.tomcat.initSQL: ALTER SESSION SET CURRENT_SCHEMA=OUR_SCHEMA
spring.datasource.initSQL: ALTER SESSION SET CURRENT_SCHEMA=OUR_SCHEMA
Also, it worked before switching to springboot config - specifying hibernate.default-schema = OUR_SCHEMA in persistence.xml was enough.
Stack:
spring-boot: 2.0.6
hibernate: 5.3.1.Final
postgresql: 42.2.5
postgis: 2.2.1

You're probably looking for the PostgreSQL search_path variable, which controls which schemas are checked when trying to resolve database object names. The path accepts several schema names, which are checked in order. So you can use the following
SET search_path=our_schema,public;
This will make PostgreSQL look for your tables (and functions!) first in our_schema, and then in public. Your JDBC driver may or may not support multiple schemas in its current_schema parameter.
Another option is to install the PostGIS extension (which provides the make_point() function) in the our_schema schema:
CREATE EXTENSION postgis SCHEMA our_schema;
This way you only have to have one schema in your search path.

JDBC param currentSchema explicitly allows specifying several schemas separating them by commas:
jdbc:postgresql://postgres-cert:5432/db?currentSchema=my,public&connectTimeout=4&ApplicationName=my-app
From https://jdbc.postgresql.org/documentation/head/connect.html
currentSchema = String
Specify the schema (or several schema separated by commas) to be set in the search-path. This schema will be used to resolve unqualified object names used in statements over this connection.
Note you probably need Postgres 9.6 or better for currentSchema support.
PS Probably better solution is to set search_path per user:
ALTER USER myuser SET search_path TO mydb,pg_catalog;

if you use hibernate.default_schema, then for native queries, you need to provide the {h-schema} placeholder, something like that
SELECT ST_MAKEPOINT(cast(longitude as float), cast(latitude as float)) FROM {h-schema}OUR_TABLE;

Related

Using postgresql extensions with Spring Boot multi tenant application

I am working on a multi tenant Spring Boot application using Postgresql and different schemas for tenants. Everything works fine until we need to use Postgresql's extensions, then we got errors about missing types.
ERROR: function text2ltree(character varying) does not exist
Hint: No function matches the given name and argument types. You might need to add explicit type casts.
Position: 266
The extension is installed using Liquibase and I have updated the changesets to perform the following:
CREATE SCHEMA myschema;
CREATE EXTENSION IF NOT EXISTS ltree WITH SCHEMA myschema;
ALTER DATABASE mydb SET search_path TO "$user", myschema
Then I can control where the extension is installed and updating the search path should avoid to specify the schema all the time. I have run the Liquibase migration and check Postgresql configuration for search_path and it works. I can run queries (using Squirrel) without the need to include the schema prefix for the ltree extension types and functions.
It does not work as expected as I still got the same error message. If I explicitly prefix the ltree types and functions with myschema, I got an error indicating that the operator is not found.
ERROR: operator does not exist: myschema.ltree <# myschema.ltree
Hint: No operator matches the given name and argument types. You might need to add explicit type casts.
Position: 263
For multi tenancy, we implement a custom org.hibernate.engine.jdbc.connections.spi.MultiTenantConnectionProvider that sets the schema in the MultitenantConnectionProvider.getAnyConnection() method.
I want to simplify the setup so I want to avoid to include always the extension's schema, besides the search_path configuration, I can not found anything related. Is there a way to implement the described setup to prevent prefixing with the extension schema in all the code?
If I install the extensions in the pg_catalog schema, then everything works as expected. See this answer.
I want to prevent to install the extensions in the pg_catalog schema so, a closer look to the PGConnection class reveals that the method to set the default schemas updates the search_path value to the schema given.
StringBuilder sb = new StringBuilder();
sb.append("SET SESSION search_path TO '");
Utils.escapeLiteral(sb, schema, getStandardConformingStrings());
sb.append("'");
stmt.executeUpdate(sb.toString());
It is escaping the schema argument, therefore if I try to use multiple schemas separated by commas, the whole expression is escaped.
The solution has been to create my own setSchema method in the MultiTenantConnectionProvider class. It does the same as the code above but adding my extensions' schema to the search_path value.

How to create a synonym for a table in PostgreSQL

I am migrating this Oracle command to PostgreSQL:
CREATE SYNONYM &user..emp FOR &schema..emp;
Please suggest to me how I can migrate the above command.
PostgreSQL does not support SYNOSYM or ALIAS. Synonym is a non SQL 2003 feature implemented by Microsoft SQL 2005 (I think). While it does provide an interesting abstraction, but due to lack of relational integrity, it can be considered a risk.
That is, you can create a synonym, advertise it to you programmers, the code is written around it, including stored procedures, then one day the backend of this synonym (or link or pointer) is changed/deleted/etc leading to a run time error. I don't even think a prepare would catch that.
It is the same trap as the symbolic links in unix and null pointers in C/C++.
But rather create a DB view and it is already updateable as long as it contains one table select only.
CREATE VIEW schema_name.synonym_name AS SELECT * FROM schema_name.table_name;
You don't need synonyms.
There are two approaches:
using the schema search path:
ALTER DATABASE xyz SET search_path = schema1, schema2, ...;
Put the schema that holds the table on the search_path of the database (or user), then it can be used without schema qualification.
using a view:
CREATE VIEW dest_schema.tab AS SELECT * FROM source_schema.tab;
The first approach is good if you have a lot of synonyms for objects in the same schema.

Accessing table without using schema name

I am new with DB2.
I am not able to get data from a table without using the schema name. If I use a schema name with table name, I am able to fetch data.
Example:
SELECT * FROM TABLE_NAME;
It is giving me error, while
SELECT FROM SCHEMA_NAME.TABLE_NAME;
is fetching a result.
What do I have to set up not to always have to use the schema name?
By default, your username is used as the schema name for unqualified object names. You can see the current schema with, e.g. VALUES CURRENT SCHEMA. You can change the current schema for you current session with SET SCHEMA new_schema_name, or via e.g. a JDBC connection parameter. Most query tools also have a place to specify/change the current schema.
See the manual page for SET SCHEMA https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11.1.0/com.ibm.db2.luw.sql.ref.doc/doc/r0001016.html
The full rules for the qualification of unqualified objects is here https://www.ibm.com/support/knowledgecenter/en/SSEPGG_11.1.0/com.ibm.db2.luw.sql.ref.doc/doc/r0000720.html#r0000720__unq-alias
E.g.
Unqualified alias, index, package, sequence, table, trigger, and view names are implicitly qualified by the default schema.
However, you can create a public alias for a table, module or sequence if you wish to be able to reference it regardless of your CURRENT SCHEMA value.
https://www.ibm.com/support/knowledgecenter/SSEPGG_11.5.0/com.ibm.db2.luw.sql.ref.doc/doc/r0000910.html
(P.S. all the above assumes you are using Db2 LUW)
Try using SET SCHEMA to set the default schema to be used in the session:
SET SCHEMA SCHEMA_NAME;
SELECT * FROM TABLE_NAME;
When using DBeaver - right click on connection > Connection Settings > Initialization, and select your default DB and default Schema:
After that, open your SQL Script and select Active DB:

SET/RESET command in ALTER DATABASE is not supported

Encountered this issue when trying to modify the search_path to my new Redshift db.
Presently, I've migrated the contents of my MySQL db into a redshift cluster via AWS' Data Migration Service. The data was imported into a schema lets call my_schema. When I try to execute queries against the cluster it requires me to prefix table names with the schema name
i.e.
select * from my_schema.my_table
I wanted to change the setup so that I can reference the table directly without needing the prefix. After a bit of looking around I found out that this was possible by modifying the search_path attribute.
First I tried doing this by running
set search_path = "$user", my_schema;
This appeared to work but then I realized that this was simply setting my_schema as the default schema in the context of the current session, I wanted it set on a database level. I found several sources saying that the way to do this was to use the alter command like so...
alter database my_db set search_path = "$user", public, my_schema
However, running this command results in the following error which somehow shows up in 0 google results:
SET/RESET commmand in ALTER DATABASE is not supported
I'm pretty baffled by how the above error hasn't ever had a post made about it but I'm also pretty interested in figuring out how to resolve my initial issue of setting a global default schema for my redshift cluster.
ALTER DATABASE SET is not supported in Redshift. However you can SET/RESET configuration parameters at USER level using the ALTER USER SET SEARCH_PATH TO <SCHEMA1>,<SCHMEA2>;
Please check: http://docs.aws.amazon.com/redshift/latest/dg/r_ALTER_USER.html
http://docs.aws.amazon.com/redshift/latest/dg/r_search_path.html
When you set the search_path to <SCHEMA1>,<SCHMEA2> in db1 for a user it is not for just current session, it will be set for all future sessions.

Joining Results from Two Separate Databases

Is it possible to JOIN rows from two separate postgres databases?
I am working with system with couple databases in one server and sometimes I really need such a feature.
According to http://wiki.postgresql.org/wiki/FAQ
There is no way to query a database other than the current one.
Because PostgreSQL loads database-specific system catalogs, it is
uncertain how a cross-database query should even behave.
contrib/dblink allows cross-database queries using function calls. Of
course, a client can also make simultaneous connections to different
databases and merge the results on the client side.
EDIT: 3 years later (march 2014), this FAQ entry has been revised and is more helpful:
How do I perform queries using multiple databases?
There is no way to directly query a database other than the current
one. Because PostgreSQL loads database-specific system catalogs, it is
uncertain how a cross-database query should even behave.
The SQL/MED support in PostgreSQL allows a "foreign data wrapper" to
be created, linking tables in a remote database to the local database.
The remote database might be another database on the same PostgreSQL
instance, or a database half way around the world, it doesn't matter.
postgres_fdw is built-in to PostgreSQL 9.3 and includes read/write
support; a read-only version for 9.2 can be compiled and installed as
a contrib module.
contrib/dblink allows cross-database queries using function calls and
is available for much older PostgreSQL versions. Unlike postgres_fdw
it can't "push down" conditions to the remote server, so it'll often
land up fetching a lot more data than you need.
Of course, a client can also make simultaneous connections to
different databases and merge the results on the client side.
Forget about dblink!
Say hello to Postgres_FDW:
To prepare for remote access using postgres_fdw:
Install the postgres_fdw extension using CREATE EXTENSION.
Create a foreign server object, using CREATE SERVER, to represent each remote database you want to connect to. Specify connection
information, except user, and password, as options of the server
object.
Create a user mapping, using CREATE USER MAPPING, for each database user you want to allow to access each foreign server. Specify
the remote user name and password to use as user and password options
of the user mapping.
Create a foreign table, using CREATE FOREIGN TABLE or IMPORT FOREIGN SCHEMA, for each remote table you want to access. The columns
of the foreign table must match the referenced remote table. You can,
however, use table and/or column names different from the remote
table's, if you specify the correct remote names as options of the
foreign table object.
Now you need only SELECT from a foreign table to access the data
stored in its underlying remote table.
It's really useful even on large data.
Yes, it is possible to do this using dblink albeit with significant performance considerations.
The following example will require the current SQL user to have permissions on both databases. If db2 is not located on the same cluster, then you will need to replace dbname=db2 with the full connection string defined in the dblink documentation.
SELECT *
FROM table1 tb1
LEFT JOIN (
SELECT *
FROM dblink('dbname=db2','SELECT id, code FROM table2')
AS tb2(id int, code text);
) AS tb2 ON tb2.column = tb1.column;
If table2 is very large, you could have performance issues because the sub-query loads up the entire table2 before performing the join.
No you can't. You could use dblink to connect from one database to another database, but that won't help if you're looking for JOIN's.
You can't use different SCHEMA's within a single database to store all you data?
Just a few steps and You can reach the goal:
follow this reference step by step
WE HAVE BEEN CONNECTED TO DB2 WITH TABLE TBL2 AND COLUMN COL2
ALSO THERE IS DB1 WITH TBL1 AND COLUMN COL1
*** connecting to second db ie db2
Now just **copy paste the 1-7 processes** (make sure u use correct username and password and ofcourse db name)
1.**CREATE EXTENSION dblink;**
2.**SELECT pg_namespace.nspname, pg_proc.proname
FROM pg_proc, pg_namespace
WHERE pg_proc.pronamespace=pg_namespace.oid
AND pg_proc.proname LIKE '%dblink%';**
3.**SELECT dblink_connect('host=localhost user=postgres password=postgres dbname=db1');**
4.**CREATE FOREIGN DATA WRAPPER postgres VALIDATOR postgresql_fdw_validator;**
5.**CREATE SERVER postgres2 FOREIGN DATA WRAPPER postgres OPTIONS (hostaddr '127.0.0.1', dbname 'db1');**
6.**CREATE USER MAPPING FOR postgres SERVER postgres2 OPTIONS (user 'postgres', password 'postgres');**
7.**SELECT dblink_connect('postgres2');**
---Now, you can SELECT the data of Database_One from Database_Two and even join both db results:
**SELECT * FROM public.dblink
('postgres2','SELECT col1,um_name FROM public.tbl1 ')
AS DATA(um_userid INTEGER),tbl2 where DATA.col1=tbl2.col2;**
You can also Check this :[How to join two tables of different databases together in postgresql [\[working finely in version 9.4\]][1]
You need to use dblink...as araqnid mentioned above, something like this works fine:
select ST.Table_Name, ST.Column_Name, DV.Table_Name, DV.Column_Name, *
from information_schema.Columns ST
full outer join dblink('dbname=otherdatabase','select Table_Name,
Column_Name from information_schema.Columns') DV(Table_Name text,
Column_Name text)
on ST.Table_Name = DV.Table_name
and ST.Column_Name = DV.Column_Name
where ST.Column_Name is null or DV.Column_Name is NULL
You have use dblink extension of postgresql.
Reference take from this Article:
DbLink extension of PostgreSQL which is used to connect one database to another database.
Install DbLink extension.
CREATE EXTENSION dblink;
Verify DbLink:
SELECT pg_namespace.nspname, pg_proc.proname
FROM pg_proc, pg_namespace
WHERE pg_proc.pronamespace=pg_namespace.oid
AND pg_proc.proname LIKE '%dblink%';
I have already prepared full demonstration on this. Please visit my post to learn step by step for executing cross database query in Postgresql.
Cannot be done? Of course we can, without special extensions. In our case, we had to compare two tables from different database servers, e.g. ACC and PROD, hence an even harder case than from most answers. Especially because ACC and PROD are deliberately on different servers to create a barrier, so you will not easily gain enough rights to perform a GRANT USAGE ON FOREIGN SERVER.
The obvious solution is to export both tables, and import both in the same database, e.g. DEV, or your own local db, under appropriate names, e.g. table1_acc and table1_prod, or schemas like acc and prod. Then, you may JOIN those with no special problems.