I already posted a similar question to the Power Bi forum, but probably it's too short-sighted to search the solution on Power Bi side.
We have a postgres schema and a postgres user that has the following rights: granted connect to db and schema, revoked create on schema, granted select privilege on all tables in schema. Nothing more. The user works in the db client, can only read from the tables in the schema, cannot create, insert, read, update, delete anything.
When we import the schema in Power Bi with our Read Only User, the relations between the tables is not displayed. The postgres documentation writes the following:
"The view table_constraints contains all constraints belonging to tables that the current user owns or has some privilege other than SELECT on."
This would mean, our Read Only User must have granted insert, delete or update permissions to make Power Bi able to display the table relations?! Our postgres db shall work as a datawarehouse and give customers the possibility to connect from their local Power Bi Desktop or their companies Power Bi instance, so we don't have too much influence on Power Bi side at all.
Is there a permission we can give the Read Only User that keeps him read only but allows to resolve the table relations in Power Bi?
Thanks and kind regards!
We tried giving the user a insert privilege on all tables, that made Power Bi resolve the table relations but is conflicting with our Read Only behave.
We read documentation and tried to find out, if there are other possibilities than the above.
That is really unfortunate, but the SQL standard explicitly requires that (e.g. in ISO/IEC 9075-11:2003, 5.58):
5.58 TABLE_CONSTRAINTS view
Function
Identify the table constraints defined on tables in this catalog that are accessible to a given user or role.
Definition
CREATE VIEW TABLE_CONSTRAINTS AS
SELECT CONSTRAINT_CATALOG, CONSTRAINT_SCHEMA, CONSTRAINT_NAME,
TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME,
CONSTRAINT_TYPE, IS_DEFERRABLE, INITIALLY_DEFERRED
FROM DEFINITION_SCHEMA.TABLE_CONSTRAINTS
WHERE ( TABLE_CATALOG, TABLE_SCHEMA, TABLE_NAME ) IN
( SELECT TP.TABLE_CATALOG, TP.TABLE_SCHEMA, TP.TABLE_NAME
FROM DEFINITION_SCHEMA.TABLE_PRIVILEGES AS TP
WHERE TP.PRIVILEGE_TYPE <> 'SELECT'
[...]
So really this is a shortcoming of Power BI, if it is supposed to work with a read-only user. On the other hand, if they want to be database agnostic, they have little choice but to use the information_schema.
But interesting as that may be, you are looking for a solution or a workaround. If you look at the view definition of information_schema.table_constraints, you will see that visibility is determined by this condition:
pg_has_role(r.relowner, 'USAGE'::text) OR
has_table_privilege(r.oid, 'INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER'::text) OR
has_any_column_privilege(r.oid, 'INSERT, UPDATE, REFERENCES'::text)
I'd say that the most promising privilege here is REFERENCES, which allows a user to create a foreign key pointing to the table. If the user doesn't have the CREATE privilege on any schema, they cannot create a table that references the table, so the privilege is effectively useless. Still, if you grant the user REFERENCES on all tables, the constraints will become visible. It is hacky, but it should be good enough.
Related
I'm stumbling into an accidental DBA role at my current company and SO has been a huge help in getting me set up with useful queries.
I'm currently trying to create some views to assist with UAR (user access review) to satisfy compliance requirements and assist users with troubleshooting access. To do so I'm trying to create a view that lists user names and their privileges on all tables in all schemas in the database. I started with a view against information_schema.table_privileges but noticed it was incomplete. If it's relevant, this is in an AWS RDS Postgres implementation.
To dive in, I performed \z schema_name.table_name and I got the below output. All looks well: I see the owning role analytics_owner with all privileges and six group roles with varying levels of privilege. (All employees and applications accessing the database are granted membership to one of these six roles.)
This is also shown by a query against pg_class:
But if I query against information_schema.table_privileges, there are no results. (In fact, if I drop the where relname = '[table]' clause there are only two tables with detail in the relevant schema in the table_privileges output — but there are about a hundred tables in the actual schema.)
The privilege detail from the \z command just isn't showing in information_schema.table_privileges. I'd prefer to write a view off of table_privileges rather than pg_class because I wouldn't have to decode and separate the aggregated ACL details. However, it looks like there is something very specific to the table_privileges view given that only a very small subset of the actual tables are in there.
Is there a query that produces the same detail in the same format as contained in information_schema.table_privileges but takes its data from pg_class? Alternatively, what is different about table_privileges that makes it not show the full detail, and is there something I can do about the granted privileges to make them visible there for ease in review?
Per Adrian Klaver's comment, the information_schema depends on the role you are logged in on when the query was run.
I'm going to guess that the answer is "no" based on the below error message (and this Google result), but is there anyway to perform a cross-database query using PostgreSQL?
databaseA=# select * from databaseB.public.someTableName;
ERROR: cross-database references are not implemented:
"databaseB.public.someTableName"
I'm working with some data that is partitioned across two databases although data is really shared between the two (userid columns in one database come from the users table in the other database). I have no idea why these are two separate databases instead of schema, but c'est la vie...
Note: As the original asker implied, if you are setting up two databases on the same machine you probably want to make two schemas instead - in that case you don't need anything special to query across them.
postgres_fdw
Use postgres_fdw (foreign data wrapper) to connect to tables in any Postgres database - local or remote.
Note that there are foreign data wrappers for other popular data sources. At this time, only postgres_fdw and file_fdw are part of the official Postgres distribution.
For Postgres versions before 9.3
Versions this old are no longer supported, but if you need to do this in a pre-2013 Postgres installation, there is a function called dblink.
I've never used it, but it is maintained and distributed with the rest of PostgreSQL. If you're using the version of PostgreSQL that came with your Linux distro, you might need to install a package called postgresql-contrib.
dblink() -- executes a query in a remote database
dblink executes a query (usually a SELECT, but it can be any SQL
statement that returns rows) in a remote database.
When two text arguments are given, the first one is first looked up as
a persistent connection's name; if found, the command is executed on
that connection. If not found, the first argument is treated as a
connection info string as for dblink_connect, and the indicated
connection is made just for the duration of this command.
one of the good example:
SELECT *
FROM table1 tb1
LEFT JOIN (
SELECT *
FROM dblink('dbname=db2','SELECT id, code FROM table2')
AS tb2(id int, code text);
) AS tb2 ON tb2.column = tb1.column;
Note: I am giving this information for future reference. Reference
I have run into this before an came to the same conclusion about cross database queries as you. What I ended up doing was using schemas to divide the table space that way I could keep the tables grouped but still query them all.
Just to add a bit more information.
There is no way to query a database other than the current one. Because PostgreSQL loads database-specific system catalogs, it is uncertain how a cross-database query should even behave.
contrib/dblink allows cross-database queries using function calls. Of course, a client can also make simultaneous connections to different databases and merge the results on the client side.
PostgreSQL FAQ
Yes, you can by using DBlink (postgresql only) and DBI-Link (allows foreign cross database queriers) and TDS_LInk which allows queries to be run against MS SQL server.
I have used DB-Link and TDS-link before with great success.
I have checked and tried to create a foreign key relationships between 2 tables in 2 different databases using both dblink and postgres_fdw but with no result.
Having read the other peoples feedback on this, for example here and here and in some other sources it looks like there is no way to do that currently:
The dblink and postgres_fdw indeed enable one to connect to and query tables in other databases, which is not possible with the standard Postgres, but they do not allow to establish foreign key relationships between tables in different databases.
If performance is important and most queries are read-only, I would suggest to replicate data over to another database. While this seems like unneeded duplication of data, it might help if indexes are required.
This can be done with simple on insert triggers which in turn call dblink to update another copy. There are also full-blown replication options (like Slony) but that's off-topic.
see https://www.cybertec-postgresql.com/en/joining-data-from-multiple-postgres-databases/ [published 2017]
These days you also have the option to use https://prestodb.io/
You can run SQL on that PrestoDB node and it will distribute the SQL query as required. It can connect to the same node twice for different databases, or it might be connecting to different nodes on different hosts.
It does not support:
DELETE
ALTER TABLE
CREATE TABLE (CREATE TABLE AS is supported)
GRANT
REVOKE
SHOW GRANTS
SHOW ROLES
SHOW ROLE GRANTS
So you should only use it for SELECT and JOIN needs. Connect directly to each database for the above needs. (It looks like you can also INSERT or UPDATE which is nice)
Client applications connect to PrestoDB primarily using JDBC, but other types of connection are possible including a Tableu compatible web API
This is an open source tool governed by the Linux Foundation and Presto Foundation.
The founding members of the Presto Foundation are: Facebook, Uber,
Twitter, and Alibaba.
The current members are: Facebook, Uber, Twitter, Alibaba, Alluxio,
Ahana, Upsolver, and Intel.
In case someone needs a more involved example on how to do cross-database queries, here's an example that cleans up the databasechangeloglock table on every database that has it:
CREATE EXTENSION IF NOT EXISTS dblink;
DO
$$
DECLARE database_name TEXT;
DECLARE conn_template TEXT;
DECLARE conn_string TEXT;
DECLARE table_exists Boolean;
BEGIN
conn_template = 'user=myuser password=mypass dbname=';
FOR database_name IN
SELECT datname FROM pg_database
WHERE datistemplate = false
LOOP
conn_string = conn_template || database_name;
table_exists = (select table_exists_ from dblink(conn_string, '(select Count(*) > 0 from information_schema.tables where table_name = ''databasechangeloglock'')') as (table_exists_ Boolean));
IF table_exists THEN
perform dblink_exec(conn_string, 'delete from databasechangeloglock');
END IF;
END LOOP;
END
$$
I'm in trouble with grant in postgresql (version 9.3).
I'm trying to restrict a ROLE 'client_1'. I want it to be able to do only select for one table. But there is inheritance between tables.
Here is my table structure:
CREATE TABLE public.table_a (...);
CREATE TABLE table_a_partitions.child_1 (...) INHERITS (public.table_a);
CREATE TABLE table_a_partitions.child_2 (...) INHERITS (public.table_a);
GRANT SELECT ON table_a_child_1 TO client_1;
It's okay when I do a select on child_2, there is an error, but if I do a SELECT * FROM table_a; for example it also reads the forbidden table child_2. I would my client access only child_1 (and some other in the future) results when he does SELECT * FROM table_a;.
Is there a simple way to solve this problem ?
Thank you
You would need to use a VIEW in PostgreSQL 9.3 to solve this problem. If you upgrade to 9.5, however, you could use row-level security.
As a note as to why, the grant check only occurs on the level of the initial relation queried. This means if you query a view, you need access to the view's contents, but the view owner (NOT YOU) needs access to the underlying relations. This allows a view to be useful for information hiding. Similarly with inheritance, this structure allows you to forbid rows to be inserted or queried directly from partitions of a table, but to allow different queries via the parent table. So this is a consequence of design priorities, not a bug.
Before row-level security, you would basically create a view and fold in user privilege criteria into the view (with partitioning/inheritance this is also a good idea for other reasons since your insert/update/delete triggers can return exactly what the db would do even though it cannot on a table).
As for row-level security, PostgreSQL 9.5 does allow you to specify row-level policies (conditions that get appended to insert/select/update/delete queries) and that provides something a little more manageable in some cases than the view approach.
I'm looking for a way to grant some right (say SELECT) on all tables of one user to some other user.
Something like that:
GRANT SELECT ON Username1.* to Username2
Important condition is that I need not only granting the right to all existing tables, but I also want, that all tables that will be created by Username1 by default should be granted with correspondent right to Username2
What I've done is creation a an external script for choosing all tables created by Username1:
SELECT table_name FROM systable WHERE creator IN (SELECT user_id from sysuser WHERE username='Username1'))
and generating GRANT statements for every of them.
But than I need to execute that script as soon as new table is created.
It is not really elegant to have that granting logic out of the database.
I would prefer having kind of trigger within the database, but Sybase IQ does not support triggers.
Anybody have idea how to solve the problem in elegant manner?
Is it possible to JOIN rows from two separate postgres databases?
I am working with system with couple databases in one server and sometimes I really need such a feature.
According to http://wiki.postgresql.org/wiki/FAQ
There is no way to query a database other than the current one.
Because PostgreSQL loads database-specific system catalogs, it is
uncertain how a cross-database query should even behave.
contrib/dblink allows cross-database queries using function calls. Of
course, a client can also make simultaneous connections to different
databases and merge the results on the client side.
EDIT: 3 years later (march 2014), this FAQ entry has been revised and is more helpful:
How do I perform queries using multiple databases?
There is no way to directly query a database other than the current
one. Because PostgreSQL loads database-specific system catalogs, it is
uncertain how a cross-database query should even behave.
The SQL/MED support in PostgreSQL allows a "foreign data wrapper" to
be created, linking tables in a remote database to the local database.
The remote database might be another database on the same PostgreSQL
instance, or a database half way around the world, it doesn't matter.
postgres_fdw is built-in to PostgreSQL 9.3 and includes read/write
support; a read-only version for 9.2 can be compiled and installed as
a contrib module.
contrib/dblink allows cross-database queries using function calls and
is available for much older PostgreSQL versions. Unlike postgres_fdw
it can't "push down" conditions to the remote server, so it'll often
land up fetching a lot more data than you need.
Of course, a client can also make simultaneous connections to
different databases and merge the results on the client side.
Forget about dblink!
Say hello to Postgres_FDW:
To prepare for remote access using postgres_fdw:
Install the postgres_fdw extension using CREATE EXTENSION.
Create a foreign server object, using CREATE SERVER, to represent each remote database you want to connect to. Specify connection
information, except user, and password, as options of the server
object.
Create a user mapping, using CREATE USER MAPPING, for each database user you want to allow to access each foreign server. Specify
the remote user name and password to use as user and password options
of the user mapping.
Create a foreign table, using CREATE FOREIGN TABLE or IMPORT FOREIGN SCHEMA, for each remote table you want to access. The columns
of the foreign table must match the referenced remote table. You can,
however, use table and/or column names different from the remote
table's, if you specify the correct remote names as options of the
foreign table object.
Now you need only SELECT from a foreign table to access the data
stored in its underlying remote table.
It's really useful even on large data.
Yes, it is possible to do this using dblink albeit with significant performance considerations.
The following example will require the current SQL user to have permissions on both databases. If db2 is not located on the same cluster, then you will need to replace dbname=db2 with the full connection string defined in the dblink documentation.
SELECT *
FROM table1 tb1
LEFT JOIN (
SELECT *
FROM dblink('dbname=db2','SELECT id, code FROM table2')
AS tb2(id int, code text);
) AS tb2 ON tb2.column = tb1.column;
If table2 is very large, you could have performance issues because the sub-query loads up the entire table2 before performing the join.
No you can't. You could use dblink to connect from one database to another database, but that won't help if you're looking for JOIN's.
You can't use different SCHEMA's within a single database to store all you data?
Just a few steps and You can reach the goal:
follow this reference step by step
WE HAVE BEEN CONNECTED TO DB2 WITH TABLE TBL2 AND COLUMN COL2
ALSO THERE IS DB1 WITH TBL1 AND COLUMN COL1
*** connecting to second db ie db2
Now just **copy paste the 1-7 processes** (make sure u use correct username and password and ofcourse db name)
1.**CREATE EXTENSION dblink;**
2.**SELECT pg_namespace.nspname, pg_proc.proname
FROM pg_proc, pg_namespace
WHERE pg_proc.pronamespace=pg_namespace.oid
AND pg_proc.proname LIKE '%dblink%';**
3.**SELECT dblink_connect('host=localhost user=postgres password=postgres dbname=db1');**
4.**CREATE FOREIGN DATA WRAPPER postgres VALIDATOR postgresql_fdw_validator;**
5.**CREATE SERVER postgres2 FOREIGN DATA WRAPPER postgres OPTIONS (hostaddr '127.0.0.1', dbname 'db1');**
6.**CREATE USER MAPPING FOR postgres SERVER postgres2 OPTIONS (user 'postgres', password 'postgres');**
7.**SELECT dblink_connect('postgres2');**
---Now, you can SELECT the data of Database_One from Database_Two and even join both db results:
**SELECT * FROM public.dblink
('postgres2','SELECT col1,um_name FROM public.tbl1 ')
AS DATA(um_userid INTEGER),tbl2 where DATA.col1=tbl2.col2;**
You can also Check this :[How to join two tables of different databases together in postgresql [\[working finely in version 9.4\]][1]
You need to use dblink...as araqnid mentioned above, something like this works fine:
select ST.Table_Name, ST.Column_Name, DV.Table_Name, DV.Column_Name, *
from information_schema.Columns ST
full outer join dblink('dbname=otherdatabase','select Table_Name,
Column_Name from information_schema.Columns') DV(Table_Name text,
Column_Name text)
on ST.Table_Name = DV.Table_name
and ST.Column_Name = DV.Column_Name
where ST.Column_Name is null or DV.Column_Name is NULL
You have use dblink extension of postgresql.
Reference take from this Article:
DbLink extension of PostgreSQL which is used to connect one database to another database.
Install DbLink extension.
CREATE EXTENSION dblink;
Verify DbLink:
SELECT pg_namespace.nspname, pg_proc.proname
FROM pg_proc, pg_namespace
WHERE pg_proc.pronamespace=pg_namespace.oid
AND pg_proc.proname LIKE '%dblink%';
I have already prepared full demonstration on this. Please visit my post to learn step by step for executing cross database query in Postgresql.
Cannot be done? Of course we can, without special extensions. In our case, we had to compare two tables from different database servers, e.g. ACC and PROD, hence an even harder case than from most answers. Especially because ACC and PROD are deliberately on different servers to create a barrier, so you will not easily gain enough rights to perform a GRANT USAGE ON FOREIGN SERVER.
The obvious solution is to export both tables, and import both in the same database, e.g. DEV, or your own local db, under appropriate names, e.g. table1_acc and table1_prod, or schemas like acc and prod. Then, you may JOIN those with no special problems.