I'm pretty frustrated - Power BI requires me to enter a value in the field "server" in order to connect to a postgres db on my machine, but I cannot determine any way to retrieve that value from Postgres. Everything I get by googling seems to involve hostname/localhost, etc., not the server name. It seems mind-boggling to me that, if I have a server on which my database is running, Postgres won't provide me with any easy way to retrieve its name (and it must have one). Getting Power BI to connect to Postgres seems way, way too complicated.
Related
My team and I are currently experiencing an issue where we can't connect to Cloud SQL's Postgres instance(s) from anything other than the psql cli tool. We get a too many connections for database "postgres" error (in PGAdmin, DBeaver, and our node typeorm/pg backend). It initially happened on our (only) Postgres database instance. After restarting, stopping and starting again, increasing machine CPU/memory proved to do nothing, I deleted the database instance entirely and created a new one from scratch.
However, after a few hours the problem came back. I know that we're not actually having too many connections as I am able to query pg_stat_activity from psql command line and see the following:
Only one of those (postgres username) connections is ours.
My coworker also can't connect at all - not even from psql cli.
If it matters, we are using PostgreSQL 13, europe-west2 (London), single zone availability, db-g1-small instance with 1.7GB memory, 10GB HDD, and we have public IP enabled and the correct IP addresses whitelisted.
I'd really appreciate if anyone has any insights into what's causing this.
EDIT: I further increased the instance size (to no longer be a shared core), and I managed to successfully connect my backend to it. However my psql cli no longer works - it appears that only the first client to connect is allowed to connect after a restart (even if it disconnects, other clients can't connect...).
From the error message, it is clear that the database "postgres" has a custom connection limit (set, for example, by ALTER DATABASE postgres CONNECTION LIMIT 1). And apparently, it is quite small. Why is everyone try to connect to that database anyway? Usually 'postgres' database is reserved for maintenance operations, and you should create other databases for daily use.
You can see the setting with:
select datconnlimit from pg_database where datname='postgres';
I don't know if the low setting is something you did, or maybe Google does it on its own for their cloud offering.
#jjanes had the right idea/mention.
I created another database within the Cloud SQL instance that wasn't named postgres and then it was fine.
It wasn't anything to do with maximum connection settings (as this was within Google Cloud SQL) or not closing connections (as TypeORM/pg does this already).
First of all i do not know if is possible what i want to achive, will describe below:
I have access of a remote PostgreSQL that holds the data i need (let's say Remote PostgreSQL 1)
have just credentials to read from database
What i want to achive is to create a localt PostgreSQL on my machine (let's say Local PostgreSQL 2)
Want to copy and check for missing data from Remote PostgreSQL 1 to Local PostgreSQL 2 in real time or at list to copy at the end of the day data
The scenario will work perfect with replication but the issue is that Remote PostgreSQL 1 is not owned by me and can not be used as real time DB, because of this trying to find a solution to get all the data from Remote PostgreSQL 1 to Local PostgreSQL 2.
Could be the following scenarios:
first time setup to downlaod all the database from Remote PostgreSQL 1 to Local PostgreSQL 2
after first time setup to check what data came new inside and add them in Local PostgreSQL 2
Would be great if this could be done on OS level on UBUNTU. My application is written in python 3 i could do scripts to do all this job but i speak of 100 millions of raws per table huge amount of data. Think will be problems to get everything from database and start to check everything what is missing and not.
Any ideas would be great.
If the owner of Remote Database 1 won't cooperate with you other than to give you read only access to the tables, then you don't have any efficient options. If the remote owner does or can be convinced to keep insertion/modification timestamp columns in all the tables (although then deletions would be a problem), or an in-database "audit" log for all the tables, you could use those. I think you have an organizational/political problem rather a programming problem.
I've recently joined a company with a mixed set of databases that include a Redshift cluster and some SQL databases. I'd like to use a single IDE to access both for analytical reporting, so I don't have to switch between tools. I'm currently using workbench, which works, but it's not clicking with me.
I do like Azure Data Studio, but it's SQL Server and Postgres only. Given the similarities between Redshift and Postgres, I thought I'd see if I could connect using the Postgres driver.
I've installed the Postgres extension and can "connect" to the database. However when I try to explore the database using the tree view, I get the error message 'Cannot Expand Node'. When I run a simple query that works in workbench, e.g.
Select * from [server].[database].[table]
I get the following Error message:
Started executing query at Line 1
cursors can only be used within the transaction that created them.
Total execution time: 00:00:00.019
I know I'm trying to do something that shouldn't be done. And if I can't, I can't. But has anyone here managed to get a redshift connection going in Azure Data Studio?
FWIW, I've come across a GitHub Repository that may be a Redshift driver for data studio - but this looks like a clone of the Postgres driver, with no activity since march (not even renaming the 'Postgres' titles to Redshift)... and therefore I'm dubious.
I am trying to setup Postgres 9.3 on an Ubuntu 14 server, and I'm feeling pretty demoralised at this point. I've previously used MySQL, so I'm happy with general database concepts, as well as client/server models etc.
I start with two users - 'root' and 'sam' (me). As 'sam' I install postgresql using apt-get. This also creates a third user called 'postgres'.
Fine.
I'm told that to use postgres you must be logged in as the postgres user, so I switch to that account. Apparently this comes with a postgres admin role (I think I'm fine with the concept of roles per se), and apparently all roles have an associated database of the same name (?). So now I have a Linux account called postgres, a role called postgres, and a database called postgres? This all seems needless but I'm assuming it's useful for reasons I don't know about (not meant sarcastically - this is usually the case when things seem overly complicated at first).
So, to create a database, do I login to the server as postgres, start postgres by typing 'psql' (which doesn't ask for a password - why doesn't the postgres account have a password?) and proceed from there? Or should I create a new role? Does that role need its own Linux user? Should the role be the same name as the database I want to create?
I appreciate this is a bit of a jumble, but my confusion is such that I'm not even sure I understand the fundamentals here. I miss MySQL.
I've been mainly using the DigitalOcean tutorial for this - which are usually very good - but it didn't really make any of this clear. I also read the postgres docs (specifically the installation and users/roles sections) which didn't help, and the google results for this are even less helpful.
This is my last hope before I go back to the safety blanket of MySQL. Any suggestions for making this click?
OS usernames and Postgres DB usernames are not related; they live in seperate universes.
one exception: if you connect from the same machine via the unix-domain socket, and you don't explicitely specify a username, your OS name is assumed to be your DB-username, too. (which in most cases is not correct)
second exception: the "postgres" username is used both as an OS-username (owner of the files, uid of the running processes) and as the DBMS superuser.
Note: "root" is a bad name for a DB-user.
Currently, I have an application that uses Firebird in embedded mode to connect to a relatively simple database stored as a file on my hard drive. I want to switch to using PostgreSQL to do the same thing (Yes, I know it's overkill). I know that PostgreSQL cannot operate in embedded mode and that is fine - I can leave the server process running and that's OK with me.
I'm trying to figure out a connection string that will achieve this, but have been unsuccessful. I've tried variations on the following:
jdbc:postgresql:C:\myDB.fdb
jdbc:postgresql://C:\myDB.fdb
jdbc:postgresql://localhost:[port]/C:\myDB.fdb
but nothing seems to work. PostgreSQL's directions don't include an example for this case. Is this even possible?
You can trick it. If you are running PostGRESQL on a UNIXlike system, then you should be able to create a RAMDISK and use that for the database storage. Here's a pretty good step by step guide for RAMdisks on Linux.
In general though, I would suggest using SQLITE for an SQL db in RAM type of application.
Postgres databases are not a single file. There will be one file for each table and each index in the data directory, inside a directory for the database. All files will be named with the object ID (OID) of db / table / index.
The JDBC urls point to the database name, not any specific file:
jdbc:postgresql:foodb (localhost is implied)
If by "disk that behaves like memory", you mean that the db only exists for the lifetime of your program, there's no reason why you can't create a db at program start and drop it at program exit. Note that this is just DDL to create the DB, not creating the data dir via the init-db program. You could connect to the default 'postgres' db, create your db then connect to it.
Firebird 2.1 onwards supports global temporary tables, which only exist for the duration of the database connection.
Syntax goes something like CREATE GLOBAL TEMPORARY TABLE ... ON COMMIT PRESERVE ROWS