I am using the PostreSQL pg_try_advisory_lock function to coordinate a cluster of micro-services, sharing the same database, to ensure that only one instance performs an ad hoc task.
Does this function contend with any internal locks PostresSQL may be using. I am concerned that using a database wide locking mechanism such as this may impact the system resources used within the database engine.
PostgreSQL does not use advisory locks internally, so there is no danger of blocking anything in the system by using them.
Related
As per standard Postgres documentation
As with the plain file-system-backup technique, this method can only support restoration of an entire database cluster, not a subset.
From this, I understood that it is not possible to setup PITR for individual databases in a cluster (a.k.a. a database instance holding multiple databases).
If my understanding is incorrect, probably the next part of the question is not relevant, but if not, here it is:
I still do not get the problem in setting this up theoretically as each database is generating its own WAL archive.
The problem here is: I am in need of setting up multiple Postgres clusters and somehow I have only 2 RHEL 7.6 machines to handle this. I am trying to reduce the number of clusters on these 2 machines to only 2. I am planning to create multiple database rather than multiple instances to handle customer applications. But that means that I have to sacrifice PITS, as PITR only can be performed on the instance/cluster level and not on the database level (as per the official documentation).
Could someone please help clarifying my misunderstanding.
You are correct, you can only do PITR on a PostgreSQL database cluster, not on an individual database.
There is only one WAL stream for the complete database cluster; WAL is not split up per database.
Don't hesitate to run several PostgreSQL clusters on a single machine if that is advantageous for you.
There is little overhead in running a second database cluster. The biggest resource that is hogged by a cluster is shared buffers, but you want that to be only a fraction of the available RAM anyway. Most of the memory should be left to the filesystem cache that is shared by all PostgreSQL clusters.
Is it possible for firebirdSQL to run 2 servers sharing 1 database file (FDB)/ repository?
No. The server needs exclusive access to the database files. In the case of the Classic architecture version, multiple fb_inet_server processes access the same files, but locks are managed through the fb_lock_mgr process.
Databases on NFS or SMB/CIFS shares are disallowed unless one explicitly disables this protection. firebird.conf includes strong warnings against doing this unless you really know what you are doing.
If you mean if two servers on different host can share the same database, then no.
Firebird either requires exclusive access to a database (SuperServer), or coordinates access to the database by different processes on the same host through a lock file (SuperClassic and ClassicServer).
In both cases the server requires certain locking and write visibility guarantees, and most networked filesystems don't provide those (or don't provide the locking semantics Firebird needs it).
If you really, really want to, you can by changing a setting in firebird.conf, but that is a road to corrupt database or other consistency problems. And therefor not something you should want to do.
Every SQL server will not allow such configuration. If you want to split load, maybe you need to look at Multi Tier architecture. Using this architecture, you can split your SQL query load to many computers.
Anyone can kindly tell me how to process distributed transaction within postgresql, which is also called "XA"? Is there any resources about it? Great thanks for any answer.
It looks like you are a bit confused. Generally database systems support two notions of distributed transaction types:
Native distributed transactions and
XA transactions.
Native distributed transactions are generally between different servers of the same RDBMS. Postgres also supports this with the dblink_exec command. Generally the connection to the other server is created by a so called database link. Postgres is a bit more clumsy to use then some other commercial grade RDBMS. You first need to install an extension to be able to use database links. However the postgres rdbms is managing the transaction.
XA transactions on the other hand are managed by the external transaction manager (TM) and each of the participating database has the role of a XA resource, which enlists with the transaction manager. The RDBMS can no longer decide itself when to commit a transaction. This is the task of the XA transaction manager. He uses a 2PC protocol to make sure the changes are applied or rolled back in a consistent manner across the databases.
On some OSes like windows a transaction manager is part of the operating system on others not. Generally java is shipped with a transaction manager and the corresponding data source needs to be configured to use XA.
I would like all queries from my Spring-Hibernate application executed in a read-only transaction to be dispatched to a PostgreSQL slave and all read-write transaction queries to a master.
While using annotation driven transactions in Spring, if the transaction is defined as read-only, the PostreSQL driver allows only select queries to be executed, which is obvious, however there is no mention of how the driver would behave in a master slave configuration. For e.g., the MySQL driver has a replication connection class which automatically dispatches read-only transaction queries to the slave.
One solution would be to use multiple Hibernate session factories and use the one pointing to the slave for selects and the other for updates, but that would be too much manual handling. How should I be designing this?
This is a surprisingly complex question and the answer is not simply easy. You need to keep in mind that you have to have this dispatched in such a way that the layer which does the dispatching knows whether a transaction is likely to be read-only or not.
The cleanest solution is probably to implement the dispatching in your middleware. This has the advantage of being a functional dispatch-- we know what we are trying to do so let's dispatch there... Of course functions can create a bit of a knowledge gap in what is read-only and what writes....
The second option is that one could probably dispatch with something like PGPool or the like. I would expect you would probably want to avoid server-side prepared queries in these cases because the more knowledge you provide the intermediate layer, the fewer problems you will have.
I have an application that can not afford to lose data, so Postgresql is my choice for database (ACID)
However, speed and query advantages of MongoDB are very attractive, but based on what I've read so far, MongoDB can report a successful write which may not have gone to disk, so I can't make it my mission critical db (I'll also need transactions)
I've seen references to people using mysql and MongoDB together, one for the transactions and the other for queries. Please not that I'm not talking about keeping some data in one DB and the rest in another. I want to use Postgresql as a gateway to data entry, and MongoDB for reads.
Are there any resources that offer an architecture/guide for Postgresql + MongoDB usage in this way? I can remember seeing this topic in Postgresql conference agenda, but I could not find the link.
I don't think you'll get much speed using MongoDB just as a cache. It's strengths are replication and horizontal scalability. On one computer you'd make Mongo and Postgres compete for memory, IO bandwidth and processor time.
As you can not afford to loose transactions you'll be better with Postgres only. Its has efficient caching, sophisticated query planner, prepared queries and wide indexing support cause that read-only queries will be very fast - really comparable to MongoDB on a single computer.
Postgres can even scale horizontally now using asynchronous, or, from version 9.1, synchronous replication.
One way to achieve this would be to set up a master-slave replication with the PostgreSQL database as master, and the MongoDB database as slave. You would then do all reads from MongoDB, and all writes to PostgreSQL.
This post discusses such a setup using a tool called Bucardo:
http://blog.endpoint.com/2011/06/mongodb-replication-from-postgres-using.html
You may also be able to do it with Tungsten Replicator, although it seems designed to be used with MySQL:
http://code.google.com/p/tungsten-replicator/wiki/TRCHeterogeneousReplication
I can remember seeing this topic in Postgresql conference agenda, but I could not find the
link.
Maybe, you are talking about this: https://www.postgresqlconference.org/content/hybrid-applications-using-mongodb-and-postgres
Depending how important transactions are to you, one option is to use MongoDb driver's safe mode and drop Postgresql.
http://www.mongodb.org/display/DOCS/getLastError+Command
How can you expect transactional consistency from Postgres but trust MongoDB for reads? How would you support rollbacks in this scenario? How do you detect when they've gotten out of sync?
I think you're better off going with memcache and implementing a higher level object cache. Alternatively, you could consider a replication slave for reads. If you have performance needs beyond what a dedicated read slave can provide, consider denormalizing your tables on your slave system.
Make sure that any of this is actually needed. For thin tables with PK lookups most modern database engines like Postgres or InnoDB are going to generally keep up with NoSQL solutions. Don't fall into the ROFLSCALE trap
http://www.youtube.com/watch?v=b2F-DItXtZs
I think you can run a mongo replica set.. Let say 3 Slave and 1 Master.. Then in your app you should run all write transactions on Postgresql and then on Mongo ReplicaSet.. After that you can query read operations on Mongo Replica set..
But Synchronizing will be a problem, you should work on it..
you may find some replacement for mongo in here or here that is safer and fast as well.
but I advise to simplify your solution instead of making a complicated design.
Visual Guide to NoSQL Systems
lucky
In mongodb we can specify writeConcern property to specify that it should write to journal/ instances and then send confirmation/ acknowledgement and i think even mongodb has teh concept of transactions. Not sure why we need postgres behind it.