I would like to use asynchronous queries in PostgreSQL. I know that some asynchronous drivers exist in many programming languages ie. asyncg (Python), vertx-postgress (Java) and so on.
Do I need to configure PostgreSQL somehow to use asynchronous features? There is an "Asynchronous Behavior" section in postgesql.conf. Do I need to uncomment and edit these values to use PostgreSQL
optimally in an asynchronous way?
The parameters in the “asynchonous behavior” section of the documentation are largely unrelated.
You should instead study the Asynchronous Command Processing section. This is about the C library, but that's probably what's used by the libraries you mention under the hood.
Don't forget that there can only be one query at a time on a PostgreSQL database connection, regardless if processing is synchronous or not.
Related
I found discussion about the topic here https://jira.mongodb.org/browse/RUST-271
The latest answer:
Hello! We don't implement the bulk write API and have no plans to add
it. From dealing with it in other drivers, we've found that the
semantics tend to be non-obvious, especially with regards to error
handling, and it doesn't provide a whole lot of technical benefit
given that it still sends different write types in separate commands
to the server (i.e. the same way as if each of insert_many,
update_many, delete_many, etc. were called separately).
But there is something still not clear for me.
Why that answer mention *_many (update_many, insert_many etc) type operations if bulk write is set of *_one (update_one, insert_one etc)?
Ok, insert_many uses bulk write under the hood. But bulk write and update many are different. Because first one uses a set of update_one operations which have different filters and update bodies for each one. And update_many has one filter for many updates.
So, update_many cannot replace bulk write. Am I right? If so, then why MongoDB rust driver does not support bulk write? If I am not right can you please explain why? Maybe I do not understand some details.
It seems if I use update_one a lot of time I get bad performance compared to bulk write. Because in case of bulk write I will connect once to MongoDB compared to update_one where I need one connection per update_one.
This is related to javascript stored function in mongodb server. I know all the details about the working and use cases. I am doubtful about one line which is in the official documentation of MongoDB.
"Note : We do not recommend using server-side stored functions if possible."
Infact what I feel, after moving to V8 JavaScript engine ( improving concurrency issues for javascript queries ) and given the fact this may save us many network round trip time, why this is not recommended by 10gen?
This is not recommended due to the fact that the javascript function needs to take write lock for the duration of it's executing meaning you'll cause potential bottle necks in your write performance.
There are some disadvantages of stored procedures in general:
https://stackoverflow.com/questions/462978/when-should-you-use-stored-procedures
Yet I understand your point concerning the network roundtrips.
I have just moved to PostgreSQL after having worked with Oracle for a few years.
I have been looking into some performance issues with prepared statements in the application (Java, JDBC) with the PostgreSQL database.
Oracle caches prepared statements in its SGA - the pool of prepared statements is shared across database connections.
PostgreSQL documentation does not seem to indicate this. Here's the snippet from the documentation (https://www.postgresql.org/docs/current/static/sql-prepare.html) -
Prepared statements only last for the duration of the current database
session. When the session ends, the prepared statement is forgotten,
so it must be recreated before being used again. This also means that
a single prepared statement cannot be used by multiple simultaneous
database clients; however, each client can create their own prepared
statement to use.
I just want to make sure that I am understanding this right, because it seems so basic for a database to implement some sort of common pool of commonly executed prepared statements.
If PostgreSQL does not cache these that would mean every application that expects a lot of database transactions needs to develop some sort of prepared statement pool that can be re-used across connections.
If you have worked with PostgreSQL before, I would appreciate any insight into this.
Yes, your understanding is correct. Typically if you had a set of prepared queries that are that critical then you'd have the application call a custom function to set them up on connection.
There are three key reasons for this afaik:
There's a long todo list and they get done when a developer is interested/paid to tackle them. Presumably no-one has thought it worth funding yet or come up with an efficient way of doing it.
PostgreSQL runs in a much wider range of environments than Oracle. I would guess that 99% of installed systems wouldn't see much benefit from this. There are an awful lot of setups without high-transaction performance requirement, or for that matter a DBA to notice whether it's needed or not.
Planned queries don't always provide a win. There's been considerable work done on delaying planning/invalidating caches to provide as good a fit as possible to the actual data and query parameters.
I'd suspect the best place to add something like this would be in one of the connection pools (pgbouncer/pgpool) but last time I checked such a feature wasn't there.
HTH
Since MongoDB does not support transactions, is there any way to guarantee transaction?
What do you mean by "guarantee transaction"?
There are two conepts in MongoDB that are similar;
Atomic operations
Using safe mode / getlasterror ...
http://www.mongodb.org/display/DOCS/Last+Error+Commands
If you simply need to know if there was an error when you run an update for example you can use the getlasterror command, from the docs ...
getlasterror is primarily useful for
write operations (although it is set
after a command or query too). Write
operations by default do not have a
return code: this saves the client
from waiting for client/server
turnarounds during write operations.
One can always call getLastError if
one wants a return code.
If you're writing data to MongoDB on
multiple connections, then it can
sometimes be important to call
getlasterror on one connection to be
certain that the data has been
committed to the database. For
instance, if you're writing to
connection # 1 and want those writes to
be reflected in reads from connection #2, you can assure this by calling getlasterror after writing to
connection # 1.
Alternatively, you can use atomic operations for cases where you need to increment a value for example (like an upvote, etc.) more about that here:
http://www.mongodb.org/display/DOCS/Atomic+Operations
As a side note, MySQL's default storage engine doesn't have transaction either! :)
http://dev.mysql.com/doc/refman/5.1/en/myisam-storage-engine.html
MongoDB only supports atomic operations. There is no ways implement transaction in the sense of ACID on top of MongoDB. Such a transaction support must be implemented in the core. But you will never see full transaction support due to the CARP theorem. You can not have speed, durability and consistency at the same time.
I think ti's one of the things you choose to forego when you choose a NoSQL solution.
If transactions are required, perhaps NoSQL is not for you. Time to go back to ACID relational databases.
Unfortunately MongoDB does't support transaction out of the box, but actually you can implement ACID optimistic transactions on top on it. I wrote an example and some explanation on a GitHub page.
I am looking for a DBI (or similar) proxy that supports both SQL restrictions and transactions. The two I know about are:
DBD::Proxy
DBD::Gofer
DBD::Proxy
The problem I have found with DBD::Proxy is that its server, DBI::ProxyServer, doesn't just restrict queries coming in over the network (which I want), but it also restricts queries generated internally by the database driver. So, for example, with DBD::Oracle, ping no longer works, as well as many other queries it issues itself.
I can't just allow them, because:
That is quite a bit of internal knowledge of DBD::Oracle and would be quite fragile.
The whitelist is query_name => 'sql', where query_name is the first word of whatever is passed to prepare. DBD::Oracle has a lot of internal queries, and the first word of many of them is select (duh).
So, it doesn't seem I can use DBD::Proxy
DBD::Gofer
I haven't tried DBD::Gofer, because the docs seem to tell me that I can't use transactions through it:
CONSTRAINTS
...
You can’t use transactions
AutoCommit only. Transactions aren’t supported.
So, before I write my own application-specific proxy (using RPC::PLServer ?), is there code out there that solves this problem?
This question would be best asked on the DBI Users mailing list, dbi-users#perl.org.
Sign up at http://dbi.perl.org/
I'm not sure what you mean about DBD::Proxy restricting queries. On the only occasion I've used it, it didn't modify the queries at all.