Context:
A client sends calls to a server to execute a job. For each job, I create a new MongoClient (with Morphia):
MongoClient mongoClient = new MongoClient("000.00.000.000", 27017);
Morphia morphia = new Morphia();
Datastore ds = morphia.createDatastore(mongoClient, "myDatastore");
//operations on the datastore: save, find, update...
The question: is it good practice / totally wrong? Or should I create only one MongoClient / Morphia instance for the whole app as a global variable, and let it be called by each job? (as described here)
The doc for the Mongo Java driver says:
The Java MongoDB driver is thread safe. If you are using in a web
serving environment, for example, you should create a single
MongoClient instance, and you can use it in every request. The
MongoClient object maintains an internal pool of connections to the
database (default pool size of 10). For every request to the DB (find,
insert, etc) the Java thread will obtain a connection from the pool,
execute the operation, and release the connection. This means the
connection (socket) used may be different each time.
So... one MongoClient per app, not per job called.
Related
I'm developing an application with SpringBoot. I already have a RestController and a RabbitMQ component that depending on the message I receive I get some data from a MongoDB and do some logic.
I set up the database as:
MongoClient mongoClient = MongoClients.create("mongodb://localhost:27017");
MongoDatabase db = mongoClient.getDatabase("databaseName");
MongoCollection<Document> collection = db.getCollection("collectionName");
Since I'm using SpringBoot I wanted to do it with Springboot and acess it in every SpringBoot component (the RestController and the RabbitMQ component).
I already understood that I have to put the settings on application.properties.
What I don't get is how do I acess the database afterwards.
Am I supposed to do a #Configuration class?
And how can I do, for example, collection.find(eq("id",userID)).first() everywhere?
Use the spring data JPA. You literally don't have to write any code.
Just follow this
I'm using OrientDB 2.2.2 jdbc driver, and using OrientDB internal pool.
When all the connections of the pool have been used, after a while the pool starts to throw this error:
ERROR c.c.e.s.l.ContentServerExtractorService - Error extracting data from content Server.Content Server database and on Internal Database.
com.orientechnologies.orient.core.exception.ODatabaseException: Database instance has been released to the pool. Get another database instance from the pool with the right username and password
Storage URL="plocal:xxx/databases/xxx"
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTxPooled.checkOpeness(ODatabaseDocumentTxPooled.java:151)
A search on OrientDB indicates that this was a problem that was supposed to be corrected:
https://github.com/wisdom-framework/wisdom-orientdb/issues/25
https://github.com/wisdom-framework/wisdom-orientdb/issues/25#issuecomment-112743409
The second link seems to indicate it might be an issue of a deprecated class, and i've reversed engineered the JDBC driver, and i verified that the driver uses that specific deprecated class to obtain the connection.
Is the OrientDB team aware of the problem?
And if so, when can this problem be corrected?
My application uses both OrientDB and PostgreSQL databases for different purposes.
It seems they were able to coexist before, but today my code stopped working. Upon debugging, it seems that the OrientDB driver is attempting to connect to my PostgreSQL database when I'm expecting the PostgreSQL driver to connect instead.
Here is the sequence of events:
OrientDB connection is made (using OrientGraphFactory.setupPool()), transaction is started.
Connection attempt is made on PostgreSQL database, error occurs when trying to create the Connection object.
Here is the segment of code that creates the PostgreSQL connection:
Class.forName("org.postgresql.Driver");
Connection connection = DriverManager.getConnection(
"jdbc:postgresql://" + [...]);
return connection;
An InvocationTargetException is thrown at the DriverManager.getConnection() line. Here is the stack trace, clearly indicating that the OrientDB driver was the one trying to connect:
Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
com.orientechnologies.orient.core.exception.ODatabaseException: Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:204)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:168)
at com.orientechnologies.orient.jdbc.OrientJdbcConnection.<init>(OrientJdbcConnection.java:62)
at com.orientechnologies.orient.jdbc.OrientJdbcDriver.connect(OrientJdbcDriver.java:52)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Indeed it seems the JDBC drivers are clashing.
I don't actually need JDBC functionality with OrientDB in this case. However, I can't find the OrientDB JAR that doesn't contain JDBC. The home page lets you download JDBC-all or JDBC-only. Where can I find a JAR with all dependencies bundled into a single JAR, but without JDBC?
This is probably not caused by the drivers themselves but by the DriverManager getting "confused" which driver handles which URL.
You can bypass the DriverManager by asking the driver directly for a connection:
Driver drv = new org.postgresql.Driver();
Properties props = new Properties();
props.put("username", "foo_user");
props.put("password", "database_password");
Connection connection = drv.getConnection("jdbc:postgresql://dbhost/db_name", props);
Unrelated, but: Class.forName("org.postgresql.Driver"); is longer necessary with current Java and driver versions.
I solved the problem here:
https://github.com/orientechnologies/orientdb/commit/8e0f4bed41999cf68ae9de229b3ff6a4260813da
It was a misunderstanding on how the DriverManager registers drivers and then calls the getConnection method.
Solutions.
My suggestion is to not use the orientdb-jdbc-all jar at all. If you don't need to work with orient embedded in your app AND access to it via JDBC it is really too big.
Instead, use your dependency management framework (maven, gradle?) to import orient jars, maybe only the orientdb-client if you need to interact with a remote db, maybe more if you need to embed orient in your app.
If you need to interact to a remote Orient Server via JDBC, use only che orientdb-jdbc. But you need the fixed one, so you should build it from source, or wait for next 2.1.8 release.
If you want to stay with the jdbc fat jar, again you can build it from source right now, or you can wait next hotfix release (2.1.8).
hope this help,
best regards
I've recently started using the new OrientGraphFactory in OrientDB 2.1 and it's been great for the most part.
I'm using scala, and doing it like so:
lazy val orientFactory = new OrientGraphFactory(url, username, password).setupPool(minConnections, maxConnections)
When I need a new database instance, I simply do:
val graph = orientFactory.getTx.asInstanceOf[TransactionalGraph]
This all seems to be working well, but I'm running into some issues at high load. I'm suspecting that perhaps I need to do some extra work with the Network Connection Pool, as specified in the OrientDB Docs: http://orientdb.com/docs/2.1/Performance-Tuning.html
The documentation suggests setting the network connection pool like so:
database = new ODatabaseDocumentTx("remote:localhost/demo");
database.setProperty("minPool", 2);
database.setProperty("maxPool", 5);
database.open("admin", "admin");
What I'm wondering is whether the OrientGraphFactory pool also serves as a network pool of sorts, creating a network connection for each database instance. Is this the case? Or do I need to additionally setup the network connection pool?
Thanks!
The network pool uses this setting to setup the maximum number of connections:
OGlobalConfiguration.CLIENT_CHANNEL_MAX_POOL.setValue( 500 );
The default is 100.
It seems to be a common and safer practice to host the database separately from Meteor apps. That is to say, have an EC2 instance for your Meteor app, and an EC2 instance for your MongoDB, and make them talk to one another.
From what I understand, people do this because it's more secure, and it allows them to deploy newer versions of their app without touching the database.
I'd like to do this with Amazon EC2 alone, as opposed to using another 3rd party service, like Compose.io.
How can I host a Meteor app and its database separately on two EC2 instances, and have them communicate with one another?
It is common practice, and people mostly do it because it offers you the ability to scale them both independently.
As to the how, you'll want to obviously configure each of your Amazon EC2 instances, installing meteor on one, and MongoDB on the other. You'll also need to configure your VPC (Amazon Virtual Private Cloud) so that your MongoDB instance accepts incoming connections on whatever port you specify (default is 27017), so that your Meteor Application can connect.
After that it's just a matter of telling your meteor app where to go to get the database connection. The most secure way of doing this will be to set a couple Environment Variables, named MONGODBSERVER and MONGODBPORT, DBUSER, DBPASSWORD, etc.
You'll then want to set some variables in your server Meteor code, using something like:
Meteor.startup(function() {
var DbUser = process.env.DBUSER;
var DbPassword = process.env.DBPASSWORD;
var MongoDBServer = process.env.MONGODBSERVER;
var MongoDBPort = process.env.MONGODBPORT;
});
And if you're using the native MongoDB Driver, connecting becomes trivial:
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect('mongodb://DbUser:DbPassword#MongoDBServer:MongoDBPort/databasename', function(err, db) {
...
});
Then it's just a matter of constructing your Mongo models using something like:
Temperatures = new Mongo.Collection('temperatures');
Temperatures._ensureIndex({temp: 1, time: 1});
And then taking action on those models in regard to the database:
Temperatures.insert({temp: ftemp, time: Math.floor(Date.now() / 1000)});
I'll also mention that http://modulus.io is a really decent Meteor hosting solution. I'd recommend them, unless you are stuck on using Amazon EC2 instances, which is fine, but more complicated for a simple application.
You need to set an Environment Variable for Mongo where it is hosted
MONGO_URL
mongodb://:#hostingproviderurl:port/xxx?autoReconnect=true&connectTimeoutMS=60000
the correct mongodb:// url string would be provided by the mongodb hosting provider.