I've recently started using the new OrientGraphFactory in OrientDB 2.1 and it's been great for the most part.
I'm using scala, and doing it like so:
lazy val orientFactory = new OrientGraphFactory(url, username, password).setupPool(minConnections, maxConnections)
When I need a new database instance, I simply do:
val graph = orientFactory.getTx.asInstanceOf[TransactionalGraph]
This all seems to be working well, but I'm running into some issues at high load. I'm suspecting that perhaps I need to do some extra work with the Network Connection Pool, as specified in the OrientDB Docs: http://orientdb.com/docs/2.1/Performance-Tuning.html
The documentation suggests setting the network connection pool like so:
database = new ODatabaseDocumentTx("remote:localhost/demo");
database.setProperty("minPool", 2);
database.setProperty("maxPool", 5);
database.open("admin", "admin");
What I'm wondering is whether the OrientGraphFactory pool also serves as a network pool of sorts, creating a network connection for each database instance. Is this the case? Or do I need to additionally setup the network connection pool?
Thanks!
The network pool uses this setting to setup the maximum number of connections:
OGlobalConfiguration.CLIENT_CHANNEL_MAX_POOL.setValue( 500 );
The default is 100.
Related
I configured a OrientDB Cluster with 3 nodes.
Suppose that the node names are:
leonardo.domain.ext
michelangelo.domain.ext
donatello.domain.ext
I have a configuration file to get this urls and create the proper url to pass to OrientGraphFactory (e.g. remote:leonardo.domain.ext/mydb)
I create the GraphFactory as following:
OrientGraphFactory factory = new OrientGraphFactory(url,username,password).setupPool(1, 10);
If leonardo is down when I try to create OrientGraphFactory I don't have any issues, but I get an Exception when I do.
factory.getNoTx();
Reading the documentation https://orientdb.gitbooks.io/orientdb-manual/content/orientdb.wiki/Distributed-Architecture-Lifecycle.html I understand that if leonardo go down (for any reason) the clients automatically reconnect to the other instances.
This is valid for the factory and the connection that it will try to create in the future or just for the active ones?
Do I have to manage to recreate the Factory in my application ?
You could use all those addresses separated by a semicolon:
remote:leonardo.domain.ext;michelangelo.domain.ext;donatello.domain.ext/mydb
OrientDB will try connecting to the first, but if it's unreachable, it will try the others in sequence.
It will also do load balancing at client side, if you are on 2.2.x version.
Hope it helps.
I'm using OrientDB 2.2.2 jdbc driver, and using OrientDB internal pool.
When all the connections of the pool have been used, after a while the pool starts to throw this error:
ERROR c.c.e.s.l.ContentServerExtractorService - Error extracting data from content Server.Content Server database and on Internal Database.
com.orientechnologies.orient.core.exception.ODatabaseException: Database instance has been released to the pool. Get another database instance from the pool with the right username and password
Storage URL="plocal:xxx/databases/xxx"
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTxPooled.checkOpeness(ODatabaseDocumentTxPooled.java:151)
A search on OrientDB indicates that this was a problem that was supposed to be corrected:
https://github.com/wisdom-framework/wisdom-orientdb/issues/25
https://github.com/wisdom-framework/wisdom-orientdb/issues/25#issuecomment-112743409
The second link seems to indicate it might be an issue of a deprecated class, and i've reversed engineered the JDBC driver, and i verified that the driver uses that specific deprecated class to obtain the connection.
Is the OrientDB team aware of the problem?
And if so, when can this problem be corrected?
My application uses both OrientDB and PostgreSQL databases for different purposes.
It seems they were able to coexist before, but today my code stopped working. Upon debugging, it seems that the OrientDB driver is attempting to connect to my PostgreSQL database when I'm expecting the PostgreSQL driver to connect instead.
Here is the sequence of events:
OrientDB connection is made (using OrientGraphFactory.setupPool()), transaction is started.
Connection attempt is made on PostgreSQL database, error occurs when trying to create the Connection object.
Here is the segment of code that creates the PostgreSQL connection:
Class.forName("org.postgresql.Driver");
Connection connection = DriverManager.getConnection(
"jdbc:postgresql://" + [...]);
return connection;
An InvocationTargetException is thrown at the DriverManager.getConnection() line. Here is the stack trace, clearly indicating that the OrientDB driver was the one trying to connect:
Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
com.orientechnologies.orient.core.exception.ODatabaseException: Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:204)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:168)
at com.orientechnologies.orient.jdbc.OrientJdbcConnection.<init>(OrientJdbcConnection.java:62)
at com.orientechnologies.orient.jdbc.OrientJdbcDriver.connect(OrientJdbcDriver.java:52)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Indeed it seems the JDBC drivers are clashing.
I don't actually need JDBC functionality with OrientDB in this case. However, I can't find the OrientDB JAR that doesn't contain JDBC. The home page lets you download JDBC-all or JDBC-only. Where can I find a JAR with all dependencies bundled into a single JAR, but without JDBC?
This is probably not caused by the drivers themselves but by the DriverManager getting "confused" which driver handles which URL.
You can bypass the DriverManager by asking the driver directly for a connection:
Driver drv = new org.postgresql.Driver();
Properties props = new Properties();
props.put("username", "foo_user");
props.put("password", "database_password");
Connection connection = drv.getConnection("jdbc:postgresql://dbhost/db_name", props);
Unrelated, but: Class.forName("org.postgresql.Driver"); is longer necessary with current Java and driver versions.
I solved the problem here:
https://github.com/orientechnologies/orientdb/commit/8e0f4bed41999cf68ae9de229b3ff6a4260813da
It was a misunderstanding on how the DriverManager registers drivers and then calls the getConnection method.
Solutions.
My suggestion is to not use the orientdb-jdbc-all jar at all. If you don't need to work with orient embedded in your app AND access to it via JDBC it is really too big.
Instead, use your dependency management framework (maven, gradle?) to import orient jars, maybe only the orientdb-client if you need to interact with a remote db, maybe more if you need to embed orient in your app.
If you need to interact to a remote Orient Server via JDBC, use only che orientdb-jdbc. But you need the fixed one, so you should build it from source, or wait for next 2.1.8 release.
If you want to stay with the jdbc fat jar, again you can build it from source right now, or you can wait next hotfix release (2.1.8).
hope this help,
best regards
I use "Liberty for Java" app and Statica service(Proxy) on Bluemix.
We set http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort as system properties in Java code every transactions.
for example:
URL url = new URL(xxx);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
........
System.setProperty("http.proxyHost", host);
System.setProperty("http.proxyPort", port);
System.setProperty("https.proxyHost", host);
System.setProperty("https.proxyPort", port);
........
DataOutputStream out = new DataOutputStream(connection.getOutputStream());
I have an issue that one transaction go from the app to a target server directly in spite of tens of thousands of transactions passed the proxy.
Question 1:
Do "Liberty for Java" app on Bluemix clear or update system properties, http.proxyHost/http.proxyPort/https.proxyHost/https.proxyPort?
I wonder "Liberty for Java" app updated with null to access outer servers in multi-thread environment.
Question 2:
Do "Liberty for Java" app on Bluemix communicate with outer servers?
I found the following log in Statica.
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.agents.na.apm.ibmserviceengage.com
https://xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.gateway.prd.na.ca.ibmserviceengage.com
( I masked a part of URL.)
P.S. We will change java code with ProxySelector class or Proxy class.
Re #1: No.
Re #2: Potentially yes. In your case, it seems your app is bound with a Monitoring & Analytics service? If so, a data collector will be installed and will send collected data to remote servers.
What's the reason that you need to set the proxy system properties in your code? Is it because you want some connections to go through the proxy and others not?
If so, then the way you do this is not right because the system proxy setting is a global setting, not a thread-scoped setting. This means if one thread sets the proxy setting, all threads will then use that proxy; if one thread unsets it, all threads will then do direct connections. That may explain why you are intermittently seeing some direct connections. The right way is to use a http client lib that supports proxy as parameters, like https://hc.apache.org/httpcomponents-client-ga/httpclient/apidocs/org/apache/http/client/config/RequestConfig.Builder.html#setProxy%28org.apache.http.HttpHost%29
If you want all connections to go through the http proxy, then you should simply set the JAVA_OPTS environment variable to pass in those system properties, e.g., "-Dhttp.proxyHost=x.x.x.x -Dhttp.proxyPort=xx".
Context:
A client sends calls to a server to execute a job. For each job, I create a new MongoClient (with Morphia):
MongoClient mongoClient = new MongoClient("000.00.000.000", 27017);
Morphia morphia = new Morphia();
Datastore ds = morphia.createDatastore(mongoClient, "myDatastore");
//operations on the datastore: save, find, update...
The question: is it good practice / totally wrong? Or should I create only one MongoClient / Morphia instance for the whole app as a global variable, and let it be called by each job? (as described here)
The doc for the Mongo Java driver says:
The Java MongoDB driver is thread safe. If you are using in a web
serving environment, for example, you should create a single
MongoClient instance, and you can use it in every request. The
MongoClient object maintains an internal pool of connections to the
database (default pool size of 10). For every request to the DB (find,
insert, etc) the Java thread will obtain a connection from the pool,
execute the operation, and release the connection. This means the
connection (socket) used may be different each time.
So... one MongoClient per app, not per job called.