How to manage multiple url with OrientGraphFactory - orientdb

I configured a OrientDB Cluster with 3 nodes.
Suppose that the node names are:
leonardo.domain.ext
michelangelo.domain.ext
donatello.domain.ext
I have a configuration file to get this urls and create the proper url to pass to OrientGraphFactory (e.g. remote:leonardo.domain.ext/mydb)
I create the GraphFactory as following:
OrientGraphFactory factory = new OrientGraphFactory(url,username,password).setupPool(1, 10);
If leonardo is down when I try to create OrientGraphFactory I don't have any issues, but I get an Exception when I do.
factory.getNoTx();
Reading the documentation https://orientdb.gitbooks.io/orientdb-manual/content/orientdb.wiki/Distributed-Architecture-Lifecycle.html I understand that if leonardo go down (for any reason) the clients automatically reconnect to the other instances.
This is valid for the factory and the connection that it will try to create in the future or just for the active ones?
Do I have to manage to recreate the Factory in my application ?

You could use all those addresses separated by a semicolon:
remote:leonardo.domain.ext;michelangelo.domain.ext;donatello.domain.ext/mydb
OrientDB will try connecting to the first, but if it's unreachable, it will try the others in sequence.
It will also do load balancing at client side, if you are on 2.2.x version.
Hope it helps.

Related

Multiple server names in a single connection string

In the PostgreSQL documentation https://www.postgresql.org/docs/10/libpq-connect.html, it has been said that multiple hosts can be specified in a single connection string such that all the hosts will be tried in order one after the other until one of the server gets succeeds.
But when i tried to implement the same setting in the tag present in my ASP.net web.config file, it is throwing error as no such host name. I am using NpgSQL provider in order to connect to PostgreSQL database.
I need to add multiple server names in the connection string such that if the server#1 fails then it should try for the next server server#2 immediately provided in the order until it succeeds
Can you please suggest on how multiple hosts can be provided in the connection string?
The Npgsql driver does not currently support this functionality. The issue tracking this is https://github.com/npgsql/npgsql/issues/732, I'm still hoping we can get this into the next release but there's a lot going on.
Load balancing and failover is avaialble in Npgsql version 6. At the time of writing v.6 is in preview.
Simple failover example (server2 is only used if a connection could not be established to server1):
Host=server1,server2;Username=test;Password=test
Example with load balancing (round robin I guess):
Host=server1,server2,server3,server4,server5;Username=test;Password=test;Load
Balance Hosts=true;Target Session Attributes=prefer-standby
https://www.npgsql.org/doc/failover-and-load-balancing.html

Newbie help - how to connect to AWS Redshift cluster (currently using Aginity)

(I'm afraid I'm probably about to reveal myself as completely unfit for the task at hand!)
I'm trying to setup a Redshift cluster and database to help manage data for a class/group project.
I have a dc2.large cluster running with either default options, or what looked like the most generic in the couple of place I was forced to make entries.
I have downloaded Aginity (Win64) as it is described as being specialized for Redshift. That said, I can't find any instructions for connecting using it. The connection dialog requests the follwoing:
Server: using the endpoint for my cluster (less :57xx at the end).
UserID: the Master username for the database defined for the cluster.
Password: to match the UserID
SSL Mode (Disable, Allow, Prefer, Require): trying various options
Database: as named in cluster setup
Port: as defined in cluster setup
I can't get it to connect ("failed to establish connection") and don't know if I'm entering something wrong in Aginity or if I haven't set up my cluster properly.
Message: Failed to establish a connection to 'abc1234-smtm.crone7m2jcwv.us-east-1.redshift.amazonaws.com'.
Type : Npgsql.NpgsqlException
Source : Npgsql
Trace : at Npgsql.NpgsqlClosedState.Open(NpgsqlConnector context, Int32 timeout)
at Npgsql.NpgsqlConnector.Open()
at Npgsql.NpgsqlConnection.Open()
at Aginity.MPP.Common.BaseDataProvider.get_Connection()
at Aginity.MPP.Common.BaseDataProvider.CreateCommand(String commandText, CommandType commandType, IDataParameter[] commandParams)
at Aginity.MPP.Common.BaseDataProvider.ExecuteReader(String commandText, CommandType commandType, IDataParameter[] commandParams)
--- Inner Exception: ---
......
It seems there is not enough information going into Aginity to authorize connection to my cluster - no account credential are supplied. For UserID, am I meant to enter the ID of a valid user? Can I use the root account? What would the ID look like? I have setup a User with FullAccess to S3 and Redshift, then entered the UserID in this format
arn:aws:iam::600123456789:user/john
along with the matching password, but that hasn't worked either.
The only training/tutorial I have been able to find/do on this is the Intro AWS direct you to, at https://qwiklabs.com/focuses/2366, which uses a web-based client that I can't find outside of the tutorial (pgweb).
Any advice what I am doing wrong, and how to do it right?
Well, I think I got it working - I haven't had a chance to see if I can actually create table yet, but it seems to be connected. I had to allow inbound traffic from outside the VPC, as per the above snapshot.
I'm guessing there's a better way than opening it up to all IP addresses, but I don't know the users' (fellow team members) IPs, and aren't they all subject to change depending on the device they're using to connect?
How does one go about getting inside the VPC to connect that way, presumably more securely?

OrientDB and PostgreSQL JDBC drivers are clashing (InvocationTargetException): is there an OrientDB JAR with everything except JDBC?

My application uses both OrientDB and PostgreSQL databases for different purposes.
It seems they were able to coexist before, but today my code stopped working. Upon debugging, it seems that the OrientDB driver is attempting to connect to my PostgreSQL database when I'm expecting the PostgreSQL driver to connect instead.
Here is the sequence of events:
OrientDB connection is made (using OrientGraphFactory.setupPool()), transaction is started.
Connection attempt is made on PostgreSQL database, error occurs when trying to create the Connection object.
Here is the segment of code that creates the PostgreSQL connection:
Class.forName("org.postgresql.Driver");
Connection connection = DriverManager.getConnection(
"jdbc:postgresql://" + [...]);
return connection;
An InvocationTargetException is thrown at the DriverManager.getConnection() line. Here is the stack trace, clearly indicating that the OrientDB driver was the one trying to connect:
Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
com.orientechnologies.orient.core.exception.ODatabaseException: Error on opening database 'jdbc:postgresql://[hostname]/[db_name]'
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:204)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.<init>(ODatabaseDocumentTx.java:168)
at com.orientechnologies.orient.jdbc.OrientJdbcConnection.<init>(OrientJdbcConnection.java:62)
at com.orientechnologies.orient.jdbc.OrientJdbcDriver.connect(OrientJdbcDriver.java:52)
at java.sql.DriverManager.getConnection(DriverManager.java:664)
at java.sql.DriverManager.getConnection(DriverManager.java:247)
Indeed it seems the JDBC drivers are clashing.
I don't actually need JDBC functionality with OrientDB in this case. However, I can't find the OrientDB JAR that doesn't contain JDBC. The home page lets you download JDBC-all or JDBC-only. Where can I find a JAR with all dependencies bundled into a single JAR, but without JDBC?
This is probably not caused by the drivers themselves but by the DriverManager getting "confused" which driver handles which URL.
You can bypass the DriverManager by asking the driver directly for a connection:
Driver drv = new org.postgresql.Driver();
Properties props = new Properties();
props.put("username", "foo_user");
props.put("password", "database_password");
Connection connection = drv.getConnection("jdbc:postgresql://dbhost/db_name", props);
Unrelated, but: Class.forName("org.postgresql.Driver"); is longer necessary with current Java and driver versions.
I solved the problem here:
https://github.com/orientechnologies/orientdb/commit/8e0f4bed41999cf68ae9de229b3ff6a4260813da
It was a misunderstanding on how the DriverManager registers drivers and then calls the getConnection method.
Solutions.
My suggestion is to not use the orientdb-jdbc-all jar at all. If you don't need to work with orient embedded in your app AND access to it via JDBC it is really too big.
Instead, use your dependency management framework (maven, gradle?) to import orient jars, maybe only the orientdb-client if you need to interact with a remote db, maybe more if you need to embed orient in your app.
If you need to interact to a remote Orient Server via JDBC, use only che orientdb-jdbc. But you need the fixed one, so you should build it from source, or wait for next 2.1.8 release.
If you want to stay with the jdbc fat jar, again you can build it from source right now, or you can wait next hotfix release (2.1.8).
hope this help,
best regards

OrientGraphFactory Pool vs Network Connection Pool

I've recently started using the new OrientGraphFactory in OrientDB 2.1 and it's been great for the most part.
I'm using scala, and doing it like so:
lazy val orientFactory = new OrientGraphFactory(url, username, password).setupPool(minConnections, maxConnections)
When I need a new database instance, I simply do:
val graph = orientFactory.getTx.asInstanceOf[TransactionalGraph]
This all seems to be working well, but I'm running into some issues at high load. I'm suspecting that perhaps I need to do some extra work with the Network Connection Pool, as specified in the OrientDB Docs: http://orientdb.com/docs/2.1/Performance-Tuning.html
The documentation suggests setting the network connection pool like so:
database = new ODatabaseDocumentTx("remote:localhost/demo");
database.setProperty("minPool", 2);
database.setProperty("maxPool", 5);
database.open("admin", "admin");
What I'm wondering is whether the OrientGraphFactory pool also serves as a network pool of sorts, creating a network connection for each database instance. Is this the case? Or do I need to additionally setup the network connection pool?
Thanks!
The network pool uses this setting to setup the maximum number of connections:
OGlobalConfiguration.CLIENT_CHANNEL_MAX_POOL.setValue( 500 );
The default is 100.

Configure AppFabric Cache without listing servers in web.config

I am trying to understand how to properly configure AppFabric Caching on a web site. We are planning to use SQL Server as the cache manager and as far as I can understand the SQL will contain a list of the cache hosts in the cluster.
However, when running
DataCacheFactory factory = new DataCacheFactory();
I get
Server collection cannot be empty.
which, I guess, is to be expected since I have not added any servers in the web.config.
However, I do not want to maintain a server list on each web server, I want that to be done centrally on the SQL Server. I assume there is a way to point to the SQL Server, but I cannot find information on how to do this.
(I have also tried with the XML configration option, but it cannot even find that file. I have checked the health of the service in power shell.)
How do I centralize the server cache host list?
We are planning to use SQL Server as the cache manager and as far as I
can understand the SQL will contain a list of the cache hosts in the
cluster.
It's false. SQL Server can perform cluster management but it's only for managing the cache hosts, and ultimately, the cache cluster. It's just for internal management and your clients can use this configuration and they don't need to have acces to Sql Server.
DataCacheFactory factory = new DataCacheFactory();
This code will try to load default datacacheclient in config. In your case, it should be empty that's why you get this error.
You can still use code to configure cache host in this way.
// Declare array for cache host(s).
DataCacheServerEndpoint[] servers = new DataCacheServerEndpoint[1];
servers[0] = new DataCacheServerEndpoint("CacheServer1", 22233);
DataCacheFactoryConfiguration factoryConfig = new DataCacheFactoryConfiguration();
factoryConfig.Servers = servers;
DataCacheFactory mycacheFactory = new DataCacheFactory(factoryConfig);
DataCache myDefaultCache = mycacheFactory.GetCache("NamedCache1");
You don't need to specify all host names here, because AppFabric Caching will route request to the correct cache host, event if it is not in your list.