Clear pinned pages in OrientDB 2.2.6 - orientdb

I'm running OrientDB 2.2.6 on Windows 7 (64 bit) and I keep getting the following warning when I run a TestNG test in Java 8.
2016-10-05 10:46:10:888 WARNI {db=orientSystemDb} Maximum amount of pinned pages is
reached, given page OReadCacheEntry{fileId=7136553380276606136, pageIndex=0,
dataPointer=OCachePointer{referrersCount=1, usagesCount=0}, dirty=false, usagesCount=1}
will not be marked as pinned which may lead to performance degradation. You may consider
to increase percent of pined pages by changing of property storage.diskCache.pinnedPages
[O2QCache]
I'd prefer not to increase the percent of pinnedPages right now since memory usage will probably be limited. What I would like to do is clear the pages that have been pinned when the database has started and see if that resolves the issue.
Here is my config file:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<orient-server>
<handlers>
<handler class="com.orientechnologies.orient.graph.handler.OGraphServerHandler">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="50" name="graph.pool.max"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<parameter value="${distributed}" name="enabled"/>
<parameter value="${ORIENTDB_HOME}/config/default-distributed-db-config.json" name="configuration.db.default"/>
<parameter value="${ORIENTDB_HOME}/config/hazelcast.xml" name="configuration.hazelcast"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OJMXPlugin">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="true" name="profilerManaged"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OAutomaticBackup">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="${ORIENTDB_HOME}/config/automatic-backup.json" name="config"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OServerSideScriptInterpreter">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="SQL" name="allowedLanguages"/>
</parameters>
</handler>
</handlers>
<network>
<protocols>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary" name="binary"/>
<protocol name="http" implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb"/>
</protocols>
<listeners>
<listener protocol="binary" socket="default" port-range="2424-2430" ip-address="0.0.0.0"/>
<listener protocol="http" port-range="2480-2485" ip-address="0.0.0.0">
<commands>
<command implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent" pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg">
<parameters>
<entry value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" name="http.cache:*.htm *.html"/>
<entry value="Cache-Control: max-age=120" name="http.cache:default"/>
</parameters>
</command>
</commands>
</listener>
</listeners>
</network>
<users>
<user resources="*" password="root" name="root"/>
<user resources="connect,server.listDatabases,server.dblist" password="guest" name="guest"/>
</users>
<properties>
<entry value="1" name="db.pool.min"/>
<entry value="50" name="db.pool.max"/>
<entry value="true" name="profiler.enabled"/>
<!-- Avoid updating versions when created or deleting edges -->
<entry value="-1" name="ridBag.embeddedToSbtreeBonsaiThreshold"/>
</properties>
</orient-server>
Here is the code for creating edges:
private int connectToAllOtherVertices(OrientBaseGraph oGraph, OrientVertex oVertex)
{
int numConnections = 0;
for(Vertex v : oGraph.getVertices())
{
if(v.equals(oVertex))
{
// skip it
continue;
}
oVertex.addEdge(String.format("connection%s", numConnections), v);
numConnections++;
}
return numConnections;
}
I'm able to create the edges but I keep getting the above warnings. How can I clear the pinned pages before running my tests?

Pinned pages is special area of disk cache which unloaded from RAM only when OrientDB storage is closed. So you may try using closeStorage method. See javadoc.

Related

Dynamic Servicefabric Settings and Overrides

Is there a way to not tell the service about the settings at all and just provide them at the application level?
I am still unhappy with how servicefabric configuration seems to work.
Near as I can tell I have to specify in the service’s settings.xml all of the possible configuration values. Then I can override those in the application’s ApplicationParameters. Per documentation this looks like it holds true for environment variables also.
The complication that creates is that our configuration is used to hydrate options in many cases with arrays.
For example consider the json:
{
"AuthorizationOptions": {
"Policies": [
{
"Name": "User",
"Groups": [ "Domain Users" ]
}
]
}
}
There are 2 arrays; that are necessary and useful. To express this in the service fabric configuration it translates to:
<Section Name="AuthorizationOptions">
<Parameter Name="Policies:0:Name" Value="User"/>
<Parameter Name="Policies:0:Groups:0" Value="Domain Users"/>
</Section>
While the translation is not pleasant in comparison to the structured object it is completely usable.
However, If I don’t specify the section and parameters in the service, I can’t seem to override them in the application. So in this case I would have to define the exact number of policies and groups per policy in the service and the application could modify the policy name, or the group values, but not the number of policies total or number of groups total.
Is there a way to not tell the service about the settings at all and just provide them at the application level?
If not what alternatives exist to make the service reusable across applications that I may want to use to provide this type of dynamic configuration differently?
The last part of the puzzle that may assist in answering this question is I am using some pre-release code to translate the service fabric settings into Microsoft.Extensions.Configuration.IConfiguration. However, that is just taking the settings it finds; it isn't the cause of the override issue I am running into.
Example Service Settings.xml:
<?xml version="1.0" encoding="utf-8" ?>
<Settings xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Section Name="AuthorizationOptions">
<!-- I should not have to provide these at the application level!
However, it fails to deploy if I don't. -->
<Parameter Name="Policies:0:Name" Value="User"/>
<Parameter Name="Policies:0:Groups:0" Value="Domain Users"/>
</Section>
</Settings>
Example Application ApplicationManifest.xml:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" ApplicationTypeName="ServiceFabric.ExampleType" ApplicationTypeVersion="1.0.0" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" DefaultValue="" />
<Parameter Name="Service.Example_InstanceCount" DefaultValue="-1" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Name" DefaultValue="Users" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Groups_0" DefaultValue="Domain Users" />
</Parameters>
<ServiceManifestImport>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="Service.ExamplePkg" ServiceManifestVersion="1.0.0" />
<ConfigOverrides>
<ConfigOverride Name="Config">
<Settings>
<Section Name="AuthorizationOptions">
<Parameter Name="Policies:0:Name" Value="[Service.Example_AuthorizationOptions_Policies_0_Name]" />
<Parameter Name="Policies:0:Groups:0" Value="[Service.Example_AuthorizationOptions_Policies_0_Groups_0]" />
</Section>
</Settings>
</ConfigOverride>
</ConfigOverrides>
<EnvironmentOverrides CodePackageRef="code">
<EnvironmentVariable Name="ASPNETCORE_ENVIRONMENT" Value="[Service.Example_ASPNETCORE_ENVIRONMENT]" />
</EnvironmentOverrides>
</ServiceManifestImport>
<DefaultServices>
<Service Name="Service.Example" ServicePackageActivationMode="ExclusiveProcess">
<StatelessService ServiceTypeName="Service.ExampleType" InstanceCount="[Service.Example_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
</ApplicationManifest>
Example Application ApplicationParameters (Local.1Node.xml):
<?xml version="1.0" encoding="utf-8"?>
<Application xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Name="fabric:/ServiceFabric.Example" xmlns="http://schemas.microsoft.com/2011/01/fabric">
<Parameters>
<Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" Value="Development" />
<Parameter Name="Service.Example_InstanceCount" Value="-1" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Name" Value="Users" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies_0_Groups_0" Value="Domain Users" />
</Parameters>
</Application>
Example Application PublishProfiles (Local.1Node.xml):
<?xml version="1.0" encoding="utf-8"?>
<PublishProfile xmlns="http://schemas.microsoft.com/2015/05/fabrictools">
<ClusterConnectionParameters />
<ApplicationParameterFile Path="..\ApplicationParameters\Local.1Node.xml" />
</PublishProfile>
Should be unrelated, but example consumption of the settings:
internal sealed class Example : StatelessService
{
public Example(StatelessServiceContext context)
: base(context)
{ }
protected override IEnumerable<ServiceInstanceListener> CreateServiceInstanceListeners()
{
return new ServiceInstanceListener[]
{
new ServiceInstanceListener(serviceContext =>
new HttpSysCommunicationListener(serviceContext, "ServiceEndpoint", (url, listener) =>
{
ServiceEventSource.Current.ServiceMessage(serviceContext, $"Starting HttpSys on {url}");
return new WebHostBuilder()
.UseHttpSys(options =>
{
options.Authentication.Schemes = AuthenticationSchemes.Negotiate; // Microsoft.AspNetCore.Server.HttpSys
options.Authentication.AllowAnonymous = false;
}).ConfigureServices(services => services.AddSingleton<StatelessServiceContext>(serviceContext))
.UseContentRoot(Directory.GetCurrentDirectory())
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.SetBasePath(Directory.GetCurrentDirectory());
config.AddServiceFabricConfiguration(FabricRuntime.GetActivationContext(), options => {
options.IncludePackageName=false;
});
})
.UseStartup<Startup>()
.UseServiceFabricIntegration(listener, ServiceFabricIntegrationOptions.None)
.UseUrls(url)
.Build();
}))
};
}
}
From that point forward everything is in the IConfiguration object as expected.
There are many ways to configure a service fabric application, and each approach will bring you to a different challenges.
SF team recommend the approach in the docs, because you can have a better version control of configurations, and makes harder to commit mistakes, as it is explicitly declared in a file, I've used a few different approaches because of limitations like yours, the follwoing approach might solve your problem:
Configure like the original approach, but with complex types stored as JSON values: it is the closest solution to the recommended design and you still can keep control of the configuration versions on source control.
It would be something like:
Settings.xml:
<?xml version="1.0" encoding="utf-8" ?>
<Settings xmlns... namespaces here...>
<Section Name="AuthorizationOptions">
<Parameter Name="Policies"/>
</Section>
</Settings>
ApplicationManifest.xml:
<?xml version="1.0" encoding="utf-8"?>
<ApplicationManifest ApplicationTypeName="ServiceFabric.ExampleType" ApplicationTypeVersion="1.0.0" xmlns:...namespaces....>
<Parameters>
<Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" DefaultValue="" />
<Parameter Name="Service.Example_InstanceCount" DefaultValue="-1" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies" DefaultValue="[]" />
</Parameters>
<ServiceManifestImport>
<ServiceManifestImport>
<ServiceManifestRef ServiceManifestName="Service.ExamplePkg" ServiceManifestVersion="1.0.0" />
<ConfigOverrides>
<ConfigOverride Name="Config">
<Settings>
<Section Name="AuthorizationOptions">
<Parameter Name="Policies" Value="[Service.Example_AuthorizationOptions_Policies]" />
</Section>
</Settings>
</ConfigOverride>
</ConfigOverrides>
<EnvironmentOverrides CodePackageRef="code">
<EnvironmentVariable Name="ASPNETCORE_ENVIRONMENT" Value="[Service.Example_ASPNETCORE_ENVIRONMENT]" />
</EnvironmentOverrides>
</ServiceManifestImport>
<DefaultServices>
<Service Name="Service.Example" ServicePackageActivationMode="ExclusiveProcess">
<StatelessService ServiceTypeName="Service.ExampleType" InstanceCount="[Service.Example_InstanceCount]">
<SingletonPartition />
</StatelessService>
</Service>
</DefaultServices>
</ApplicationManifest>
ApplicationParameters.xml
<?xml version="1.0" encoding="utf-8"?>
<Application Name="fabric:/ServiceFabric.Example" xmlns:...namespaces....>
<Parameters>
<Parameter Name="Service.Example_ASPNETCORE_ENVIRONMENT" Value="Development" />
<Parameter Name="Service.Example_InstanceCount" Value="-1" />
<Parameter Name="Service.Example_AuthorizationOptions_Policies" Value="[{'Name': 'User','Groups': ['Domain Users']}, {'Name': 'Admin','Groups': ['Administrators']}]" />
</Parameters>
</Application>
In your service code:
public class Policy
{
public string Name { get; set; }
public string[] Groups { get; set; }
}
var settings = this.Context.CodePackageActivationContext.GetConfigurationPackageObject("Config").Settings;
var authOptions = settings.Sections["AuthorizationOptions"].Parameters["Policies"].Value;
var obj = JsonConvert.DeserializeObject<Policy[]>(authOptions);
You could go a level further and store the entire
AuthorizationOptions as a JSON, but like said previously, more
generic it become, easier will be to commit mistakes and harder to find
configuration issues.

Start OrientDB studio for plocal DB

I'm able to start my orientDb (2.2.6) instance using a plocal connection and able to add vertices to the DB but I can't access the studio. When I go to http://localhost:2480, the page is blank. Isn't studio ready and available 'out of the box'?
orientdb-server-config.xml is:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<orient-server>
<network>
<protocols>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary" name="binary"/>
<protocol name="http" implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb"/>
</protocols>
<listeners>
<listener protocol="binary" socket="default" port-range="2424-2430" ip-address="0.0.0.0"/>
<listener protocol="http" port-range="2480-2485" ip-address="0.0.0.0">
<commands>
<command implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent" pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg">
<parameters>
<entry value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" name="http.cache:*.htm *.html"/>
<entry value="Cache-Control: max-age=120" name="http.cache:default"/>
</parameters>
</command>
</commands>
</listener>
</listeners>
</network>
<users>
<user resources="*" password="root" name="root"/>
<user resources="connect,server.listDatabases,server.dblist" password="guest" name="guest"/>
</users>
<properties>
<entry value="1" name="db.pool.min"/>
<entry value="50" name="db.pool.max"/>
<entry value="true" name="profiler.enabled"/>
</properties>
</orient-server>
I get these warnings whenever I try to access localhost:2480 from Chrome:
2016-09-28 16:56:01:756 WARNI path variable points to 'src/site' but it doesn't exists [OServerCommandGetStaticContent]
2016-09-28 16:56:01:756 WARNI path variable points to 'src/site' but it isn't a directory [OServerCommandGetStaticContent]
What am I missing?

OrientDB: getting OLowDiskSpaceException when trying to restore a database backup

I backed up my database and I'm now upgrading to v2.1.16.
I've tested my backup and restore routine on a local Vagrant virtual server and everything went fine.
I'm now trying to run the same procedure on my Amazon AWS server, and the process fails due to a OLowDiskSpaceException.
Here is the message I'm getting:
Started import of database 'remote:localhost/mydatabase' from /home/ubuntu/orientdb_
update_16.04.27.11.10.+0000.gz...
Non merge mode (-merge=false): removing all default non security classesError on
database import happened just before line 0, column 1
com.orientechnologies.orient.core.exception.OLowDiskSpaceException: Error occurr
ed while executing a write operation to database 'mydatabase' due to limited free sp
ace on the disk (-26 MB). The database is now working in read-only mode. Please
close the database (or stop OrientDB), make room on your hard drive and then reo
pen the database. The minimal required space is 100 MB. Required space is now se
t to 100MB (you can change it by setting parameter storage.diskCache.diskFreeSpa
ceLimit) .
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginat
edStorage.checkLowDiskSpaceAndFullCheckpointRequests(OAbstractPaginatedStorage.j
ava:2890)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginat
edStorage.updateRecord(OAbstractPaginatedStorage.java:943)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.exe
cuteSaveRecord(ODatabaseDocumentTx.java:2005)
at com.orientechnologies.orient.core.tx.OTransactionNoTx.saveRecord(OTra
nsactionNoTx.java:159)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.sav
e(ODatabaseDocumentTx.java:2568)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.sav
e(ODatabaseDocumentTx.java:121)
at com.orientechnologies.orient.core.record.impl.ODocument.save(ODocumen
t.java:1768)
at com.orientechnologies.orient.core.record.impl.ODocument.save(ODocumen
t.java:1764)
at com.orientechnologies.orient.core.metadata.schema.OSchemaShared$1.cal
l(OSchemaShared.java:1213)
at com.orientechnologies.orient.core.db.OScenarioThreadLocal.executeAsDi
stributed(OScenarioThreadLocal.java:71)
at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.saveI
nternal(OSchemaShared.java:1208)
at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.relea
seSchemaWriteLock(OSchemaShared.java:642)
at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.relea
seSchemaWriteLock(OSchemaShared.java:631)
at com.orientechnologies.orient.core.metadata.schema.OSchemaShared.dropC
lass(OSchemaShared.java:557)
at com.orientechnologies.orient.core.metadata.schema.OSchemaProxy.dropCl
ass(OSchemaProxy.java:156)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDropClass.ex
ecute(OCommandExecutorSQLDropClass.java:122)
at com.orientechnologies.orient.core.sql.OCommandExecutorSQLDelegate.exe
cute(OCommandExecutorSQLDelegate.java:90)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginat
edStorage.executeCommand(OAbstractPaginatedStorage.java:1547)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginat
edStorage.command(OAbstractPaginatedStorage.java:1528)
at com.orientechnologies.orient.core.command.OCommandRequestTextAbstract
.execute(OCommandRequestTextAbstract.java:67)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkP
rotocolBinary.command(ONetworkProtocolBinary.java:1323)
at com.orientechnologies.orient.server.network.protocol.binary.ONetworkP
rotocolBinary.executeRequest(ONetworkProtocolBinary.java:400)
at com.orientechnologies.orient.server.network.protocol.binary.OBinaryNe
tworkProtocolAbstract.execute(OBinaryNetworkProtocolAbstract.java:223)
at com.orientechnologies.common.thread.OSoftThread.run(OSoftThread.java:
77)
Error: com.orientechnologies.orient.core.db.tool.ODatabaseExportException: Error
on importing database 'mydatabase' from file: /home/ubuntu/orientdb_update_16.04.27
.11.10.+0000.gz
Error: com.orientechnologies.orient.core.exception.OLowDiskSpaceException: Error
occurred while executing a write operation to database 'mydatabase' due to limited
free space on the disk (-26 MB). The database is now working in read-only mode.
Please close the database (or stop OrientDB), make room on your hard drive and t
hen reopen the database. The minimal required space is 100 MB. Required space is
now set to 100MB (you can change it by setting parameter storage.diskCache.disk
FreeSpaceLimit) .
The weird thing is I have plenty of free space available. Running 'df -h' returns:
Filesystem Size Used Avail Use% Mounted on
udev 492M 12K 492M 1% /dev
tmpfs 100M 336K 99M 1% /run
/dev/xvda1 7.8G 2.2G 5.2G 30% /
none 4.0K 0 4.0K 0% /sys/fs/cgroup
none 5.0M 0 5.0M 0% /run/lock
none 497M 0 497M 0% /run/shm
none 100M 0 100M 0% /run/user
Why is OrientDB convinced I have less than 100MB of free space, when in fact I have over 5GB?
EDIT: here is my configuration
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<orient-server>
<handlers>
<handler class="com.orientechnologies.orient.graph.handler.OGraphServerHandler">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="50" name="graph.pool.max"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<parameter value="${DISTRIBUTED}" name="enabled"/>
<parameter value="${ORIENTDB_HOME}/config/distributed-db-config.json" name="configuration.db.default"/>
<parameter value="${ORIENTDB_HOME}/config/hazelcast.xml" name="configuration.hazelcast"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OJMXPlugin">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="true" name="profilerManaged"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OAutomaticBackup">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="4h" name="delay"/>
<parameter value="backup" name="target.directory"/>
<parameter value="${DBNAME}-${DATE:yyyyMMddHHmmss}.zip" name="target.fileName"/>
<parameter value="9" name="compressionLevel"/>
<parameter value="1048576" name="bufferSize"/>
<parameter value="" name="db.include"/>
<parameter value="" name="db.exclude"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.handler.OServerSideScriptInterpreter">
<parameters>
<parameter value="true" name="enabled"/>
<parameter value="SQL" name="allowedLanguages"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.token.OrientTokenHandler">
<parameters>
<parameter value="false" name="enabled"/>
<parameter value="" name="oAuth2Key"/>
<parameter value="60" name="sessionLength"/>
<parameter value="HmacSHA256" name="encryptionAlgorithm"/>
</parameters>
</handler>
<handler class="com.orientechnologies.orient.server.plugin.livequery.OLiveQueryPlugin">
<parameters>
<parameter value="false" name="enabled"/>
</parameters>
</handler>
</handlers>
<network>
<sockets>
<socket implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory" name="ssl">
<parameters>
<parameter value="false" name="network.ssl.clientAuth"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore"/>
<parameter value="password" name="network.ssl.keyStorePassword"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore"/>
<parameter value="password" name="network.ssl.trustStorePassword"/>
</parameters>
</socket>
<socket implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory" name="https">
<parameters>
<parameter value="false" name="network.ssl.clientAuth"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore"/>
<parameter value="password" name="network.ssl.keyStorePassword"/>
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore"/>
<parameter value="password" name="network.ssl.trustStorePassword"/>
</parameters>
</socket>
</sockets>
<protocols>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary" name="binary"/>
<protocol implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb" name="http"/>
</protocols>
<listeners>
<listener protocol="binary" socket="default" port-range="2424-2430" ip-address="0.0.0.0"/>
<listener protocol="http" socket="default" port-range="2480-2490" ip-address="0.0.0.0">
<commands>
<command implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent" pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg GET|*.json GET|*.woff GET|*.woff2 GET|*.ttf GET|*.svgz" stateful="false">
<parameters>
<entry value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" name="http.cache:*.htm *.html"/>
<entry value="Cache-Control: max-age=120" name="http.cache:default"/>
</parameters>
</command>
<command implementation="com.orientechnologies.orient.graph.server.command.OServerCommandGetGephi" pattern="GET|gephi/*" stateful="false"/>
</commands>
<parameters>
<parameter value="utf-8" name="network.http.charset"/>
<parameter value="true" name="network.http.jsonResponseError"/>
</parameters>
</listener>
</listeners>
</network>
<storages/>
<users>
<user resources="*" password="..." name="root"/>
<user resources="connect,server.listDatabases,server.dblist" password="guest" name="guest"/>
</users>
<properties>
<entry value="1" name="db.pool.min"/>
<entry value="50" name="db.pool.max"/>
<entry value="true" name="profiler.enabled"/>
<entry value="info" name="log.console.level"/>
<entry value="fine" name="log.file.level"/>
</properties>
</orient-server>

How to setup embedded master/master replication with OrientDB?

My goal is to have two nodes. Node A should write some vertices and node B should be able to read those vertices.
So far my hazelcast discovery works just fine. OrientDB will also hot deploy any found database from nodeA to nodeB. Unfortunately any write on Node A will not be replicated to nodeB.
My setup:
orientdb-server-config.xml:
Simplified version (copied from orientdb-community tar.gz)
%NODENAME% will be replaced with nodeA or nodeB.
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<orient-server>
<handlers>
<handler
class="com.orientechnologies.orient.graph.handler.OGraphServerHandler">
<parameters>
<parameter name="enabled" value="true" />
<parameter name="graph.pool.max" value="50" />
</parameters>
</handler>
<!-- CLUSTER PLUGIN, TO TURN ON SET THE 'ENABLED' PARAMETER TO 'true' -->
<handler
class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<parameter name="nodeName" value="%NODENAME%" />
<parameter name="enabled" value="true" />
<parameter name="configuration.db.default"
value="config/default-distributed-db-config.json" />
<parameter name="configuration.hazelcast" value="config/hazelcast.xml" />
</parameters>
</handler>
</handlers>
<network>
<sockets>
<socket
implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory"
name="ssl">
<parameters>
<parameter value="false" name="network.ssl.clientAuth" />
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore" />
<parameter value="password" name="network.ssl.keyStorePassword" />
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore" />
<parameter value="password" name="network.ssl.trustStorePassword" />
</parameters>
</socket>
<socket
implementation="com.orientechnologies.orient.server.network.OServerSSLSocketFactory"
name="https">
<parameters>
<parameter value="false" name="network.ssl.clientAuth" />
<parameter value="config/cert/orientdb.ks" name="network.ssl.keyStore" />
<parameter value="password" name="network.ssl.keyStorePassword" />
<parameter value="config/cert/orientdb.ks" name="network.ssl.trustStore" />
<parameter value="password" name="network.ssl.trustStorePassword" />
</parameters>
</socket>
</sockets>
<protocols>
<!-- Default registered protocol. It reads commands using the HTTP protocol
and write data locally -->
<protocol name="binary"
implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary" />
<protocol name="http"
implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb" />
</protocols>
<listeners>
<listener protocol="binary" ip-address="0.0.0.0" port-range="2424-2430"
socket="default" />
<!-- <listener protocol="binary" ip-address="0.0.0.0" port-range="2434-2440"
socket="ssl"/> -->
<listener protocol="http" ip-address="0.0.0.0" port-range="2480-2490"
socket="default">
<parameters>
<!-- Connection's custom parameters. If not specified the global configuration
will be taken -->
<parameter name="network.http.charset" value="utf-8" />
<parameter value="true" name="network.http.jsonResponseError" />
<!-- Define additional HTTP headers to always send as response -->
<!-- Allow cross-site scripting -->
<!-- parameter name="network.http.additionalResponseHeaders" value="Access-Control-Allow-Origin:
*;Access-Control-Allow-Credentials: true" / -->
</parameters>
<commands>
<command
pattern="GET|www GET|studio/ GET| GET|*.htm GET|*.html GET|*.xml GET|*.jpeg GET|*.jpg GET|*.png GET|*.gif GET|*.js GET|*.css GET|*.swf GET|*.ico GET|*.txt GET|*.otf GET|*.pjs GET|*.svg GET|*.json GET|*.woff GET|*.woff2 GET|*.ttf GET|*.svgz"
implementation="com.orientechnologies.orient.server.network.protocol.http.command.get.OServerCommandGetStaticContent">
<parameters>
<!-- Don't cache html resources in development mode -->
<entry name="http.cache:*.htm *.html"
value="Cache-Control: no-cache, no-store, max-age=0, must-revalidate\r\nPragma: no-cache" />
<!-- Default caching -->
<entry name="http.cache:default" value="Cache-Control: max-age=120" />
</parameters>
</command>
<command pattern="GET|gephi/*"
implementation="com.orientechnologies.orient.graph.server.command.OServerCommandGetGephi" />
</commands>
</listener>
</listeners>
<cluster>
</cluster>
</network>
<!-- <storages> <storage name="db_%NODENAME%" path="%DB_PATH%" userName="admin"
userPassword="admin" loaded-at-startup="true" /> </storages> -->
<users>
<user name="root" password="finger" resources="*" />
<user name="admin" password="finger" resources="*" />
</users>
<properties>
<!-- DATABASE POOL: size min/max -->
<entry name="db.pool.min" value="1" />
<entry name="db.pool.max" value="50" />
<!-- PROFILER: configures the profiler as <seconds-for-snapshot>,<archive-snapshot-size>,<summary-size> -->
<entry name="profiler.enabled" value="true" />
<!-- <entry name="profiler.config" value="30,10,10" /> -->
<entry name="plugin.directory" value="%PLUGIN_DIRECTORY%" />
<!-- LOG: enable/Disable logging. Levels are: finer, fine, finest, info,
warning -->
<entry name="log.console.level" value="%CONSOLE_LOG_LEVEL%" />
<entry name="log.file.level" value="%FILE_LOG_LEVEL%" />
</properties>
</orient-server>
default-distributed-db-config.json:
{
"autoDeploy": true,
"hotAlignment": true,
"executionMode": "synchronous",
"readQuorum": 1,
"writeQuorum": 1,
"failureAvailableNodesLessQuorum": false,
"readYourWrites": true,
"servers": {
"*": "master"
},
"clusters": {
"internal": {
},
"index": {
},
"*": {
"servers": ["<NEW_NODE>"]
}
}
}
Start OServer:
OServer server = OServerMain.create();
server.startup(getOrientServerConfig());
server.activate();
Wait until both nodes have been started.
Each node connects to the graph database.
OrientGraphFactory factory = new OrientGraphFactory("plocal:" + new File("databases/db_testdb").getAbsolutePath());
NodeA will add new vertices using getNoTx.
NodeB will just read the graph and count the found vertices using getNoTx.
I created a very basic maven project which contains two tests that will start nodeA and nodeB.
It is mandatory to set the ORIENTDB_HOME property for each node. The property must be set that way so that the $ORIENTDB_HOME/databases folder can be located. I have updated the maven project.
Each node can set the property for example this way:
String orientdbHome = new File("").getAbsolutePath();
System.setProperty("ORIENTDB_HOME", orientdbHome);

StackOverflow Error when opening Document DB with distribution active

I am trying to get Orient DB running embedded and distributed. Right now I am facing a StackOverflowError whenever I open my local DB if the hazelcast plugin is active. When I turn the hazelcast plugin off, everything is fine.
OrientDB is getting stuck in this loop:
at com.orientechnologies.orient.server.distributed.ODistributedAbstractPlugin.onOpen(ODistributedAbstractPlugin.java:141)
at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:107)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:118)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
at com.orientechnologies.orient.server.OServer.openDatabase(OServer.java:657)
at com.orientechnologies.orient.server.distributed.conflict.ODefaultReplicationConflictResolver.startup(ODefaultReplicationConflictResolver.java:74)
at com.orientechnologies.orient.server.distributed.OStorageSynchronizer.<init>(OStorageSynchronizer.java:56)
at com.orientechnologies.orient.server.distributed.ODistributedAbstractPlugin.getDatabaseSynchronizer(ODistributedAbstractPlugin.java:197)
at com.orientechnologies.orient.server.distributed.ODistributedAbstractPlugin.onOpen(ODistributedAbstractPlugin.java:141)
at com.orientechnologies.orient.core.db.raw.ODatabaseRaw.open(ODatabaseRaw.java:107)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
at com.orientechnologies.orient.core.db.record.ODatabaseRecordAbstract.open(ODatabaseRecordAbstract.java:118)
at com.orientechnologies.orient.core.db.ODatabaseWrapperAbstract.open(ODatabaseWrapperAbstract.java:49)
at com.orientechnologies.orient.server.OServer.openDatabase(OServer.java:657)
at com.orientechnologies.orient.server.distributed.conflict.ODefaultReplicationConflictResolver.startup(ODefaultReplicationConflictResolver.java:74)
at com.orientechnologies.orient.server.distributed.OStorageSynchronizer.<init>(OStorageSynchronizer.java:56)
at com.orientechnologies.orient.server.distributed.ODistributedAbstractPlugin.getDatabaseSynchronizer(ODistributedAbstractPlugin.java:197)
at com.orientechnologies.orient.server.distributed.ODistributedAbstractPlugin.onOpen(ODistributedAbstractPlugin.java:141)
The ODatabaseRaw.open has a list of listeners, when Hazelcast is active the plugin is registered as a listener, and will be called everytime the method runs.
Anyone has any ideas?
Edit:
I kind of got it to work, I had taken the distributed configuration from github. When I remove the following line it works:
<parameter name="conflict.resolver.impl" value="com.orientechnologies.orient.server.distributed.conflict.ODefaultReplicationConflictResolver" />
I am not really sure if this is such a good solution though, so feel free to point out better solutions =)
My complete configuration looks like this:
db.config:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<orient-server>
<network>
<protocols>
<protocol name="binary" implementation="com.orientechnologies.orient.server.network.protocol.binary.ONetworkProtocolBinary"/>
<protocol name="http" implementation="com.orientechnologies.orient.server.network.protocol.http.ONetworkProtocolHttpDb"/>
</protocols>
<listeners>
<listener ip-address="0.0.0.0" port-range="2480-2490" protocol="http"/>
</listeners>
</network>
<users>
<user name="root" password="ThisIsA_TEST" resources="*"/>
</users>
<properties>
<entry name="server.cache.staticResources" value="false"/>
<entry name="server.database.path" value="."/>
<entry name="log.console.level" value="info"/> <entry name="log.file.level" value="fine"/>
</properties>
<handlers>
<handler class="com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin">
<parameters>
<!-- <parameter name="alias" value="europe1" /> -->
<parameter name="enabled" value="true" />
<parameter name="configuration.db.default" value="src/main/resources/default-distributed-db-config.json" />
<parameter name="configuration.hazelcast" value="src/main/resources/hazelcast.xml" />
<parameter name="alignment.startup" value="true" />
<parameter name="alignment.timer" value="120000" />
</parameters>
</handler>
</handlers>
</orient-server>
default-distributed-db-config.json (there is a '}' missing in the example on github):
{
"synchronization" : true,
"clusters" : {
"internal" : { "synchronization" : false },
"ODistributedConflict" : { "synchronization" : false },
"*" : {
"synchronization" : true,
"master" : "$auto",
"synch-replicas" : 0,
"asynch-replicas" : "100%" },
"index" : { "synchronization" : false }
}
}
hazelcast.xml
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<hazelcast xsi:schemaLocation="http://www.hazelcast.com/schema/config hazelcast-basic.xsd"
xmlns="http://www.hazelcast.com/schema/config" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<group>
<name>orientdb</name>
<password>orientdb</password>
</group>
<network>
<port auto-increment="true">5701</port>
<join>
<multicast enabled="false">
<multicast-group>224.2.2.3</multicast-group>
<multicast-port>54327</multicast-port>
</multicast>
<tcp-ip enabled="true">
<interface>127.0.0.1</interface>
</tcp-ip>
</join>
</network>
</hazelcast>
edit2:
Getting another StackOverflowError when two instances try to sync:
java.lang.StackOverflowError
-> com.hazelcast.core.DistributedTask.get(DistributedTask.java:126)
-> com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin$1.done(OHazelcastPlugin.java:204)
-> com.hazelcast.impl.ExecutorManager$3.run(ExecutorManager.java:355)
-> com.hazelcast.impl.executor.ParallelExecutorService$ParallelExecutorImpl$ExecutionSegment.run(ParallelExecutorService.java:212)
-> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
-> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
-> java.lang.Thread.run(Thread.java:722)
-> com.hazelcast.impl.ExecutorThreadFactory$1.run(ExecutorThreadFactory.java:38)
null
-> com.hazelcast.core.DistributedTask.get(DistributedTask.java:126)
-> com.orientechnologies.orient.server.hazelcast.OHazelcastPlugin$1.done(OHazelcastPlugin.java:204)
-> com.hazelcast.impl.ExecutorManager$3.run(ExecutorManager.java:355)
-> com.hazelcast.impl.executor.ParallelExecutorService$ParallelExecutorImpl$ExecutionSegment.run(ParallelExecutorService.java:212)
-> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
-> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
-> java.lang.Thread.run(Thread.java:722)
-> com.hazelcast.impl.ExecutorThreadFactory$1.run(ExecutorThreadFactory.java:38)
Does not seem to very stable yet when running distributed and embedded. A shame really, liked it so far...