I am new to orient-db and have run into a major block even trying to open a simple in memory database.
Here is my two lines of code (in java)
OrientGraphFactory factory = new
OrientGraphFactory("memory:test").setupPool(1,10);
// EVERY TIME YOU NEED A GRAPH INSTANCE
OrientGraph g = factory.getTx();
try {
} finally {
g.shutdown();
}
I get the following error:
Exception in thread "main" com.orientechnologies.orient.core.exception.OStorageException: Cannot open local storage 'test' with mode=rw
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.open(OAbstractPaginatedStorage.java:210)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.open(ODatabaseDocumentTx.java:223)
at com.orientechnologies.orient.core.db.OPartitionedDatabasePool.acquire(OPartitionedDatabasePool.java:287)
at com.tinkerpop.blueprints.impls.orient.OrientBaseGraph.<init>(OrientBaseGraph.java:163)
at com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.<init>(OrientTransactionalGraph.java:78)
at com.tinkerpop.blueprints.impls.orient.OrientGraph.<init>(OrientGraph.java:128)
at com.tinkerpop.blueprints.impls.orient.OrientGraphFactory.getTx(OrientGraphFactory.java:74)
Caused by: com.orientechnologies.orient.core.exception.OStorageException:
Cannot open the storage 'test' because it does not exist in path: test
at
com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage .open(OAbstractPaginatedStorage.java:154)
... 7 more
What 'path' is it talking about? How is a path even relevant when trying to open a simple in memory database? Furthermore I have also tried this with plocal:/..... ,,, and I always get the above error.
Regards,
Bhargav.
Try to create the database first :
OrientGraphNoTx graph = new OrientGraphNoTx ("memory:test");
Then use the pool :
OrientGraphFactory factory = new OrientGraphFactory ("memory:test").setupPool (1, 10);
By the way which db version are you using ?
Databases created as in-memory only needs to be created first and the pool didn't allow it (fixed in last snapshot). Try acquiring an instance from the factory without pool, like:
OrientGraphFactory factory = newOrientGraphFactory("memory:test");
factory.getTx().shutdown(); // AUTO-CREATE THE GRAPH IF NOT EXISTS
factory.setupPool(1,10);
// EVERY TIME YOU NEED A GRAPH INSTANCE
OrientGraph g = factory.getTx();
try {
} finally {
g.shutdown();
}
Related
I'm using haddop-connectors
project for writing BLOBs to Google Cloud Storage.
I'd like to make sure that a BLOB with a specific target name that is being written in a concurrent context is either written in FULL or not appearing at all as visible in case that an exception has occurred while writing.
In the code below, in case that that an I/O exception occurs, the BLOB written will appear on GCS because the stream is being closed in finally:
val stream = fs.create(path, overwrite)
try {
actions.map(_ + "\n").map(_.getBytes(UTF_8)).foreach(stream.write)
} finally {
stream.close()
}
The other possibility would be to not close the stream and let it "leak" so that the BLOB does not get created. However this is not really a valid option.
val stream = fs.create(path, overwrite)
actions.map(_ + "\n").map(_.getBytes(UTF_8)).foreach(stream.write)
stream.close()
Can anybody share with me a recipe on how to write to GCS a BLOB either with hadoop-connectors or cloud storage client in an atomic fashion?
I have used reflection within hadoop-connectors to retrieve an instance of com.google.api.services.storage.Storage from the GoogleHadoopFileSystem instance
GoogleCloudStorage googleCloudStorage = ghfs.getGcsFs().getGcs();
Field gcsField = googleCloudStorage.getClass().getDeclaredField("gcs");
gcsField.setAccessible(true);
Storage gcs = (Storage) gcsField.get(googleCloudStorage);
in order to have the ability to make a call based on an input stream corresponding to the data in memory.
private static StorageObject createBlob(URI blobPath, byte[] content, GoogleHadoopFileSystem ghfs, Storage gcs)
throws IOException
{
CreateFileOptions createFileOptions = new CreateFileOptions(false);
CreateObjectOptions createObjectOptions = objectOptionsFromFileOptions(createFileOptions);
PathCodec pathCodec = ghfs.getGcsFs().getOptions().getPathCodec();
StorageResourceId storageResourceId = pathCodec.validatePathAndGetId(blobPath, false);
StorageObject object =
new StorageObject()
.setContentEncoding(createObjectOptions.getContentEncoding())
.setMetadata(encodeMetadata(createObjectOptions.getMetadata()))
.setName(storageResourceId.getObjectName());
InputStream inputStream = new ByteArrayInputStream(content, 0, content.length);
Storage.Objects.Insert insert = gcs.objects().insert(
storageResourceId.getBucketName(),
object,
new InputStreamContent(createObjectOptions.getContentType(), inputStream));
// The operation succeeds only if there are no live versions of the blob.
insert.setIfGenerationMatch(0L);
insert.getMediaHttpUploader().setDirectUploadEnabled(true);
insert.setName(storageResourceId.getObjectName());
return insert.execute();
}
/**
* Helper for converting from a Map<String, byte[]> metadata map that may be in a
* StorageObject into a Map<String, String> suitable for placement inside a
* GoogleCloudStorageItemInfo.
*/
#VisibleForTesting
static Map<String, String> encodeMetadata(Map<String, byte[]> metadata) {
return Maps.transformValues(metadata, QuickstartParallelApiWriteExample::encodeMetadataValues);
}
// A function to encode metadata map values
private static String encodeMetadataValues(byte[] bytes) {
return bytes == null ? Data.NULL_STRING : BaseEncoding.base64().encode(bytes);
}
Note in the example above, that even if there are multiple callers trying to create a blob with the same name in parallel, ONE and only ONE will succeed in creating the blob. The other callers will receive 412 Precondition Failed.
GCS objects (blobs) are immutable 1, which means they can be created, deleted or replaced, but not appended.
The Hadoop GCS connector provides the HCFS interface which gives the illusion of appendable files. But under the hood, it is just one blob creation, GCS doesn't know if the content is complete or not from the application's perspective, just as you mentioned in the example. There is no way to cancel a file creation.
There are 2 options you can consider:
Create a temp blob/file, copy it to the final blob/file, then delete the temp blob/file, see 2. Note that there is no atomic rename operation in GCS, rename is implemented as copy-then-delete.
If your data fits into the memory, first read up the stream and buffer the bytes in memory, then create the blob/file, see 3.
GCS connector should also work with the 2 options above, but I think GCS client library gives you more control.
I need to run a raw sql query, but I'm getting an error when I try to open the connection to the database. "The connection was not closed. The connection's current state is open."
_loginValidator and _contactService are passed into the controller through DI:
services.AddScoped<ILoginValidator, LoginValidator>();
services.AddScoped<IContactService, ContactService>();
The two lines below are in an action function of the controller. If I switch the two lines, the error goes away...:
var validationErrors = _loginValidator.Validate(id, "");
var user = _contactService.GetContact(id);
Here is _loginValidator.Validate. If I comment out the second line, the error goes away...:
public LoginValidationResult Validate(int userId, string encryptedPassword)
{
var vr = new LoginValidationResult();
var user = _context.Users.Include(u => u.LoginUserQuestionAnswers).FirstOrDefault(u => u.Id == userId);
//...
}
Here is _contactService.GetContact. This is where I get the error:
public ContactDto GetContact(int id)
{
var conn = _context.Database.GetDbConnection();
//ERROR HERE!!!
conn.Open();
//work on conn, for example: ExecuteReader
conn.Close();
}
Notes:
If I comment out the _context line in the Validate(...) function, I do not get the error.
If I switch the two lines I listed in the action function, I do not get the error.
I think the problem is that EntityCore is not closing the connection after I finish using it in _loginValidator.Validate(...)
Anyone know how I can deal with this problem?
DB Connection is an unmanaged resource and you need to close it yourself. The best practice is to use a using statement for your DB connections.
See these links:
http://stackoverflow.com/questions/35077000/entity-framework-7-get-database-time
https://msdn.microsoft.com/en-us/data/dn456849.aspx
The connection being left open after the FirstOrDefault query is a bug. I filed https://github.com/aspnet/EntityFramework/issues/6581 for it and we just triaged it for the 1.0.2 release.
To workaround the bug for now I think you can check if the connection is already open and, if so, don't try to open it again.
I'm having a problem with lots of connections being opened to the mongo db.
The readme on the Github page for the C# driver gives the following code:
using MongoDB.Bson;
using MongoDB.Driver;
var client = new MongoClient("mongodb://localhost:27017");
var server = client.GetServer();
var database = server.GetDatabase("foo");
var collection = database.GetCollection("bar");
collection.Insert(new BsonDocument("Name", "Jack"));
foreach(var document in collection.FindAll())
{
Console.WriteLine(document["Name"]);
}
At what point does the driver open the connection to the server? Is it at the GetServer() method or is it the Insert() method?
I know that we should have a static object for the client, but should we also have a static object for the server and database as well?
Late answer... but the server connection is created at this point:
var client = new MongoClient("mongodb://localhost:27017");
Everything else is just getting references for various objects.
See: http://docs.mongodb.org/ecosystem/tutorial/getting-started-with-csharp-driver/
While using the latest MongoDB drivers for C#, the connection happens at the actual database operation. For eg. db.Collection.Find() or at db.collection.InsertOne().
{
//code for initialization
//for localhost connection there is no need to specify the db server url and port.
var client = new MongoClient("mongodb://localhost:27017/");
var db = client.GetDatabase("TestDb");
Collection = db.GetCollection<T>("testCollection");
}
//Code for db operations
{
//The connection happens here.
var collection = db.Collection;
//Your find operation
var model = collection.Find(Builders<Model>.Filter.Empty).ToList();
//Your insert operation
collection.InsertOne(Model);
}
I found this out after I stopped my mongod server and debugged the code with breakpoint. Initialization happened smoothly but error was thrown at db operation.
Hope this helps.
I am trying to load my RDF model into JENA SDB. I have done the connection (and I think it should be fine).
I do not know why but it does not work right. I think it is because of the ARQ library which I have imported into my project but I do not know.
Here the code :
String RDF_FILE = "./prova_rules_M_rdf.owl";
String className = "com.mysql.jdbc.Driver";
String DB_URL = "jdbc:mysql://localhost:3306/prova_rules";
String DB_USER = "root";
String DB_PASSWD = "";
// create store description
StoreDesc storeDesc = new StoreDesc(LayoutType.LayoutTripleNodesHash,DatabaseType.MySQL);
// load database driver
try {
Class.forName(className);
System.out.println("JDBC driver load successfully!");
} catch (Exception e) {
e.printStackTrace();
}
// create SDBConnection
SDBConnection sdbConnection = new SDBConnection(DB_URL,DB_USER,DB_PASSWD);
// connect to store
Store store = SDBFactory.connectStore(sdbConnection,storeDesc);
// connect store to dataset
Dataset dataset = SDBFactory.connectDataset(store);
// prepare the model
Model tmpModel = ModelFactory.createDefaultModel();
FileManager.get().readModel(tmpModel,RDF_FILE,"OWL");
System.out.println(tmpModel.size());
// add the model into the dataset
dataset.getDefaultModel().add(tmpModel);
// all done ... hopefully
store.close();
}
Here the error:
Exception in thread "main" java.lang.NoSuchMethodError: com.hp.hpl.jena.query.ARQ.getContext()Lcom/hp/hpl/jena/sparql/util/Context;
at com.hp.hpl.jena.sdb.SDB.initWorker(SDB.java:87)
at com.hp.hpl.jena.sdb.SDB.(SDB.java:61)
at com.hp.hpl.jena.sdb.store.StoreFactory.(StoreFactory.java:40)
at com.hp.hpl.jena.sdb.SDBFactory.connectStore(SDBFactory.java:117)
at CreateOntModel.main(CreateOntModel.java:94)
NoSuchMethodError indicates that you have an inconsistent set of jars on the runtime classpath (Eclipse build path). Make sure you have only one copy of each Jena jar on the project build path as well.
Using maven with Eclipse will manage the dependencies automatically.
We have two different query strategies that we'd ideally like to operate in conjunction on our site without opening redundant connections. One strategy uses the enterprise library to pull Database objects and Execute_____(DbCommand)s on the Database, without directly selecting any sort of connection. Effectively like this:
Database db = DatabaseFactory.CreateDatabase();
DbCommand q = db.GetStoredProcCommand("SomeProc");
using (IDataReader r = db.ExecuteReader(q))
{
List<RecordType> rv = new List<RecordType>();
while (r.Read())
{
rv.Add(RecordType.CreateFromReader(r));
}
return rv;
}
The other, newer strategy, uses a library that asks for an IDbConnection, which it Close()es immediately after execution. So, we do something like this:
DbConnection c = DatabaseFactory.CreateDatabase().CreateConnection();
using (QueryBuilder qb = new QueryBuilder(c))
{
return qb.Find<RecordType>(ConditionCollection);
}
But, the connection returned by CreateConnection() isn't the same one used by the Database.ExecuteReader(), which is apparently left open between queries. So, when we call a data access method using the new strategy after one using the old strategy inside a TransactionScope, it causes unnecessary promotion -- promotion that I'm not sure we have the ability to configure for (we don't have administrative access to the SQL Server).
Before we go down the path of modifying the query-builder-library to work with the Enterprise Library's Database objects ... Is there a way to retrieve, if existent, the open connection last used by one of the Database.Execute_______() methods?
Yes, you can get the connection associated with a transaction. Enterprise Library internally manages a collection of transactions and the associated database connections so if you are in a transaction you can retrieve the connection associated with a database using the static TransactionScopeConnections.GetConnection method:
using (var scope = new TransactionScope())
{
IEnumerable<RecordType> records = GetRecordTypes();
Database db = DatabaseFactory.CreateDatabase();
DbConnection connection = TransactionScopeConnections.GetConnection(db).Connection;
}
public static IEnumerable<RecordType> GetRecordTypes()
{
Database db = DatabaseFactory.CreateDatabase();
DbCommand q = db.GetStoredProcCommand("GetLogEntries");
using (IDataReader r = db.ExecuteReader(q))
{
List<RecordType> rv = new List<RecordType>();
while (r.Read())
{
rv.Add(RecordType.CreateFromReader(r));
}
return rv;
}
}