Defining Query for OrientDB - using JSON data - orientdb

I have a question concerning using orientDB for JSON data. In my scenario,
I receive entities which are serialized into JSON. This JSON data should
be stored in orientDB. The relevant code part to create documents
in orientDB looks as follows:
//JSON string created by using Jackson
String jsonStr = "...";
//here some dummy name "test"
ODocument doc = new ODocument("test");
doc.fromJSON(jsonStr);
doc.save();
In the following, I give an example for classes I'm working with
(I left out constructors, getters and setters and other fields which
are not relevant for the example):
class AbstractEntity {
private String oidString;
}
class A extends AbstractEntity {
private List<B> bs;
}
class B extends AbstractEntity {
private List<C> cs;
}
class C extends AbstractEntity {
private List<D> ds;
}
class D extends AbstractEntity {
private int type;
}
As the classes use type List, Jackson needs to store
additional type information in the JSON representation, to be able
to deserialize the data properly.
{
"oidString" : "AAA_oid1",
"bs" : [ "java.util.ArrayList", [ {
"oidString" : "b_oid1",
"cs" : null
}, {
"oidString" : "b_oid2",
"cs" : [ "java.util.ArrayList", [ {
"oidString" : "c_oid1",
"ds" : [ "java.util.ArrayList", [ ] ]
}, {
"oidString" : "c_oid2",
"ds" : [ "java.util.ArrayList", [ {
"oidString" : "d_oid1",
"type" : 2
} ] ]
} ] ]
} ] ]
}
However, I have problems querying such a document if I try to e.g. find all D instances that contain a certain type. I tried to simplify my query by first listing all D instances that can be found for a specific A:
OQuery<?> query = new OSQLSynchQuery<ODocument>(
"select bs.cs.ds from test where oidString = 'AAA_oid1'"
);
This returns: {"#type":"d","#rid":"#-2:0","#version":0,"bs":[null,null]}
The additional type information ("java.util.ArrayList") seems to cause problems for orientDB. If I rewrite my example and only use ArrayList directly, hence, the additional type information is omitted, the query above shows something as a result.
Is there a general solution to this problem? I have to work with JSON data and that JSON data will contain additional type information (it has to).
Can't deal orientDB with this situation?

Related

AppSync: pipeline resolver #return null result

I'm successfully using a pipeline resolver to persist a parent/child relationship, except when the list of child items is empty and I #return early.
I'm guessing the issue is around my response mappers and use of $ctx.prev vs $ctx.result but I can't figure it out.
The pipeline looks like this:
BEFORE template: {}
Function 1:
request = PutItem the parent
response = $utils.toJson($ctx.result)
Function 2:
request = TransactWriteItems (foreach UpdateItem) the children
response = $utils.toJson($ctx.prev.result)
AFTER template: $utils.toJson($ctx.prev.result)
When I call the mutation with
{"parentAttribute":"foo", "children": [{"childAttribute": "bar"}]}
I get a good response like:
{
"data": {
"createFoo": {
"parentAttribute": "foo",
"children": [
{
"childAttribute": "bar"
}
]
}
}
}
If no children, Function 2 request mapper does #return to avoid "TransactWriteItems must have at least one operation" error.
In this scenario I am hoping for the above response to the mutation, just with children: []
Instead, I get:
{
"data": {
"createFoo": null
}
}
The data has been written correctly; if I query it I get back the parent with empty list of children.
How do I get this pipeline to execute so that it returns the combined parent+child data whether the child array is populated or not?
Detail
The schema is something like:
type Foo {
id: String!
attr1: String
bars: [Bar]
}
type Bar {
id: String!
attr2: String
}
type Mutation {
createFoo(foo: Foo): Foo
}
And a dynamodb representation like this:
pk
sk
attr1
attr2
FOO#1
METADATA#FOO#1
Lorem
FOO#1
BAR#1
Ipsum
While the pipeline looks like:
before.vtl
{}
createParent-request.vtl
{
"version" : "2017-02-28",
"operation" : "PutItem",
"key" : {
"pk" : $util.dynamodb.toDynamoDBJson(...),
"sk" : $util.dynamodb.toDynamoDBJson(...)
},
"attributeValues" : {
"data" : $util.dynamodb.toDynamoDBJson(...)
}
}
createParent-response.vtl
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
$utils.toJson($ctx.result)
createChildren-request.vtl
#if($ctx.args.fooInput.children.size() > 0)
{
"version": "2018-05-29",
"operation": "TransactWriteItems",
"transactItems": [
#foreach( $child in $ctx.args.fooInput.children )
{
"table": "${table}",
"operation": "UpdateItem",
"key": {
"pk" : $util.dynamodb.toDynamoDBJson(...),
"sk" : $util.dynamodb.toDynamoDBJson(...)
},
"update": {
"expression": "SET #data = :data",
"expressionNames": {
"#data": "data"
},
"expressionValues": {
":data":
$util.dynamodb.toDynamoDBJson(...)
}
}
}
#if( $foreach.hasNext ),#end
#end
]
}
#else
#return
#end
createChildren-response.vtl
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
$utils.toJson($ctx.prev.result)
after.vtl
#if($ctx.error)
$utils.error($ctx.error.message, $ctx.error.type)
#end
$utils.toJson($ctx.prev.result)
I figured it out. For the expected behaviour, one needs the 'after' mapper to return the necessary JSON to populate the overall mutation response. In my example above, after.vtl needs to return a parent and nothing else matters (in particular, the result of the individual function response mappers).
I ended up putting the output of the 'create parent' operation into ctx.stash then returning ctx.stash in after.vtl, setting the other resolvers to {}.
Note that, if your response has subtypes (with their own resolvers) and you return it sparse, AppSync will call the resolver. In the context of my example, it's enough to return the parent without any children and then the normal query resolver for "get children of a parent" will execute to populate the final response.

How MongoClient::save(...) might change the _id field of document parameter

I have a class User that embeds a JsonObject to represent the user's fields. This class looks like that:
class User {
private JsonObject data;
public User(...) {
data = new JsonObject();
data.put("...", ...).put(..., ...);
}
public String getID() { return data.getString("_id"); }
// more getters, setters
// DB access methods
public static userSave(MongoClient mc, User user){
// some house keeping
mc.save("users", user.jsonObject(), ar -> {
if(ar.succeeded()) { ... } else { ... }
});
}
}
I've just spent more than half a day trying to figure out why a call to user.getID() sometimes produced the following error: ClassCastException: class io.vertx.core.json.JsonObject cannot be cast to class java.lang.CharSequence. I narrowed down to the userSave() method and more specifically to MongoClient::save() which actually produces a side effect which transforms the data._id from something like
"_id" : "5ceb8ebb9790855fad9be2fc"
into something like
"_id" : {
"$oid" : "5ceb8ebb9790855fad9be2fc"
}
This is confirmed by the vertx documentation which states that "This operation might change _id field of document parameter". This actually is also true for other write methods like inserts.
I came with two solutions and few questions about doing the save() properly while keeping the _id field up to date.
S1 One way to achieve that is to save a copy of the Json Object rather than the object itself, in other words : mc.save("users", user.jsonObject().copy(), ar -> {...});. This might be expensive on the long run.
S2 An other way is to "remember" _id and then to reinsert it into the data object in the if(ar.succeeded()) {data.put("_id", oidValue); ...} section. But as we are asynchronous, I don't think that the interval between save() and the data.put(...) is atomic ?
Q1: Solution S1 make the assumption that the ID doesn't change, i.e., the string 5ceb8ebb9790855fad9be2fc will not change. Do we have a warranty about this ?
Q2: What is the right way to implement the saveUser() properly ?
EDIT: The configuration JSON object user for the creation of the MongoClient is as follows (in case there is something wrong) :
"main_pool" : {
"pool_name" : "mongodb",
"host" : "localhost",
"port" : 27017,
"db_name" : "appdb",
"username" : "xxxxxxxxx",
"password" : "xxxxxxxxx",
"authSource" : "admin",
"maxPoolSize" : 5,
"minPoolSize" : 1,
"useObjectId" : true,
"connectTimeoutMS" : 5000,
"socketTimeoutMS" : 5000,
"serverSelectionTimeoutMS" : 5000
}

how to retrieve and inject sections from and to JSON

If I have an incoming JSON of following structure
[
{
"personId" : 12,
"name": "John Doe",
"age": 48,
"birthdate": "12/1/1954",
"relationships": [
{
"relationType":"parentOf",
"value" : "Johnny walker"
},
{
"relationType":"sonOf",
"value" : "Charles Smith"
}
]
},
{
"personId" : 13,
"name": "Merry Christmas",
"age": 28,
"birthdate": "12/1/1985",
"relationships": [
{
"relationType":"sisteOf",
"value" : "Will Smith"
},
{
"relationType":"cousinOf",
"value" : "Brad Pitt"
}
]
}
]
And requirement is that for each Person record controller will have to carve out relationships array and store each record from it in a separate relationship table with personId association while persisting this incoming JSON.
And subsequently when querying these persons records system will have to lookup relationships for each person from relationships table and inject them to form the same above looking JSON to give back to UI for rendering.
What's the best efficient way to perform this "carve out" and later "inject" operations using Play framework in Scala? (using Slick in persistent layer APIs) I have looked at this JSON transformation link and json.pickBranch in there but not quite sure if that'll be fully applicable here for "carve out" and "inject" use cases for preparing JSON shown in the example. are there any elegant ideas?
One way, which is pretty straightforward, is to use case classes along with Play Json inceptions
import play.api.libs.json.Json
case class Relationship(relationType: String, value: String)
object Relationship {
implicit val RelationshipFormatter = Json.format[Relationship]
}
case class Person(personId: String, name: String, age: Int, birthdate: String, relationships: Seq[Relationship]) {
def withRelationships(relationship: Seq[Relationship]) = copy(relationships = relationships ++ relationship)
}
object Person {
implicit val PersonFormatter = Json.format[Person]
}
Now you can convert a json value to Person by using the following code, provided that jsValue is a json value of type JsValue (which in play controllers you can get by request.body.asJson):
Json.fromJson[Person](jsValue)
In Play controllers, you can
For converting a Person to json you can use the following code provided that person is a value of type Person in your context:
Json.toJson(person)
The only remaining thing is your Slick schemas which is pretty straight forward.
One option is to use a simple schema for Person, without relations and one schema for Relation with a foreign key to Person table. Then you need to find all relations associated with a specific Person, namely person and then append them to that person by calling the withRelationships method which gives you a new Person which you can serve as json:
val person = .... // find person
val relations = .... // find relationships associated with this person
val json = Json.toJson(person.withRelationships(relations))

GORM get/find resource by ID using MongoDB in Grails

Grails makes it easy to get a domain object by ID (handy for building a REST API).
A controller to retrieve a resource can be as simple as:
MetricController.groovy
import grails.converters.JSON
class MetricController {
def index() {
def resource = Metric.get(params.id)
render resource as JSON
}
}
When using the Grails plugin for MongoDB GORM (compile ":mongodb:1.2.0"), the default id type of Long needs to be changed to type String or ObjectId.
Metric.groovy
import org.bson.types.ObjectId
class Metric {
static mapWith = "mongo"
ObjectId id
String title
}
However, doing a .get(1) will now result in:
Error 500: Internal Server Error
URI
/bow/rest/metric/1
Class
java.lang.IllegalArgumentException
Message
invalid ObjectId [1]
I took a guess and changed the controller to use findById:
def resource = Metric.findById(new ObjectId(new Date(), params.id.toInteger()))
That fixed the error, but it fails to find the object (always returns null).
For example, using the id of "-1387348672" does not find this test object:
{ "class" : "Metric",
"id" : { "class" : "org.bson.types.ObjectId",
"inc" : -1387348672,
"machine" : 805582250,
"new" : false,
"time" : 1371329632000,
"timeSecond" : 1371329632
},
"title" : "Test"
}
The ObjectId.inc field may not even be the correct field to use for the resource ID.
So, what is the simplest way to retrieve a domain object by ID when using MongoDB?
When a domain object is persisted in MongoDB, it is stored as a document with an ObjectId as a unique 12 bytes BSON primary key. For example, if you have a domain object Product like
import org.bson.types.ObjectId
class Product {
ObjectId id
String name
static mapWith = "mongo"
}
then the persisted entity in MongoDB would look like below if you save with a name as "TestProduct".
//db.product.find() in mongodb
{
"_id" : ObjectId("51bd047c892c8bf0b3a58b21"),
"name" : "TestProduct",
"version" : 0
}
The _id becomes your primary key for that document. To get this document you need the primary key ObjectId. Speaking from a RESTful context, you atleast need the 12 bytes hexcode 51bd047c892c8bf0b3a58b21 as part of the request.
So in the above case, you can fetch that particular document by doing something like
Product.get(new ObjectId("51bd047c892c8bf0b3a58b21"))
Product.findById(new ObjectId("51bd047c892c8bf0b3a58b21"))
Have a look at the API for ObjectId which would make clear how to retrieve a document.
When you retrieve the document as JSON, then it just shows the ObjectId class with its elements.
{
"class": "com.example.Product",
"id": {
"class": "org.bson.types.ObjectId",
"inc": -1280996575,
"machine": -1993569296,
"new": false,
"time": 1371341948000,
"timeSecond": 1371341948
},
"name": "TestProduct"
}
For completeness, here's the domain with a controller to get a resource by ID string (instead of ObjectID).
Example:
Metric.get("51bf8ccc30040460f5a05579")
Domain
import org.bson.types.ObjectId
class Metric {
ObjectId id
String title
static mapWith = "mongo"
def out() {
return [
id: id as String, //convert Mongo ObjectId to 12-byte hex BSON
title: title
]
}
}
The out() method is used to render the resource showing its ID string (not its ObjectID).
Controller
import grails.converters.JSON
class MetricController {
def index() {
def resource = Metric.read(params.id)
render resource.out() as JSON
}
}
Example JSON response for /rest/metric/51bf8ccc30040460f5a05579
{ "id": "51bf8ccc30040460f5a05579", "title": "Accounts" }

Nested document insert into MongoDB with C#

I am trying to insert nested documents in to a MongoDB using C#. I have a collection called categories. In that collection there must exist documents with 2 array, one named categories and one named standards. Inside those arrays must exist new documents with their own ID's that also contain arrays of the same names listed above. Below is what I have so far but I am unsure how to proceed. If you look at the code what I want to do is add the "namingConventions" document nested under the categories array in the categories document however namingConventions must have a unique ID also.
At this point I am not sure I have done any of this the best way possible so I am open to any and all advice on this entire thing.
namespace ClassLibrary1
{
using MongoDB.Bson;
using MongoDB.Driver;
public class Class1
{
public void test()
{
string connectionString = "mongodb://localhost";
MongoServer server = MongoServer.Create(connectionString);
MongoDatabase standards = server.GetDatabase("Standards");
MongoCollection<BsonDocument> categories = standards.GetCollection<BsonDocument>("catagories");
BsonDocument[] batch = {
new BsonDocument { { "categories", new BsonArray {} },
{ "standards", new BsonArray { } } },
new BsonDocument { { "catagories", new BsonArray { } },
{ "standards", new BsonArray { } } },
};
categories.InsertBatch(batch);
((BsonArray)batch[0]["categories"]).Add(batch[1]);
categories.Save(batch[0]);
}
}
}
For clarity this is what I need:
What I am doing is building a coding standards site. The company wants all the standards stored in MongoDB in a tree. Everything must have a unique ID so that on top of being queried as a tree it can be queried by itself also. An example could be:
/* 0 */
{
"_id" : ObjectId("4fb39795b74861183c713807"),
"catagories" : [],
"standards" : []
}
/* 1 */
{
"_id" : ObjectId("4fb39795b74861183c713806"),
"categories" : [{
"_id" : ObjectId("4fb39795b74861183c713807"),
"catagories" : [],
"standards" : []
}],
"standards" : []
}
Now I have written code to make this happen but the issue seems to be that when I add object "0" to the categories array in object "1" it is not making a reference but instead copying it. This will not due because if changes are made they will be made to the original object "0" so they will not be pushed to the copy being made in the categories array, at least that is what is happening to me. I hope this clears up what I am looking for.
So, based on your latest comment, it seems as though this is the actual structure you are looking for:
{
_id: ObjectId(),
name: "NamingConventions",
categories: [
{
id: ObjectId(),
name: "Namespaces",
standards: [
{
id: ObjectId(),
name: "TitleCased",
description: "Namespaces must be Title Cased."
},
{
id: ObjectId().
name: "NoAbbreviations",
description: "Namespaces must not use abbreviations."
}
]
},
{
id: ObjectId(),
name: "Variables",
standards: [
{
id: ObjectId(),
name: "CamelCased",
description: "variables must be camel cased."
}
]
}
]
}
Assuming this is correct, then the below is how you would insert one of these:
var collection = db.GetCollection("some collection name");
var root = new BsonDocument();
root.Add("name", "NamingConventions");
var rootCategories = new BsonArray();
rootCategories.Add(new BsonDocument
{
{ "id": ObjectId.GenerateNewId() },
{ "name", "Namespaces" },
{ "standards", new BsonArray() }
});
root.Add("categories", rootCategories);
//etc...
collection.Save(root);
Hope that helps, if not, I give up :).
So, I guess I'm confused by what you are asking. If you just want to store the namingConventions documents inside the array, you don't need a collection for them. Instead, just add them to the bson array and store them.
var categoriesCollection = db.GetCollection<BsonDocument>("categories");
var category = new BsonDocument();
var namingConventions = new BsonArray();
namingConventions.Add(new BsonDocument("convention1", "value"));
category.Add("naming_conventions", namingConventions);
categoriesCollection.Insert(category);
This will create a new document for a category, create an array in it called naming_conventions with a single document in it with an element called "convention1" and a value of "value".
I also am not quite sure what you are trying to accomplish. Perhaps if you posted some sample documents in JSON format we could show you the C# code to write documents that match that.
Alternatively, if you wish to discuss your schema, that could also be better done in the context of JSON rather than C#, and once a schema has been settled on then we can discuss how to write documents to that schema in C#.
One thing that didn't sound right in your original description was the statement "in that collection must exist 2 arrays". A collection can only contain documents, not arrays. The documents themselves can contain arrays if you want.
var filter = Builders<CollectionDefination>.Filter.Where(r => r._id== id);
var data = Builders<CollectionDefination>.Update.Push(f=>
f.categories,categoriesObject);
await _dbService.collection.UpdateOneAsync( filter,data);
Note: Make sure embedded document type [Categories] is array.