Retrieving Mongo Documents based on nested objectIds by passing list of ids - mongodb

Question

I have two collections in MongoDB, Parent an Child. Below is the structure of the document.
Parent: {
'_id': 'some_value',
'name': 'some_value',
'child': {
'_id': 'some_value',
'name': 'some_value',
'key':'value'
}
}
I am trying pass list of child Ids in MongoRepository method in order to retrieve Parent objects but getting null values. Below is my code.
import org.bson.types.ObjectId;
class MyRepository extends CrudRepository<Parent,Long> {
#Query("{'child._id': {$in : ?0 }}")
List<Parent> findByChild_IdIn(List<ObjectId> childIds);
}
I am invoking my method as shown below.
import org.bson.types.ObjectId;
List<String> childrenIds = getChildrenIdList();
List<ObjectId> childIds = childrenIds.stream().map(childrenId -> new ObjectId(childrenId)).collect(Collectors.toList());
List<Parent> parentsList = myRepository.findByChild_IdIn(childIds);
What am I doing wrong here? Why it is giving null values.

Answer 1

TL;DR
Annotate the id field in the inner document with #MongoId.
Longer answer
As mentioned in this answer, you would usually not require an _id field in a child document:
The _id field is a required field of the parent document, and is typically not necessary or present in embedded documents. If you require a unique identifier, you can certainly create them, and you may use the _id field to store them if that is convenient for your code or your mental model; more typically, they are named after what they represent (e.g. "username", "otherSystemKey", etc). Neither MongoDB itself, nor any of the drivers will automatically populate an _id field except on the top-level document.
In other words, unlike in RDMBS "normalized" schemas, here you would not require the children documents to have a unique id.
You may still want though that your children documents contain some kind of "id" field that refers to a document in another collection (eg "GoodBoys").
But indeed, for whatever reason, you may require a "unique" field in those inner children documents.
The following model would support the proposed structure:
#Data
#Builder
#Document(collection = "parents")
public class Parent {
#Id
private String id;
private String name;
private Child child;
#Data
#Builder
public static class Child {
#MongoId
private String id;
private String name;
private String key;
}
}
You can retrieve parents by list of ids with either of:
public interface ParentsRepository extends MongoRepository<Parent, String> {
// QueryDSL
List<Parent> findByChildIdIn(List<String> ids);
// Native Query
#Query("{'child._id': {$in: ?0}}")
List<Parent> findByChildrenIds(List<String> ids);
}
Add a few parents with something like:
parentsRepository.saveAll(Arrays.asList(
Parent.builder()
.name("parent1")
.child(Parent.Child.builder().id(new ObjectId().toString()).name("child1").key("value1").build())
.build(),
Parent.builder()
.name("parent2")
.child(Parent.Child.builder().id(new ObjectId().toString()).name("child2").key("value2").build())
.build()
));
The resulting entries in MongoDB will look like this:
{
"_id" : ObjectId("5e07384596d9077ccae89a8c"),
"name" : "parent1",
"child" : {
"_id" : "5e07384596d9077ccae89a8a",
"name" : "child1",
"key" : "value1"
},
"_class" : "com.lubumbax.mongoids.model.Parent"
}
Note two important things here:
The parent id in Mongo is an actual ObjectId that is de facto unique.
The child id in Mongo is a string that we can assume as being unique.
This approach works fine for this case in which we either have inserted the children ids from Java to MongoDB with new ObjectId().toString() or in whatever other way, as long as the resulting children id is just a string in MongoDB.
That means that the children ids are not strictly said an ObjectId in MongoDB.
#Id in the Child
If we annotate the children id field with #MongoId, the resulting query will be something like:
StringBasedMongoQuery: Created query Document{{child._id=Document{{$in=[5e0740e41095314a3401e49c, 5e0740e41095314a3401e49d]}}}} for Document{{}} fields.
MongoTemplate : find using query: { "child._id" : { "$in" : ["5e0740e41095314a3401e49c", "5e0740e41095314a3401e49d"]}} fields: Document{{}} for class: com.lubumbax.mongoids.model.Parent in collection: parents
If instead we annotate the children id field with #Id, the resulting query will be:
StringBasedMongoQuery: Created query Document{{child._id=Document{{$in=[5e0740e41095314a3401e49c, 5e0740e41095314a3401e49d]}}}} for Document{{}} fields.
MongoTemplate : find using query: { "child._id" : { "$in" : [{ "$oid" : "5e0740e41095314a3401e49c"}, { "$oid" : "5e0740e41095314a3401e49d"}]}} fields: Document{{}} for class: com.lubumbax.mongoids.model.Parent in collection: parents
Note the $oid there. The MongoDB Java driver expects the id attribute in MongoDB to be an actual ObjectId, so it tries to "cast" our string to an $oid.
The problem with that is that in MongoDB, the _id attribute of the children is simply a string and not a MongoDB's ObjectId. Thus, our repository method won't find our parent!
All ObjectId()
What happens if we insert a new document to MongoDB where the child _id is an actual ObjectId rather than a string?:
> db.parents.insert({
name: "parent3",
child: {
_id: ObjectId(),
name: "child3",
key: "value3"
}
});
The resulting entry is:
> db.parents.find({});
{
"_id" : ObjectId("5e074233669d34403ed6bcd2"),
"name" : "parent3",
"child" : {
"_id" : ObjectId("5e074233669d34403ed6bcd1"),
"name" : "child3",
"key" : "value3"
}
}
If we try to find this one now whith the #MongoId annotated child _id field, we won't find it!
The resulting query would be:
StringBasedMongoQuery: Created query Document{{child._id=Document{{$in=[5e074233669d34403ed6bcd1]}}}} for Document{{}} fields.
MongoTemplate : find using query: { "child._id" : { "$in" : ["5e074233669d34403ed6bcd1"]}} fields: Document{{}} for class: com.lubumbax.mongoids.model.Parent in collection: parents
Why? Because now the _id attribute in MongoDB is an actual ObjectId and we are trying to query it as a plain string.
We might be able to workaround that by tweaking the query with SpEL, but IMHO we are entering "Land of Pain".
That document would be found though, as we might expect, if we annotated the child _id field with #Id:
StringBasedMongoQuery: Created query Document{{child._id=Document{{$in=[5e074233669d34403ed6bcd1]}}}} for Document{{}} fields.
MongoTemplate : find using query: { "child._id" : { "$in" : [{ "$oid" : "5e074233669d34403ed6bcd1"}]}} fields: Document{{}} for class: com.lubumbax.mongoids.model.Parent in collection: parents
Once again, and as suggested at the top of this answer, I would discourage you from using ObjectId in children documents.
Some conclusions
As stated above, I would discourage anyone from using ObjectId in children documents.
A 'parents' entry in our MongoDB collection is what is unique. What that document contains may, or may not, represent a Parent entity in our Java application.
This is one of the pilars of NoSQL, in comparison to RDBMS where we would "tend to" normalize our schemas.
From that point of view, there is not such a thing as "part of the information in a document is unique" like, the nested children are unique.
The children would better be called "children element" (maybe there is a better name for this) rather than "chilren documents", because they are not actual documents but rather a part of a "parents" document (for those parents that happen to have a "child" element in their structure).
If we still want to somehow "link" the nested children elements to "exclusive" (or unique) documents in another collection, we can do so indeed (see below).
We just
Refer from nested element to documents in another collection
That in my opinion is a better practise.
The idea is that a document in a collection contains all we need in order to represent an entity.
For instance, in order to represent parents, apart of their names, age, and any other information specific of the parents, we might just want to know the name of their child (in a world where parents have only one child).
If we needed to access to more information of those children at some point we may have another collection with children detailed information. We could thus "link" or "refer" from the nested children in the parents collection to the documents in the children collection.
Our Java model would look something like this for the Children:
#Data
#Builder
#Document(collection = "children")
public class Child {
#Id
private String id;
private String name;
private String key;
private Integer age;
}
A Parent entity would not have any "hard" relation to a Child entity:
#Data
#Builder
#Document(collection = "parents")
public class Parent {
#Id
private String id;
private String name;
private ChildData child;
#Data
#Builder
public static class ChildData {
private String id;
private String name;
}
}
Note that in this case I prefer to name the nested children objects as Parent.ChildData, because it contains just the information I need about a child as to represent a Parent entity.
Note as well that I am not annotating the nested child id field with #Id. Per convention, MappingMongoConverter in this case will anyways map a field called id to a mongo _id field.
Given that there is no relation in MongoDB (as we understand them in RDBMS) between a nested child id and the children ids, we may even rename the ChildData.id field to ChildData.link.
You can see an example of this idea in the LinkitAir PoC.
There, the Flight entity model (stored in MongoDB in a flights collection) contains a nested AirportData document that via its code field "refers" to Airport entities (stored in MongoDB in a airports collection).