Given the following exemplary JSON document, which is a list of polymorphic objects of type A and B:
[ {
"a" : 1,
"type" : "A"
}, {
"b" : true,
"type" : "B"
}, {
"b" : false,
"type" : "B"
}, {
"a" : 2,
"type" : "A"
} ]
How would I be able to select the As and the Bs to be able to document them differently.
I put an example project on github: https://github.com/dibog/spring-restdocs-polymorphic-list-demo
Here is an excerpt of me trying to document the fetch method:
.andDo(document("fetch-tree",
responseFields(
beneathPath("[0]").withSubsectionId("typeA"),
fieldWithPath("type")
.type(JsonFieldType.STRING)
.description("only node types 'A' and 'B' are supported"),
fieldWithPath("a")
.type(JsonFieldType.NUMBER)
.description("specific field for node type A")
),
responseFields(
beneathPath("[1]").withSubsectionId("typeB"),
fieldWithPath("type")
.type(JsonFieldType.STRING)
.description("only node types 'A' and 'B' are supported"),
fieldWithPath("b")
.type(JsonFieldType.BOOLEAN)
.description("specific field for node type A")
)))
But I get the following error message:
org.springframework.restdocs.payload.PayloadHandlingException: [0] identifies multiple sections of the payload and they do not have a common structure. The following non-optional uncommon paths were found: [[0].a, [0].b]
It looks like that [0] or [1] does not work and is interpreted as [].
What would be the best way to handle this situation?
Thanks,
Dieter
It looks like that [0] or [1] does not work and is interpreted as [].
That's correct. Adding support for indices is being tracked by this issue.
What would be the best way to handle this situation?
The beneathPath method that you've tried to use above returns an implementation of a strategy interface, PayloadSubsectionExtractor. You could provide your own implementation of this interface and, in the extractSubsection(byte[], MediaType) method, extract the JSON for a particular element in the array and return it as a byte[].
Related
When I asked a question earlier about querying in MongoDB resolved the respective issue, but another question stemmed from the original idea.
Under similar conditions, suppose that I'm trying to query:
Example: {
"AttributeA": type,
"AttributeB": type,
"AttributeC": type,
"AttributeD": type
etc...
}
But I want to find all elements given the conditions where:
(Attribute A matches criteria1 or Attribute B matches criteria2)
and
(Attribute C matches criteria3 or Attribute D matches criteria4 or Attribute E matches criteria5)
The $in operator only tracks an $or conditional given that the attributes are the same (eg. referring to previous question of AttributeC matching criteria 3, 4, or 5). So the general layout in this new query would be more like:
db.Example.find({
$or:[ {AttrA : "criteria1"}, {AttrB : "criteria2"}],
$or:[ {AttrC : "criteria3"}, {AttrD : "criteria4"}, {AttrE : "criteria5"} ]
})
But under the conditions above it seems impossible without a duplicate "$or" operator unless I do some boolean algebra and separate it into:
((A+B)*(C+D+E) = AC + AD + AE + BC + BD + BE) aka
AttrA matches ... and AttrC matches ...
or
AttrA matches ... and AttrD matches ...
or
...
AttrB matches ... and AttrE matches ...
meaning that the format would look like
db.Example.find({
$or:[
$and:[{AttrA : "criteria1"}, {AttrC : "criteria3"}],
$and:[{AttrA : "criteria1"}, {AttrD : "criteria4"}],
...,
$and:[{AttrB : "criteria2"}, {AttrE : "criteria5"}
]
})
Though I'm not even sure if the mongoDB system allows for duplicate "$and"s either.
Could there be an easier way or am I overcomplicating the conditional queries?
There is no need to manually distribute the conditions here. You must use the $and explicitly rather than implying on the implicit one:
db.Example.find({
$and: [
{ $or:[ {AttrA : "criteria1"}, {AttrB : "criteria2"}] },
{ $or:[ {AttrC : "criteria3"}, {AttrD : "criteria4"}, {AttrE : "criteria5"} ] }
]
})
Playground example here. This is covered in the documentation here.
The general problem that is encountered, at least with Javascript that includes the shell, is that duplicate field names are not supported. Consider this example using Node directly:
$ node
Welcome to Node.js v16.17.0.
Type ".help" for more information.
> let x = { x: 123, y: 456, x: 789 }
undefined
> x
{ x: 789, y: 456 }
>
I got the unusual json (actually from IBM Bluemix), shown below,
Thank goodness, trusty and heartwarming SwiftyJSON was able to get the values, like this...
let mauves = json["blue"][0]["brown"][0]["mauve"]
However, notice there are weird sort of "empty unnamed array nested things" in the JSON (hence the [0] calls to Swifty).
My question, in short,
is this valid json?
Even if valid, is it "crappy"? Or am I wrong, it's totally idiomatic? (Maybe I've just been dating the wrong services for decades, I don't know.)
I appreciate that running it through online validators seems to say "valid" (except this one http://json.parser.online.fr gives red things), but, you know, who trusts online services? Ask experts on SO....)
--
{
"red" : 1,
"green" : 4,
"blue" : [
{
"yellow" : "word",
"brown" : [
{
"orange" : "1826662593",
"gold" : "23123",
"mauve" : [
{
"a" : "Beagle",
"b" : 0.979831
},
{
"a" : "Chow",
"b" : 0.937588
},
{
"a" : "Hound",
"b" : 0.987798
}
]
}
]
}
]
}
--
The JSON is valid. The blue member contains an array with 1 element (at index [0] which is the yellow object, and this is repeated for orange.
When I paste it into json.parser.online.fr it reports it as valid for me - are you accidentally including other text around it?
The JSON is perfectly valid - your validators are not lying to you. I don't know if this JSON contains real keys, or if the names have been changed to protect the innocent (it certainly looks like nonsense), but in a real world situation there are frequently arrays that contain one element (because they might contain zero or many elements!).
I have a collection with following document:
{
"_id" : ObjectId("51f1fd2b8188d3117c6da352"),
"cust_id" : "abc1234",
"ord_date" : ISODate("2012-10-03T18:30:00Z"),
"status" : "A",
"price" : 27,
"items" : [{
"sku" : "mmm",
"qty" : 5,
"price" : 2.5
}, {
"sku" : "nnn",
"qty" : 5,
"price" : 2.5
}]
}
I want to use "$where" in the fields of "items", so something like this:
{$where:"this.items.sku==mmm"}
How can I do it? It works when the field is not of array type.
You don't need a $where operator to do this; just use a query object of:
{ "items.sku": mmm }
As for why your $where isn't working, the value of that operator is executed as JavaScript, so that's not going to check each element of the items array, it's just going to treat items as a normal object and compare its sku property (which is undefined) to mmm.
You are comparing this.items.sku to a variable mmm, which isn't initialized and thus has the value unefined. What you want to do, is iterate the array and compare each entry to the string 'mmm'. This example does this by using the array method some which returns true, when the passed function returns true for at least one of the entries:
{$where:"return this.items.some(function(entry){return entry.sku =='mmm'})"}
But really, don't do this. In a comment to the answer by JohnnyHK you said "my service is just a interface between user and mongodb, totally unaware what the field client want's to store". You aren't really explaining your use-case, but I am sure you can solve this better.
The $where operator invokes the Javascript engine even though this
trivial expression could be done with a normal query. This means unnecessary performance overhead.
Every single document in the collection is passed to the function, so when you have an index, it can not be used.
When the javascript function is generated from something provided by the client, you must be careful to sanetize and escape it properly, or your application gets vulnerable to code injection.
I've been reading through your comments in addition to the question. It sounds like your users can generically add some attributes, which you are storing in an array within a document. Your client needs to be able to query an arbitrary pair from the document in a generic manner. The pattern to achieve this is typically as follows:
{
.
.
attributes:[
{k:"some user defined key",
v:"the value"},
{k: ,v:}
.
.
]
}
Note that in your case, items is attributes. Now to get the document, your query will be something like:
eg)
db.collection.find({attributes:{$elemMatch:{k:"sku",v:"mmm"}}});
(index attributes.k, attributes.v)
This allows your service to provide a way to query the data, and letting the client specify what the k,v pairs are. The one caveat with this design is always be aware that documents have a 16MB limit (unless you have a use case that makes GridFS appropriate). There are functions like $slice which may help with controlling this.
Ok there are a couple of things going on here..I have two collections: test and test1. The documents in both collections have an array field (tags and tags1, respectively) that contains some tags. I need to find the intersection of these tags and also fetch the whole document from collection test1 if even a single tag matches.
> db.test.find();
{
"_id" : ObjectId("5166c19b32d001b79b32c72a"),
"tags" : [
"a",
"b",
"c"
]
}
> db.test1.find();
{
"_id" : ObjectId("5166c1c532d001b79b32c72b"),
"tags1" : [
"a",
"b",
"x",
"y"
]
}
> db.test.find().forEach(function(doc){db.test1.find({tags1:{$in:doc.tags}})});
Surprisingly this doesn't return anything. However when I try it with a single document, it works:
> var doc = db.test.findOne();
> db.test1.find({tags1:{$in:doc.tags}});
{ "_id" : ObjectId("5166c1c532d001b79b32c72b"), "tags1" : [ "a", "b", "x", "y" ] }
But this is part of what I need. I need intersection as well. So I tried this:
> db.test1.find({tags1:{$in:doc.tags}},{"tags1.$":1});
{ "_id" : ObjectId("5166c1c532d001b79b32c72b"), "tags1" : [ "a" ] }
But it returned just "a" whereas "a" and "b" both were in tags1. Does positional operator return just the first match? Also, using $in won't exactly give me an intersection..How can I get an intersection (should return "a" and "b") irrespective of which array is compared against the other.
Now say there's an operator that can do this..
> db.test1.find({tags1:{$intersection:doc.tags}},{"tags1.$":1});
{ "_id" : ObjectId("5166c1c532d001b79b32c72b"), "tags1" : [ "a", "b" ] }
My requirement is, I need the entire tags1 array PLUS this intersection, in the same query like this:
> db.test1.find({tags1:{$intersection:doc.tags}},{"tags1":1, "tags1.$":1});
{ "_id" : ObjectId("5166c1c532d001b79b32c72b"), "tags1": [ "a", "b", "x", "y" ],
"tags1" : [ "a", "b" ] }
But this is an invalid json. Is renaming key possible, or this is possible only through aggregation framework (and across different collections?)? I tried the above query with $in. But it behaved as if it totally ignored "tags:1" projection.
PS: I am going to have at least 10k docs in test1 and very few (<10) in test. And this query is in real-time, so I want to avoid mapreduce :)
Thanks for any help!
In newer versions you can use aggregation to accomplish this.
db.test.aggregate(
{
$match: {
tags1: {
$in: doc.tags
}
}
},
{
$project: {
tags1: 1,
intersection: {
$setIntersection: [doc.tags, "$tags1"]
}
}
}
);
As you can see, the match portion is exactly the same as your initial find() query. The project portion generates the result fields. In this case, it selects tags1 from the matching documents and also creates intersection from the input and the matching docs.
Mongo doesn't have any inherent ability to retrieve array intersections. If you really need to use ad-hoc querying get the intersection on the client side.
On the other hand, consider using Map-Reduce and storing it's output as a collection. You can augment the returned objects in the finalize section to add the intersecting tags. Cron MR to run every few seconds. You get the benefit of a permanent collection you can query from on the client side.
If you want to have this in realtime you should consider to move away from Serverside Javascript which is only run with one thread and should be quite slow (single threaded) (this is no longer true for v2.4, http://docs.mongodb.org/manual/core/server-side-javascript/).
The positional operator only returns the first matching/current value. Without knowing the internal implementation, from the point of performance it doesn't even makes sense to look for further matching criteria if the document was already evaluated as match. So I doubt that you can go for this.
I don't know if you need the cartesian product for your search, but I would consider joining your few test one document tags into one and then have some $in search for it on test1, returning all matching documents. On your local machine you could have multiple threads which generate the intersection for your document.
Depending on how frequent your test1 and test collection changes, you're performing this query you might precalculate this information. Which would allow to easily do a query on the field which contains the intersection information.
The document is invalid because you have two fields names tags1
I am working with MongoDB and Perl. Here is my data structure:
{
"_id" : ObjectId("501976f8005c8b541d000000"),
"err_id" : "err",
"solution" : [
{
"attachment" : "attach",
"macr" : "macrs",
"yammer" : "yam",
"resolution" : "l",
"salesforce" : "salesforce",
"username" : "bob"
},
{
"attachment" : "attach",
"macr" : "macrs",
"yammer" : "yam",
"resolution" : "losssss",
"salesforce" : "salesforce",
"username" : "bob"
}
]
}
As you can see, I have an array with objects inside. I have created this using the Perl MongoDB library.
I am familiar with some syntax for manipulating arrays in the Perl MongoDB lib. For example, I use this to find entries with a username the same as $username.
$users->find({"solution.username" => $username});
I thought removing an element would be as simple:
$users->remove({"solution.username" => $username});
But alas, it is not so. I have tried this and using pull, but to no avail! I've had a hard time finding this. Does anybody know the syntax to remove an array element based on the contents of one of its fields?
The MongoDB::Collection remove() method will remove documents matched by your query .. so definitely not what you are looking for.
To delete specific fields you should use $unset.
Your solution.usernames are actually in an array, so you would have to include an array index for the fields to delete, eg:
$users->update({"_id" => '123'}, {
'$unset' => {
'solution.0.username' => 1,
'solution.1.username' => 1
}
});
I'm not aware of a shorter syntax to unset all fields matching username within the solution array, but you can add multiple solution.#.username fields to the $unset command.
My example above deletes the first two username entries from the array. If the matching document(s) had more than two username entries, each time you ran this update you would delete up to two more entries (if they exist).