I'm trying to get all the activities from a user or its teams also filtered by some types if some properties are set.
This is what I have right now:
Activity.withCriteria{
and{
or {
eq 'user',myUser
"in" 'team',userTeams
}
and{
if (showA || showB || showC){
or{
if (showA){
"in" "a", myAList
}
if (showB){
"in" "b", myBList
}
if (showC){
"in" "c",myCList
}
}
}
}
}
order "date","desc"
maxResults maxElements
}
Executing that, what I get it's the OR of user and team block and the showA, showB, showC block instead of the AND of those two blocks.
I'm using grails 2.2.1 (also using MongoDB GORM 1.2.0 without Hibernate)
EDIT:
I have been able to see the query that's sent to MongoDB and it's not doing the first part of the criteria.
This is the query that's being passed to MongoDB:
query: { query: { $or: [ { a: { $in: [ "5191e2c7c6c36183687df8b6", "5191e2c7c6c36183687df8b7", "5191e2c7c6c36183687df8b8" ] } }, { b: { $in: [ "5191e2c7c6c36183687df8b9", "5191e2c7c6c36183687df8ba", "5191e2c7c6c36183687df8bb" ] } }, { c: { $in: [ "5191e2c7c6c36183687df8b5" ] } } ] }, orderby: { date: -1 } } ntoreturn: 10 ntoskip: 0
EDIT: I have just seen that a JIRA has already been raised and it seems that's a MongoDB plugin problem...
http://jira.grails.org/browse/GPMONGODB-296
You can think in your criteria in a SQL perspective.
and ((user = 'myUserValue'
or team in (...))
and(a in (...)
or b in (...)
or c in (...)))
So your or is applied to user and team, but I think you want something like:
or {
and {
eq 'user',myUser
"in" 'team',userTeams
}
and{
if (showA || showB || showC){
or{
if (showA){
"in" "a", myAList
}
if (showB){
"in" "b", myBList
}
if (showC){
"in" "c",myCList
}
}
}
}
}
So the key here is that the block you declare is applied to what you have inside.
EDIT:
A good tip to inspect a criteria is to enable the output of sql's generated by Hibernate. This can be done in DataSource.groovy
dataSource {
logSql = true
}
hibernate {
format_sql = true
}
With the new version of Mongo for Grails this has been fixed, so now it's working with version 1.3.0.
http://grails.org/plugin/mongodb
Related
//Sample Collection
{
//fields
"roleList" : [ROLE_1, ROLE_2, ROLE_3],
"siteList" : [
{
"role" : ROLE_1,
//fields
},
{
"role" : ROLE_2,
//fields
},
]
}
//Expected Output
{
//fields
"roleDiff":[ROLE_3] // roleList subtracted by set of roles present in siteList
}
//Script that works with Studio 3t
db.getCollection("SAMPLE_COLLECTION").aggregate(
// Pipeline
[
// Stage 1
{
$project: {
"roleList":1,
"siteRoles":{$ifNull: [{$reduce:{
input:"$siteList",
initialValue:[],
in:{$setUnion:["$$value", {
"$split": ["$$this.role", " "]}]}
}
}, []]}
}
},
// Stage 2
{
$project: {
"roleDiff":{ $setDifference:["$roleList", "$siteRoles"]
}
}
},
],
// Options
{
}
// Created with Studio 3T, the IDE for MongoDB - https://studio3t.com/
);
ArrayOperators.Reduce reduce = ArrayOperators.Reduce.arrayOf("siteList").withInitialValue(Collections.EMPTY_SET)
.reduce(SetOperators.SetUnion.arrayAsSet(StringOperators.valueOf("$$this.role").split(" ")).union(ArrayOperators.Reduce.Variable.VALUE.getTarget()));
ProjectionOperation projectionOperationOne = Aggregation.project().andInclude(/*Some fields,*/ "roleList").and(ConditionalOperators.ifNull(reduce).then(Collections.EMPTY_LIST)).as("siteRoles");
ProjectionOperation projectionOperationTwo = Aggregation.project().andInclude(/*Some fields*/).and(SetOperators.SetDifference.arrayAsSet("roleList").differenceTo("siteRoles")).as("roleDiff");
Aggregation aggregation = Aggregation.newAggregation(projectionOperationOne, projectionOperationTwo);
AggregationResults<SiteDiff> siteDiff = mongoTemplate.aggregate(aggregation, SampleCollection.class, SiteDiff.class);
The Java code above thows exception
org.springframework.data.mapping.context.InvalidPersistentPropertyPath: No property '$$value' found on class Did you mean: ?
The query works fine with Studio3T. My intention here is to get the difference between 2 String arrays "roleList" and "siteRoles" along with some other fields in the record.
"siteRoles" has to be derived from "siteList" which is an array of object. Since $addToSet works only with $group operation, I'm finding it difficult to extract the role from "siteList". I used reduce here combining it with setUnion. There problem was role is a string and I had to convert it to array. Only way I could find was to use $split and use " " as delimiter as I'm sure role will not have space.
Finally, the script worked in studio3t but the java version is not getting executed.
If you refer to a field in an embedded array, it should give you the entire array of those values.
The aggregation should only need 1 stage:
{$project: {roleDiff: { $setDifference: ["$roleList","$siteList.role"]}}}
In my DynamoDB every document has several fields, one of the fields is a document called "engines" that holds several documents (all the engines) that hold several fields, as the picture shows below:
I would like to get all the couples of (engine,definitions) that their definition date is greater than a specific date.
I tried:
cursor=collection.find(
{'engines': { "$elemMatch" :
{ "definitions" :
{'$gt': startdate} } } }
,{'engines':{'$elemMatch':1}},{'engines':{'$elemMatch':{'definitions':1}}} )
but I get:
TypeError: skip must be an instance of int
Can someone help with the query?
You've mixed up the closing } and ended up passing {'engines':{'$elemMatch':{'definitions':1}}} as a skip argument value.
I think you meant:
cursor = collection.find(
{
'engines': {
"$elemMatch": {
"definitions": {
'$gt': startdate
}
}
}
},
{
'engines': {
'$elemMatch': {
'definitions': 1
}
}
}
)
I am trying to use the db.collection.distinct(field, query) command, documented here. I am trying to call this with the C# driver, documented here.
Currently I am using the code:
_repository.Search(item.searchCriteria).Select(i => i.messageId).Distinct().ToList()
where messageId is a string and the Search function does:
//Create search accross all properties of type.
public IQueryable<SearchType> Search(SearchType entity)
{
Type entityType = entity.GetType();
var propertiesToSearch = entityType.GetProperties(BindingFlags.GetProperty | BindingFlags.Instance | BindingFlags.Public);
query = _collection.AsQueryable();
query = query.WhereAnd(
query.ElementType,
propertiesToSearch.Select(p => new SearchCriteria()
{
Column = p.Name,
Value = p.GetValue(entity),
Operation = WhereOperation.Equal
}).ToArray());
return query;
}
So this should get converted to:
db.collection.distinct("messageId", { $and: [ { prop1: "" }, { prop2: "" } ] })
I am getting the following error when this is run though:
"Distinct is only supported for a single field. Projections used with Distinct must resolve to a single field in the document."
I am using Mongo 2.4.9 and the official C# driver 1.8.3
The .distinct() method is an older implementation that is more of a convenience method wrapping mapReduce. For anything more involved that simple operations you should use .aggregate().
So the shell equivalent:
db.collection.aggregate([
{ "$match": { "$and": [ { "prop1": "" }, { "prop2": "" } ] } },
{ "$group": { "_id": "$messageId" } }
])
The documents are just formed as a chain of BSON documents. There are various examples here.
I have a collection of documents in the following form:
{ _id: ObjectId(...)
, title: "foo"
, tags: ["bar", "baz", "qux"]
}
The query should find all documents with any of these tags. I currently use this query:
{ "tags": { "$in": ["bar", "hello"] } }
And it works; all documents tagged "bar" or "hello" are returned.
However, I want to sort by relevance, i.e. the more matching tags the earlier the document should occur in the result. For example, a document tagged ["bar", "hello", "baz"] should be higher in the results than a document tagged ["bar", "baz", "boo"] for the query ["bar", "hello"]. How can I achieve this?
MapReduce and doing it client-side is going to be too slow - you should use the aggregation framework (new in MongoDB 2.2).
It might look something like this:
db.collection.aggregate([
{ $match : { "tags": { "$in": ["bar", "hello"] } } },
{ $unwind : "$tags" },
{ $match : { "tags": { "$in": ["bar", "hello"] } } },
{ $group : { _id: "$title", numRelTags: { $sum:1 } } },
{ $sort : { numRelTags : -1 } }
// optionally
, { $limit : 10 }
])
Note the first and third pipeline members look identical, this is intentional and needed. Here is what the steps do:
pass on only documents which have tag "bar" or "hello" in them.
unwind the tags array (meaning split into one document per tags element
pass on only tags exactly "bar" or "hello" (i.e. discard the rest of the tags)
group by title (it could be also by "$_id" or any other combination of original document
adding up how many tags (of "bar" and "hello") it had
sort in descending order by number of relevant tags
(optionally) limit the returned set to top 10.
You could potentially use MapReduce for something like that. You'd process each document in the Map step, figuring out how many tags match the query, and assign a score. Then you could sort based on that score.
http://www.mongodb.org/display/DOCS/MapReduce
Something that complex should be done after querying. Either server-side through db.eval (if your client supports this) or just clientside. Here's an example for what you're looking for.
It will retreive all posts with the tags you specified, then sorts them according to the amount of matches.
remove the db.eva( part and translate it to the language your client uses to query to get the clientside effect (
db.eval(function () {
var tags = ["a","b","c"];
return db.posts.find({tags:{$in:tags}}).toArray().sort(function(a,b){
var matches_a = 0;
var matches_b = 0;
a.tags.forEach(function (tag) {
for (t in tags) {
if (tag == t) {
matches_a++;
} else {
matches_b++;
}
}
});
b.tags.forEach(function(tag) {
for (t in tags) {
if (tag == t) {
matches_b++;
} else {
matches_a++;
}
}
});
return matches_a - matches_b;
});
});
I am currently evaluating the efficiency of different databases for a use case. In Mongodb, would like to store around 1 million objects with the following structure. Each object will have between 5 and 10 objects in the foo array.
{
name:"my name",
foos:[
{
foo:"...",
bar:"..."
},
{
foo:"...",
bar:"..."
},
{
foo:"...",
bar:"..."
}
]
}
I often need to search for objects which where the foos collection contains an object with a specific property, e.g.:
// mongo collection
[
{
name:'my name',
foos:[
{
foo:'one_foo',
bar:'a_bar'
},
{
foo:'two_foo',
bar:'b_bar'
}
]
},
{
name:'another name',
foos:[
{
foo:'another foo',
bar:'a_bar'
},
{
foo:'just another foo',
bar:'c_bar'
}
]
}
]
// search (pseudo code)
{ foos: {$elemMatch: {bar: 'c_bar'}} }
// returns
{
name:'another name',
foos:[
{
foo:'another foo',
bar:'a_bar'
},
{
foo:'just another foo',
bar:'c_bar'
}
]
}
Can this efficiently be done with mongo and how should the indexes be set?
I don't want you to evaluate performance for me, just an idea how mongo performs for my use case or how optimization could look like.
MongoDB has documentation explaining how to create indexes on embedded documents, through dot notation:
Dot Notation (Reaching into Objects)
> db.blogposts.findOne()
{ title : "My First Post", author: "Jane",
comments : [{ by: "Abe", text: "First" },
{ by : "Ada", text : "Good post" } ]
}
> db.blogposts.find( { "comments.by" : "Ada" } )
> db.blogposts.ensureIndex( { "comments.by" : 1 } );
As for the performance characteristic... just test it with your dataset.