I have a search controller where I want to exclude some documents by an array of ids. Next is my aggregation:
var aggregation = [
{
$geoNear:{
query : {
//_id:{$nin:user.friends}, <---- user.friends is an array of id's that I want to exclude
"profile.age":{$gte: minAge, $lte: maxAge},
"approved":true
},
near: { type: "Point", coordinates: [ user.location[0], user.location[1]] },
limit:100,
maxDistance:radius*1000,
distanceField: "dist.calculated",
includeLocs: "dist.location",
distanceMultiplier:1/1000,
spherical: true
}
},
{
$project:{
online:1,
promoted:1,
location:1,
profile:1,
dist:1
}
},
{ $sort : { online:-1, promoted:-1} }
];
How can I achieve that the result would exclude some doc's having an array 'user.friends' populated with the id's that needs to be excluded.
_id:{$nin:user.friends} is not working
I would assume your array user.friends is an array of strings, So since it checks against the _id which is an Object, not a String, the query cannot work as you expect it to work.
So what you have to is to convert your array user.fridns into an array of Objects. Something like:
user.friends = user.friends.map(x => ObjectId(x) );
Related
I want to filter my collection by aggregation for one of many ObjectIDs.
Because of some DocumentDB restrictions I can not build a single pipeline with uncorrelated subqueries. So my fix is to do it in two queries.
for example: I have an aggregation that returns all teamIds, for some conditions as an array of Object with the IDs.
[{_id: ObjectID("abcdef")}, {_id: ObjectID("ghijkl")}, {_id: ObjectID("vwxyz")}, ...]
I now want to have a second aggregation filter another collection using the ObjectIDs.
This would work in Mongo Compass:
{
"team": {
"$in": [ObjectId("60aabcb05c7462f42b3d7zyx"), ObjectId("60aabc7b05c7462f42b3dxyz")]
},
....
}
My issue is that i can not find the correct syntax for JS to generate such a pipeline.
What ever I try, JS always converts my Array of ObjectIDs to something like this:
{
"team": {
"$in": [{
"_id": "60aabcb05c7462f42b3d7zyx"
},{
"_id": "60aabc7b05c7462f42b3dxyz"
}]
},
I fixed it like this. I am not 100% why this syntax works because it is still just an array of objects, formatted like before, but I guess there is some stuff mongoose does, that is opaque to me.
let teams = await TeamMgmt.getTeamsAggregatedByFilter( teamFilter )
// make an array of ObjectIds so we can filter for them.
let idArray = []
Object.keys( teams ).map( function ( key, index ) {
idArray.push( new mongoose.Types.ObjectId( teams[ index ]._id.toString() ) )
} );
const shiftFilter = [
{
'$match': {
'team': {
"$in": idArray
},
....
}
I am using a Mongo pipeline within an aggregation lookup on 2 collections, Locations and Places.
I am trying to return all the places which these locations are near.
The error I get is 'MongoError: 'near' field must be point'
I believe this is because I am trying to use the $point variable in the pipeline from the let in the lookup and I am doing something wrong here. All the answers I see on here have static coordinates but I want to use the ones from the lookup.
This is the code I have:
return await this.placeModel.aggregate([{
$lookup : {
from : "locations",
let : {point : "location.coordinates"},
pipeline: [ {
$geoNear: {
distanceField: "distance",
near: { type: "Point", coordinates: "$point" },
maxDistance: 20,
spherical: true
}
}],
as : "places"
}
}]);
}
I have a mongoose Place model and Location model. Each model has a GeoJson point that looks like this:
location: {
type: { type: String },
coordinates: []
},
How do I reference the point properly if at all possible.
I have a MongoDB collection which contains a location (GeoJSON Point) and other fields to filter on.
{
"Location" : {
"type" : "Point",
"coordinates" : [
-118.42359,
33.974563
]
},
"Filters" : [
{
"k" : 1,
"v" : 5
},
{
"k" : 2,
"v" : 8
}
]
}
My query uses the aggregate function because it performs a sequence of filtering, sorting, grouping, etc... The first step where it's filtering is where I'm having trouble performing the geo near operation.
$geoNear: {
spherical: true,
near: [-118.236391, 33.782092],
distanceField: 'Distance',
query: {
// Filter by other fields.
Filters: {
$all: [
{ $elemMatch: { k: 1 /* Bedrooms */, v: 5 } }
]
}
},
maxDistance: 8046
},
For indexing I tried two approaches:
Approach #1: Create two separate indexes, one with the Location field and one with the fields we subsequently filter on. This approach is slow, with very little data in my collection it takes 3+ seconds to query within a 5 mile radius.
db.ResidentialListing.ensureIndex( { Location: '2dsphere' }, { name: 'ResidentialListingGeoIndex' } );
db.ResidentialListing.ensureIndex( { "Filters.k": 1, "Filters.v": 1 }, { name: 'ResidentialListingGeoQueryIndex' } );
Approach #2: Create one index with both the Location and other fields we filter on. Creating the index never completed, as it generated a ton of warnings about "Insert of geo object generated a high number of keys".
db.ResidentialListing.ensureIndex( { Location: '2dsphere', "Filters.k": 1, "Filters.v": 1 }, { name: 'ResidentialListingGeoIndex' } );
The geo index itself seems to work fine, if I only perform the $geoNear operation and don't try to query after then it executes in 60ms. However, as soon as I try to query on other fields after is when it gets slow. Any ideas would be appreciated on how to set up the query and indexes correctly so that it performs well...
I wanted to do this:
model.User.aggregate([
//step 1 match criteria
{
$match: criteria
},
//step 2 skip
{
$skip: offset
},
//step 3 limit
{
$limit: limit
},
//step 4 sort by computed distance
{
$geoNear : {
near: {type: 'Point', coordinates: coords },
distanceField: 'currentCity.computed_distance',
includeLocs: 'currentCity.loc',
spherical: true,
uniqueDocs: true,
distanceMultiplier: 3963.2, //convert to miles (this number is the radius of the earth in miles)
}
}
],function(err,users){
if (err) return res.error(err);
if (!users.length) return res.error('no matched criteria');
res.apiResponse(users);
});
but the documentation of $geoNear states:
You can only use $geoNear as the first stage of a pipeline.
Reading the documentation, i see that i can simply move $match inside of $geoNear via the query option. likewise, $limit may be placed inside of $geoNear via the limit option. the one problem is, there is no equivalent for the $skip option, so it looks as if facilitating pagination would not be possible? i'm really confused here, why $geoNear can't be the 4th step in the pipeline. The goal of the query is simply to find the best n matches, where n = limit, then sort by nearest in proximity. is this even possible? I am having trouble finding an answer for this specific use case.
I suppose one solution might be to perform a query to select only ids matching documents, convert to list of ids, then do the aggregates with an $in query like so:
model.User.find(criteria).skip(offset).limit(limit).select('_id').exec(function (err, userIds) {
var ids = [];
userIds.forEach(function(u){
ids.push(u._id);
});
model.User.aggregate([
{
$geoNear : {
query: { _id: {$in: $ids } },
near: {type: 'Point', coordinates: coords },
distanceField: 'currentCity.computed_distance',
includeLocs: 'currentCity.loc',
spherical: true,
uniqueDocs: true,
distanceMultiplier: 3963.2, //convert to miles (this number is the radius of the earth in miles)
}
}
],function(err,users){
if (err) return res.error(err);
if (!users.length) return res.error('no matched criteria');
res.apiResponse(users);
});
});
This would work, but ideally i could do it in 1 query if possible. any ideas greatly appreciated.
One solution is this one:
result = db.cafes.aggregate([{
'$geoNear': {
'near': {
'type': 'Point',
'coordinates': [
-73.991084,
40.735863]},
'spherical': True,
'distanceField': 'dist',
'num': 20}
}, {
'$skip': 10
}])
There is also a better solution with this approach:
ids = [42]
result = db.command(
'geoNear', 'cafes',
near={
'type': 'Point',
'coordinates': [
-73.991084,
40.735863]},
spherical=True,
minDistance=268,
query={
'_id': {
'$nin': ids}},
num=10)
And a really nice explanation on speed and issues over here:
https://emptysqua.re/blog/paging-geo-mongodb/
I have a document structured as follows:
{
_id: "someid",
games: [{
slug: "world-of-warcraft",
class: 'shaman'
}, {
slug: 'starcraft-ii',
class: 'zerg'
}],
roles: {
'starcraft-ii': ['player'],
'world-of-warcraft': ['player']
}
}
I am trying to filter it so that only starcraft-ii within the games array will show up for all players in the role of player in starcraft-ii. I do the following:
function getUsersInGame(slug) {
return Users.find({
'games.slug': slug,
[`roles.${slug}`]: 'player'
}, {
fields: {
'games.$': 1,
'roles': 1
}
});
}
However, this does not match within the games array, and instead returns a 1-element array with world-of-warcraft instead.
What is the appropriate way to filter this array in mongo?
Use $elemMatch in your fields, since the $ will return the first element of the array. So your query should look like this:
function getUsersInGame(slug) {
return Users.find(
{
'"roles.'+ slug + '"': { $in : ['player']}
},
{
'games': {
$elemMatch: {'slug': slug}
},
'roles': 1
});
Please note the difference from the docs:
"The $ operator projects the array elements based on some condition from the query statement.
The $elemMatch projection operator takes an explicit condition argument. This allows you to project based on a condition not in the query..."