I'm trying to do a $lookup from a collection whose documents look this way:
{
"_id": {"$oid":"5f1957c7cdf25116937ed3ef"},
"idSensor": {"$numberLong":"3"},
...
"data":
{
"inicio":"2019-11-28T16:09:08+01:00",
"fin":"2019-11-28T16:09:18+01:00",
...
},
...
}
I deliver them to the $unwind stage just as "array", and then I try a $match:
[
...
{
"$match":
{
"$expr":
{
"$and":
[
{"array.idSensor": 3},
{"$gt": ["array.data.inicio", "ISODate('2021-02-28T23:59:59+01:00')"]},
{"$lt": ["array.data.fin", "ISODate('2021-03-15T00:00:01+01:00')"]}
]
}
}
},
...
]
Here's where I'm getting an error message:
"FieldPath field names may not contain '.'."
Filtering just by "array.idSensor" is OK, I checked it, so the problem lies within the dates.
Thanks in advance for your help.
Related
I'll start with the example as it's easier to explain for me.
[
{
"_id": 100,
"narr": [
{
"field": 1
}
]
},
{
"_id": 101,
"narr": [
{
"field": 1,
},
{
"field": 2
}
]
}
]
Goal is to find document exactly with values specified by me for a field.
Example:
for lookup = [1] find document with _id=100.
for lookup = [1,2] find document with _id=101.
So far I came up with (for second example with [1,2]):
db.col.find(
{
"narr": {
"$all": [
{
"$elemMatch": {
"field": {
"$in": [1, 2]
}
}
}
]
}
}
)
But it also includes document with _id=100. How can I make it perform strict match?
Building whole arrays won't work as there are multiple fields with unknown values in each nested structure.
Without considering duplication in the field and your input, you can simply do a find on narr.field. It is like performing search on an array with values from field.
db.collection.find({
$expr: {
$eq: [
"$narr.field",
[
1,
2
]
]
}
})
Here is the Mongo playground for your reference.
If duplication may happens, try to use $setEquals.
db.collection.find({
$expr: {
"$setEquals": [
"$narr.field",
[
1,
2
]
]
}
})
Here is the Mongo playground for your reference.
I know that by the title it is not very clear what is my problem, so let me explain it with an example.
Let's suppose I have a collection in a mongo database called tweets whose elements look like this:
{
"id": "tweet_id",
"text": "this is the tweet's text",
"user": {
"id": "user_id",
"name": "user_name",
}
}
Let's suppose that we have 100 documents that look like that, and 10 users with diferent ids and names.
How would look a query to know see the different user_id s that exist in the collection and their names?
The result I want would look like this:
{"user_id1": "user_name1"}
{"user_id2": "user_name2"}
...
{"user_id10": "user_name10"}
Thank you for your help
You can use this aggregation query:
First $group by user.id to get all differents user ids with the name.
And then use $replaceRoot with $arrayToObject to get the desired output format.
db.collection.aggregate([
{
"$group": {
"_id": "$user.id",
"name": {
"$first": "$user.name"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$arrayToObject": [
[
{
"k": "$_id",
"v": "$name"
}
]
]
}
}
}
])
Example here
Using MongoDB 4.2 and MongoDB Atlas to test aggregation pipelines.
I've got this products collection, containing documents with this schema:
{
"name": "TestProduct",
"relatedList": [
{id:ObjectId("someId")},
{id:ObjectId("anotherId")}
]
}
Then there's this cities collection, containing documents with this schema :
{
"name": "TestCity",
"instructionList": [
{ related_id: ObjectId("anotherId"), foo: bar},
{ related_id: ObjectId("someId"), foo: bar}
{ related_id: ObjectId("notUsefulId"), foo: bar}
...
]
}
My objective is to join both collections to output something like this (the operation is picking each related object from the instructionList in the city document to put it into the relatedList of the product document) :
{
"name": "TestProduct",
"relatedList": [
{ related_id: ObjectId("someId"), foo: bar},
{ related_id: ObjectId("anotherId"), foo: bar},
]
}
I tried using the $lookup operator for aggregation like this :
$lookup:{
from: 'cities',
let: {rId:'$relatedList._id'},
pipeline: [
{
$match: {
$expr: {
$eq: ["$instructionList.related_id", "$$rId"]
}
}
},
]
}
But it's not working, I'm a bit lost with this complex pipeline syntax.
Edit
By using unwind on both arrays :
{
{$unwind: "$relatedList"},
{$lookup:{
from: "cities",
let: { "rId": "$relatedList.id" },
pipeline: [
{$unwind:"$instructionList"},
{$match:{$expr:{$eq:["$instructionList.related_id","$$rId"]}}},
],
as:"instructionList",
}},
{$group: {
_id: "$_id",
instructionList: {$addToSet:"$instructionList"}
}}
}
I am able to achieve what I want, however,
I'm not getting a clean result at all :
{
"name": "TestProduct",
instructionList: [
[
{
"name": "TestCity",
"instructionList": {
"related_id":ObjectId("someId")
}
}
],
[
{
"name": "TestCity",
"instructionList": {
"related_id":ObjectId("anotherId")
}
}
]
]
}
How can I group everything to be as clean as stated for my original question ?
Again, I'm completely lost with the Aggregation framework.
the operation is picking each related object from the instructionList in the city document to put it into the relatedList of the product document)
Given an example document on cities collection:
{"_id": ObjectId("5e4a22a08c54c8e2380b853b"),
"name": "TestCity",
"instructionList": [
{"related_id": "a", "foo": "x"},
{"related_id": "b", "foo": "y"},
{"related_id": "c", "foo": "z"}
]}
and an example document on products collection:
{"_id": ObjectId("5e45cdd8e8d44a31a432a981"),
"name": "TestProduct",
"relatedList": [
{"id": "a"},
{"id": "b"}
]}
You can achieve try using the following aggregation pipeline:
db.products.aggregate([
{"$lookup":{
"from": "cities",
"let": { "rId": "$relatedList.id" },
"pipeline": [
{"$unwind":"$instructionList"},
{"$match":{
"$expr":{
"$in":["$instructionList.related_id", "$$rId"]
}
}
}],
"as":"relatedList",
}},
{"$project":{
"name":"$name",
"relatedList":{
"$map":{
"input":"$relatedList",
"as":"x",
"in":{
"related_id":"$$x.instructionList.related_id",
"foo":"$$x.instructionList.foo"
}
}
}
}}
]);
To get a result as the following:
{ "_id": ObjectId("5e45cdd8e8d44a31a432a981"),
"name": "TestProduct",
"relatedList": [
{"related_id": "a", "foo": "x"},
{"related_id": "b", "foo": "y"}
]}
The above is tested in MongoDB v4.2.x.
But it's not working, I'm a bit lost with this complex pipeline syntax.
The reason why it's slightly complex here is because you have an array relatedList and also an array of subdocuments instructionList. When you refer to instructionList.related_id (which could mean multiple values) with $eq operator, the pipeline doesn't know which one to match.
In the pipeline above, I've added $unwind stage to turn instructionList into multiple single documents. Afterward, using $in to express a match of single value of instructionList.related_id in array relatedList.
I believe you just need to $unwind the arrays in order to lookup the relation, then $group to recollect them. Perhaps something like:
.aggregeate([
{$unwind:"relatedList"},
{$lookup:{
from:"cities",
let:{rId:"$relatedList.id"}
pipeline:[
{$match:{$expr:{$eq:["$instructionList.related_id", "$$rId"]}}},
{$unwind:"$instructionList"},
{$match:{$expr:{$eq:["$instructionList.related_id", "$$rId"]}}},
{$project:{_id:0, instruction:"$instructionList"}}
],
as: "lookedup"
}},
{$addFields: {"relatedList.foo":"$lookedup.0.instruction.foo"}},
{$group: {
_id:"$_id",
root: {$first:"$$ROOT"},
relatedList:{$push:"$relatedList"}
}},
{$addFields:{"root.relatedList":"$relatedList"}},
{$replaceRoot:{newRoot:"$root"}}
])
A little about each stage:
$unwind duplicates the entire document for each element of the array,
replace the array with the single element
$lookup can then consider each element separately. The stages in $lookup.pipeline:
a. $match so we only unwind the document with matching ID
b. $unwind the array so we can consider individual elements
c. repeat the $match so we are only left with matching elements (hopefully just 1)
$addFields assigns the foo field retrieved from the lookup to the object from relatedList
$group collects together all of the documents with the same _id (i.e. that were unwound from a single original document), stores the first as 'root', and pushes all of the relatedList elements back into an array
$addFields moves the relatedList in to root
$replaceRoot returns the root, which should now be the original document with the matching foo added to each relatedList element
How to search all the collections of a database-name: test for a word having length say 6
The sample data in collections is like:
{
"_id": {
"$oid": "5e0983863bcf0dab51f2872b"
},
"word": "never",
"wordset_id": "a42b50e85e",
"meanings": [{
"id": "1f1bca9d9f",
"def": "not ever",
"speech_part": "adverb",
"synonyms": ["ne'er"]
}, {
"id": "d35f973ed0",
"def": "not at all",
"speech_part": "adverb"
}]
}
How can I query for all words with length of 6 across all collections of test?
I have tried this way but it is giving only first collection results:
#app.......
def fn():
collections=db.collection_names()
for collection in collections:
data = db[collection].aggregate([
{
"$match": {
"$expr": {"$eq": [{"$strLenCP": "$word"}, n]}
}
}
])
return render_template('lettersearch.html',data=data)
when I am printing data I get all cursors as:
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F5B470>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F76BA8>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F6E5F8>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F8A6A0>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F8E048>
How to iterate over these objects and render in template as data?
How to search all the collections of a database
You can achieve this in two steps:
Retrieve all collections in the database - db.getCollectionNames()
For each collection run the query below
Query:
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$strLenCP: "$word"
},
6
]
}
}
}
]);
For each collection use $strLenCP with this aggregation:
db.collection.aggregate([
{
$match: {
$expr: {$eq: [{$strLenCP: "$word"}, 6]}
}
}
])
WHAT I WANT TO ACHIEVE
Let us say I have this object:
{
"_id" : ObjectId("5aec063380a7490014e88792"),
"personal_info" : {
"dialing_code": "+44",
"phone_number": "67467885664"
}
}
I need to concat the two values personal_info.dialing_code (+44) and phone_number (67467885664) into one. +4467467885664 and compare it to a value. I need to retrieve a specific record from the database that will match the said value.
PROBLEM
I am having trouble concatinating two fields inside a subdocument and I am receiving this error:
{
"name": "MongoError",
"message": "$concat only supports strings, not object",
"ok": 0,
"errmsg": "$concat only supports strings, not object",
"code": 16702,
"codeName": "Location16702"
}
ATTEMPT #1
I have tried this:
UserModel.aggregate([
{ $unwind: '$personal_info' },
{$project: {
concat_p: {$concat: [
'$personal_info.dialing_code',
'$personal_info.phone_number'
]}
}}
])
It is giving me an error as mentioned above and in result I cannot do a $match right after.
ATTEMPT #2
I also tried this:
UserModel.aggregate([
{ $unwind: '$personal_info' },
{$project: {
p_dialing_code: '$personal_info.dialing_code',
p_phone_number: '$personal_info.phone_number',
concat_p: {$concat: [
'$p_dialing_code',
'$p_phone_number'
]}
}}
])
I have successfully took out the subdocument values one level however when I tried concatinating, it is producing me null values. This is the result I am getting:
{
"_id": "5af0998036daa90014129d6e",
"p_dialing_code": "+44",
"p_phone_number": "13231213213244",
"concat_p": null
}
I know how to do it on the $match pipeline but I have no luck concatinating the values inside the subdocument. Clearly, I need to do this first before I can compare. Thanks
It seems like you have different types under personal_info.dialing_code and personal_info.phone_number fields. In your example $concat is applied to every document in your collection and that's why you're getting an exception since $concat strictly expects its parameters to be strings.
So it will be working fine for document posted in your question but will throw an exception for something like this:
{
"_id" : ObjectId("5aec063380a7490014e88792"),
"personal_info" : {
"dialing_code": {},
"phone_number": "67467885664"
}
}
One way to fix this is to add $match condition before $project and use $type operator to get only documents having strings on those fields you want to concatenate.
db.UserModel.aggregate([
{
$match: {
$expr: {
$and: [
{ $eq: [ { $type: "$personal_info.dialing_code" }, "string" ] },
{ $eq: [ { $type: "$personal_info.phone_number" }, "string" ] }
]
}
}
},
{$project: {
concat_p: {$concat: [
"$personal_info.dialing_code",
"$personal_info.phone_number"
]}
}}
])