search through all collections of a database in Mongodb (python) - mongodb

How to search all the collections of a database-name: test for a word having length say 6
The sample data in collections is like:
{
"_id": {
"$oid": "5e0983863bcf0dab51f2872b"
},
"word": "never",
"wordset_id": "a42b50e85e",
"meanings": [{
"id": "1f1bca9d9f",
"def": "not ever",
"speech_part": "adverb",
"synonyms": ["ne'er"]
}, {
"id": "d35f973ed0",
"def": "not at all",
"speech_part": "adverb"
}]
}
How can I query for all words with length of 6 across all collections of test?
I have tried this way but it is giving only first collection results:
#app.......
def fn():
collections=db.collection_names()
for collection in collections:
data = db[collection].aggregate([
{
"$match": {
"$expr": {"$eq": [{"$strLenCP": "$word"}, n]}
}
}
])
return render_template('lettersearch.html',data=data)
when I am printing data I get all cursors as:
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F5B470>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F76BA8>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F6E5F8>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F8A6A0>
<pymongo.command_cursor.CommandCursor object at 0x00000258F4F8E048>
How to iterate over these objects and render in template as data?

How to search all the collections of a database
You can achieve this in two steps:
Retrieve all collections in the database - db.getCollectionNames()
For each collection run the query below
Query:
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$strLenCP: "$word"
},
6
]
}
}
}
]);

For each collection use $strLenCP with this aggregation:
db.collection.aggregate([
{
$match: {
$expr: {$eq: [{$strLenCP: "$word"}, 6]}
}
}
])

Related

MongoDB query slow in join using $ne to look for not empty arrays

I'm new to Mongo and I have a slow query when using $ne in a match pipeline (to get the records that match only and not all the ones where the array is empty)
The query is as follow:
db.EN.aggregate([
{
$lookup: {
from: 'csv_import',
let: {pn:'$ICECAT-interface.Product.#Prod_id'},
pipeline: [{
$match: {
$expr: {
$eq: ["$$pn","$part_no"]
}
}
}],
as: 'part_number_info'
}
}, { $match: { part_number_info: { $ne: [] } } }
]).pretty();
When I remove the { $match: { part_number_info: { $ne: [] } } } the query executes in 21 seconds, vs almost 2 hours when executed using the $ne clause.
There's an index already on ICECAT-interface.Product.#Prod_id, and here are the 2 collections structure sample:
csv_import:
{
"_id": "ObjectId(\"6348339cc6e5c8ce0b7da5a4\")",
"index": 23679,
"product_id": 4019734,
"part_no": "CP-HAR-EP-ADVANCED-REN-1Y",
"vendor_standard": "Check Point"
}
EN:
[{
"_id": "1414",
"ICECAT-interface": {
"#xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance",
"#xsi:noNamespaceSchemaLocation": "https://data.icecat.biz/xsd/ICECAT-interface_response.xsd",
"Product": {
"#Code": "1",
"#HighPic": "https://images.icecat.biz/img/norm/high/1414-HP.jpg",
"#HighPicHeight": "400",
"#HighPicSize": "43288",
"#HighPicWidth": "400",
"#ID": "1414",
"#LowPic": "https://images.icecat.biz/img/norm/low/1414-HP.jpg",
"#LowPicHeight": "200",
"#LowPicSize": "17390",
"#LowPicWidth": "200",
"#Name": "C6614NE",
"#IntName": "C6614NE",
"#LocalName": "",
"#Pic500x500": "https://images.icecat.biz/img/gallery_mediums/img_1414_medium_1480667779_072_2323.jpg",
"#Pic500x500Height": "500",
"#Pic500x500Size": "101045",
"#Pic500x500Width": "500",
"#Prod_id": "C6614NE",
SOLUTION
I did add an index on part_no field in csv_import and I changed the order of the query to be smaller to large (EN is 27GB and csv_import is a few MB)
Final query: (includes the suggestion made by nimrod serok
db.csv_import.aggregate([
{
$lookup: {
from: 'EN',
let: {pn:'$part_no'},
pipeline: [{
$match: {
$expr: {
$eq: ["$$pn","$ICECAT-interface.Product.#Prod_id"]
}
}
}],
as: 'part_number_info'
}
},{$match: {"part_number_info.0": {$exists: true}}}
])
A better option is to use:
{$match: {"part_number_info.0": {$exists: true}}}
See how it works on the playground example

$elemMatch doesn't work on nested documents in MongoDB

Stack Overflow!
I have a very strange problem with using $elemMatch in MongoDB. I added multiple documents to a collection. Some of these documents were added using import feature in MongoDB Compass (Add Data -> Import File -> JSON) and some of them were added using insertMany().
Here is an example structure of a single document:
{
"id": "1234567890",
"date": "YYYY-MM-DD",
"contents": {
"0": {
"content": {
"id": "1111111111",
"name": "Name 1"
}
},
"1": {
"content": {
"id": "2222222222",
"name": "Name 2"
}
},
"2": {
"content": {
"id": "3333333333",
"name": "Name 3"
}
}
}
}
The thing is, when I use the following find query using this filter:
{date: "<some_date_here>", "contents": {
$elemMatch: {
"content.id": <some_id_here>
}
}}
ONLY documents that were imported from MongoDB Compass are showing up. Documents that were added by Mongosh or by NodeJS driver (doesn't matter), do NOT show up.
Am I missing something obvious here? What should I do in order to make all documents in a collection (that matches filter) to show up?
Simple filters that do not include $elemMatch work well and all documents that match the filtering rules show up. Problem seems to be with $elemMatch.
I tried adding the same batch of documents using different methods but only direct importing a JSON file in MongoDB Compass make them appear using a filter mentioned above.
Thank you for your help!
$elemMatch if for matching array , and in this case you don't have array
first you should convert contents object to array and then check the query for example id with filter and use match to find all doc that have specific data and size of new filters array
db.collection.aggregate([
{
"$addFields": {
"newField": {
"$objectToArray": "$contents"
}
}
},
{
"$addFields": {
"newField": {
"$filter": {
"input": "$newField",
"as": "z",
"cond": {
$eq: [
"$$z.v.content.id",
"1111111111"
]
}
}
}
}
},
{
"$addFields": {
"newField": {
$size: "$newField"
}
}
},
{
$match: {$and:[ {newField: {
$gt: 0
}},{date:{$gt:Date}}]}
},
{$project:{
contents:1,
date:1,
id:1,
}}
])
https://mongoplayground.net/p/pue4QPp1dYR
in mongoplayground I don't add filter of date

Mongodb find document from multiple collections

in nodejs and mongodb, usin mongoose ,
how can I query multiple collections?
for example: I have 3 collections:
mycollection1, mycollection2, mycollection3
I want to create query like findOne or findMany on mycollection*
and the query will return al the documents that exist in those collections
*(the same as I can do in Elasticsearch)
Thanks,
Larry
you can use $unionWith
db.coll1.aggregate([
{
"$unionWith": {
"coll": "coll2"
}
},
{
"$unionWith": {
"coll": "coll3"
}
},
{
"$match": {
"$expr": {
"$eq": [
"$a",
1
]
}
}
}
])
Test it here
Its better to do the filters before the union, and use this(if you have filters) (the above filters after), you can take the $match and add it to the each union.
{ $unionWith: { coll: "<collection>", pipeline: [ <stage1>, ... ] } }

how to project fields using another field's value in mongo db?

I have a mongo document like this:
{"_id": {"$oid":"xx"} ,"start": "a", "elements": {"a":"large object", "b": "large object"}
My expected query result is to project only the start element, in this case, it is {"elements.a:"large object"}. But with the value of "start" unknow before the query, I don't know how to write the query.
2 undesirable alternatives:
One way I could figure is to query start once with _id, and project for start to get "a", and another for elements.a。(
Another way is query all, and get the start element in code. But I don't want to query all at once for the document may be very large)
You can make use of $objectToArray, $arrayToObject and $filter operators.
The below query will be helpful:
db.collection.aggregate([
{
$project: {
elements: {
$arrayToObject: {
$filter: {
input: {
$objectToArray: "$elements"
},
as: "e",
cond: {
$eq: [
"$$e.k",
"$start"
]
}
}
}
}
}
}
])
Output:
[
{
"_id": 1,
"elements": {
"a": "large object"
}
}
]
MongoPlayGroundLink
I hope, this is what you want.

unable to access date value after lookup in mongodb

Main Collection
{
"_id": ObjectID("5ea1a07bfd7e4965408a5171"),
"data": [],
"history_id": ObjectID("5e4e755b380054797d9db627"),
"sender_id": ObjectID("5e4e74eb380054797d9db623"),
"text": "Hi tester",
"date": 1587650683434
}
History collection
{
"_id": ObjectID("5ea1afd4f4151402efd234e3"),
"user_id": [
"5e4a8d2d3952132a08ae5764"
],
"dialog_id": ObjectID("5e4e755b380054797d9db627"),
"date": 1587549034211,
"__v": 1
}
const messages = await MainModal.aggregate([
{
$lookup: {
from: 'history',
localField: 'history_id',
foreignField: 'history_id',
as: 'History'
}
},
{ $unwind: '$History' },
{ "$match" : { date: {$gt: "History.date" } } }, // this is not working
])
I am getting value inside $history but enable to fetch matched record. I don't know why i read somewhere $gt does work on number my date is number too
when i set string instead "History.date" it does work but after putting this doesnot work
Basically my idea is not to display those value whose date are $lt from history collection and where user not in 5e4a8d2d3952132a08ae5764
You are mixing the query language with aggregation operators. In order to use $match to compare two fields from the same document, you will need to use $expr along with aggregation operators.
For that final match stage, use
{ "$match":{ "$expr":{ "$gt":[ "$date", "$History.date" ] } } }
Edit
Adding an additional comparison between other fields would need to use $and or $or, like:
{ "$match":{
"$expr":{
"$and":[
{"$gt":[ "$date", "$History.date" ] },
{"$not": {"$eq":[ "$sender_id", "$History.user_id" ]}}
]
}
}}
Playground