MongoDB query or aggregation to skip sub-documents - mongodb

I'd like to create a query or aggregation where the returned documents do not include sub-documents. I do not know that a given field will be a sub-document ahead of time (or I would just use the projection to skip them). So for example, if I have a document like this:
{
_id: 1,
field1: "a",
field2: "b",
field3: {
subfield1: "c",
subfield2: "d"
}
}
When my query returns this document, it either skips field3, or replaces field3's value with something else (e.g. a string = "field_is_an_object").
As I said, I don't know ahead of time which fields will be sub-documents (or "object" types). The $redact operator was the closest I could find, but I couldn't figure out a syntax to get it to work.

There are at least two ways you can achieve what you want:
The first one is pretty concise and requires just one aggregation stage which, however, is a little bit more complex and harder to understand:
db.collection.aggregate({
$replaceRoot: { // create a new top level document
"newRoot": { // ...which shall be
$arrayToObject: { // ...created from an array
$filter: { // ...that again should contain only those elements
input: { // ...from our input array
$objectToArray: "$$ROOT" // ...which is our respective top level document transformed into an array of key-value pairs
},
cond: { // ...where
$ne: [ { $type: "$$this.v" }, "object" ] // ...the "v" (as in "value" field is not an object)
}
}
}
}
}
})
The second one I can think of is way more verbose but pretty easy to understand by adding the stages step-by-step (as always with the aggregation framework).
db.collection.aggregate({
$project: {
"tmp": { // we create a temporary field
$objectToArray: "$$ROOT" // that contains our respective root document represented as an array of key-value pairs
}
}
}, {
$unwind: "$tmp" // flatten the temporary array into multiple documents
}, {
$match: {
"tmp.v": { $not: { $type: "object" } } // filter all documents out that we do not want in our result
}
}, {
$group: { // group all documents together again
"_id": "$_id", // into one bucket per original document ("_id")
"tmp": {
$push: "$tmp" // and create an array with all the key-value pairs that have survived our $match stage
}
}
}, {
$replaceRoot: { // create a new top level document...
"newRoot": {
$arrayToObject: "$tmp" // ...out of the data we have left in our array
}
}
})

Related

can't get data for unknown key in mongodb query

I want to get a subarray from a document using mongodb queries but I don't know the key for it. My data looks something like this:
{ 'log file name I dont know': [Array] }
'log file name I dont know' is the unique key at that level. Is there something like "get value for the first key" in mongodb aggregation pipeline?
1 Maybe something like this:
db.collection.aggregate([
{
$group: {
_id: "$_id",
new: {
$first: "$$ROOT"
}
}
},
{
"$project": {
"map": {
"$objectToArray": "$new"
}
}
},
{
$unwind: "$map"
},
{
$match: {
"map.k": {
"$ne": "_id"
},
"map.v": 4
}
}
])
Explained:
Add the root document to new key named "new"
Convert the object named "new" to array named "map"
$unwind the array
Remove the array elements with k:"_id" , so in the final result to have only:
map.k -> the unknown keys
map.v -> the array values for the unknown keys
Finally you can add in the $match stage the searched element , in the example:"map.v":4
playground1
2 Here is the option when you want to preserve the strange structure with the unknown key after the match stage:
db.collection.aggregate([
{
$group: {
_id: "$_id",
new: {
$first: "$$ROOT"
}
}
},
{
"$project": {
"map": {
"$objectToArray": "$new"
}
}
},
{
$match: {
"map.v": 1
}
},
{
"$project": {
"map": {
"$arrayToObject": "$map"
}
}
},
{
$replaceRoot: {
newRoot: "$map"
}
}
])
Explained:
Add the root document to new key named "new"
Convert the object named "new" to array named "map"
Match the necessary documents(in the example this is array element 1 ( "map.v":1 )
Convert back the array to object.
Replace the root tocument with the "map docunent" so it looks as original collection
playground2
Few words: File name value is not recommended to be a key , best practice is the filename to be key/value so if you want to search on filename value to be possible to create index on that filename key field , so your json structure to looke something like:
{ "filename":"The unknown file name" , "theArray":[1,2,3] }

How do I Convert a subset of fields in all documents to $toDouble?

I have a database with many documents structured like so:
Date:2021-01-02T08:00:00.000+00:00
FIT_ME44_Volume:"984385"
LT_MJ01:"0"
LT_MJ08:"0"
LT_MJ16:"-34.526024"
FIT_ME56_Volume:"0"
FIT_MJ22_Volume:"9538598"
LT_MJ26:"-61.803848"
FIT_ME52_Volume:"2734271"
LT_ME16:"0"
FIT_MJ28_Volume:"0"
LT_ME29:"2.10552"
LT_ME02:"2.005206"
LT_ME50:"8.732683"
FIT_MJ13_Volume:"0"
FIT_ME02_Volume:"1131376"
FIT_ME23_Volume:"2585415"
LT_ME03:"6.918576"
FIT_MJ08_Volume:"0"
FIT_MJ18_Volume:"0"
QQCr_Total_Plant:"5471052"
FIT_ME03_Volume:"103164"
FIT_ME51_Volume:"3587575"
LT_ME06:"24.423439"
FIT_ME46_Volume:"1619"
What I would like to do is convert all fields in all documents except Date from string to double. I can do this on a field by field basis, but how do I accomplish this in bulk?
You can try an update with aggregation pipeline query starting from MongoDB 4.2,
$objectToArray convert root document from object to key-value array of object format
$map to iterate loop of above converted array
return k key as it is
$cond to check is key in the provided array then don't convert means ignore and return existing value, you can also add fields name if don't want to convert
else $toDouble to convert a string value to double
$arrayToObject back to convert above key-value array of objects to an real object format
$replaceRoot to replace above converted object to root
db.collection.updateMany(
{},
[{
$replaceRoot: {
newRoot: {
$arrayToObject: {
$map: {
input: { $objectToArray: "$$ROOT" },
in: {
k: "$$this.k",
v: {
$cond: [
{ $in: ["$$this.k", ["Date", "_id"]] },
"$$this.v",
{ $toDouble: "$$this.v" }
]
}
}
}
}
}
}
}]
)
Playground

How to group documents of a collection to a map with unique field values as key and count of documents as mapped value in mongodb?

I need a mongodb query to get the list or map of values with unique value of the field(f) as the key in the collection and count of documents having the same value in the field(f) as the mapped value. How can I achieve this ?
Example:
Document1: {"id":"1","name":"n1","city":"c1"}
Document2: {"id":"2","name":"n2","city":"c2"}
Document3: {"id":"3","name":"n1","city":"c3"}
Document4: {"id":"4","name":"n1","city":"c5"}
Document5: {"id":"5","name":"n2","city":"c2"}
Document6: {"id":"6,""name":"n1","city":"c8"}
Document7: {"id":"7","name":"n3","city":"c9"}
Document8: {"id":"8","name":"n2","city":"c6"}
Query result should be something like this if group by field is "name":
{"n1":"4",
"n2":"3",
"n3":"1"}
It would be nice if the list is also sorted in the descending order.
It's worth noting, using data points as field names (keys) is somewhat considered an anti-pattern and makes tooling difficult. Nonetheless if you insist on having data points as field names you can use this complicated aggregation to perform the query output you desire...
Aggregation
db.collection.aggregate([
{
$group: { _id: "$name", "count": { "$sum": 1} }
},
{
$sort: { "count": -1 }
},
{
$group: { _id: null, "values": { "$push": { "name": "$_id", "count": "$count" } } }
},
{
$project:
{
_id: 0,
results:
{
$arrayToObject:
{
$map:
{
input: "$values",
as: "pair",
in: ["$$pair.name", "$$pair.count"]
}
}
}
}
},
{
$replaceRoot: { newRoot: "$results" }
}
])
Aggregation Explanation
This is a 5 stage aggregation consisting of the following...
$group - get the count of the data as required by name.
$sort - sort the results with count descending.
$group - place results into an array for the next stage.
$project - use the $arrayToObject and $map to pivot the data such
that a data point can be a field name.
$replaceRoot - make results the top level fields.
Sample Results
{ "n1" : 4, "n2" : 3, "n3" : 1 }
For whatever reason, you show desired results having count as a string, but my results show the count as an integer. I assume that is not an issue, and may actually be preferred.

How to sort a dictionary keys and pick the first in MongoDb?

I'm running the following query as described in the docs.
db.getCollection('things')
.find(
{ _id: UUID("...") },
{ _id: 0, history: 1 }
)
It produces a single element that, when unfolded in the GUI, shows the dictonary history. When I unfold that, I get to see the contents: bunch of keys and correlated values.
Now, I'd like to sort the keys alphabetically and pick n first ones. Please note that it's not an array but a dictionary that is stored. Also, it would be great if I could flatten the structure and pop up my history to be the head (root?) of the document returned.
I understand it's about projection and slicing. However, I'm not getting anywhere, despite many attempts. I get syntax errors or a full list of elements. Being rather nooby, I fear that I require a few pointers on how to diagnose my issue to begin with.
Based on the comments, I tried with aggregate and $sort. Regrettably, I only seem to be sorting the current output (that produces a single document due to the match condition). I want to access the elements inside history.
db.getCollection('things')
.aggregate([
{ $match: { _id: UUID("...") } },
{ $sort: { history: 1 } }
])
I'm sensing that I should use projection to pull out a list of elements residing under history but I'm getting no success using the below.
db.getCollection('things')
.aggregate([
{ $match: { _id: UUID("...") } },
{ $project: { history: 1, _id: 0 } }
])
It is a long process to just sort object properties by alphabetical order,
$objectToArray convert history object to array in key-value format
$unwind deconstruct above generated array
$sort by history key by ascending order (1 = ascending, -1 = descending)
$group by _id and reconstruct history key-value array
$slice to get your number of properties from dictionary from top, i have entered 1
$arrayToObject back to convert key-value array to object format
db.getCollection('things').aggregate([
{ $match: { _id: UUID("...") } },
{ $project: { history: { $objectToArray: "$history" } } },
{ $unwind: "$history" },
{ $sort: { "history.k": 1 } },
{
$group: {
_id: "$_id",
history: { $push: "$history" }
}
},
{
$project: {
history: {
$arrayToObject: { $slice: ["$history", 1] }
}
}
}
])
Playground
There is another option, but as per MongoDB, it can not guarantee this will reproduce the exact result,
$objectToArray convert history object to array in key-value format
$setUnion basically this operator will get unique elements from an array, but as per experience, it will sort elements by key ascending order, so as per MongoDB there is no guarantee.
$slice to get your number of properties from dictionary from top, i have entered 1
$arrayToObject back to convert key-value array to object format
db.getCollection('things').aggregate([
{ $match: { _id: UUID("...") } },
{
$project: {
history: {
$arrayToObject: {
$slice: [
{ $setUnion: { $objectToArray: "$history" } },
1
]
}
}
}
}
])
Playground

Project array elements as individual fields

I have a collection with a field in each document that looks like:
{
person = [{"name":"Alex"},{"name":"Betsy"},{"name":"Chauncy"}]
}
Documents can have different length for this array
I'm trying to project the individual element of this array as fields within the same document with the key as Name {index} and the corresponding value for the name. I can't seem to generate the projected fields given an array.
Expected output:
[{
"name1": "Alex",
"name2": "Betsy",
"name3": "Chauncy"
}]
You can use aggregation
$unwind to deconstruct the array and it helps to get the index too
$group to reconstruct the array with key (k): value(v) pair which helps in next stage
$arrayToObject since group already made k:v pair, this helps to make array to opbect
$replaceRoot helps to make the object as root
Here is the code
db.collection.aggregate([
{
$unwind: { path: "$person", includeArrayIndex: "index" }
},
{
$group: {
_id: null,
person: {
$push: {
k: { "$concat": [ "name", { $toString: "$index" } ] },
v: "$person.name"
}
}
}
},
{
$project: {
person: { "$arrayToObject": "$person" }
}
},
{
"$replaceRoot": { "newRoot": "$person" }
}
])
Working Mongo playground