Mongo does not allow documents to have dots in their keys (see MongoDB dot (.) in key name or https://softwareengineering.stackexchange.com/questions/286922/inserting-json-document-with-in-key-to-mongodb ).
However we have a huge mongo database where some documents do contain dots in their keys. These documents are of the form:
{
"_id" : NumberLong(2761632),
"data" : {
"field.with.dots" : { ... }
}
}
I don't know how these records got inserted. I suspect that we must have had the check_keys mongod option set to false at some point.
My goal is to find the offending documents, to update them and remove the dots. I haven't found how to perform the search query. Here is what I tried so far:
db.collection.find({"data.field.with.dots" : { $exists : true }})
db.collection.find({"data.field\uff0ewith\uff0edots" : { $exists : true}})
You can use $objectToArray to get your data in form of keys and values. Then you can use $filter with $indexOfBytes to check if there are any keys with . inside of it . In the next step you can use $size to filter out those documents where remaining array is empty (no fields with dots), try:
db.col.aggregate([
{
$addFields: {
dataKv: {
$filter: {
input: { $objectToArray: "$data" },
cond: {
$ne: [ { $indexOfBytes: [ "$$this.k", "." ] } , -1 ]
}
}
}
}
},
{
$match: {
$expr: {
$ne: [ { $size: "$dataKv" }, 0 ]
}
}
},
{
$project: {
dataKv: 0
}
}
])
Mongo playground
Related
I am trying to update the property name of the json in mongodb document.
{
"_id" : ObjectId("1234556789"),
"apps" : [
{
"_id" : 101,
"regions" : [
"WANAE",
"WANAF"
]
},
{
"_id" : 102,
"regions" : [
"WANAE",
"WANAF"
]
}
]
}
in the above josn, I want to change apps regions to codes. Treid below queries but did not work
db.packs.updateMany( {}, { $rename: { 'apps.$.regions': 'apps.$.codes' } } );
db.packs.updateMany( {}, { $rename: { 'apps.$[].regions': 'apps.$[].codes' } } );
any help
Update: As Joe suggested, I have a aggregation that changes the document with the changes needed and I tried updating the entire collection like below with the aggregated result
db.packs.aggregate([
{
$addFields: {
apps: {
$map: {
input: "$apps",
as: "app",
in: {
_id: "$$app._id",
did: "$$app.did",
name: "$$app.name",
codes: "$$app.regions"
}
}
}
}
},
{
$project:{
"apps.regions":0
}
},
{
$out:"packs"
}
])
As per the documentation, $out should replace the existing collection if it is exists but I received an error that says I have to supply a new collection name Please supply a collection that does not already exist to the $out stage.. Isn't $Out replace the exiting packs with new aggregated results
When you reference a field in an array of objects, like "$apps.regions", the value is an array containing all of the values of that field from all of the elements.
If you set the value of regions directly, each sub document will contain an array of arrays, probably not what you want.
renaming the field in the entire array of objects will require iterating the array, perhaps with $map or $reduce.
If you are using MongoDB 4.2, you can do that with a pipeline in an update:
db.packs.updateMany( {}, [
{$set: {
"apps": {
$map: {
input: "$apps",
in: {
$mergeObjects: [
"$$this",
{
codes: "$$this.regions"
}
]
}
}
}
}},
{$unset: "apps.regions"}
]}
If you are using an earlier version, you'll need to do that with aggregation, perhaps with $out, and then replace the original collection with the updated one.
I have the below structure for my collection:
{
"price":123,
"totalPrices": [
{
"totPrice":123
}
]
}
I am trying to query for all the documents in my collection where price is not equals to totalPrice.totPrice (so above should not be returned).
But it keeps returning the documents which have equal prices as well (such as above sample).
This is the query I'm using:
{
$where : "this.price!== this.totalPrices.totPrice",
totalPrice:{$size:1}
}
What am I doing wrong :(
First, you need to match the size of the array totalPrices is equal to 1. Second, you need to unwind the totalPrices, since it's an array field. Last, you should match the equality of price and totalPrices.totPrice. Try the below code:
db.collection.aggregate([
{
$match: {
$expr: {
$eq: [
{
$size: "$totalPrices"
},
1
]
}
}
},
{
$unwind: "$totalPrices"
},
{
$match: {
$expr: {
$ne: [
"$price",
"$totalPrices.totPrice"
]
}
}
}
])
MongoPlayGroundLink
In a mongodb collection , i have following documents :
{"id":"1234","name":"John","stateCode":"CA"}
{"id":"1234","name":"Smith","stateCode":"CA"}
{"id":"1234","name":"Tony","stateCode":"GA"}
{"id":"3323", "name":"Neo","stateCode":"OH"}
{"id":"3323", "name":"Sam","stateCode":"US"}
{"id":"4343","name":"Bruce","stateCode":"NV"}
I am trying to write a mongo aggregate query which do following things:
match based on id field
Give more priority to document having values other than "NV" or "GA" in stateCode field.
If all the document have values either "NV" or "GA" then ignore the priority.
If any of the document have stateCode other than "NV" or "GA" , then return those document.
Example 1:
id = "1234"
then return
{"id":"1234","name":"John","stateCode":"CA"}
{"id":"1234","name":"Smith","stateCode":"CA"}
Example 2:
id = "4343"
then return
{"id":"4343","name":"Bruce","stateCode":"NV"}
Could you please help with a query to achieve this.
I tried with a query , but i am stuck with error:
Failed to execute script.
Error: command failed: {
"ok" : 0,
"errmsg" : "input to $filter must be an array not string",
"code" : 28651,
"codeName" : "Location28651"
} : aggregate failed
Query :
db.getCollection('emp').aggregate([{$match:{
'id': "1234"
}
},
{
$project: {
"data": {
$filter: {
input: "$stateCode",
as: "data",
cond: { $ne: [ "$data", "GA" ],$ne: [ "$data", "NV" ] }
}
}
}
}
])
I actually recommend you split this into 2 queries, first try to find documents with a different status code and if that fails then retrieve the rest.
With that said here is a working pipeline that does it in one go, Due to the fact we cant know in advance whether the condition is true or not we need to iterate all the documents who match the id, this fact makes it VERY inefficient in the case the id is shared by many documents, if this is not possible then using this pipeline is fine.
db.getCollection('emp').aggregate([
{
$match: {
'id': "1234"
}
},
{ //we have to group so we can check
$group: {
_id: null,
docs: {$push: "$$ROOT"}
}
},
{
$addFields: {
highPriorityDocs: {
$filter: {
input: "$docs",
as: "doc",
cond: {$and: [{$ne: ["$$doc.stateCode", "NV"]}, {$ne: ["$$doc.stateCode", "GA"]}]}
}
}
}
},
{
$project: {
finalDocs: {
$cond: [ // if size of high priority docs gt 0 return them.
{$gt: [{$ize: "$highPriorityDocs"}, 0]},
"$highPriorityDocs",
"$docs"
]
}
}
},
{
$unwind: "$finalDocs"
},
{
$replaceRoot: {newRoot: "$finalDocs"}
}
])
The last two stages are just to restore the original structure, you can drop them if you don't care about it.
Mongodb: 4.0.13
I'm having troubles in understand and get working $expr with arrays.
Let' start and create a new collection (dbRepeatElement) with following document:
db.testRepeatElement.insert([
{
"data" : {
"FlsResSemires_2" : {
"Sospensione" : [
{
"DataInizio" : 1548806400000,
"DataFine" : 1549065600000,
"Motivazione" : "1"
}
]
}
},
"derived" : {
"DATAFINEANNORIFERIMENTO" : 1609372800000,
"regione190" : "190",
"REGAOEROG" : "190209820300",
"REGASLEROG" : "190209"
}
}
])
In a bigger aggregation, following part is not working:
db.testRepeatElement.aggregate([
{
$match: {
$expr: {
$gt: ["$data.FlsResSemires_2.Sospensione.DataInizio", "$derived.DATAFINEANNORIFERIMENTO"]
}
}
}
])
Result: return a match ( wrong! just check dates)
Reading mongodb documentation seems to be, using combination with arrays, aggregation and $expr does not return expected result and you have to specify with element of the array you want to check, like:
db.testRepeatElement.aggregate([
{
$match: {
$expr: {
$gt: ["$data.FlsResSemires_2.0.Sospensione.DataInizio", "$derived.DATAFINEANNORIFERIMENTO"]
}
}
}
])
Result: return no match (right!)
Question: my requirement is to check every element in the array, so how to solve this, without using $unwind? Why there is this kind of result ?
The $filter aggregation operator is used to do the match operation on array elements. The following aggregation query will result only the Sospensione array elements which match the $gt condition:
db.testRepeatElement.aggregate( [
{
$addFields: {
"data.FlsResSemires_2.Sospensione": {
$filter: {
input: "$data.FlsResSemires_2.Sospensione",
cond: {
$gt: [ "$$this.DataInizio", "$derived.DATAFINEANNORIFERIMENTO" ]
}
}
}
}
},
{
$match: {
$expr: {
$gt: [ { $size: "$data.FlsResSemires_2.Sospensione" }, 0 ]
}
}
}
] ).pretty()
I've following mongodb query:
db
.getCollection("entries")
.find({
$and: [
{
"array.attribute_1": {
$exists: true,
$not: {
$size: 0
}
}
},
{
$or: [
{ "array.attribute_2": { $exists: true, $size: 0 } },
{ "array.attribute_2": { $exists: true, $eq: {} } }
]
},
]
})
And example of my document:
{
_id: 'foo',
array: [
{attribute_1: [], attribute_2: []},
{attribute_1: ['bar'], attribute_2: []}
]
}
In my understanding my query should find all entries that have at least one element within array that has existent and not empty attribute_1 and existent empty array or empty object attribute_2. However, this query finds all entries that has all elements within array that has existent and not empty attribute_1 and existent empty array or empty object attribute_2. As such, my foo entry won't be found.
What should be the correct formula for my requirements?
$find would find the first document with the matching criteria and in your case that first document contains all the arrays. You either need to use $project with $filter or aggregation with $unwind and $match.
Something like this:
db.collection.aggregate([
{ $unwind: "$array" },
{
$match: {
$and: [
{ "array.attribute_1.0": { $exists: true }},
{
$or: [
{ "array.attribute_2.0": { $exists: false } },
{ "array.attribute_2.0": { $eq: {} } }
]
}
]
}
}
])
You can see it working here
Also since you are trying to find out if array is empty and exists at the same time using .0 with $exists is a quick and one statement way to get the same result as with both $exists and $size.