I recently started working with MongoDB (version 3.2.9) and wanted to try the validation capabilities it had, as I like database-level consistency.
I know I can, and should only, do this validation in the business logic, as I already do.
But it feels nice having an additional layer of security.
I tried adding a collection category with an _id of type 'int', name of type 'string' and desc of type 'string'.
These values should all exist, but restrictions on the content are not given.
Now, following the manual, I ended up with the following code:
db.createCollection('category', {
validator: { $or:
[
{ _id: { $and: [ { $exists: true },
{ $type: 'int' } ] } },
{ name: { $and: [ { $exists: true },
{ $type: 'string' } ] } },
{ desc: { $and: [ { $exists: true },
{ $type: 'string' } ] } }
]
}
})
However, with this simple structure already, MongoDB complains:
{ "ok" : 0, "errmsg" : "unknown operator: $and", "code" : 2 }
Now, I have found this similar question, but just went and tried to put multiple constraints directly inside the $or arrays:
db.createCollection('category', {
validator: { $or:
[
{ _id: { $exists: true } },
{ _id: { $type: 'int' } },
{ name: { $exists: true } },
{ name: { $type: 'string' } },
{ desc: { $exists: true } },
{ desc: { $type: 'string' } }
]
}
})
But now, while it does create the collection like it should, I can now execute any of the following insert commands:
db.category.insert({})
db.category.insert({_id: 17, name: '', desc: 'valid'})
db.category.insert({_id: 42, name: 42, desc: ''})
db.category.insert({_id: 43, name: '', desc: 42})
db.category.insert({_id: '', name: 42, desc: 42})
db.category.find() now returns
{ "_id" : ObjectId("57e19650b10ab85eca323684") }
{ "_id" : 17, "name" : "", "desc" : "valid" }
{ "_id" : 42, "name" : 42, "desc" : "" }
{ "_id" : 43, "name" : "", "desc" : 42 }
{ "_id" : "", "name" : 42, "desc" : 42 }
As a last resort, I tried changing the $or operator to an $and, as it should need all rules to be valid, not just at least one.
db.createCollection('category', {
validator: { $and:
[
{ _id: { $exists: true } },
{ _id: { $type: 'int' } },
{ name: { $exists: true } },
{ name: { $type: 'string' } },
{ desc: { $exists: true } },
{ desc: { $type: 'string' } }
]
}
})
With this approach which seems the most rational to me, it however does not work at all. No matter which of the above mentioned inserts I tried using, mind you there's one valid out of the 5, none works, all giving me the same error:
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 121,
"errmsg" : "Document failed validation"
}
})
As stated above, a call to db.version() returns 3.2.9, and I am running the default MongoDB 64-bit distribution on a Windows machine with only the --dbpath parameter set to a previously empty directory.
Please provide the 32-bit integer value using the function NumberInt(). Otherwise, it is interpreted as Double.
The following insert should successfully insert a document into collection.
Success scenario:-
db.category.insert({_id: NumberInt(17), name: "aaaa", desc: "valid"})
Failure scenario:-
db.category.insert({_id: NumberInt(42), name: 42, desc: ''})
Related
I went through this link: How to rename a document field in a MongoDB?, but its not working fine for me. I've Mongo Document like below. I am using MongoDB server version: 4.0.3.
{
"_id" : ObjectId("5cb825e566135255e0bf38a4"),
"firstName" : "John",
"lastName" : "svc_user",
.....
.....
"status" : "A",
"effDate" : ISODate("2012-08-24T01:46:33.000Z"),
"department" : [
{
"deptName" : "KG",
....
....
},
...
....
.....
],
...
...
...
}
I executed the below query:
db.employee.update({}, {$rename:{"department.deptName":"department.departmentName"}}, false, true);
Error:
cannot use the part (department of department.deptName) to traverse the element
In Mongo 4.2 you could run this one:
db.collection.updateMany(
{
department: { $exists: true },
"department.deptName": { $exists: true }
},
[{
$set: {
department: {
$map: {
input: "$department",
in: { departmentName: "$$this.deptName" }
}
}
}
}]
)
I assume in 4.0.3. you have to modify the field name one-by-one with a loop in JavaScript
I want to know how to write the mongodb script for updating the field in my db
my collection is something like this
{
_id: 1,
name: xyz,
answer: [
{ type: 'C',
from: 0
},
{ type: 'M',
from: 0
},
{ type: 'P',
from: 0
}
........ so on
]
}
.
.
.
and other objects
I want to add a field called "test" in each object of answer array whose "type" value is not 'C'
I am new to this. Can someone help me how should i do this.
Are you looking for this:
db.col.updateOne(
{ name: "xyz" },
{ "$set": { 'answer.$[i].test': null } },
{ arrayFilters: [{ "i.type": { $ne: "C" } }] }
)
Result:
{
"name" : "xyz",
"answer" : [
{
"type" : "C",
"from" : 0.0
},
{
"type" : "M",
"from" : 0.0,
"test" : null
},
{
"type" : "P",
"from" : 0.0,
"test" : null
}
]
}
If you like to update all documents in your collection use updateMany() and skip filter { name: "xyz" }
Of course you can run the update manually like this:
db.col.find().forEach(function (doc) {
doc.answer.forEach(function (i) {
if (i.type != "C")
i.test = null;
})
db.col.updateOne(
{ _id: doc._id },
{ $set: doc }
)
})
but you have to admit, arrayFilters is shorter and more convenient.
I have documents representing Users with onboarding data stored as a nested object:
{
"_id" : ObjectId("5c7eb0132e6f793bcc7f4bf7"),
"userName" : "sample_user_name",
"onBoarding" : {
"completed" : ISODate("2019-03-05T17:46:28.803Z"),
"stepId" : 8,
"started" : null
}
}
But due to a bug we are missing a date when onboarding was started, I would like to "retrieve" this information by running an update operation where "started" will be set to the same date as "completed". I have a query like:
db.getCollection('user').updateMany(
{
$and: [
{"onBoarding.started": {$exists: false}},
{"onBoarding.completed": {$exists: true}}
]},
{
$set: { "onBoarding.started": "$onBoarding.completed" }
})
This however, sets "started" to "$onBoarding" literally (as a string).
{
"_id" : ObjectId("5c7eb0132e6f793bcc7f4bf7"),
"userName" : "sample_user_name",
"onBoarding" : {
"completed" : ISODate("2019-03-05T17:46:28.803Z"),
"stepId" : 8,
"started" : "$onBoarding"
}
}
How should I write it for mongo to take a value from "onBoarding.completed" and copy this value to "onBoarding.started"??
Expected result document should look like:
{
"_id" : ObjectId("5c7eb0132e6f793bcc7f4bf7"),
"userName" : "sample_user_name",
"onBoarding" : {
"completed" : ISODate("2019-03-05T17:46:28.803Z"),
"stepId" : 8,
"started" : ISODate("2019-03-05T17:46:28.803Z")
}
}
You need to use an aggregation pipeline to be able to use the value of another field :
db.user.updateMany(
{ <your query selector > },
[
{ $set: { onBoarding.started: "$onBoarding.completed" } },
]
)
Be aware that here, $set refers to the aggregation pipeline stage and not the update operator $set : https://docs.mongodb.com/manual/reference/method/db.collection.updateMany/index.html#update-with-aggregation-pipeline
The exists operator checks for existence of a field. If a field has a value of null, the query still returns true (because the field still exists, only its value is null).
The following queries behave differently with this input document: { _id: 1, fld1: 123, fld2: null }
db.test.find( { fld2: { exists: false } } ) returns false.
db.test.find( { fld2: null } } ) returns true.
Coming back to the data in question - the following query / script will update all the documents with following condition: ( "onBoarding.started" is null or the field doesn't exist ) and ( "onBoarding.completed" field exists and is not null ).
db.test.find( { $and: [ { $or: [ { "onBoarding.started": null }, { "onBoarding.started": { $exists: false } } ] }, { $and: [ { "onBoarding.completed": { $exists: true } }, { "onBoarding.completed": { $ne: null } } ] } ] } ).forEach( doc => db.test.updateOne( { _id: doc._id }, { $set: { "onBoarding.started" : doc.onBoarding.completed } } ) )
Using MongoDB version 4.2, the same update can be run as follows. Note the update uses an Aggregation stage (see documetation on update).
db.test.updateMany(
{ $and: [ { $or: [ { "onBoarding.started": null }, { "onBoarding.started": { $exists: false } } ] }, { $and: [ { "onBoarding.completed": { $exists: true } }, { "onBoarding.completed": { $ne: null } } ] } ] },
[
{ $addFields:
{ "onBoarding.started" : "$onBoarding.completed" }
}
]
)
I'm trying the aggregation below, but do not appear to be getting the expected result using $ne and null.
I have tried other solutions like using a combination of $cond, $not, and $eq to no avail. Using $gt:[ "$unloadeddate", null] seems to give some results, but that does not seem to be proper syntax and I am concerned it's not trustworthy to run properly over entire dataset.
Also, querying as follows:
db.getCollection('esInvoices').find({"esBlendTickets.loadeddate":{$ne:null}})
... returns results so not sure why the same query in the aggregate function not working.
Any help is appreciated!!
The first part works...
"unloadeddate": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esBlendTickets.ticketdate", null]
},
then: "$esBlendTickets.ticketdate"
}, {
case: {
"$ne":[ "$esDeliveryTickets.ticketdate", null]
},
then: "$esDeliveryTickets.ticketdate"
}],
default: null
}
},
"loadeddate": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esBlendTickets.loadeddate", null]
},
then: "$esBlendTickets.loadeddate"
}, {
case: {
"$ne":[ "$esDeliveryTickets.loadeddate", null]
},
then: "$esDeliveryTickets.loadeddate"
}],
default: null
}
},
... but this second part (essentially the same logic except for the resulting value) does not work as expected...
"stagename": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esDeliveryTickets.ticketdate", null]
},
then: "Invoiced"
}, {
case: {
"$ne":[ "$esBlendTickets.ticketdate", null]
},
then: "Invoiced"
}],
default: "Invoiced-Only"
}
}
Complete Aggregation:
db.esInvoices.aggregate([ {
$addFields: {
// A single invoice will not have both a blend ticket and delivery ticket associated so looping tough each case should work.
"unloadeddate": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esBlendTickets.ticketdate", null]
},
then: "$esBlendTickets.ticketdate"
}, {
case: {
"$ne":[ "$esDeliveryTickets.ticketdate", null]
},
then: "$esDeliveryTickets.ticketdate"
}],
default: null
}
},
"loadeddate": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esBlendTickets.loadeddate", null]
},
then: "$esBlendTickets.loadeddate"
}, {
case: {
"$ne":[ "$esDeliveryTickets.loadeddate", null]
},
then: "$esDeliveryTickets.loadeddate"
}],
default: null
}
},
"stagedate": "$InvoiceHeader.InvDate",
"stagename": { "$switch": {
branches:[ {
case: {
"$ne":[ "$esDeliveryTickets.ticketdate", null]
},
then: "Invoiced"
}, {
case: {
"$ne":[ "$esBlendTickets.ticketdate", null]
},
then: "Invoiced"
}],
default: "Invoiced-Only"
}
}
}}])
Think I just ran into the same problem you were having. Using Mongo 3.4 at the moment. From what I can tell the query $ne behaves differently from the aggregate $ne when you are comparing it to null. Threw me off for a little bit.
Specifically, a { $ne: [ '$field', null ] } predicate in the aggregate pipeline will return true when $field is undefined.
However, when using queries (not $aggregate), the { field: { $ne: null } predicate will return false for those same documents when $field is undefined.
My work-around was to use a $project with { field: { $ifNull: [ '$field': null ] } } in a prior step to turn undefined instances of that field into explicit nulls, which will then make the aggregate $ne work as I needed. For whatever reason, $ifNull works with null, undefined, and missing fields. Not sure why $ne is different.
Here's an example to reproduce.
db.test.insertMany([
{ a: 1, b: 'string' },
{ a: 2, b: null },
{ a: 3 },
])
db.test.find({ b: { $ne: null }}, { _id: 0 })
/*
returns:
{
"a" : 1.0,
"b" : "string"
}
*/
db.test.aggregate([
{ $project: {
_id: 0,
a: 1,
b: 1,
switched: { $switch: {
branches: [
{ case: { $ne: [ '$b', null ] }, then: 'cased' },
],
default: 'default',
}}
}}
])
/*
returns:
{
"a" : 1.0,
"b" : "string",
"switched" : "cased"
},
{
"a" : 2.0,
"b" : null,
"switched" : "default"
},
{
"a" : 3.0,
"switched" : "cased" <--
}
*/
The problem is missing fields are undefined while null fields are, well, nulls. When you write "$ne":[ "$esDeliveryTickets.ticketdate", null] you don't filter the former but only the latter.
But undefined is "smaller" than null. In order to filter them both you just need to make lte/gt instead of eq/ne. So this query returns all existing values:
See example:
db.test.insertOne(
{
"a" : 10,
"b" : null
})
...
db.getCollection('mytest').find( {},
{
"a" : { $lte : ["$a", null] },
"b" : { $lte : ["$b", null] },
"c" : { $lte : ["$c", null] },
})
// {
// "_id" : ObjectId("606724f38ec4d26b981b5a1c"),
// "a" : false,
// "b" : true,
// "c" : true
// }
In your case:
"stagename": { "$switch": {
branches:[ {
case: {
"$gt":[ "$esDeliveryTickets.ticketdate", null]
},
then: "Invoiced"
}, {
case: {
"$gt":[ "$esBlendTickets.ticketdate", null]
},
then: "Invoiced"
}],
default: "Invoiced-Only"
}
}
and this one returns only missing (or null) values:
"stagename": { "$switch": {
branches:[ {
case: {
"$lte":[ "$esDeliveryTickets.ticketdate", null]
},
then: "Invoiced"
}, {
case: {
"$lte":[ "$esBlendTickets.ticketdate", null]
},
then: "Invoiced"
}],
default: "Invoiced-Only"
}
}
Basically, I am new to MongoDB and learning from documents. I wanted to use validator, validationLevel, validationAction while defining a collection in MongoDB. So I can get an error if validated data is not inserted or updated. I have followed this document for reference. However, I got success in defining collection but when I add the wrong record it doesn't give me error like it is mentioned in documents. Everytime new record is getting inserted. Can you guys please help me in this whats wrong with my code.
I am using the following command to create a collection
db.createCollection( "contacts",
{
validator: { $and:
[
{ phone: { $type: "string" } },
{ email: { $regex: /#mongodb\.com$/ } },
{ status: { $in: [ "Unknown", "Incomplete" ] } }
],
validationAction: "error"
}
}
)
I have also used the same command with validationLevel to strict like below but it also didn't work out.
db.createCollection( "contacts",
{
validator: { $and:
[
{ phone: { $type: "string" } },
{ email: { $regex: /#mongodb\.com$/ } },
{ status: { $in: [ "Unknown", "Incomplete" ] } }
],
validationAction: "error",
validationLevel: "strict"
}
}
)
A command I used to insert record is as below
db.contacts.insert( { name: "Amanda", status: "Updated" } )
Below is the expected output as per the document.
WriteResult({
"nInserted" : 0,
"writeError" : {
"code" : 121,
"errmsg" : "Document failed validation"
}
})
But I end up getting success output as below
WriteResult({ "nInserted" : 1 })
I guess you validation rule is not applied to the collection. You can check for the same using
getCollectionInfos({name:'constacts'})
here is the link.
Error is in the validationAction and validationLevel.
use this
db.createCollection( "contacts", { validator:
{ $and:
[
{ phone: { $type: "string" } },
{ email: { $regex: /#mongodb\.com$/ } },
{ status: { $in: [ "Unknown", "Incomplete" ] } }
]
},
validationLevel : "strict",
validationAction : "error"
}
)
insted of
db.createCollection( "contacts",
{
validator: { $and:
[
{ phone: { $type: "string" } },
{ email: { $regex: /#mongodb\.com$/ } },
{ status: { $in: [ "Unknown", "Incomplete" ] } }
],
validationAction: "error",
validationLevel: "strict"
}
}
)
Quote from documentation:
You cannot specify a validator for collections in the admin, local,
and config databases. You cannot specify a validator for system.*
collections.
https://docs.mongodb.com/v3.2/reference/method/db.createCollection/