I was checking logic for querying on non-values and noticed when using the mongo shell, it differentiates between undefined and null values.
> use test
> db.test.insert({ a : 1, b : null, c : undefined })
> db.test.insert({ a : null, b : undefined, c : 1 })
> db.test.insert({ a : undefined, b : 1, c : null })
When you query on the collection, you get this:
> db.test.find()
{ "_id" : ObjectId("52d95575c9333565e80ccb22"), "a" : 1, "b" : null, "c" : null }
{ "_id" : ObjectId("52d9557fc9333565e80ccb23"), "a" : null, "b" : null, "c" : 1 }
{ "_id" : ObjectId("52d95586c9333565e80ccb24"), "a" : null, "b" : 1, "c" : null }
When you query on null however, it only retrieves records who was explicitly set to null.
> db.test.find({ a : null })
{ "_id" : ObjectId("52d9557fc9333565e80ccb23"), "a" : null, "b" : null, "c" : 1 }
Is this a bug in MongoDB? How can I properly query for null/ undefined/ non-set fields?
EDIT
So I can query for not-set values with this:
db.test.find({ $or : [ { a : null }, { a : { $exists : false } } ] })
But in this example though, it still only returns the single record:
{ "_id" : ObjectId("52d9557fc9333565e80ccb23"), "a" : null, "b" : null, "c" : 1 }
Anyone know why/ how MongoDB differentiates between undefined and null? Is this an issue with how the data was entered in MongoDB?
If you want to return a document where a field exists AND is not null, use { a : {$ne : null}}
Undefined and null values are different, but the shell shows them both as null - https://jira.mongodb.org/browse/SERVER-6102
In case you wonder, why MongoDB casts undefined to null instead of just ignoring such properties - ignoreUndefined flag could solve this behaviour.
https://mongodb.github.io/node-mongodb-native/2.1/reference/connecting/connection-settings/
You should be using $exists to check for values that don't have the field.
Related
I am performing a query to show the objects from a specific collection of data in mongodb using the query db.collection.find()
On performing the query over a db with collection called User, keys with no value are ommitted, hence the length of objects are not the same.
for example:
when I do : db.User.find()
I get this for the first two objects :
{
"_id" : ObjectId("5f6be9aba8fced67a9af8154"),
"username" : "nestor",
"first_name" : "Nestor",
"email" : "nestor#example.com",
}
{
"_id" : ObjectId("5f6bee1767e194695a0a3516"),
"username" : "salma",
"first_name" : "Salma",
"last_name" : "Driss",
"email" : "salma#example.com",
}
from the example above, the key last_name is omitted in first object since it has no value, but the second one has it present since it has a value. I expect to have 'last_name' : null if last_name value is not present.
So I should expect to get this:
{
"_id" : ObjectId("5f6be9aba8fced67a9af8154"),
"username" : "nestor",
"first_name" : "Nestor",
"last_name" : null,
"email" : "nestor#example.com",
}
{
"_id" : ObjectId("5f6bee1767e194695a0a3516"),
"username" : "salma",
"first_name" : "Salma",
"last_name" : "Driss",
"email" : "salma#example.com",
}
What query would be useful to help me solve this problem?
You can project the result with this:
db.collection.aggregate([
{ $set: { last_name: { $ifNull: ["$last_name", null] } } }
])
The { item : null } query matches documents that either contain the item field whose value is null or that do not contain the item field. MongoDB docs
db.User.find( { item: null } )
You can also test this yourself by following the link to the MongoDB docs as above.
I'm reading MongoDB BSON Comparison docs.
But I couldn't understand clearly.
When comparing values of different BSON types, MongoDB uses the following comparison order, from lowest to highest:
MinKey (internal type)
Null
Numbers (ints, longs, doubles, decimals)
Symbol, String
...
Is there any implicit type conversion and how it works?
Could you give me a few examples?
There is no type conversion.
When two items are compared:
If they have different types, the order of the types referenced becomes the order of the items.
If they have the same type, the respective values are compared, and their order becomes the order of the items.
Try:
> db.foo.insertMany([{a:1},{a:[]},{a:'1'},{a:{}},{a:true},{a:false},{a:/test/},{a:null},{a:[1]}])
{
"acknowledged" : true,
"insertedIds" : [
ObjectId("5e8aeae7d11542e71728da71"),
ObjectId("5e8aeae7d11542e71728da72"),
ObjectId("5e8aeae7d11542e71728da73"),
ObjectId("5e8aeae7d11542e71728da74"),
ObjectId("5e8aeae7d11542e71728da75"),
ObjectId("5e8aeae7d11542e71728da76"),
ObjectId("5e8aeae7d11542e71728da77"),
ObjectId("5e8aeae7d11542e71728da78"),
ObjectId("5e8aeae7d11542e71728da79")
]
}
> db.foo.find().sort({a: 1})
{ "_id" : ObjectId("5e8aeae7d11542e71728da72"), "a" : [ ] }
{ "_id" : ObjectId("5e8aeae7d11542e71728da78"), "a" : null }
{ "_id" : ObjectId("5e8aeae7d11542e71728da71"), "a" : 1 }
{ "_id" : ObjectId("5e8aeae7d11542e71728da79"), "a" : [ 1 ] }
{ "_id" : ObjectId("5e8aeae7d11542e71728da73"), "a" : "1" }
{ "_id" : ObjectId("5e8aeae7d11542e71728da74"), "a" : { } }
{ "_id" : ObjectId("5e8aeae7d11542e71728da76"), "a" : false }
{ "_id" : ObjectId("5e8aeae7d11542e71728da75"), "a" : true }
{ "_id" : ObjectId("5e8aeae7d11542e71728da77"), "a" : /test/ }
>
The behavior of the empty array does not seem to match the documentation.
According to information in the MongoDB manpage on cursor.sort (last paragraph on comparing values), NULL values are smaller than any other values in terms of the sort() function.
The question is, are missing attributes regarded as NULL? According to my testing not so:
> db.sort.find().sort({id:1})
{ "_id" : ObjectId("5269554df18e7d2f4bd1241d"), "a" : "Anonym" }
{ "_id" : ObjectId("52695684f18e7d2f4bd12421"), "a" : "Bnonym", "d" : "iii" }
{ "_id" : ObjectId("52695892f18e7d2f4bd12422"), "id" : null, "a" : "Bnonym", "d" : "iii" }
{ "_id" : ObjectId("5269591bf18e7d2f4bd12425"), "id" : null, "a" : "ZZZaaa", "d" : "iii" }
{ "_id" : ObjectId("526954d4f18e7d2f4bd1241b"), "id" : 7, "a" : "Jozo" }
The order seems to be:
undefined/missing sort attribute
null value
the rest according to the order from manpage
Q: The question is, whether there is some undocumented type different from null for missing attributes under the hood?
As far as I can tell (I haven't reviewed the source code specifically), this is undefined and depends on a variety of factors. The most striking one is that reversing the sort does not, in general, reverse the order of null / undefined elements:
> db.sort2.find().sort({"a" : 1});
{ "_id" : ObjectId("526a4d45b4e29833675fb089"), "a" : null }
{ "_id" : ObjectId("526a4d4cb4e29833675fb08b") }
{ "_id" : ObjectId("526a4d48b4e29833675fb08a"), "a" : 21433 }
>
> db.sort2.find().sort({"a" : -1});
{ "_id" : ObjectId("526a4d48b4e29833675fb08a"), "a" : 21433 }
{ "_id" : ObjectId("526a4d45b4e29833675fb089"), "a" : null }
{ "_id" : ObjectId("526a4d4cb4e29833675fb08b") }
Also, when I created an index on a for that test collection, the order changed(!). It didn't change back, however, when I dropped the index. For me, the lack of documentation alone would be enough not to rely on that behavior, but it seems largely unpredictable in even the most simple scenario.
A well-defined and important difference of null and non-existant fields is when using sparse indexes: In a sparse index, null is a valid value and will actually be indexed (it has a value, after all), while a non-existant field is not added to the index at all.
When I issue the following query:
db.users.find({"pic.status" : {$ne : null} }, {"pic" : 1}).toArray()
I expect to receive all users whose pic.status is NOT null. However, the actual result looks something like this:
{
"_id" : ObjectId("4f1e1ab9cdf9dbaa160000bf"),
"pic" : {
"id" : "4f1e1ab9cdf9dbaa160000be",
"status" : null
}
},
{
"_id" : ObjectId("4f1e28480eaf38193d00006f"),
"pic" : {
"id" : "4f1e28480eaf38193d00006e",
"status" : null
}
}
That is, I receive users whose pic.status IS null. How can I fix this?
I know it's an old question, and might not be relevant anymore, but as there was no accepted answer, I thought I'll comment for anyone looking for answer.
I was able to reproduce the problem on MongoDB version 2.4.9.
> db.sourceListings.findOne({displayed:{$ne:null}});
{
<.. other stuff went here..>
"displayed" : null
}
The problem disappears on Version 2.6.1:
> db.sourceListings.findOne();
{
<.. other stuff ..>
"displayed" : null
}
> db.sourceListings.findOne({displayed:{$ne:null}});
null
Probably was fixed somewhere in between these two versions.
I cannot seem to reproduce this. What version of Mongo are you using? (I am using 2.1.1-pre) Here are the steps that I took. The following is from the JS shell:
> db.users.save({
"_id" : 1,
"pic" : {
"id" : "4f1e1ab9cdf9dbaa160000be",
"status" : null
}
});
> db.users.save({
"_id" : 2,
"pic" : {
"id" : "4f1e28480eaf38193d00006e",
"status" : null
}
});
> db.users.save({
"_id" : 3,
"pic" : {
"id" : "4f1e28480eaf38193d00006e",
"status" : "Something"
}
});
> db.users.find({"pic.status":{$ne:null}}, {pic:1}).toArray()
[
{
"_id" : 3,
"pic" : {
"id" : "4f1e28480eaf38193d00006e",
"status" : "Something"
}
}
]
Only the document containing "pic.status":"Something" is returned.
My only thought is, are you absolutely certain that the value of (null) in the query is the same as what is saved in the documents? Were the documents saved using the JS shell, or were they saved using a driver in a different language? Theoretically, null, should be null, should be null, in any language, but I know that different languages represent "no value" differently. In python, for example, the value for "no value" is (None). (null) is not recognised in python.
In a python shell, I attempted to save a document with (null) as a value, but received an error:
In [13]: coll.save({"_id":5, "pic":{"id":5, "status":null}})
---------------------------------------------------------------------------
NameError Traceback (most recent call last)
/Users/mbastien/mongodb-osx-x86_64-2.0.1/bin/<ipython-input-13-1ad232456c88> in <module>()
----> 1 coll.save({"_id":5, "pic":{"id":5, "status":null}})
NameError: name 'null' is not defined
I then inserted (with the Python shell) a status of 'null' ('null' being a string)
In [15]: coll.save({"_id":5, "pic":{"id":5, "status":'null'}})
Out[15]: 5
Not surprisingly, when I reran the query in the JS shell, this document was returned, because 'null' != null
> db.users.find({"pic.status":{$ne:null}}, {pic:1}).toArray()
[
{
"_id" : 3,
"pic" : {
"id" : "4f1e28480eaf38193d00006e",
"status" : "Something"
}
},
{
"_id" : 5,
"pic" : {
"status" : "null",
"id" : 5
}
}
]
Is this similar to what you are experiencing, or can you reproduce this 100% in the JS shell? Hopefully, we will be able to get to the root of this issue!
Use $nin with a single element in the array. It'll behave as you expect. KI with many of the people who use mongo.
My collection (MongoDB v 2.0.2) has following records:
db.organization.find({})
{ "_id" : 1001, "path" : [ ], "parent" : null }
{ "_id" : 1002, "path" : [ 1001 ], "parent" : NumberLong(1001) }
organization has indexes:
db.organization.ensureIndex({"path":1});
db.organization.ensureIndex({"parent":1},{sparse:false});
(note I put awarnes sparse : false - to grant that null is indexed)
But, executing:
db.organization.find({"parent":null})
Returns empty set. What is wrong? Thank you in advance
I had the same issue. After reading the following documents
querying and nulls
BSON specification
I tried to query for the different BSON element types and found that my null was represented as a BSON element type 6 (undefined, deprecated) instead of the expected BSON element type 10 (null).
db.collection.find({ field: { "$type" : 6} };
Just checked following script at 2.0 and 2.0.2:
db.items.insert({ "_id" : 1001, "path" : [ ], "parent" : null })
db.items.insert({ "_id" : 1002, "path" : [ 1001 ], "parent" : NumberLong(1001) })
db.items.ensureIndex({"path":1});
db.items.ensureIndex({"parent":1},{sparse:false});
db.items.find({"parent":null})
actually returns one document that you expect:
{ "_id" : 1001,
"path" : [],
"parent" : null }
Also you can look into this doc about querying and nulls, probably should help you avoid possible future mistakes.