Is there a way to handle numbers, which are stored as strings, like integers in mongodb?
e.g.
{
"someKey" : "45646764646"
}
I would like to perform $gte or $lte operations on that value, but it's not possible as long the value for "someKey" is a string. I also would like to avoid of using a dbcursor and make comparisons with java e.g.
If your field someKey contains numbers, consider saving it as NumberLong() . Your driver should allow you to represent that field in your application as a string on most all 10gen drivers.
I would recommend, if possible, converting the strings to ints when you store them in the database. This could prevent confusion in the future, and would save a lot of space due to the storage differences.
Related
I have a MongoDB Collection build like this :
When I try to filter by ID, I got inconsistent results. For exemple, when I try to get the first entry, by typing in filter query :
{_id:209383449381830657}
I got no result found.
But if I type for exemple the third, It work correctly.
{_id:191312485326913536}
I searched if it's due to too large int but no, all _id value are Int64 and it's the same code that generate all entries.
I really don't know why I got this result and that why I am asking here.
EDIT:
All entries have same type.
No limit are set in query.
If I type {_id:{"$gte":209383449381830657}} it found the entry, but don't if I type {_id:{"$eq":209383449381830657}}
MongoDB Compass uses mongo's node.js driver.
Both 209383449381830657 and 191312485326913536 exceed the javascript max safe integer of (2^53-1).
Javascript does not handle numbers larger than that in a consistent manner.
Note that in your documents, those numbers are reported as $numberLong, indicating that they are not using javascript's default floating point numeric representation.
To query these number consistently, use the NumberLong constructor when querying, like
{_id:NumberLong("209383449381830657")}
It depends on the version of MongoDB Compass you have (I assume it's +- latest)
As I see, you are using NumberLong for _id field.
You might use NumberLong("...") to handle such fields in MongoDB.
So, better to try search like this
{_id: NumberLong("209383449381830657")}
Can't say for sure how MongoDB Compass handles number you are trying to pass here.
As it should automatically cast values to int32 or int64, but it might be a bit tricky. So, better to define it by yourself to avoid unexpected results.
Some time ago I read an article that it tries to cast to int32, but if it's more than int32 might handle -> then it uses this function
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isSafeInteger
and tries to cast it to int64. My speculation is that problem lives here, but can't say for sure.
In my case the _id contain hex digits.
Filter {_id: ObjectId("62470af3036b9f058552032e")} works
I have a little bit strange situation.
I persist objects in collection "refs" explicitly setting _id.
So I have objects with very big id's.
db.refs.find().sort({_id: -1});
// {_id: 9200000000165761625}
// ...
But when I try to find object with biggest id in mongo shell it returns nothing:
db.refs.find({_id: 9200000000165761625}); // nothing
But!
db.refs.find({_id: 9200000000165761625}).count(); // return 1
How could this happen?
i could not reproduce your problem. i was able to successfully query on the _id value you specified.
ensure that when you are querying you are passing correct collection name
JavaScript currently only has a single numeric type Number, which represents all values as 64-bit floating point values. The maximum safe integer representation in JavaScript's native Number type is 253-1 or 9007199254740991 (as returned by the constant Number.MAX_SAFE_INTEGER).
Any integer values beyond the safe range cannot be represented distinctly, so two or more mathematical values will map to the same JavaScript Number.
You can see this effect in the mongo shell with values adjacent to your provided _id (which is larger than the safe integer size):
> 9200000000165761624
9200000000165762000
> 9200000000165761625
9200000000165762000
> 9200000000165761626
9200000000165762000
However, these driver/client limitations are distinct from the underlying data types used in MongoDB's BSON format for documents. BSON has a 64-bit integer type which represents the full range of values: up to 263-1 for 64-bit integers.
Your example _id is within the 64-bit integer range so you should be able to insert or update this using a driver with support for 64-bit integers, but would not be able to safely query or manipulate long values in the mongo shell or other JavaScript environments. To avoid unexpected outcomes you may want to use a different data type for these long _id values.
I have two type of documents in a mongodb collection:
one where key sessions has a simple value:
{"sessions": NumberLong("10000000000001")}
one where key sessions has an array of values.
{"sessions": [NumberLong("10000000000001")]}
Is there any way to retrieve all documents from the second category, ie. only documents whose value is an arary and not a simple value?
You can use this kind of query for that:
db.collectionName.find( { $where : "Array.isArray(this.sessions)" } );
but you'd better convert all the records to one type to keep the things consistent.
This code can be simple like this:
db.c.find({sessions:{$gte:[]}});
Explanation:
Because you only want to retrieve documents whose sessions data type is array, and by the feature of $gte (if data types are different between tow operands, it returns false; Double, Integer32, Integer64 are considered as same data type.), giving an empty array as the opposite operand will help to retrieve all results by required.
Also , $gt, $lt, $lte for standard query (attention: different behaviors to operaors with same name in expression of aggregation pipeline) have the same feature. I proved this by practice on MongoDB V2.4.8, V2.6.4.
Whats the best option to use for MongoID data type for the regular MongoDB TEXT data type.
Wondering why MongoID doesnt have a data type TEXT.
is it okay to use STRING type and store large amounts of data.
P.S coming from SQL background.
According to the mongoid documentation all fields are strings, unless we explicitly specify an other data types. Unlike SQL's varchar and text differences, strings in mongo have no limitation (the only limitation is that of the 16MB maximum document size) so there is no need to worry about size.
Yes, strings in MongoDB have unlimited length (up to document max size, of course (16MB)). So there was no reason to introduce separate TEXT column type, as do relational DBs.
Just use string type.
You can use String datatype only. There is no reason to use any other datatype as string provides you unlimited length. You can directly use String type for MongoDB text fields.
Let's say I have two collections:
Products and Categories.
The latter collection's documents have 2 fields:
_id (BSON ObjectId)
Name (String)
The latter collection's documents have 3 fields:
_id (BSON ObjectId)
Name (String)
Products (Array of Strings)
Assume I have the following Product document:
{ "_id" : ObjectId("AAA"), "name" : "Shovel" }
Let's say I have the following Category document:
{ "_id" : ObjectId("BBB"), "Name" : "Gardening", "Products" : ["AAA"] }
For purposes of this example, assume that AAA and BBB are legitimate ObjectId's - example: ObjectId("523c7df5c30cc960b235ddee") where they would equal the inner ObjectId's string.
Should the Products field be stored as ObjectId(...)'s rather than as Strings?
I don't think it really matters that much.
I'm pretty sure that the ObjectId format encodes a hex number, so it is probably slightly more efficient with memory and bandwidth. I have done it both ways. As long as you decide, for each field, how you are going to encode it, either will work just fine.
As long as you consistently use the same type (so that comparisons happen correctly), the difference is:
An ObjectId cannot be compared to a String representation of the same ObjectId value. Thus, ObjectId("523c7df5c30cc960b235ddee") is not equal to "523c7df5c30cc960b235ddee".
ObjectIds, when stored natively, will be stored as 12 bytes, plus field name
An ObjectId, when stored as a string, will be commonly stored in 24 bytes (as it will be converted to a hexadecimal number), plus field name
Comparisons can be made more SLIGHTLY more efficiently with the 12 byte number, as it's comparing fewer bytes. It won't matter in most types of usage though, so it's a micro-optimization (but something you should know)
Bonus -- if you don't use short abbreviated field names, the size benefit of using an ObjectId natively as 12 bytes really won't matter, as the field names will far outweigh the size of bytes when stored as a string.
I'd recommend storing them as native ObjectIds. Some drivers can optionally and transparently translate to an ObjectId to a String and back so that the client code can more easily manipulate it. The C# driver for example can do this, and I've used it so that when serializing to JSON, the ObjectId is in a simple format that is easily consumed in JavaScript.
This will matter most when you try to find the details of a product starting from the Categories collection.
Since there are no server side JOIN in Mongo, your code will have to match documents together. ObjectIDs are encoded as 12 bytes, which you can easilly compare in any language. Using either strings or object ids does not really matter.
The real issue you are facing is one of data normalization (or lack thereof). If you store the Name field in your Categories documents, instead of the ObjectID, you will be able to return the products names in a single call (instead of multiple calls, 1 for each products of the category).
It feels wrong the first time you do it. After all, you will have to update many documents if you ever change the name of a product, which might or might not be frequent. You have to model your data by thinking of the way your application will use it.
Finally, index the Name attribute in the Prodcuts collection. Getting the details of a product, starting with the string you found in a Categories document will be fast.
Another way to do it is to not to have a Categories collection at all, but to add a Category attribute to your Products document. You can find documents that have the {'Category':'Gardening'}. Indexing the Category field will probably be a good idea.
Again, ObjectID or String does not matter much. It is about modeling your data thinking of how your application will use it.