I'm having a problem handling integers with Mongodb that are too large for the standard Int64 format. To give an example of the sorts of numbers we're talking about here: I mean something like 10683171315666225459389678813960790592
Now, I've considered turning this into a Decimal128 and that seems to work. The problem I run into is that I think MongoDB is truncating these values when I convert them. I'm using:
db.NuAIS.find().forEach( function (x) { x.HilbertVal = parseFloat(x.HilbertVal); db.NuAIS.save(x); });
So I read the db, convert each of those string values to Decimal and get: 1.0683171315666225e+37. My assumption was that MongoDB was storing the actual value despite what it declared.
Here's the thing though: these values seem truncated since, when I try to search using the original string on MongoCompass, I get multiple values returned for a supposedly unique string. I assume that mongoDB has truncated the decimals after X strings and so is just returning ones that match the first X digits of the search string, which means you get multiple matches instead of one.
Is my assumption here right? Is there a way to avoid this with decimals or just to skip the whole problem altogether by having ints larger than 64 bits? I'd leave it as a string but I need to work with them as digits, find things that fall within a range and such.
Thank you
Related
As a general understanding, numeric operations are faster than string operations. It is faster to compute 2 < 5 than "hello" < "world".
Also, since the length of the _id is 12. If it is stored as a string where a character can take 2 bytes and thus the entire string would take 24 bytes. However a number of 8 bytes can very easily store this value of 12 digits. Basically, a number type would take lesser space.
So with these logics, why does mongodb store _id as string type ObjectIDs and not numeric ObjectID?
Your understanding is incorrect.
It is much better than Strings. It just takes 12 bytes in memory as mentioned in the documentation
Java implementation for the same.
Refer toHexString in the github link, that's what returned to you when you ask for String. It actually takes 24bytes whereas ObjectId takes only 12bytes.
I have a MongoDB Collection build like this :
When I try to filter by ID, I got inconsistent results. For exemple, when I try to get the first entry, by typing in filter query :
{_id:209383449381830657}
I got no result found.
But if I type for exemple the third, It work correctly.
{_id:191312485326913536}
I searched if it's due to too large int but no, all _id value are Int64 and it's the same code that generate all entries.
I really don't know why I got this result and that why I am asking here.
EDIT:
All entries have same type.
No limit are set in query.
If I type {_id:{"$gte":209383449381830657}} it found the entry, but don't if I type {_id:{"$eq":209383449381830657}}
MongoDB Compass uses mongo's node.js driver.
Both 209383449381830657 and 191312485326913536 exceed the javascript max safe integer of (2^53-1).
Javascript does not handle numbers larger than that in a consistent manner.
Note that in your documents, those numbers are reported as $numberLong, indicating that they are not using javascript's default floating point numeric representation.
To query these number consistently, use the NumberLong constructor when querying, like
{_id:NumberLong("209383449381830657")}
It depends on the version of MongoDB Compass you have (I assume it's +- latest)
As I see, you are using NumberLong for _id field.
You might use NumberLong("...") to handle such fields in MongoDB.
So, better to try search like this
{_id: NumberLong("209383449381830657")}
Can't say for sure how MongoDB Compass handles number you are trying to pass here.
As it should automatically cast values to int32 or int64, but it might be a bit tricky. So, better to define it by yourself to avoid unexpected results.
Some time ago I read an article that it tries to cast to int32, but if it's more than int32 might handle -> then it uses this function
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/isSafeInteger
and tries to cast it to int64. My speculation is that problem lives here, but can't say for sure.
In my case the _id contain hex digits.
Filter {_id: ObjectId("62470af3036b9f058552032e")} works
I have a Mongodb Collection in which a decimal amount is stored as a string.
I would like to be able to query treating that field as a number.
Is that possible without elaborate code? I would expect to be able to do. tonumber(field):{$gt:100}.
But I can't find anything.
I have a little bit strange situation.
I persist objects in collection "refs" explicitly setting _id.
So I have objects with very big id's.
db.refs.find().sort({_id: -1});
// {_id: 9200000000165761625}
// ...
But when I try to find object with biggest id in mongo shell it returns nothing:
db.refs.find({_id: 9200000000165761625}); // nothing
But!
db.refs.find({_id: 9200000000165761625}).count(); // return 1
How could this happen?
i could not reproduce your problem. i was able to successfully query on the _id value you specified.
ensure that when you are querying you are passing correct collection name
JavaScript currently only has a single numeric type Number, which represents all values as 64-bit floating point values. The maximum safe integer representation in JavaScript's native Number type is 253-1 or 9007199254740991 (as returned by the constant Number.MAX_SAFE_INTEGER).
Any integer values beyond the safe range cannot be represented distinctly, so two or more mathematical values will map to the same JavaScript Number.
You can see this effect in the mongo shell with values adjacent to your provided _id (which is larger than the safe integer size):
> 9200000000165761624
9200000000165762000
> 9200000000165761625
9200000000165762000
> 9200000000165761626
9200000000165762000
However, these driver/client limitations are distinct from the underlying data types used in MongoDB's BSON format for documents. BSON has a 64-bit integer type which represents the full range of values: up to 263-1 for 64-bit integers.
Your example _id is within the 64-bit integer range so you should be able to insert or update this using a driver with support for 64-bit integers, but would not be able to safely query or manipulate long values in the mongo shell or other JavaScript environments. To avoid unexpected outcomes you may want to use a different data type for these long _id values.
I am using Mongoid, which is on top of the Ruby MongDB driver. Even though my Map's emit is giving out a parseInt(num), and the Reduce's return is giving back also a parseInt(num), the final results still are floats.
Is that particular to MongoDB? Any way to make it integer instead?
The parseInt function officially takes a string as parameter. This string is parsed as if it were an integer, thus ignoring everything after the first non-numeric character. If you provide a floating point number, it will be converted to a string before it is parsed.
The parseInt functions returns a Number, not an integer. Number is the only numeric data type in JavaScript; there is no distinction between integers and floats.
So while parseInt will remove any decimals, the data type doesn't change. Therefore Mongoid doesn't know whether to treat the result as a float or an integer. You're responsible for converting the result to an integer, as you can see in this example.
Update
I came across the NumberLong type, which represents a 64-bit integer. If you return new NumberLong(num) from your reduce function, Mongoid may treat it as an integer type.
Note that you'll need MongoDB 1.6 for this to work in the MongoDB Shell. I don't know whether Mongoid supports it yet.