I'm trying to run a fairly simple query using MGO that only has one condition included: published field must be less than or equal to the current time.
I have a test document in my database that was created as follows:
db.articles.insert([{
"title": "Woo this is a test title",
"published": ISODate("2017-01-02T12:00:00Z")
}])
My querying code is:
now := time.Now().Format(time.RFC3339)
articlesCollection.Find(bson.M{"published": bson.M{"$lte": now}}).
Sort("-published").All(&frontPageArticles.Articles)
But I get no records returned.
I'm comfortable with Mongo, but very new to Go (and thus mgo) - I'm sure I'm doing something wrong at a basic level, but am not sure what. Can anyone help?
The solution is simple: don't format your time, just pass a time.Time value:
now := time.Now()
The mgo package will encode the time in the appropriate format. In your example you passed the time as a string which would only work if the published field in MongoDB would also be a string in the same format (and not a MongoDB date).
Related
In Golang I read a value from a database.
findOptions := options.FindOneOptions{}
findOptions.SetSort(bson.D{{"foo", -1}})
var valueFromDatabase *mongo.SingleResult
valueFromDatabase = clients.MongoClient.Database("foo").Collection("foo").FindOne(context.TODO(), bson.M{})
Is it possible to get a specific position, much like you would in an array?
Not working sample code, how it should fetch the values:
valueFromDatabase.NestedObject.Array[0].Value
Background is. A generic approach should be used so that the solution works independently of the structure of the BSON document. Which fields are to be read out is known from a slice.
The following approach was tried: The conversion to a json destroys the Mongo encryption.
I'm trying to migrate data from Parse to a new project that uses Mongo as its database (without Parse/Parse Server). Since the schemas are different between the two projects, I'm manually writing a migration script to achieve this.
As I understand it, Parse appears to use 10-character-long IDs for their objects (combinations of digits, lower-case letters, and upper-case letters), while Mongo uses 24-character-long IDs (12 bytes represented as hex).
Right now, when migrating data for a document from the old project to the new one, I'm using a function that converts the Parse ID to a unique Mongo ObjectId (it converts each character to a 2-digit hex value, then pads the 20-character string with 4 zeroes).
Is this a good approach? I'm avoiding using Mongo's automatic ObjectId generation in case I ever need to re-migrate any of the old Parse documents and find the matching document in the new database. I know automatically generated ObjectIds in Mongo also embed some other information like creation dates, but I don't think this would be important and I can just use my custom ObjectId generator? However, I'm not sure about the implications for performance/if I'm just going about this migration the wrong way.
The approach i recommend is letting Mongo auto-generate the ids and then storing Parse's ids in a new field called parseID for future reference if needed.
For example:
PARSE DATA:
"_id": ObjectId(1234567890),
"title": "Mongo Migrate",
"description": "Migrating from Parse to Mongo"
MONGO DATA:
"_id": ObjectId(1ad83e4k2ab8e0daa8ebde7), //mongo generated
"parseId":ObjectId(1234567890),
"title": "Mongo Migrate",
"description": "Migrating from Parse to Mongo"
Then if you need to match a document between the two databases later, you can write a script that goes along the lines of Parse.find({"_id": Mongo.parseId}).....
MongoDB uses _id as primary key by default. _id has to be unique to avoid collision. The way you are generating unique ObjectId to _id is fine. As long as they are unique, you could even reduce the 20-character pad to save space.
I need monitor the time of the records been created, for further query and modify.
first thing flashed in my mind is give the document a "createDateTime" field, with the default value of "new Date()", but Mongodb said the document _id has a timestamp embedded with, and the id was generated when the document was created, so it sounds dummy to add a new field for that.
for too many times, I've seen people set a "createDateTime" for their data, and I don't know if they know about the details of mongodb's _id.
I want know should I use the _id as a "createDateTime" field? what is the best practice?
and the pros and cons.
thanks for any tips.
I'd actually say it depends on how you want to use the date.
For example, it's not actionable using the aggregation framework Date operators.
This will fail for example:
db.test.aggregate( { $group : { _id: { $year: "$_id" } } })
The following error occurs:
"errmsg" : "exception: can't convert from BSON type OID to Date"
(The date cannot be extracted from the ObjectId.)
So, operations that are normally simple date operations become much more complex if you wanted to do any sort of date math in an aggregation. It would be far easier to have a createDateTime stamp. Counting the number of documents created in a particular year and month would be simple using aggregation with a distinct createdDateTime field.
You can sort on an ObjectId, to some degree. The remaining 8 bytes of the ObjectId aren't sortable in a meaningful way. Most MongoDB drivers default to creating the ObjectId within the driver and not on the database. So, if you've got multiple clients (like web servers for example) creating new documents (and new ObjectIds), the time stamps will only be as accurate as the various servers.
Also, depending the precision you'd need, an ISODate value is stored using 8 bytes, rather than the 4 used in an ObjectId.
Yes, you should. There is no reason not to do, besides the human readability while directly looking into the database. See also here and here.
If you want to use the aggregation framework to group by the date within _id, this is not possible yet as WiredPrairie correctly said. There is an open jira ticket for that, you might watch. But of course you can do this with Map-Reduce and ObjectID.getTimestamp(). An example for that can be found here.
I have a collection of data and I want to get it sorted by insertion time. I have not any additional fields to store the insert time. But as I found out I can get this time from Id.
I have tried this code:
return bookmarks.find({}, {sort: {_id.getTimestamp(): 1}, limit: 10});
or
return bookmarks.find({}, {sort: {ObjectId(_id).getTimestamp(): 1}, limit: 10});
but get the error message:
=> Your application has errors. Waiting for file change.
Is there any way to sort collection by insertion datetime using only id field ?
At the moment this isn't possible with Meteor, even if it is with MongoDB. The ObjectID's created with meteor don't bear a timestamp. See http://docs.meteor.com/#collection_object_id
The reason for this is client side code can insert code and it can arrive late on the server, hence there is no guarantee the timestamp portion of the ObjectID will be accurate. In addition to the latency the client side's date is used meaning if they're off it's going to get you incorrect data. I think this is the reason they use an ObjectID but it is completely random.
If you want to sort by date you have to store the time/date separately.
The part what i striked out is not accurate. Meteor use it is own id generation which is based on a random string that is while does not apply the doc what i linked before. Check sasha.sochka's comment under.
It is nearly but not 100% good if you just sort for the _id field . While as it is constructed the first 4 byte is the timestamp in secs (so sorting for the getTimestamps value is not better). Under one second resolution you cannot get the exact order, as it is mentioned in the documentation: http://docs.mongodb.org/manual/reference/object-id/#objectid
It is still true that you can try to check the exact order of the insert/update ops against your collection in the oplog, if you have an oplog, but as it is a capped collection anyway you will see the recent operations only. http://docs.mongodb.org/manual/core/replica-set-oplog/.
I'm inserting a Mongo doc with the following time-stamp:
val format = new java.text.SimpleDateFormat("yyyyMMddHHmmss")
format.format(new Date()).toLong
Here's what the section looks like from Mongo's shell:
"{Timestamp" : NumberLong("20130919161948")}"
Based on a few tests, it appears to me that I can simply compare 2 documents by Timestamp by simply checking > or < for the yyyyMMddHHmmss format.
Please let me know if this time-stamp is OK for Mongo. Will I be able to query with it?
Mongo will not understand this as a timestamp, but as a number. As you set your date with a format going from year to seconds, you will be able to query mongo using > or < to know if it is before or after.
However if you want to mongo to treat the data as a date, you will need to use the appropriate bson date format. By having mongo treat it as a date, you will have all Mongo date operations available, like extracting year, day of week, etc.. read more
If you are using casbah, and Joda, you can enable serialization and deserialization by an explicit call:
import com.mongodb.casbah.conversions.scala._
RegisterJodaTimeConversionHelpers()
Read more here.
#Kevin, I think you are right. java.util.Date is supported in BSON object.
Using NumberLong to represent timestamp allows you to do range queries, but with BSON date type, date operation in aggregation framework becomes possible, which is more powerful.