I'm using PrestoDB to query some MongoDB collections. MongoDB has a getTimestamp() method to get the timestamp portion of an ObjectId. How can I get a similar timestamp on PrestoDB?
It's not implemented in Presto, but there is a PR: https://github.com/prestosql/presto/pull/3089
You can implement this with eg
#ScalarFunction("get_timestamp")
#SqlType(StandardTypes.TIMESTAMP_WITH_TIME_ZONE) // ObjectId's timestamp is a point in time
public static long getTimestamp(#SqlType("ObjectId") Slice value)
{
int epochSeconds = new ObjectId(value.getBytes()).getTimestamp();
return DateTimeEncoding.packDateTimeWithZone(TimeUnit.SECONDS.toMillis(epochSeconds), UTC_KEY);
}
-- add this in the https://github.com/prestosql/presto/blob/master/presto-mongodb/src/main/java/io/prestosql/plugin/mongodb/ObjectIdFunctions.java class
Related
I have the following struct:
type TypeIncidence struct {
Number int bson:"number" json:"number"
Description string bson:"description" json:"description"
Date_time_stamp string bson:"dateTimeStamp" json:"date_time_stamp"
}
and I want insert a document in a collection:
type TypeIncidence struct {
Number int `bson:"number" json:"number"`
Description string `bson:"description" json:"description"`
Date_time_stamp **string?**
}
var incidence TypeIncidence
incidence.Number = 1
Description =" Text"
Date_time_stamp = **string?**
What data type would I have to use in a Golang structure to store date_time_struct a string?
If I want to store with the following format 'YYYY-MM-DD hh:mm:ss', what module and/or function should I use in golang? (in local machine or server converting zone time)
Thanks in advance
You can use time.Time:
CreatedAt time.Time `json:"created_at" bson:"created_at"`
However, I would recommend that you store Epoch Unix timestamp (the number of seconds since Jan 1st 1970) because it is universal:
CreatedAt int64 `json:"created_at" bson:"created_at"`
I have tried in the past to store time.Time in MongoDB through Golang but then I had trouble when I parsed the same information into a datetime object in Python. If you would like to be compatible across languages and technologies, storing the Epoch Unix timestamp would be a great option.
I am trying to retrieve records from mongodb collection after certain date but the date field is stored as a string in mongodb collection. The below query doesn't work well I guess because it does a string comparison. How can I convert the string date from mongo and then compare with input date.
`mongoOperations.find(query(where("lastUpdated").gte(inputTimeStamp).and("status").in("COMPLETED")), Cart.class);`
Something like this worked for me. This may not be the best way to do it because it has potential of sql injection but I made sure data is sanitized before it reaches here.
String queryStr = "{\"$expr\": {\"$gte\": [{ \"$dateFromString\": { \"dateString\": \"$lastUpdated\",timezone:\"America/New_York\" }}, new Date(\"%s\") ]},status:{$in:[\"COMPLETED\"]}}";
BasicQuery query = new BasicQuery(String.format(queryStr,timeStamp));
return mongoTemplate.find(query, Cart.class);
I would like to change order my documents in Mongo DB using Go. I have valid json string code and i can marshal it successfully in to map[string]int. The sample of this type just like:
[{year 1}, {lastupdated -1}]. The value presents order year field ascending and lastupdated field descending. This struct is aspect of MongoDB understands. Also I'm pass this data into bson.D type. Here is my code:
if queries["order"] != nil {
var unmarshalledOrder map[string]int
json.Unmarshal(queries["order"].([]byte), &unmarshalledOrder)
docRes := make(bson.D, 0)
for field, sort := range unmarshalledOrder {
docRes = append(docRes, bson.DocElem{field, sort})
}
log.Println(docRes)
}
When I print docRes, everthing goes well. But i pass the data to options.Sort function, the function throws interface conversion: interface {} is runtime.errorString, not string panic. Is it a bug on mongo go driver or am i wrong?
Can you post the code you've written which uses the driver? Based on the use of bson.DocElem, I think you're using mgo, but mgo's Query.Sort method takes strings, not documents (https://pkg.go.dev/github.com/globalsign/mgo?tab=doc#Query.Sort).
I am new to both Morphia and MongoDB. Is there a way to check using Morphia that a certain field in my database is not null and also exists. For example from the following record of a user from a collection of users in database:
{ "_id" : ObjectId("51398e6e30044a944cc23e2e"),
"age" : 21 ,
"createdDate" : ISODate("2013-03-08T07:08:30.168Z"),
"name" : "Some name" }
How would I use a Morphia query to check if field "createdDate" is not null and exists.
EDIT:
I am looking for a solution in Morphia. So far I have come up with this:
query.and(query.criteria("createdDate").exists(),
query.criteria("createdDate").notEqual(null));
From documentation, I learnt Morphia does not store empty or null fields. Hence the justification for notEqual(null).
EDIT 2: From the answers I can see the problem needs more explanation. I cannot modify the createdDate. To elaborate: the example above is less complex than my actual problem. My real problem has sensitive fields which I cannot modify. Also to complicate things a bit more, I do not have control over the model otherwise I could have used #PrePersist as proposed in one of the answers.
Is there a way to check for null and non existing field when I have no control over the model and I am not allowed to modify fields?
From the documentation, Morphia does not store Null/Empty values (by default) so the query
query.and(
query.criteria("createdDate").exists(),
query.criteria("createdDate").notEqual(null)
);
will not work since it seems you are not able to query on null, but can query for a specific value.
However, since you can only query for a specific value, you can devise a workaround where you can update the createdDate field with a date value that is never used in your model. For example, if you initialize a Date object with 0, it will be set to the beginning of the epoch, Jan 1st 1970 00:00:00 UTC. The hours you get is the localized time offset. It will be sufficient if your update only involves modifying the matching element(s) in mongo shell, hence it would look similarly to this:
db.users.update(
{"createdDate": null },
{ "$set": {"createdDate": new Date(0)} }
)
You can then use the Fluent Interface to query on that specific value:
Query<User> query = mongoDataStore
.find(User.class)
.field("createdDate").exists()
.field("createdDate").hasThisOne(new Date(0));
It would be much simpler when defining your model to include a prePersist method that updates the createdDate field. The method is tagged with the #PrePersist annotation so that the date is set on the order prior to it being saved. Equivalent annotations exist for #PostPersist, #PreLoad and #PostLoad.
#Entity(value="users", noClassNameStored = true)
public class User {
// Properties
private Date createdDate;
...
// Getters and setters
..
#PrePersist
public void prePersist() {
this.createdDate = (createdDate == null) ? new Date() : createdDate;
}
}
When you first create your Morphia instance, before calling morphia.mapPackage() do this:
morphia.getMapper().getOptions().setStoreNulls(true);
to have Morphia store null values.
Anyway, you should be able to query non-null values with:
query.field("createdDate").notEqual(null);
In mongo you can use this query:
db.MyCollection.find({"myField" : {$ne : null}})
This query will return objects that have the field 'myField' and has a value that is not null.
I need to iterate through all of the collections in my MongoDB database and get the time when each of the collections was created (I understand that I could get the timestamp of each object in the collection, but I would rather not go that route if a simpler/faster method exists).
This should give you an idea of what I'm trying to do:
MongoDatabase _database;
// code elided
var result = _database.GetAllCollectionNames().Select(collectionName =>
{
_database.GetCollection( collectionName ) //.{GetCreatedDate())
});
As far as I know, MongoDB doesn't keep track of collection creation dates. However, it's really easy to do this yourself. Add a simple method, something like this, and use it whenever you create a new collection:
public static void CreateCollectionWithMetadata(string collectionName)
{
var result = _db.CreateCollection(collectionName);
if (result.Ok)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
collectionMetadata.Insert(new { Id = collectionName, Created = DateTime.Now });
}
}
Then whenever you need the information just query the collectionMetadata collection. Or, if you want to use an extension method like in your example, do something like this:
public static DateTime GetCreatedDate(this MongoCollection collection)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
var metadata = collectionMetadata.FindOneById(collection.Name);
var created = metadata["Created"].AsDateTime;
return created;
}
The "creation date" is not part of the collection's metadata. A collection does not "know" when it was created. Some indexes have an ObjectId() which implies a timestamp, but this is not consistent and not reliable.
Therefore, I don't believe this can be done.
Like Mr. Gates VP say, there is no way using the metadata... but you can get the oldest document in the collection and get it from the _id.
Moreover, you can insert an "empty" document in the collection for that purpose without recurring to maintain another collection.
And it's very easy get the oldest document:
old = db.collection.find({}, {_id}).sort({_id: 1}).limit(1)
dat = old._id.getTimestamp()
By default, all collection has an index over _id field, making the find efficient.
(I using MongoDb 3.6)
Seems like it's some necroposting but anyway: I tried to find an answer and got it:
Checked it in Mongo shell, don't know how to use in C#:
// db.payload_metadata.find().limit(1)
ObjectId("60379be2bec7a3c17e6b662b").getTimestamp()
ISODate("2021-02-25T12:45:22Z")