Spring Data MongoDB - Is there a way to have an implicit timestamp? - mongodb

Using my MongoDB and Spring Data MongoDB I am currently looking for a timestamp to be visible in the collection documents.
For now the document looks like this:
{
"_id":"f84fd693-e04b-4acb-9390-32ee755c1506",
"name":"Herbert",
"age":{"$numberInt":"21"},
"_class":"com.alemannigame.backend.domain.Character"
}
However I'd like to have a "timestamp": "1988-03-12T02:30:12+00:00" (example format) in it as well. Is there a way to do so without having to write logic in a Service to actually add a timestamp manually?
I thought about something like:
#Document(withTimestamp: true) // this
data class Character(
#Id
val id: String,
val name: String,
val age: Int
)
Could not find anything similar in the interwebs! Nifty solutions are welcome!

So I found something out about Lifecycle Events for Spring Data MongoDB (https://docs.spring.io/spring-data/mongodb/docs/current/reference/html/#mongo.aggregation).
This gave me several interceptors before a document is saved to MongoDB.
It's very straight forward to use. I chose the onBeforeConvert hook so I can manipulate my model before it is being saved.
As you can see I added the timestamp of the event to my source instance. I like Unix timestamps and since the event object already has that ready I reused it.
#Component
class MongoSaveInterceptor : AbstractMongoEventListener<Character>() {
override fun onBeforeConvert(event: BeforeConvertEvent<Character>) {
val source = event.source
source.timestamp = event.timestamp
}
}
When using MongoDB's implicit ObjectId (which I do not - I use an own UUID id) I think that event.timestamp is used here.

Related

BsonInvalidOperationException in ktor

Being new to MongoDB, I'm currently integrating the kMongo library to my ktor project, and trying to create a database to read & write event models to.
Following the instructions for object mapping in the kMongo user manual, I've created a mongoId field which gets serialised as a String named _id.
My event model is a data class, nested in sealed classes but gets serialised correctly by KotlinX-Serialization. The model looks as such:
sealed class Event {
#SerialName("_id") abstract val mongoId: String
abstract val id: ID.Event
abstract val dateTime: LocalDateTime
fun asString() = id.toString()
sealed class Hiring : Event() {
#SerialName("_id") abstract override val mongoId: String
abstract override val id: ID.Event
abstract override val dateTime: LocalDateTime
#Serializable
data class Start(
override val id: ID.Event,
override val dateTime: LocalDateTime,
val hiringDetailsId: ID.HiringDetails
) : Hiring() {
#SerialName("_id") override val mongoId: String = id.asString()
}
...
In a repository class, I initialise MongoDB and use the generic, parameter-less find() on a collection to retrieve all Event models from the database:
...
private val kmongo = KMongo.createClient().coroutine.client
private val db = kmongo.getDatabase("test")
private val eventCollection = db.getCollection<Event>().coroutine
...
override suspend fun getAllEvents() = eventCollection.find().toList()
Then inside of the Main class, I try to load the Event data on a click trigger:
...
val id = ID.Event(UUID())
...
it.on.click {
runBlocking {
val events = eventRepo.getAllEvents().toString()
logger.debug { events }
}
}
The strange part starts here, the server starts correctly and MongoDB is initialised correctly, but as soon as I try to do the read on the click trigger, I am presented with following error:
org.bson.BsonInvalidOperationException: readString can only be called when CurrentBSONType is STRING, not when CurrentBSONType is DOCUMENT.
at org.bson.AbstractBsonReader.verifyBSONType(AbstractBsonReader.java:689)
at org.bson.AbstractBsonReader.checkPreconditions(AbstractBsonReader.java:721)
at org.bson.AbstractBsonReader.readString(AbstractBsonReader.java:456)
at com.github.jershell.kbson.FlexibleDecoder.decodeString(BsonFlexibleDecoder.kt:130)
at kotlinx.serialization.encoding.AbstractDecoder.decodeStringElement(AbstractDecoder.kt:58)
at kotlinx.serialization.internal.AbstractPolymorphicSerializer.deserialize(AbstractPolymorphicSerializer.kt:52)
at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:257)
at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:16)
at org.litote.kmongo.serialization.SerializationCodec.decode(SerializationCodec.kt:66)
at com.mongodb.internal.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52)
at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42)
at org.bson.internal.LazyCodec.decode(LazyCodec.java:48)
at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:104)
at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42)
at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:51)
at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:535)
at com.mongodb.internal.connection.InternalStreamConnection.access$500(InternalStreamConnection.java:86)
at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:520)
at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:498)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:821)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:785)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:160)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:573)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297)
at com.mongodb.internal.connection.AsynchronousSocketChannelStream$AsynchronousSocketChannelAdapter.read(AsynchronousSocketChannelStream.java:144)
at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:118)
at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:107)
at com.mongodb.internal.connection.InternalStreamConnection.readAsync(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.InternalStreamConnection.access$600(InternalStreamConnection.java:86)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:775)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:760)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishRead(UnixAsynchronousSocketChannelImpl.java:447)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:195)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:217)
at java.base/sun.nio.ch.KQueuePort$EventHandlerTask.run(KQueuePort.java:312)
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
According to the stacktrace, something seems to go wrong in the BSON filtering part, despite there being none. When I use the MongoDB compass to validate the object inside of the database, I can see that everything is initialised and written perfectly fine:
The normal id field is used in my software internally as an ID.Event object type whilst the _id is used by Mongo internally.
Can someone point me to what the potential issue could be here?
I'm not familiar with Kotlin, but I'd like to dive into this a bit further:
If it wouldn't unwrap the second time, the Mongo Compass would likely reveal the _id or id field to contain a bracket { while these are currently mapped as expected (a String for _id and an object for id).
To confirm, the current structure of your document is (eg here in the playground):
{
_id: "7d51",
id: {
id: "7d51"
},
hiringDetailsId: {
id: "8392"
}
}
We can see that in your screenshot from Compass where the _id field shows the value being the string directly whereas the other two fields show that the values are Objects (that each contain { id: "<string>" } values).
The error is specifically stating that the code is expecting a string but finding a document:
BsonInvalidOperationException: readString can only be called when CurrentBSONType is STRING, not when CurrentBSONType is DOCUMENT.
I can't speak to the internal unpacking, but it really feels to me like the nested id.id (and potentially also hiringDetailsId.id) is the problem here. Even if it isn't directly related, it would seem to be an opportunity to simplify the schema unless there is a compelling reason to introduce that extra level of nesting.

ReactiveMongoRepository / MongoRepository does not return _id field

I think this issue probably has to do with my Mongo Document Koltin Data class, but for our business case we need to allow the user to add on any JSON fields to describe their RF data set.
Extending the BasicDBObject was the best way I have found.
The mono being returned when I save a SigMfMetaDocument does not contain the _id field.
I cannot figure out why the save method does not return a Mono wrapping a SigMfDocument with and _id
If there is a better way to create a Type for ReactiveMongoRepository that can dynamically accept any fields I am all ears.
#Document(collection = "sigmfmeta")
class SigMfMetaDocument : BasicDBObject {
#Id
#JsonProperty("id")
val id: String? = UUID.randomUUID().toString()
constructor(map: Map<String, Any>) : super(map)
constructor() : super()
constructor(key: String, value: Object): super()
}
#Repository
interface SigMfMetaRepository : ReactiveMongoRepository<SigMfMetaDocument, String>
So I found a way to solve this for my use case. I was originally assuming the description in the documentation for the save method would apply
(Saves a given entity. Use the returned instance for further operations as the save operation might have changed the entity instance completely).
My thought Mongo auto inserting the _id value would apply to this description.
I changed my model to:
#Document(collection = "sigmfmeta")
class SigMfMetaDocument : BasicBSONObject {
constructor(map: Map<String, Any>) : super(map) {
val id = ObjectId()
this.put("_id", id)
}
constructor() : super()
}
This way I have the _id value after saving for some business logic. Again I defined my Model this way because the metadata file we are accepting needs to allow a client to add any fields they wish to describe a binary file of RF measurement data.

Using java.util.Set domain property with Grails 3.1.x and Mongo 5.0.x plugin

I'm trying to create an embedded collection in a Grails GORM Mongo domain class.
class User {
String name
Set<String> friends = []
}
I want to store a Set (a non-duplicated list) of other names of users.
When I try to save the User domain class:
new User(name: 'Bob').save(failOnError: true)
I get the error.
org.bson.codecs.configuration.CodecConfigurationException: Can't find a codec for interface java.util.Set.
Changing the Set to List works fine but I don't want duplicates and don't want to have to manage that with a List.
Is there a way GORM will use the underlying Mongo $addToSet functionality.
It might be a GORM MongoDB issue. You can create an issue here with reproducing the issue.
But for now, you can do the workaround this problem using the List like this:
class User {
String name
List<String> friends = []
void removeDuplicate() {
this.friends?.unique()
}
def beforeInsert() {
removeDuplicate()
}
def beforeUpdate() {
removeDuplicate()
}
}

How to map Grails domain object to a specific mongodb table name

I've setup a simple grails app looking at a mongodb.
My domain object looks like this:
class GoogleSearch {
String _id;
String id;
String query;
String site;
Object results;
Date date;
static mapping = {
table 'google_searches'
}
static constraints = {
}
}
However when I run the grails app up, it keeps reading/writing to a table named "googleSearch"
Does anyone know how I can override this default naming? Is it a gorm/mongodb thing?
Cheers
Basics of MongoDB. There is no concept of table. It is always collections. :)
Refer mapping as collection 'google_searches'.
For more details you can refer Grails MongoDB plugin.

How do I get the date a MongoDB collection was created using MongoDB C# driver?

I need to iterate through all of the collections in my MongoDB database and get the time when each of the collections was created (I understand that I could get the timestamp of each object in the collection, but I would rather not go that route if a simpler/faster method exists).
This should give you an idea of what I'm trying to do:
MongoDatabase _database;
// code elided
var result = _database.GetAllCollectionNames().Select(collectionName =>
{
_database.GetCollection( collectionName ) //.{GetCreatedDate())
});
As far as I know, MongoDB doesn't keep track of collection creation dates. However, it's really easy to do this yourself. Add a simple method, something like this, and use it whenever you create a new collection:
public static void CreateCollectionWithMetadata(string collectionName)
{
var result = _db.CreateCollection(collectionName);
if (result.Ok)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
collectionMetadata.Insert(new { Id = collectionName, Created = DateTime.Now });
}
}
Then whenever you need the information just query the collectionMetadata collection. Or, if you want to use an extension method like in your example, do something like this:
public static DateTime GetCreatedDate(this MongoCollection collection)
{
var collectionMetadata = _db.GetCollection("collectionMetadata");
var metadata = collectionMetadata.FindOneById(collection.Name);
var created = metadata["Created"].AsDateTime;
return created;
}
The "creation date" is not part of the collection's metadata. A collection does not "know" when it was created. Some indexes have an ObjectId() which implies a timestamp, but this is not consistent and not reliable.
Therefore, I don't believe this can be done.
Like Mr. Gates VP say, there is no way using the metadata... but you can get the oldest document in the collection and get it from the _id.
Moreover, you can insert an "empty" document in the collection for that purpose without recurring to maintain another collection.
And it's very easy get the oldest document:
old = db.collection.find({}, {_id}).sort({_id: 1}).limit(1)
dat = old._id.getTimestamp()
By default, all collection has an index over _id field, making the find efficient.
(I using MongoDb 3.6)
Seems like it's some necroposting but anyway: I tried to find an answer and got it:
Checked it in Mongo shell, don't know how to use in C#:
// db.payload_metadata.find().limit(1)
ObjectId("60379be2bec7a3c17e6b662b").getTimestamp()
ISODate("2021-02-25T12:45:22Z")