MongoJack with Kotlin - mongodb

I might be asking for trouble with this setup but I am trying to get MongoDB with MongoJack to work with Kotlin classes.
I've managed to overcome a few problems already for which I will outline my solutions if others have the same problem but I am still struggling with one issue.
Kotlin class:
data class TestDocument (
#ObjectId #JsonProperty("_id") var id:String? = null,
#JsonProperty("name") var name:String,
var date: Date = Date(),
var decimal: BigDecimal = BigDecimal.valueOf(123.456)
)
If I use the default setup (using the ObjectMapper mongoJack configured) like this:
object MongoClientTestDefault {
val log: Log = Logging.getLog(MongoClientTest::class.java)
val mongoClient : MongoClient
val database: MongoDatabase
val testCol: MongoCollection<TestDocument>
init {
val mongoUrl = "mongodb://localhost:27017/"
log.info("Initialising MongoDB Client with $mongoUrl")
try {
mongoClient = MongoClients.create(mongoUrl)
database = mongoClient.getDatabase("unitTest")
testCol = JacksonMongoCollection.builder()
.build(
mongoClient,
"unitTest",
"testCol",
TestDocument::class.java,
UuidRepresentation.STANDARD
)
log.info("Successfully initialised MongoDB client")
}catch(e: Exception){
log.error(Strings.EXCEPTION,e)
log.error("Mongo URL: ", mongoUrl)
throw e //we need to throw this as otherwise it complains that the val fields haven't been initialised
}
}
}
Then the serialization (or persisting to Mongo) works ok but the deserialization from Mongo to the Kotlin class fails because without the Kotlin jackson module Jackson and MongoJack don't know how to handle the Kotlin constructor it seems.
To fix that you have to use the jacksonObjectMapper() and pass that to the builder:
testCol = JacksonMongoCollection.builder()
.withObjectMapper(jacksonObjectMapper)
.build(
mongoClient,
"unitTest",
"testCol",
TestDocument::class.java,
UuidRepresentation.STANDARD
)
That will solve the deserialization but now the Dates are causing problems.
So in order to fix that I do the following
object MongoClientTest {
val log: Log = Logging.getLog(MongoClientTest::class.java)
val mongoClient : MongoClient
val database: MongoDatabase
val testCol: MongoCollection<TestDocument>
var jacksonObjectMapper = jacksonObjectMapper()
init {
val mongoUrl = "mongodb://localhost:27017/"
MongoJackModule.configure(jacksonObjectMapper)
log.info("Initialising MongoDB Client with $mongoUrl")
try {
mongoClient = MongoClients.create(mongoUrl)
database = mongoClient.getDatabase("unitTest")
testCol = JacksonMongoCollection.builder()
.withObjectMapper(jacksonObjectMapper)
.build(
mongoClient,
"unitTest",
"testCol",
TestDocument::class.java,
UuidRepresentation.STANDARD
)
log.info("Successfully initialised MongoDB client")
}catch(e: Exception){
log.error(Strings.EXCEPTION,e)
log.error("Mongo URL: ", mongoUrl)
throw e //we need to throw this as otherwise it complains that the val fields haven't been initialised
}
}
}
I take the jacksonObjectMapper and call MongoJackModule.configure(jacksonObjectMapper) and then pass it to the builder.
Now serialization (persiting to Mongo) and deserialization (loading from Mongo and converting to Pojo) works but there is one Exception thrown (which seems to be handled).
var testDoc = TestDocument(name="TestName")
MongoClientTest.testCol.drop()
var insertOneResult = MongoClientTest.testCol.insertOne(testDoc)
This call works and it creates the document in Mongo but this exception is printed:
java.lang.UnsupportedOperationException: Cannot call setValue() on constructor parameter of com.acme.MongoJackTest$TestDocument
at com.fasterxml.jackson.databind.introspect.AnnotatedParameter.setValue(AnnotatedParameter.java:118)
at org.mongojack.internal.stream.JacksonCodec.lambda$getIdWriter$4(JacksonCodec.java:120)
at org.mongojack.internal.stream.JacksonCodec.generateIdIfAbsentFromDocument(JacksonCodec.java:77)
at com.mongodb.internal.operation.Operations.bulkWrite(Operations.java:450)
at com.mongodb.internal.operation.Operations.insertOne(Operations.java:375)
at com.mongodb.internal.operation.SyncOperations.insertOne(SyncOperations.java:177)
at com.mongodb.client.internal.MongoCollectionImpl.executeInsertOne(MongoCollectionImpl.java:476)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:459)
at com.mongodb.client.internal.MongoCollectionImpl.insertOne(MongoCollectionImpl.java:453)
at org.mongojack.MongoCollectionDecorator.insertOne(MongoCollectionDecorator.java:528)
It is trying to create and set the _id on the TestDocument but struggles with the primary constructor of the Kotlin class.
I've tried to add a 'default aka empty' constructor:
data class TestDocument (
#ObjectId #JsonProperty("_id") var id:String? = null,
#JsonProperty("name") var name:String,
var date: Date = Date(),
var decimal: BigDecimal = BigDecimal.valueOf(123.456)
){
#JsonCreator constructor(): this(name = "")
}
But that doesn't fix it.
Any ideas how I could fix this?
This is the code in MongoJack that is catching the exception:
private Consumer<BsonObjectId> getIdWriter(final T t) {
final Optional<BeanPropertyDefinition> maybeBpd = getIdElementSerializationDescription(t.getClass());
return maybeBpd.<Consumer<BsonObjectId>>map(beanPropertyDefinition -> (bsonObjectId) -> {
try {
if (bsonObjectId != null) {
beanPropertyDefinition.getMutator().setValue(
t,
extractIdValue(bsonObjectId, beanPropertyDefinition.getRawPrimaryType())
);
}
} catch (Exception e) {
e.printStackTrace();
}
}).orElseGet(() -> (bsonObjectId) -> {
});
}
I guess it could just be ignored but that could flood the logs.

Related

Migration not starting with mongock 4.3.8

I am trying to create indexes on a mongodb collection using mongock version 4.3.8. The mongockLock and mongockChangeLog collections are eventually created, but the migration itself does not run. mongockChangeLog table is empty. Query db.BOOKS.getIndexes(); returns a single string: with an index for id (the value of name in it is equal to _id_), which is most likely generated automatically. There are no obvious errors in the logs either.
Everything else seems to be set up correctly, I don't understand why my migration is not running.
build.gradle.kts:
implementation("com.github.cloudyrock.mongock:mongodb-springdata-v3-driver:4.3.8")
implementation("com.github.cloudyrock.mongock:mongock-spring-v5:4.3.8")
implementation(platform("com.github.cloudyrock.mongock:mongock-bom:4.3.8"))
application.yml:
mongock:
change-logs-scan-package:
- com.dreamland.configuration.migration
config file:
#EnableMongock
#Configuration
#EnableMongoAuditing
class MongoConfig
Migration file:
#ChangeLog(order = "1668471203")
class IndexChangeLog {
companion object : KLogging()
#ChangeSet(order = "1668471204", id = "1668471204_create_indexes", author = "Irish")
fun createIndexes(mongockTemplate: MongockTemplate) {
val indexOps = mongockTemplate.indexOps(BookObjectDocument::class.java)
Index().on("createdAt", Sort.Direction.DESC).let { indexOps.ensureIndex(it) }.also { log(it) }
Index().on("createdBy", Sort.Direction.ASC).let { indexOps.ensureIndex(it) }.also { log(it) }
Index().on("type", Sort.Direction.DESC).let { indexOps.ensureIndex(it) }.also { log(it) }
}
private fun log(indexName: String) {
logger.info { "Index '$indexName' was created successfully" }
}
}
Document class:
#Document(collection = "BOOKS_V2")
data class BookObjectDocument(
val type: BookObjectType?,
val description: String?
) {
#Id
lateinit var id: String
#CreatedDate
lateinit var createdAt: Instant
#CreatedBy
var createdBy: String? = null
}

LIveData is observed more than twice everytime from acivity and emitting previous data

I follow MVVM Login API, with Retrofit ,My problem is livedata is observed more than twice and always emitting previous response when observed from activity, But inside Repository its giving correct response
I tried a lot of solutions from stackoverflow and other websites but still no luck, Tried removing observers also but still getting previous data ,so plz suggest a working solution, I will post my code below,
LoginActivity.kt
private lateinit var loginViewModel: LoginViewModel
loginViewModel = ViewModelProvider(this, ViewModelProvider.NewInstanceFactory()).get(
LoginViewModel::class.java
)
loginViewModel.login(userEmail, pwd)
loginViewModel.getLoginRepository().observe(this, Observer {
val loginResult = it ?: return#Observer
val accessToken = loginResult.user?.jwtToken.toString()
})
val statusMsgObserver = Observer<String> { statusMsg ->
showToast(statusMsg)
})
val errorMsgObserver = Observer<String> { errorMsg ->
// Update the UI
showToast(errorMsg)
})
loginViewModel.getStatusMessage()?.observe(this, statusMsgObserver)
loginViewModel.getErrorStatusMessage()?.observe(this, errorMsgObserver)
LoginViewModel.kt:
class LoginViewModel: ViewModel() {
private var loginRepository: LoginRepository? = null
private var _mutableLiveData = MutableLiveData<LoginAPIResponse?>()
val liveData: LiveData<LoginAPIResponse?> get() = _mutableLiveData
private var responseMsgLiveData:MutableLiveData<String>?= null
private var errorResponseMsgLiveData:MutableLiveData<String>?= null
fun login(username: String, password: String) {
loginRepository = LoginRepository.getInstance()!!
/* Query data from Repository */
//val _mutableLiveData: MutableLiveData<Response<LoginAPIResponse?>?>? = loginRepository?.doLogin(username, password)
_mutableLiveData = loginRepository?.doLogin(username, password)!!
responseMsgLiveData = loginRepository?.respMessage!!
errorResponseMsgLiveData = loginRepository?.loginResponseErrorData!!
}
fun getLoginRepository(): LiveData<LoginAPIResponse?> {
return liveData
}
fun getStatusMessage(): LiveData<String>? {
return responseMsgLiveData
}
fun getErrorStatusMessage(): LiveData<String>? {
return errorResponseMsgLiveData
}
}
LoginRepository.kt:
class LoginRepository {
private val loginApi: ApiEndpoints = RetrofitService.createService(ApiEndpoints::class.java)
val responseData = MutableLiveData<LoginAPIResponse?>()
var respMessage = MutableLiveData<String>()
var loginResponseErrorData = MutableLiveData<String>()
fun doLogin(username: String, password: String)
: MutableLiveData<LoginAPIResponse?> {
respMessage.value = null
loginResponseErrorData.value = null
val params = JsonObject()
params.addProperty("email", username)
params.addProperty("password",password)
val jsonParams = JsonObject()
jsonParams.add("user",params)
loginApi.loginToServer(jsonParams).enqueue(object : Callback<LoginAPIResponse?> {
override fun onResponse( call: Call<LoginAPIResponse?>, response: Response<LoginAPIResponse?> ) {
responseData.value = response.body()
respMessage.value = RetrofitService.handleError(response.code())
val error = response.errorBody()
if (!response.isSuccessful) {
val errorMsg = error?.charStream()?.readText()
println("Error Message: $errorMsg")
loginResponseErrorData.value = errorMsg
} else {
println("API Success -> Login, $username, ${response.body()?.user?.email.toString()}")
}
}
override fun onFailure(call: Call<LoginAPIResponse?>, t: Throwable) {
println("onFailure:(message) "+t.message)
loginResponseErrorData.value = t.message
responseData.value = null
}
})
return responseData
}
companion object {
private var loginRepository: LoginRepository? = null
internal fun getInstance(): LoginRepository? {
if (loginRepository == null) {
loginRepository = LoginRepository()
}
return loginRepository
}
}
}
In onDestroy(),I have removed the observers,
override fun onDestroy() {
super.onDestroy()
loginViewModel.getLoginRepository()?.removeObservers(this)
this.viewModelStore.clear()
}
In LoginActivity, when I observe loginResult it gives previous emitted accessToken first and then again called and giving current accessToken, Similarly observer is called more than twice everytime.
But inside repository,its giving recent data, plz check my code and suggest where I have to correct to get correct recent livedata
Finally i found the solution, In LoginRepository, I declared responseData outside doLogin(), It should be declared inside doLogin()
Since it was outside the method, it always gave previous data first and then current data,
Once I declare inside method problem was solved and now it is working Perfect!!!

store Spark RDD in mongodb

How can i store spark streaming data in the mongodb.
in java this is done like :
data.foreachRDD(
new Function<JavaRDD<String>, Void>() {
Mongo mongo = new Mongo("localhost", 27017);
DB db = mongo.getDB("mongodb");
DBCollection collection = db.getCollection("fb");
public Void call(JavaRDD<String> data) throws Exception {
if(data!=null){
List<String>result=data.collect();
for (String temp :result) {
System.out.println(temp);
DBObject dbObject = (DBObject)JSON.parse(temp.toString());
collection.insert(dbObject);
}
System.out.println("Inserted Data Done");
} else {
System.out.println("Got no data in this window");
}
return null;
}
}
);
where i want to store data in mongodb but in scala. the above code is in java.
//remove if not needed
import scala.collection.JavaConversions._
data.foreachRDD(new Function[JavaRDD[String], Void]() {
var mongo: Mongo = new Mongo("localhost", 27017)
var db: DB = mongo.getDB("mongodb")
var collection: DBCollection = db.getCollection("fb")
def call(data: JavaRDD[String]): Void = {
if (data != null) {
val result = data.collect()
for (temp <- result) {
println(temp)
val dbObject = JSON.parse(temp.toString).asInstanceOf[DBObject]
collection.insert(dbObject)
}
println("Inserted Data Done")
} else {
println("Got no data in this window")
}
null
}
})

Defer val definition in Scala

Have the following:
var jdt : JDateTime = null
try {
jdt = new JDateTime(timeString, "YYYY-MM-DD hh:mm:ss,mss")
} catch {
case e : Exception => return Option.empty
}
I would like jdt to be val, as it is a constant value. Is there any scala syntax trick that can work here? Like, in case of exception, set null etc.
Not a syntax trick, just library usage:
import scala.util.Try
val jdt = Try(new JDateTime(timeString, "YYYY-MM-DD hh:mm:ss,mss")).toOption.orNull
Alhough, I would suggest leaving it at Option instead of using null.
This is also allowed since try produces a result:
val jdt =
try {
new JDateTime(timeString, "YYYY-MM-DD hh:mm:ss,mss")
} catch {
case e : Exception => return Option.empty
}
Even in java I can do this with final since I defer setting the variable until I have to:
final Date date;
try {
date = new SimpleDateFormat().parse("");
} catch (ParseException e) {
return null;
}

Update Mongo document field from Grails app

I have a requirement where I have to check in the database if the value of a boolean variable(crawled) is false. If so I have to set it to true. I am finding the record based on the value of a string variable(website). I referenced this link but it didn't help. Please tell me what I am doing wrong.
I tried this:
def p = Website.findByWebsite(website);
if(p['crawled'] == true) {
println "already crawled"
} else {
mongo.website.update( p, new BasicDBObject( '$set', new BasicDBObject( 'crawled', 'false' ) ) )
println "updated object"
}
It gives me an error No such property: mongo for class: cmsprofiler.ResourceController
My domain class is as follows:
class Website{
String website
User user
Boolean crawled
static belongsTo = [user: User]
static constraints = {
website( url:true, unique: ['user'])
}
static hasMany = [resource: Resource]
static mapping = {resource cascade:"all-delete-orphan" }
}
you should be using
Website.mongo.update(...)
or let the framework inject it:
class ResourceController {
def mongo
def list(){
mongo.website.update(...)
}
}
This worked for me. Posting it here in case anyone else has similar requirement.
#Grab(group='com.gmongo', module='gmongo', version='0.9.3')
import com.gmongo.GMongo
#Transactional(readOnly = true)
class ResourceController {
def mongo = new GMongo()
def db = mongo.getDB("resources")
def p = Website.findByWebsite(website)
if(p['crawled']==false){
db.website.update([crawled:false],[$set:[crawled:true]])
}