Overwrite duplicate values in Scala List - scala

I have the following Comet server:
object UserServer extends LiftActor with ListenerManager {
private var users: List[UserItem] = Nil
def createUpdate = users
override def lowPriority = {
case UserItem(user, room, active, stamp) => {
users :+= UserItem(user, room, active, stamp);
updateListeners()
}
}
}
Currently each time the form is submitted a new UserItem is added to the users list. What I'm trying to do is each time the server receives a new user, instead of concatenating to the list it should overwrite an existing item with the same user and room handle.
So if the list contains the following:
UserItem("jam_2323", "demo-room", "James", "1320073365")
UserItem("jim_4533", "demo-room", "Jim", "1320073365")
The next time these users submit the form the above two items in the list will be replaced with the new stamp value:
UserItem("jam_2323", "demo-room", "James", "1320073435")
UserItem("jim_4533", "demo-room", "Jim", "1320073435")
Thanks in advance for any help, much appreciated :)

This sounds like a classic case where you need a map, rather than a list. I don't know about the details of Lift / Comet, but I guess you want something like
case class User(id: String)
case class Activity(room: String, active: String, stamp: String)
var lastUserActivity = Map[User, Activity]()
...
case UserItem(id, room, active, stamp) => {
lastUserActivity += User(id) -> Activity(room, active, stamp)
}

If you adjust UserItem from being a straight case class (I assume) to being one where you have overriden equals to ignore the stamp field, then you could make users into a Set.
Alternatively, you could filter the List to remove the old matching values before appending.

Related

BsonInvalidOperationException in ktor

Being new to MongoDB, I'm currently integrating the kMongo library to my ktor project, and trying to create a database to read & write event models to.
Following the instructions for object mapping in the kMongo user manual, I've created a mongoId field which gets serialised as a String named _id.
My event model is a data class, nested in sealed classes but gets serialised correctly by KotlinX-Serialization. The model looks as such:
sealed class Event {
#SerialName("_id") abstract val mongoId: String
abstract val id: ID.Event
abstract val dateTime: LocalDateTime
fun asString() = id.toString()
sealed class Hiring : Event() {
#SerialName("_id") abstract override val mongoId: String
abstract override val id: ID.Event
abstract override val dateTime: LocalDateTime
#Serializable
data class Start(
override val id: ID.Event,
override val dateTime: LocalDateTime,
val hiringDetailsId: ID.HiringDetails
) : Hiring() {
#SerialName("_id") override val mongoId: String = id.asString()
}
...
In a repository class, I initialise MongoDB and use the generic, parameter-less find() on a collection to retrieve all Event models from the database:
...
private val kmongo = KMongo.createClient().coroutine.client
private val db = kmongo.getDatabase("test")
private val eventCollection = db.getCollection<Event>().coroutine
...
override suspend fun getAllEvents() = eventCollection.find().toList()
Then inside of the Main class, I try to load the Event data on a click trigger:
...
val id = ID.Event(UUID())
...
it.on.click {
runBlocking {
val events = eventRepo.getAllEvents().toString()
logger.debug { events }
}
}
The strange part starts here, the server starts correctly and MongoDB is initialised correctly, but as soon as I try to do the read on the click trigger, I am presented with following error:
org.bson.BsonInvalidOperationException: readString can only be called when CurrentBSONType is STRING, not when CurrentBSONType is DOCUMENT.
at org.bson.AbstractBsonReader.verifyBSONType(AbstractBsonReader.java:689)
at org.bson.AbstractBsonReader.checkPreconditions(AbstractBsonReader.java:721)
at org.bson.AbstractBsonReader.readString(AbstractBsonReader.java:456)
at com.github.jershell.kbson.FlexibleDecoder.decodeString(BsonFlexibleDecoder.kt:130)
at kotlinx.serialization.encoding.AbstractDecoder.decodeStringElement(AbstractDecoder.kt:58)
at kotlinx.serialization.internal.AbstractPolymorphicSerializer.deserialize(AbstractPolymorphicSerializer.kt:52)
at kotlinx.serialization.encoding.Decoder$DefaultImpls.decodeSerializableValue(Decoding.kt:257)
at kotlinx.serialization.encoding.AbstractDecoder.decodeSerializableValue(AbstractDecoder.kt:16)
at org.litote.kmongo.serialization.SerializationCodec.decode(SerializationCodec.kt:66)
at com.mongodb.internal.operation.CommandResultArrayCodec.decode(CommandResultArrayCodec.java:52)
at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:60)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42)
at org.bson.internal.LazyCodec.decode(LazyCodec.java:48)
at org.bson.codecs.BsonDocumentCodec.readValue(BsonDocumentCodec.java:104)
at com.mongodb.internal.operation.CommandResultDocumentCodec.readValue(CommandResultDocumentCodec.java:63)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:87)
at org.bson.codecs.BsonDocumentCodec.decode(BsonDocumentCodec.java:42)
at com.mongodb.internal.connection.ReplyMessage.<init>(ReplyMessage.java:51)
at com.mongodb.internal.connection.InternalStreamConnection.getCommandResult(InternalStreamConnection.java:535)
at com.mongodb.internal.connection.InternalStreamConnection.access$500(InternalStreamConnection.java:86)
at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:520)
at com.mongodb.internal.connection.InternalStreamConnection$2$1.onResult(InternalStreamConnection.java:498)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:821)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback$MessageCallback.onResult(InternalStreamConnection.java:785)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
at java.base/sun.nio.ch.Invoker.invokeDirect(Invoker.java:160)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.implRead(UnixAsynchronousSocketChannelImpl.java:573)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:276)
at java.base/sun.nio.ch.AsynchronousSocketChannelImpl.read(AsynchronousSocketChannelImpl.java:297)
at com.mongodb.internal.connection.AsynchronousSocketChannelStream$AsynchronousSocketChannelAdapter.read(AsynchronousSocketChannelStream.java:144)
at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:118)
at com.mongodb.internal.connection.AsynchronousChannelStream.readAsync(AsynchronousChannelStream.java:107)
at com.mongodb.internal.connection.InternalStreamConnection.readAsync(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.InternalStreamConnection.access$600(InternalStreamConnection.java:86)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:775)
at com.mongodb.internal.connection.InternalStreamConnection$MessageHeaderCallback.onResult(InternalStreamConnection.java:760)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:645)
at com.mongodb.internal.connection.InternalStreamConnection$5.completed(InternalStreamConnection.java:642)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:250)
at com.mongodb.internal.connection.AsynchronousChannelStream$BasicCompletionHandler.completed(AsynchronousChannelStream.java:233)
at java.base/sun.nio.ch.Invoker.invokeUnchecked(Invoker.java:129)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finishRead(UnixAsynchronousSocketChannelImpl.java:447)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.finish(UnixAsynchronousSocketChannelImpl.java:195)
at java.base/sun.nio.ch.UnixAsynchronousSocketChannelImpl.onEvent(UnixAsynchronousSocketChannelImpl.java:217)
at java.base/sun.nio.ch.KQueuePort$EventHandlerTask.run(KQueuePort.java:312)
at java.base/sun.nio.ch.AsynchronousChannelGroupImpl$1.run(AsynchronousChannelGroupImpl.java:113)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at java.base/java.lang.Thread.run(Thread.java:833)
According to the stacktrace, something seems to go wrong in the BSON filtering part, despite there being none. When I use the MongoDB compass to validate the object inside of the database, I can see that everything is initialised and written perfectly fine:
The normal id field is used in my software internally as an ID.Event object type whilst the _id is used by Mongo internally.
Can someone point me to what the potential issue could be here?
I'm not familiar with Kotlin, but I'd like to dive into this a bit further:
If it wouldn't unwrap the second time, the Mongo Compass would likely reveal the _id or id field to contain a bracket { while these are currently mapped as expected (a String for _id and an object for id).
To confirm, the current structure of your document is (eg here in the playground):
{
_id: "7d51",
id: {
id: "7d51"
},
hiringDetailsId: {
id: "8392"
}
}
We can see that in your screenshot from Compass where the _id field shows the value being the string directly whereas the other two fields show that the values are Objects (that each contain { id: "<string>" } values).
The error is specifically stating that the code is expecting a string but finding a document:
BsonInvalidOperationException: readString can only be called when CurrentBSONType is STRING, not when CurrentBSONType is DOCUMENT.
I can't speak to the internal unpacking, but it really feels to me like the nested id.id (and potentially also hiringDetailsId.id) is the problem here. Even if it isn't directly related, it would seem to be an opportunity to simplify the schema unless there is a compelling reason to introduce that extra level of nesting.

Kmongo: how to add unique field

I have a simple user data class that looks like:
#Serializable
data class User(
#SerialName("_id")
val _id: Id<User> = newId(),
val email: String,
var password: String,
var tokens: Array<String> = arrayOf()
)
And I'd like the email value to be unique, i've tried a unique annotation which seemed most appropiate, but with no success.
I've also tried google and the KMongo website but I could not find an answer.
You need to create an index on the field (or combination of fields) that you want to ensure uniqueness on.
db.getCollection<User>().createIndex(User::email,
indexOptions = IndexOptions().unique(true))

Updating a many-to-many join table in slick 3.0

I have a database structure with a many-to-many relationship between Dreams and Tags.
Dreams and Tags are kept in separate tables, and there is a join table between them as usual in this kind of situation, with the class DreamTag representing the connection:
protected class DreamTagTable(tag: Tag) extends Table[DreamTag](tag, "dreamtags") {
def dreamId = column[Long]("dream_id")
def dream = foreignKey("dreams", dreamId, dreams)(_.id)
def tagId = column[Long]("tag_id")
def tag = foreignKey("tags", tagId, tags)(_.id)
// default projection
def * = (dreamId, tagId) <> ((DreamTag.apply _).tupled, DreamTag.unapply)
}
I have managed to perform the appropriate double JOIN to retrieve a Dream with its Tags, but I struggled to do it in a fully non-blocking manner.
Here is my code for performing the retrieval, as this may shed some light on things:
def createWithTags(form: DreamForm): Future[Seq[Int]] = db.run {
logger.info(s"Creating dream [$form]")
// action to put the dream
val dreamAction: DBIO[Dream] =
dreams.map(d => (d.title, d.content, d.date, d.userId, d.emotion))
.returning(dreams.map(_.id))
.into((fields, id) => Dream(id, fields._1, fields._2, fields._3, fields._4, fields._5))
.+=((form.title, form.content, form.date, form.userId, form.emotion))
// action to put any tags that don't already exist (create a single action)
val tagActions: DBIO[Seq[MyTag]] =
DBIO.sequence(form.tags.map(text => createTagIfNotExistsAction(text)))
// zip allows us to get the results of both actions in a tuple
val zipAction: DBIO[(Dream, Seq[MyTag])] = dreamAction.zip(tagActions)
// put the entries into the join table, if the zipAction succeeds
val dreamTagsAction = exec(zipAction.asTry) match {
case Success(value) => value match {
case (dream, tags) =>
DBIO.sequence(tags.map(tag => createDreamTagAction(dream, tag)))
}
case Failure(exception) => throw exception
}
dreamTagsAction
}
private def createTagIfNotExistsAction(text: String): DBIO[MyTag] = {
tags.filter(_.text === text).result.headOption.flatMap {
case Some(t: MyTag) => DBIO.successful(t)
case None =>
tags.map(t => (t.text))
.returning(tags.map(_.id))
.into((text, id) => MyTag(id, text)) += text
}
}
private def createDreamTagAction(dream: Dream, tag: MyTag): DBIO[Int] = {
dreamTags += DreamTag(dream.id, tag.id)
}
/**
* Helper method for executing an async action in a blocking way
*/
private def exec[T](action: DBIO[T]): T = Await.result(db.run(action), 2.seconds)
Scenario
Now I'm at the stage where I want to be able to update a Dream and the list of Tags, and I'm struggling.
Given a situation where the existing list of tags is ["one", "two", "three"] and is being updated to ["two", "three", "four"] I want to:
Delete the Tag for "one", if no other Dreams reference it.
Not touch the entries for "two" and "three", as the Tag and DreamTag entries already exist.
Create Tag "four" if it doesn't exist, and add an entry to the join table for it.
I think I need to do something like list1.diff(list2) and list2.diff(list1) but that would require getting performing a get first, which seems wrong.
Perhaps my thinking is wrong - Is it best to just clear all entries in the join table for this Dream and then create every item in the new list, or is there a nice way to diff the two lists (previous and existing) and perform the deletes/adds as appropriate?
Thanks for the help.
N.B. Yes, Tag is a super-annoying class name to have, as it clashes with slick.lifted.Tag!
Update - My Solution:
I went for option 2 as mentioned by Richard in his answer...
// action to put any tags that don't already exist (create a single action)
val tagActions: DBIO[Seq[MyTag]] =
DBIO.sequence(form.tags.map(text => createTagIfNotExistsAction(text)))
// zip allows us to get the results of both actions in a tuple
val zipAction: DBIO[(Int, Seq[MyTag])] = dreamAction.zip(tagActions)
// first clear away the existing dreamtags
val deleteExistingDreamTags = dreamTags
.filter(_.dreamId === dreamId)
.delete
// put the entries into the join table, if the zipAction succeeds
val dreamTagsAction = zipAction.flatMap {
case (_, tags) =>
DBIO.sequence(tags.map(tag => createDreamTagAction(dreamId, tag)))
}
deleteExistingDreamTags.andThen(dreamTagsAction)
I struggled to do it in a fully non-blocking manner.
I see you have an eval call which is blocking. I looks like this can be replaced with a flatMap:
case class Dream()
case class MyTag()
val zipAction: DBIO[(Dream, Seq[MyTag])] =
DBIO.successful( (Dream(), MyTag() :: MyTag() :: Nil) )
def createDreamTagAction(dream: Dream)(tag: MyTag): DBIO[Int] =
DBIO.successful(1)
val action: DBIO[Seq[Int]] =
zipAction.flatMap {
case (dream, tags) => DBIO.sequence(tags.map(createDreamTagAction(dream)))
}
Is it best to just clear all entries in the join table for this Dream and then create every item in the new list, or is there a nice way to diff the two lists (previous and existing) and perform the deletes/adds as appropriate?
Broadly, you have three options:
Look in the database to see what tags exist, compare them to what you want the state to be, and compute a set of insert and delete actions.
Delete all the tags and insert the state you want to reach.
Move the problem to SQL so you insert tags where they don't already exist in the table, and delete tags that don't exist in your desired state. You'd need to look at the capabilities of your database and likely need to use Plain SQL in Slick to get the effect. I'm not sure what the insert would be for adding tags (perhaps a MERGE or upsert of some kind), but deleting would be of the form: delete from tags where tag not in (1,2) if you wanted a final state of just tags 1 and 2.
The trades off:
For 1, you need to run 1 query to fetch existing tags, and then 1 query for the deletes, and at least 1 for the inserts. This will change the smallest number of rows, but will be the largest number of queries.
For 2, you'll be executing at least 2 queries: a delete and 1 (potentially) for a bulk insert. This will change the largest number of rows.
For 3, you'll be executing a constant 2 queries (if your database can carry out the logic for you). If this is even possible, the queries will be more complicated.

Play, Scala, Ebean: Checking if relationship exists

Consider a hypothetical situation where I have two models: Company and User, defined like so
case class Company(name: String) extends Model {
#Id
var id: Long = _
#OneToMany(fetch = FetchType.LAZY)
var admins: util.List[User] = new util.ArrayList[User]()
}
case class User(email: String) {
#Id
var id: Long = _
}
Next, I have a request coming in and I want to check if user_id 200 is an admin of a company_id 100. The obvious solution is to fetch the company with that id, and then check iteratively in the admins list if a user_id but that is quite inefficient. What's the best way to go about this?
I think that the easiest way to solve this problem is to simply add relation from user to company. If you need calling such query then such relation has sense:
Here is relation added to User class:
#ManyToOne
var company:Company = _
And here is example how to check if user(id=200) is admin of company(id=100):
val user = Ebean.find(classOf[User], 200L)
println(user.company.id==100L)
Second option here is using RawSql. We can do it in a way similar to this:
val sql="select u.company_id as id from user u where u.id=200"
val rawSql = RawSqlBuilder.parse(sql).create()
val query = Ebean.find(classOf[Company])
query.setRawSql(rawSql)
val list = query.findList()
println(list(0).id==100L)

Lazyloading collection in play-salat

Is it possible to load a collection lazy with Sala?
e.g. I have an object like
Example 1 (in this case, the whole user list is loaded when retrieving the object)
case class Test(
#Key("_id") _id: ObjectId = new ObjectId,
name: String,
users: List[User]) {
}
or Example 2 (the object is loaded without the list, but no idea how to get the users list)
case class Test(
#Key("_id") _id: ObjectId = new ObjectId,
name: String) {
#Persist val users: List[User] = List()
}
How can I load the object in the first example without the users list?
or: How can I load the users list in the second example?
Thanks in advance!
Salat author here.
Salat doesn't have anything like ORM lazy loading. The #Persist annotation is meant to persist fields outside of the constructor, but suppresses deserialization because only fields in the constructor will be deserialized.
But you can easily decide when making the query whether you want the list of users or not.
case class Test(#Key("_id") id = new ObjectId, name: String, users: List[User] = Nil)
You can persist the users as embedded documents inside the test document, and then use the second argument of the query, the ref, to exclude (0) or include (1) fields in the object.
TestDAO.find(/* query */, MongoDBObject("users" -> 0))
The other strategy is to break out user documents into a child collection - see https://github.com/novus/salat/wiki/ChildCollection for more information. In this example, Test is the "parent" and User is the "child".
The strategy there is that in the parent DAO, when saving, you override the save methods to save users using the child DAO, and then save the parent object with users set to Nil.
Then, by default, a Test instance is retrieved with users set to Nil.
If you want to retrieve Test with users, you will need to add a find method to your DAO that manually:
find the test document
use the _id field of the test document to query for user documents by parent id - this will yield List[User]
deserialize the test document to an instance of Test using grater[Test] and copy it with the list of users