I'm doing my first application with Scala and Play! Where I use a WebSocket exactly like they do in the chatroom example, so far I can send messages to my server but I don't seem to understand where I can "handle" this messages that I get from my client, also I wanted to know if I can send Json Arrays from my server to my client:
#Singleton
class HomeController #Inject()(cc: ControllerComponents)
(implicit actorSystem: ActorSystem,
mat: Materializer,
executionContext: ExecutionContext)
extends AbstractController(cc) {
private type WSMessage = String
private val logger = Logger(getClass)
private implicit val logging = Logging(actorSystem.eventStream, logger.underlyingLogger.getName)
// chat room many clients -> merge hub -> broadcasthub -> many clients
private val (chatSink, chatSource) = {
// Don't log MergeHub$ProducerFailed as error if the client disconnects.
// recoverWithRetries -1 is essentially "recoverWith"
val source = MergeHub.source[WSMessage]
.log("source")
.recoverWithRetries(-1, { case _: Exception ⇒ Source.empty })
val sink = BroadcastHub.sink[WSMessage]
source.toMat(sink)(Keep.both).run()
}
private val userFlow: Flow[WSMessage, WSMessage, _] = {
Flow.fromSinkAndSource(chatSink, chatSource)
}
def index: Action[AnyContent] = Action { implicit request: RequestHeader =>
val webSocketUrl = routes.HomeController.chat().webSocketURL()
logger.info(s"index: ")
Ok(views.html.index(webSocketUrl))
}
def chat(): WebSocket = {
WebSocket.acceptOrResult[WSMessage, WSMessage] {
case rh if sameOriginCheck(rh) =>
Future.successful(userFlow).map { flow =>
Right(flow)
}.recover {
case e: Exception =>
val msg = "Cannot create websocket"
logger.error(msg, e)
val result = InternalServerError(msg)
Left(result)
}
case rejected =>
logger.error(s"Request ${rejected} failed same origin check")
Future.successful {
Left(Forbidden("forbidden"))
}
}
}
}
Btw I'm sending the messages from my client through a Jquery function.
Edit The way I want to handle this messages is by passing them as parameters to a function, the function will return an Array of strings or integers which I want to return to the client
All you need to do is add a flow stage to your between your source and sink which can store in DB and return some other message to clients.
Here's how you can fix the above problem:
Do whatever you want with the received message here.
val flow = Flow[WSMessage].map { element =>
println(s"Message: $element")
element
}
Note that for persisting messages in the database, you will most likely want to use an async stage, roughly:
val flow = Flow[WSMessage].mapAsync(parallelism) { element =>
println(s"Message: $element")
// assuming DB.write() returns a Future[Unit]
DB.write(element).map(_ => element)
}
And finally, you need to add the flow stage to the stream pipeline.
source.via(flow).toMat(sink)(Keep.both).run()
Related
I am trying to split a big chunk of text into multiple paragraphs and process it concurrently by calling an external API.
An immutable list is updated each time the response comes from the API for the paragraph.
Once the paragraphs are processed and the list is updated, I would like to ask the Actor for the final status to be used in the next steps.
The problem with the below approach is that I would never know when all the paragraphs are processed.
I need to get back the targetStore once all the paragraphs are processed and the list is final.
def main(args: Array[String]) {
val source = Source.fromFile("input.txt")
val extDelegator = new ExtractionDelegator()
source.getLines().foreach(line => extDelegator.processParagraph(line))
extDelegator.getFinalResult()
}
case class Extract(uuid: UUID, text: String)
case class UpdateList(text: String)
case class DelegateLambda(text: String)
case class FinalResult()
class ExtractionDelegator {
val system = ActorSystem("ExtractionDelegator")
val extActor = system.actorOf(Props(classOf[ExtractorDelegateActor]).withDispatcher("fixed-thread-pool"))
implicit val executionContext = system.dispatchers.lookup("fixed-thread-pool")
def processParagraph(text: String) = {
extActor ! Extract(uuid, text)
}
def getFinalResult(): java.util.List[String] = {
implicit val timeout = Timeout(5 seconds)
val askActor = system.actorOf(Props(classOf[ExtractorDelegateActor]))
val future = askActor ? FinalResult()
val result = Await.result(future, timeout.duration).asInstanceOf[java.util.List[String]]
result
}
def shutdown(): Unit = {
system.terminate()
}
}
/* Extractor Delegator actor*/
class ExtractorDelegateActor extends Actor with ActorLogging {
var targetStore:scala.collection.immutable.List[String] = scala.collection.immutable.List.empty
def receive = {
case Extract(uuid, text) => {
context.actorOf(Props[ExtractProcessor].withDispatcher("fixed-thread-pool")) ! DelegateLambda(text)
}
case UpdateList(res) => {
targetStore = targetStore :+ res
}
case FinalResult() => {
val senderActor=sender()
senderActor ! targetStore
}
}
}
/* Aggregator actor*/
class ExtractProcessor extends Actor with ActorLogging {
def receive = {
case DelegateLambda(text) => {
val res =callLamdaService(text)
sender ! UpdateList(res)
}
}
def callLamdaService(text: String): String = {
//THis is where external API is called.
Thread.sleep(1000)
result
}
}
Not sure why you want to use actors here, most simple would be to
// because you call external service, you have back async response most probably
def callLamdaService(text: String): Future[String]
and to process your text you do
implicit val ec = scala.concurrent.ExecutionContext.Implicits.global // use you execution context here
Future.sequence(source.getLines().map(callLamdaService)).map {results =>
// do what you want with results
}
If you still want to use actors, you can do it replacing callLamdaService to processParagraph which internally will do ask to worker actor, who returns result (so, signature for processParagraph will be def processParagraph(text: String): Future[String])
If you still want to start multiple tasks and then ask for result, then you just need to use context.become with receive(worker: Int), when you increase amount of workers for each Extract message and decrease amount of workers on each UpdateList message. You will also need to implement then delayed processing of FinalResult for the case of non-zero amount of processing workers.
I use ask mode send a request to an actor called actor-A in a function called fun-F. The actor will get an ID generated by another system in an async way, when this is completed, I will forward a message contains this ID to another actor called actor-B, the actor-B will do some DB operations and then send back the DB operation result in a message to the sender, since in my case I use forward mode, so the actor-B recognize the sender as fun-F, the akka will give the fun-F a temporary actor name, so the returned value should be delivered to the temp actor.
My question is:
If I use sync-method to get the ID from another system, then forward this message to the actor-B, after actor-B's DB operation, the result can be delivered to the value of fun-F, and the fun-F is defined as temporary actor Actor[akka://ai-feedback-service/temp/$b] by the akka framework runtime.
If I use async-method to get the ID from another system, when it completed, I will forward the message in the oncompleted {} code block in another call back thread, the DB operation in actor-B is handled successfully, but the returned value cannot be delivered to the value defined in the fun-F, and in this case the fun-F is defined as Actor[akka://ai-feedback-service/deadLetters] by the akka framwork runtime. So the actor-B lose its way and do not know how to get back or where should this message be delivered, and this will cause an Ask time out exception throws in my log.
How can I handled this issue? or how can I avoid this dead letter ask time out exception?
Below is my code:
// this is the so-called fun-F [createFeedback]
def createFeedback(query: String,
response: String,
userId: Long,
userAgent: String,
requestId: Long,
errType: Short,
memo: String): Future[java.lang.Long] = {
val ticket = Ticket(userId,
requestId,
query,
response,
errType,
userAgent,
memo)
val issueId = (jiraActor ? CreateJiraTicketSignal(ticket))
.mapTo[CreateFeedbackResponseSignal].map{ r =>
r.issueId.asInstanceOf[java.lang.Long]
}
issueId
}
//this is the so-called actor-A [jiraActor]
//receive method are run in its parent actor for some authorization
//in this actor only override the handleActorMsg method to deal msg
override def handleActorMsg(msg: ActorMsgSignal): Unit = {
msg match {
case s:CreateJiraTicketSignal =>
val issueId = createIssue(cookieCache.cookieContext.flag,
cookieCache.cookieContext.cookie,
s.ticket)
println(s">> ${sender()} before map $issueId")
issueId.map{
case(id:Long) =>
println(s">> again++issueId = $id ${id.getClass}")
println(s">>> $self / ${sender()}")
println("again ++ jira action finished")
dbActor.forward(CreateFeedbackSignal(id,s.ticket))
case(message:String) if(!s.retry) =>
self ! CreateJiraTicketSignal(s.ticket,true)
case(message:String) if(s.retry) =>
log.error("cannot create ticket :" + message)
}
println(s">> after map $issueId")
}
//this is the so-called actor-B [dbActor]
override def receive: Receive = {
case CreateFeedbackSignal(issueId:Long, ticket:Ticket) =>
val timestampTicks = System.currentTimeMillis()
val description: String = Json.obj("question" -> ticket.query,
"answer" -> ticket.response)
.toString()
dao.createFeedback(issueId,
ticket.usrId.toString,
description,
FeedbackStatus.Open.getValue
.asInstanceOf[Byte],
new Timestamp(timestampTicks),
new Timestamp(timestampTicks),
ticket.usrAgent,
ticket.errType,
ticket.memo)
println(s">> sender = ${sender()}")
sender() ! (CreateFeedbackResponseSignal(issueId))
println("db issue id is " + issueId)
println("db action finished")
}
To avoid the dead letters issue, do the following:
For every request, use an identifier (probably the requestId) that you can associate with the ultimate target for the request. That is, tie the requestId that you're passing to the createFeedback method to the caller (ActorRef) of that method, then pass this id through your messaging chain. You can use a map to hold these associations.
Change CreateFeedbackResponseSignal(issueId) to include the requestId from the Ticket class: CreateFeedbackResponseSignal(requestId, issueId).
When dealing with the asynchronous result of a Future from inside an actor, pipe the result of the Future to self instead of using a callback.
With this approach, the result of createIssue will be sent to jiraActor when the result is available. jiraActor then sends that result to dbActor.
jiraActor will be the sender in dbActor. When jiraActor receives the result from dbActor, jiraActor can look up the reference to the target in its internal map.
Below is a simple example that mimics your use case and is runnable in ScalaFiddle:
import akka.actor._
import akka.pattern.{ask, pipe}
import akka.util.Timeout
import language.postfixOps
import scala.concurrent._
import scala.concurrent.duration._
case class Signal(requestId: Long)
case class ResponseSignal(requestId: Long, issueId: Long)
object ActorA {
def props(actorB: ActorRef) = Props(new ActorA(actorB))
}
class ActorA(dbActor: ActorRef) extends Actor {
import context.dispatcher
var targets: Map[Long, ActorRef] = Map.empty
def receive = {
case Signal(requestId) =>
val s = sender
targets = targets + (requestId -> s)
createIssue(requestId).mapTo[Tuple2[Long, Long]].pipeTo(self) // <-- use pipeTo
case ids: Tuple2[Long, Long] =>
println(s"Sending $ids to dbActor")
dbActor ! ids
case r: ResponseSignal =>
println(s"Received from dbActor: $r")
val target = targets.get(r.requestId)
println(s"In actorA, sending to: $target")
target.foreach(_ ! r)
targets = targets - r.requestId
}
}
class DbActor extends Actor {
def receive = {
case (requestId: Long, issueId: Long) =>
val response = ResponseSignal(requestId, issueId)
println(s"In dbActor, sending $response to $sender")
sender ! response
}
}
val system = ActorSystem("jiratest")
implicit val ec = system.dispatcher
val dbActor = system.actorOf(Props[DbActor])
val jiraActor = system.actorOf(Props(new ActorA(dbActor)))
val requestId = 2L
def createIssue(requestId: Long): Future[(Long, Long)] = {
println(s"Creating an issue ID for requestId[$requestId]")
Future((requestId, 99L))
}
def createFeedback(): Future[Long] = {
implicit val timeout = Timeout(5.seconds)
val res = (jiraActor ? Signal(requestId)).mapTo[ResponseSignal]
res.map(_.issueId)
}
createFeedback().onComplete { x =>
println(s"Done: $x")
}
Running the above code in ScalaFiddle results in the following output:
Creating an issue ID for requestId[2]
Sending (2,99) to dbActor
In dbActor, sending ResponseSignal(2,99) to Actor[akka://jiratest/user/$b#-710097339]
Received from dbActor: ResponseSignal(2,99)
In actorA, sending to: Some(Actor[akka://jiratest/temp/$a])
Done: Success(99)
I have a basic scala akka http CRUD application. See below for the relevant classes.
I'd simply like to write an entity id and some data (as json) to a Kafka topic whenever, for example, an entity is created/updated.
I'm looking at http://doc.akka.io/docs/akka-stream-kafka/current/producer.html, but am new to scala and akka, and unsure of how to integrate it into my application?
For example, from the docs above, this is the example of a producer writing to kafka, so I think I need to something similar, but whereabouts in my application should this go? Can I just add another map call in the create method in my service after I have created the user?
Many thanks!
val done = Source(1 to 100)
.map(_.toString)
.map { elem =>
new ProducerRecord[Array[Byte], String]("topic1", elem)
}
.runWith(Producer.plainSink(producerSettings))
Or do I need to do something like the example here https://github.com/hseeberger/accessus in the bindAndHandle() method in my Server.scala?
WebServer.scala
object System {
implicit val system = ActorSystem()
implicit val dispatcher = system.dispatcher
implicit val actorMaterializer = ActorMaterializer()
}
object WebServer extends App {
import System._
val config = new ApplicationConfig() with ConfigLoader
ConfigurationFactory.setConfigurationFactory(new LoggingConfFileConfigurationFactory(config.loggingConfig))
val injector = Guice.createInjector(new MyAppModule(config))
val routes = injector.getInstance(classOf[Router]).routes
Http().bindAndHandle(routes, config.httpConfig.interface, config.httpConfig.port)
}
Router.scala
def routes(): Route = {
post {
entity(as[User]) { user =>
val createUser = userService.create(user)
onSuccess(createUser) {
case Invalid(y: NonEmptyList[Err]) => {
throw new ValidationException(y)
}
case Valid(u: User) => {
complete(ToResponseMarshallable((StatusCodes.Created, u)))
}
}
}
} ~
// More routes here, left out for example
}
Service.scala
def create(user: User): Future[MaybeValid[User]] = {
for {
validating <- userValidation.validateCreate(user)
result <- validating match {
case Valid(x: User) =>
userRepo.create(x)
.map(dbUser => Valid(UserConverters.fromUserRow(x)))
case y: DefInvalid =>
Future{y}
}
} yield result
}
Repo.scala
def create(user: User): Future[User] = {
mutateDbProvider.db.run(
userTable returning userTable.map(_.userId)
into ((user, id) => user.copy(userId = id)) +=
user.copy(createdDate = Some(Timestamp.valueOf(LocalDateTime.now())))
)
}
Since you have written your Route to unmarshall just 1 User from the Entity I don't think you need Producer.plainSink. Rather, I think Producer.send will work just as well. Also, as a side note, throwing exceptions is not "idiomatic" scala. So I changed the logic for invalid user:
val producer : KafkaProducer = new KafkaProducer(producerSettings)
val routes : Route =
post {
entity(as[User]) { user =>
val createUser = userService.create(user)
onSuccess(createUser) {
case Invalid(y: NonEmptyList[Err]) =>
complete(BadRequest -> "invalid user")
case Valid(u: User) => {
val producerRecord =
new ProducerRecord[Array[Byte], String]("topic1",s"""{"userId" : ${u.userId}, "entity" : "User"}""")
onComplete(producer send producerRecord) { _ =>
complete(ToResponseMarshallable((StatusCodes.Created, u)))
}
}
}
}
}
I started playing around scala and came to this particular boilerplate of web socket chatroom in scala.
They use MessageHub.source() and BroadcastHub.sink() as their Source and Sink for sending the messages to all connected clients.
The example is working fine for exchanging messages as it is.
private val (chatSink, chatSource) = {
// Don't log MergeHub$ProducerFailed as error if the client disconnects.
// recoverWithRetries -1 is essentially "recoverWith"
val source = MergeHub.source[WSMessage]
.log("source")
.recoverWithRetries(-1, { case _: Exception ⇒ Source.empty })
val sink = BroadcastHub.sink[WSMessage]
source.toMat(sink)(Keep.both).run()
}
private val userFlow: Flow[WSMessage, WSMessage, _] = {
Flow.fromSinkAndSource(chatSink, chatSource)
}
def chat(): WebSocket = {
WebSocket.acceptOrResult[WSMessage, WSMessage] {
case rh if sameOriginCheck(rh) =>
Future.successful(userFlow).map { flow =>
Right(flow)
}.recover {
case e: Exception =>
val msg = "Cannot create websocket"
logger.error(msg, e)
val result = InternalServerError(msg)
Left(result)
}
case rejected =>
logger.error(s"Request ${rejected} failed same origin check")
Future.successful {
Left(Forbidden("forbidden"))
}
}
}
I want to store the messages that are exchanged in the chatroom in a DB.
I tried adding map and fold functions to source and sink to get hold of the messages that are sent but I wasn't able to.
I tried adding a Flow stage between MergeHub and BroadcastHub like below
val flow = Flow[WSMessage].map(element => println(s"Message: $element"))
source.via(flow).toMat(sink)(Keep.both).run()
But it throws a compilation error that cannot reference toMat with such signature.
Can someone help or point me how can I get hold of messages that are sent and store them in DB.
Link for full template:
https://github.com/playframework/play-scala-chatroom-example
Let's look at your flow:
val flow = Flow[WSMessage].map(element => println(s"Message: $element"))
It takes elements of type WSMessage, and returns nothing (Unit). Here it is again with the correct type:
val flow: Flow[Unit] = Flow[WSMessage].map(element => println(s"Message: $element"))
This will clearly not work as the sink expects WSMessage and not Unit.
Here's how you can fix the above problem:
val flow = Flow[WSMessage].map { element =>
println(s"Message: $element")
element
}
Not that for persisting messages in the database, you will most likely want to use an async stage, roughly:
val flow = Flow[WSMessage].mapAsync(parallelism) { element =>
println(s"Message: $element")
// assuming DB.write() returns a Future[Unit]
DB.write(element).map(_ => element)
}
UPDATE
This was answered here #
https://groups.google.com/forum/#!topic/play-framework/P-tG6b_SEyg
--
First Play/Scala app here and I am struggling a bit. What I am trying to achieve is collaboration between users on specific channels (websocket urls). I am trying to follow the example from the book "Play Framework Essentials" and trying to adapt to my use case. Depending on the request from the client, I would either like to broadcast the information to all the clients talking on this channel or only send information back to the client that sent the initial request. The broadcast part is working but I am unable to figure out how to send the information back to just the client of the request themselves.
In my controller I have
def ws(channelId: String) = WebSocket.tryAccept[JsValue] { implicit request =>
getEnumerator(channelId).map { out =>
val in = Iteratee.foreach[JsValue] { jsMsg =>
val id = (jsMsg \ "id").as[String]
// This works and broadcasts that the user has connected
connect(id, request.session.get("email").get)
// Not sure how to return the result of this just to the client of the request
retrieve(id)
}
Right((in, out))
}
}
private def getEnumerator(id: String): Future[Enumerator[JsValue]] = {
(myActor ? GetEnumerator(id)).mapTo[Enumerator[JsValue]]
}
private def connect(id: String, email: String): Unit = {
(myActor ! Connect(id, email))
}
// I have a feeling this is an incorrect return type and I need to return a Future[JsValue]
//and somehow feed that to the enumerator
private def retrieve(id: String): Future[Enumerator[JsValue]] = {
(myActor ? RetrieveInfo(id)).mapTo[Enumerator[JsValue]]
}
In my Actor
class MyActor extends Actor {
var communicationChannels = Map.empty[String, CommunicationChannel]
override def receive: Receive = {
case GetEnumerator(id) => sender() ! getOrCreateCommunicationChannel(id).enumerator
case Connect(id, email) => getOrCreateCommunicationChannel(id).connect(email)
case RetrieveInfo(id) => sender() ! Json.toJson(Info(id, "details for " + id))
}
private def getOrCreateCommunicationChannel(id: String): CommunicationChannel = {
communicationChannels.getOrElse(id, {
val communicationChannel = new CommunicationChannel
communicationChannels += id -> communicationChannel
communicationChannel
})
}
}
object MyActor {
def props: Props = Props[MyActor]
class CommunicationChannel {
val (enumerator, channel) = Concurrent.broadcast[JsValue]
def connect(email: String) = {
channel.push(Json.toJson(Connected(email)))
}
}
}
Where Connect, Connected, Info etc are just case classes with Reads and Writes defined
Can someone please tell me if it is possible to push message to specific user(s) with this approach rather than broadcast it to everyone? Or if I need to implement this in another way
Also is there a way to broadcast to everyone except yourself
Any help will be greatly appreciated
thanks!!