I am trying to play around with ZIO http using their simples hello world example. I have a Java-written service which does some logic, and it expecting a handler function, so it can call it when result is ready. How do I user it together with ZIO http ?
I want something like this:
object HelloWorld extends App {
def app(service: JavaService) = Http.collect[Request] {
case Method.GET -> Root / "text" => {
service.doSomeStuffWIthCallback((s:String) => Response.text(s))
}
}
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
Server.start(8090, app(new JavaService)).exitCode
}
Basically I want to send ZIO HTTP response from the callback function, I am just not sure how to go about that. Thanks.
EDIT:
I coudn't get the right types from your code, so I decided to simplify the whole thing, and arrived to this:
val content: HttpData[Blocking, Throwable] = HttpData.fromStream {
ZStream.fromEffect(doSomeStuffWrapped)
}
def doSomeStuffWrapped = {
UIO.effectAsync[String] { cb =>
cb(
IO.succeed("TEST STRING")
)
}
}
However, the issue here is that types do not match, HttpData.fromStream requires ZStream of byte
Here is the link to my gist:
https://gist.github.com/pmkyl/a37ff8b49e013c4e2e6f8ab5ad83e258
You should wrap your Java service with callback in an effect using effectAsync:
def doSomeStuffWrapped(service: JavaService): Task[String] = {
IO.effectAsync[Throwable, String] { cb =>
service.doSomeStuffWithCallback((s: String) => {
// Success case
cb(IO.succeed(s))
// Optional error case?
// cb(IO.fail(someException))
})
}
}
def app(service: JavaService) = Http.collectM[Request] {
case Method.GET -> Root / "text" => {
doSomeStuffWrapped(service)
.fold(err => {
// Handle errors in some way
Response.text("An error occured")
}, successStr => {
Response.text(successStr)
})
}
}
You might want to see this article presenting different options for wrapping impure code in ZIO: https://medium.com/#ghostdogpr/wrapping-impure-code-with-zio-9265c219e2e
In ZIO-http v1.0.0.0-RC18 HttpData.fromStream can also take ZStream[R, E, String] as input with Http charset which defaults to CharsetUtil.UTF_8 however you can pass any charset to the HttpData.fromStream as its second argument. you can find the solution below
val stream: ZStream[Any, Nothing, String] = ZStream.fromEffect(doSomeStuffWrapped)
val content: HttpData[Any, Nothing] = HttpData.fromStream(stream)
def doSomeStuffWrapped = {
UIO.effectAsync[String] { cb =>
cb(
IO.succeed("TEST STRING"),
)
}
}
// Create HTTP route
val app = Http.collect[Request] {
case Method.GET -> !! / "health" => Response.ok
case Method.GET -> !! / "file" => Response(data = content)
}
// Run it like any simple app
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
Server.start(8090, app.silent).exitCode
However in previous versions you could have done something like given below to make it work
val stream: ZStream[Any, Nothing, Byte] =
ZStream.fromEffect(doSomeStuffWrapped).mapChunks(_.map(x => Chunk.fromArray(x.getBytes(HTTP_CHARSET))).flatten)
val content: HttpData[Any, Nothing] = HttpData.fromStream(stream)
def doSomeStuffWrapped = {
UIO.effectAsync[String] { cb =>
cb(
IO.succeed("TEST STRING"),
)
}
}
// Create HTTP route
val app = Http.collect[Request] {
case Method.GET -> !! / "health" => Response.ok
case Method.GET -> !! / "file" => Response(data = content)
}
// Run it like any simple app
override def run(args: List[String]): URIO[zio.ZEnv, ExitCode] =
Server.start(8090, app.silent).exitCode
Here is also another way of achieving the same result:
case class MyService(name: String) {
def imDone[R, E](s: String => Unit): Unit = s(name)
}
val s: MyService = MyService("test")
val app: Http[Any, Nothing, Request, UResponse] = Http.collectM[Request] { case Method.GET -> Root / "text" =>
ZIO.effectAsync[Any, Nothing, UResponse] { cb =>
s.imDone { b =>
cb(IO.succeed(Response.text(b)))
}
}
}
Related
I want to refactor this code into something more readable and better in general. I know that in Scala there are normally neat ways of doing things but for me it's getting a bit messy (BTW I'm using the Play library in the code). this is a snippet of my code:
class HomeController #Inject()
(cc: ControllerComponents)
(implicit val config: Configuration)
extends AbstractController(cc) {
def removeIdElement(uid: String) =
HAction(uid, "AuthEvent", 1, "login", parse.text).async {
implicit request: Request[String] =>
val promise = Promise[Result]()
Future {
val removeId = request.body.toLong
println(s"remove id $removeId")
promise completeWith {
idElementsDAO.remove(removeId, uid.toLong) map {
_ => Ok("")
} recover {
case t: Throwable =>
val errorMessage: String = getMessageFromThrowable(t)
println("remove id element failure " + errorMessage)
BadRequest(errorMessage)
}
}
} recover {
case t: Throwable =>
val errorMessage: String = getMessageFromThrowable(t)
println("remove id element failure " + errorMessage)
promise.success(BadRequest(errorMessage))
}
promise.future
}
}
Assuming that idElementsDAO.remove return a Future, this is probably more idiomatic:
def removeIdElement(uid: String) =
HAction(uid, "AuthEvent", 1, "login", parse.text).async {implicit request =>
val removeId = request.body.toLong
println(s"remove id $removeId")
idElementsDAO.remove(removeId, uid.toLong)
.map(_ => NoContent) // probably more correct than `Ok("")`
.recover {
case t: Throwable =>
val errorMessage: String = getMessageFromThrowable(t)
println("remove id element failure " + errorMessage)
BadRequest(errorMessage)
}
}
No need for the Promise or the call to Future {...} (Future.apply).
Keep in mind, it's probably not the best idea to directly pass the underlying error of any Throwable directly to the http client (browser?).
If you add generic error handling code to the global error handler (for unexpected errors) that logs the error and sends a generic message to the front-end, you can then write it even cleaner like this:
def removeIdElement(uid: String) =
HAction(uid, "AuthEvent", 1, "login", parse.text).async {implicit request =>
val removeId = request.body.toLong
println(s"remove id $removeId")
for {
_ <- idElementsDAO.remove(removeId, uid.toLong)
} yield NoContent
}
https://www.playframework.com/documentation/2.6.x/ScalaErrorHandling
Here is a simpler version of your code:
class HomeController #Inject()(cc: ControllerComponents)(implicit val config: Configuration)
extends AbstractController(cc) {
def removeIdElement(uid: String) = HAction(uid, "AuthEvent", 1, "login", parse.text).async {
implicit request: Request[String] =>
Future {
val removeId = request.body.toLong
println(s"Removing id $removeId")
removeId
}.flatMap(id => idElementsDAO.remove(id, uid.toLong))
.map(_ => Ok(""))
.recover {
case t: Throwable =>
val errorMessage = getMessageFromThrowable(t)
println(s"Removing id element failed: ${errorMessage}")
BadRequest(errorMessage)
}
}
}
In the above code, a Promise is not needed, and the recover combinator is not repeated.
I am struggling to use flatMap and map with Play framework + Scala. This method has a bunch of other issues, but I am trying to go through them one for each time. The first thing that I cannot figure out how to implement is how to fill a Seq inside nested flatMap and map and return a Json output. Here is my method:
def getRacks(at: String) = Action.async { implicit request: Request[AnyContent] =>
var rackSeq: Seq[Rack] = Seq.empty
var gpuSeq: Seq[Gpu] = Seq.empty
rackRepository.get(Util.toTime(at)).flatMap { resultRack: Seq[RackRow] =>
resultRack.map { r: RackRow =>
gpuRepository.getByRack(r.id).map { result: Seq[GpuRow] =>
result.map { gpuRow: GpuRow =>
gpuSeq = gpuSeq :+ Gpu(gpuRow.id, gpuRow.rackId, gpuRow.produced, Util.toDate(gpuRow.installedAt))
println(gpuRow)
}
}
val rack = Rack(r.id, r.produced, Util.toDate(r.currentHour), gpuSeq)
rackSeq = rackSeq :+ rack
}
println("rackSeq: " + rackSeq)
Future.successful(Ok(Json.toJson(rackSeq)).as(JSON))
}.recover {
case pe: ParseException => BadRequest(Json.toJson("Error on parse String to time."))
case e: Exception => BadRequest(Json.toJson("Error to get racks."))
case _ => BadRequest(Json.toJson("Unknow error to get racks."))
}
}
I was expecting that rackSeq will be filled with GpuRow. but my output is like this:
rackSeq: List(Rack(rack-1,0.2,2018-01-23T14:15:00.79Z,List()))
GpuRow(rack-1-gpu-0,rack-1,0.2,1515867048515)
How to evaluate both lists to the output?
Instead of mutating variables, stay within the context of a Future and perform transformations until you reach the desired result. Assuming the following types...
rackRepository.get(Util.toTime(at)) // Future[Seq[RackRow]]
gpuRepository.getByRack(r.id) // Future[Seq[GpuRow]]
...you could do this instead:
def gpuRowToGpu(gpuRow: GpuRow): Gpu = {
Gpu(gpuRow.id, gpuRow.rackId, gpuRow.produced, Util.toDate(gpuRow.installedAt))
}
def getRacks(at: String) = Action.async { implicit request: Request[AnyContent] =>
rackRepository.get(Util.toTime(at)).flatMap { resultRack: Seq[RackRow] =>
val seqFutRack: Seq[Future[Rack]] = resultRack.map { r: RackRow =>
gpuRepository.getByRack(r.id).map { result: Seq[GpuRow] =>
val gpus = result.map(gpuRowToGpu) // Seq[Gpu]
Rack(r.id, r.produced, Util.toDate(r.currentHour), gpus)
} // Future[Rack]
}
val futSeqRack: Future[Seq[Rack]] = Future.sequence(seqFutRack)
futSeqRack.map(racks => Ok(Json.toJson(racks)).as(JSON))
}.recover {
...
}
}
I got the error
found : scala.concurrent.Future[Option[models.ProcessTemplatesModel]]
required: Option[models.ProcessTemplatesModel]
My function is below
def createCopyOfProcessTemplate(processTemplateId: Int): Future[Option[ProcessTemplatesModel]] = {
val action = processTemplates.filter(_.id === processTemplateId).result.map(_.headOption)
val result: Future[Option[ProcessTemplatesModel]] = db.run(action)
result.map { case (result) =>
result match {
case Some(r) => {
var copy = (processTemplates returning processTemplates.map(_.id)) += ProcessTemplatesModel(None, "[Copy of] " + r.title, r.version, r.createdat, r.updatedat, r.deadline, r.status, r.comment, Some(false), r.checkedat, Some(false), r.approvedat, false, r.approveprocess, r.trainingsprocess)
val composedAction = copy.flatMap { id =>
processTemplates.filter(_.id === id).result.headOption
}
db.run(composedAction)
}
}
}
}
what is my problem in this case?
edit:
my controller function looks like this:
def createCopyOfProcessTemplate(processTemplateId: Int) = Action.async {
processTemplateDTO.createCopyOfProcessTemplate(processTemplateId).map { process =>
Ok(Json.toJson(process))
}
}
Is there my failure?
According to the your code - there are the following issues:
You use two db.run which return futures, but inner future will
not complete. For resolving it you should compose futures with
flatMap or for-comprehension.
You use only one partial-function case Some(_) => for pattern matching
and don't handle another value None.
You can use only one db.run and actions composition.
Your code can be like as:
def createCopyOfProcessTemplate(processTemplateId: Int): Future[Option[ProcessTemplatesModel]] = {
val action = processTemplates.filter(...).result.map(_.headOption)
val composedAction = action.flatMap {
case Some(r) =>
val copyAction = (processTemplates returning processTemplates...)
copyAction.flatMap { id =>
processTemplates.filter(_.id === id).result.headOption
}
case _ =>
DBIO.successful(None) // issue #2 has been resolved here
}
db.run(composedAction) // issue #3 has been resolved here
}
We get rid of issue #1 (because we use actions composition).
Let me start by saying that i am very new to akka-http, none of the books i have covered the marsheling topic well. So it is bit of a blackbox for me. I was able to obtain the following (Un)Marsheller which is capable of returning both json and protobuf based on a request header.
This part of the code works fine and i have a get route defined in akka-http and it works fine.
trait PBMarshaller {
private val protobufContentType = ContentType(MediaType.applicationBinary("octet-stream", Compressible, "proto"))
private val applicationJsonContentType = ContentTypes.`application/json`
implicit def PBFromRequestUnmarshaller[T <: GeneratedMessage with Message[T]](companion: GeneratedMessageCompanion[T]): FromEntityUnmarshaller[T] = {
Unmarshaller.withMaterializer[HttpEntity, T](_ => implicit mat => {
case entity#HttpEntity.Strict(`applicationJsonContentType`, data) =>
val charBuffer = Unmarshaller.bestUnmarshallingCharsetFor(entity)
FastFuture.successful(JsonFormat.fromJsonString(data.decodeString(charBuffer.nioCharset().name()))(companion))
case entity#HttpEntity.Strict(`protobufContentType`, data) =>
FastFuture.successful(companion.parseFrom(CodedInputStream.newInstance(data.asByteBuffer)))
case entity =>
Future.failed(UnsupportedContentTypeException(applicationJsonContentType, protobufContentType))
})
}
implicit def PBToEntityMarshaller[T <: GeneratedMessage]: ToEntityMarshaller[T] = {
def jsonMarshaller(): ToEntityMarshaller[T] = {
val contentType = applicationJsonContentType
Marshaller.withFixedContentType(contentType) { value =>
HttpEntity(contentType, JsonFormat.toJsonString(value))
}
}
def protobufMarshaller(): ToEntityMarshaller[T] = {
Marshaller.withFixedContentType(protobufContentType) { value =>
HttpEntity(protobufContentType, value.toByteArray)
}
}
Marshaller.oneOf(protobufMarshaller(), jsonMarshaller())
}
}
the issue i am facing is on the post route.
(post & entity(as[PropertyEntity])) { propertyEntity =>
complete {
saveProperty(propertyEntity)
}
}
During compilation time, i get the following error
Error:(20, 24) could not find implicit value for parameter um: akka.http.scaladsl.unmarshalling.FromRequestUnmarshaller[PropertyEntity]
(post & entity(as[PropertyEntity])) { propertyEntity =>
I am not sure exactly what i am missing. Do i need to define an implicit FromRequestUnmarshaller ? if so what should it have?
i was able to hack something together that works for the moment, but i still don't know how to create a general Unmarshaller that can decode any ScalaPB case class
implicit val um:Unmarshaller[HttpEntity, PropertyEntity] = {
Unmarshaller.byteStringUnmarshaller.mapWithCharset { (data, charset) =>
val charBuffer = Unmarshaller.bestUnmarshallingCharsetFor(data)
JsonFormat.fromJsonString(data.decodeString(charBuffer.nioCharset().name()))(PropertyEntity)
/*PropertyEntity.parseFrom(CodedInputStream.newInstance(data.asByteBuffer))*/
}
}
even this i don't know how to have both decoders enabled at the same time. so i have commented one out.
The below code does streaming back to client, in, what I gather is a more idiomatic way than using Java's IO Streams. It, however, has an issue: connection is kept open after stream is done.
def getImage() = Action { request =>
val imageUrl = "http://hereandthere.com/someimageurl.png"
Ok.stream({ content: Iteratee[Array[Byte], Unit] =>
WS.url(imageUrl).withHeaders("Accept"->"image/png").get { response => content }
return
}).withHeaders("Content-Type"->"image/png")
}
this is intended for streaming large (>1 mb) files from internal API to requester.
The question is, why does it keep the connection open? Is there something it expects from upstream server? I tested the upstream server using curl, and the connection does close - it just doesn't close when passed through this proxy.
The reason that the stream doesn't finish is because an EOF isn't sent to the iteratee that comes back from WS.get() call. Without this explicit EOF, the connection stays open - as it's in chunked mode, and potentially a long-running, comet-like connection.
Here's the fixed code:
Ok.stream({ content: Iteratee[Array[Byte], Unit] =>
WS.url(imageUrl)
.withHeaders("Accept"->"image/png")
.get { response => content }
.onRedeem { ii =>
ii.feed(Input.EOF)
}
}).withHeaders("Content-Type"->"image/png")
Here is a modified version for play 2.1.0. See https://groups.google.com/forum/#!msg/play-framework/HwoRR-nipCc/gUKs9NexCx4J
Thanks Anatoly G for sharing.
def proxy = Action {
val url = "..."
Async {
val iterateePromise = Promise[Iteratee[Array[Byte], Unit]]
val resultPromise = Promise[ChunkedResult[Array[Byte]]]
WS.url(url).get { responseHeaders =>
resultPromise.success {
new Status(responseHeaders.status).stream({ content: Iteratee[Array[Byte], Unit] =>
iterateePromise.success(content)
}).withHeaders(
"Content-Type" -> responseHeaders.headers.getOrElse("Content-Type", Seq("application/octet-stream")).head,
"Connection" -> "Close")
}
Iteratee.flatten(iterateePromise.future)
}.onComplete {
case Success(ii) => ii.feed(Input.EOF)
case Failure(t) => resultPromise.failure(t)
}
resultPromise.future
}
}
Update for play 2.2.x:
def proxy = Action.async {
val url = "http://localhost:9000"
def enumerator(chunks: Iteratee[Array[Byte], Unit] => _) = {
new Enumerator[Array[Byte]] {
def apply[C](i: Iteratee[Array[Byte], C]): Future[Iteratee[Array[Byte], C]] = {
val doneIteratee = Promise[Iteratee[Array[Byte], C]]()
chunks(i.map {
done =>
doneIteratee.success(Done[Array[Byte], C](done)).asInstanceOf[Unit]
})
doneIteratee.future
}
}
}
val iterateePromise = Promise[Iteratee[Array[Byte], Unit]]()
val resultPromise = Promise[SimpleResult]()
WS.url(url).get {
responseHeaders =>
resultPromise.success(new Status(responseHeaders.status).chunked(
enumerator({
content: Iteratee[Array[Byte], Unit] => iterateePromise.success(content)
}
)).withHeaders(
"Content-Type" -> responseHeaders.headers.getOrElse("Content-Type", Seq("application/octet-stream")).head,
"Connection" -> "Close"))
Iteratee.flatten(iterateePromise.future)
}.onComplete {
case Success(ii) => ii.feed(Input.EOF)
case Failure(t) => throw t
}
resultPromise.future
}
if anyone has a better solution, it interests me greatly!