Streaming ByteArrayOutputStream in Action Result in scala Play Framework - scala

I am trying to convert an svg image to a jpeg image in scala play framework.
I used batik and it worked ok.
Now I like to stream the output in the action result, instead of converting ByteArrayOutputStream to a
ByteArray which loads the entire output in memory.
How can I do that?
Here is the project code, which works without streaming the output:
build.sbt
name := "svg2png"
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayScala)
resolvers += Resolver.sonatypeRepo("snapshots")
scalaVersion := "2.12.3"
libraryDependencies ++= Seq(
jdbc,
guice,
"org.apache.xmlgraphics" % "batik-transcoder" % "1.11",
"org.apache.xmlgraphics" % "batik-codec" % "1.11"
)
/project/plugins.sbt
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.6.6")
/project/build.properties
sbt.version=1.0.2
/conf/routes
GET / controllers.Application.index
GET /image controllers.Application.getImage
/conf/application.conf
play.filters.enabled += "play.filters.cors.CORSFilter"
play.filters {
hosts {
allowed = ["."]
}
headers {
contentSecurityPolicy = null
}
}
play.i18n {
langs = [ "en" ]
}
contexts {
imageService {
fork-join-executor {
parallelism-factor = 4.0
parallelism-max = 8
}
}
}
/app/controllers/Application.scala
package controllers
import play.api.mvc._
import javax.inject._
import scala.concurrent.ExecutionContext
#Singleton
class Application #Inject()(imageService: services.ImageService,
cc: ControllerComponents)(implicit exec: ExecutionContext) extends AbstractController(cc) {
def index = Action {
Ok("test app").as(HTML)
}
def getImage : Action[AnyContent] = Action.async {
imageService.getImage.map{res => Ok(res).as("image/jpeg") }
}
}
/app/services/ImageService.scala
package services
import java.io.{ByteArrayOutputStream, StringReader}
import com.google.inject.{Inject, Singleton}
import scala.concurrent.{ExecutionContext, Future}
import akka.actor.ActorSystem
import org.apache.batik.transcoder.image.JPEGTranscoder
import org.apache.batik.transcoder.TranscoderInput
import org.apache.batik.transcoder.TranscoderOutput
#Singleton
class ImageService #Inject()(actorSystem: ActorSystem) {
implicit val AnalyticsServiceExecutionContext: ExecutionContext = actorSystem.dispatchers.lookup("contexts.imageService")
def getImage: Future[Array[Byte]] = {
Future {
val t: JPEGTranscoder = new JPEGTranscoder
t.addTranscodingHint(JPEGTranscoder.KEY_QUALITY, new java.lang.Float(0.8))
val imageSVGString: String =
"""<svg width="1000" height="1000" viewBox="0 0 1000 1000" version="1.1"
| xmlns="http://www.w3.org/2000/svg"
| xmlns:xlink="http://www.w3.org/1999/xlink">
| <circle cx="500" cy="500" r="300" fill="lightblue" />
|</svg>
""".stripMargin
val input: TranscoderInput = new TranscoderInput(new StringReader(imageSVGString))
val outputStream = new ByteArrayOutputStream
val output: TranscoderOutput = new TranscoderOutput(outputStream)
t.transcode(input, output)
outputStream.toByteArray
}
}
}

This works for me:
svg match {
case None => NotFound
case Some(svg) =>
val svgImage = new TranscoderInput(new StringReader(svg))
val pngOstream = new ByteArrayOutputStream
val outputPngImage = new TranscoderOutput(pngOstream)
val converter = fileExtension match {
case "png" => new PNGTranscoder()
case _ => new JPEGTranscoder()
}
if(converter.isInstanceOf[JPEGTranscoder]){
converter.addTranscodingHint(JPEGTranscoder.KEY_QUALITY, (0.8).toFloat)
}
converter.transcode(svgImage, outputPngImage)
Ok(pngOstream.toByteArray).as("image/" + fileExtension)
}

Related

ZIO: How to return JSON ? [instead of using case class in ZIO-Http use schema to map?]

I tried directly getting body of JSON in code which I then want to convert to Avro to write to a kafka topic.
Here is my code with case class:
import zhttp.http._
import zio._
import zhttp.http.{Http, Method, Request, Response, Status}
import zhttp.service.Server
import zio.json._
import zio.kafka._
import zio.kafka.serde.Serde
import zio.schema._
case class Experiments(experimentId: String,
variantId: String,
accountId: String,
deviceId: String,
date: Int)
//case class RootInterface (events: Seq[Experiments])
object Experiments {
implicit val encoder: JsonEncoder[Experiments] = DeriveJsonEncoder.gen[Experiments]
implicit val decoder: JsonDecoder[Experiments] = DeriveJsonDecoder.gen[Experiments]
implicit val codec: JsonCodec[Experiments] = DeriveJsonCodec.gen[Experiments]
implicit val schema: Schema[Experiments] = DeriveSchema.gen
}
object HttpService {
def apply(): Http[ExpEnvironment, Throwable, Request, Response] =
Http.collectZIO[Request] {
case req#(Method.POST -> !! / "zioCollector") =>
val c = req.body.asString.map(_.fromJson[Experiments])
for {
u <- req.body.asString.map(_.fromJson[Experiments])
r <- u match {
case Left(e) =>
ZIO.debug(s"Failed to parse the input: $e").as(
Response.text(e).setStatus(Status.BadRequest)
)
case Right(u) =>
println(s"$u + =====")
ExpEnvironment.register(u)
.map(id => Response.text(id))
}
}
yield r
}
}
// val experimentsSerde: Serde[Any, Experiments] = Serde.string.inmapM { string =>
// //desericalization
// ZIO.fromEither(string.fromJson[Experiments].left.map(errorMessage => new RuntimeException(errorMessage)))
// } { theMatch =>
// ZIO.effect(theMatch.toJson)
//
// }
object ZioCollectorMain extends ZIOAppDefault {
def run: ZIO[Environment with ZIOAppArgs with Scope, Any, Any] = {
Server.start(
port = 9001,
http = HttpService()).provide(ZLayerExp.layer)
}
}
I'm looking into Zio-Json but no success yet, any help is appreciated !
We could also schema something to get the avro generic record
here's my json :
{
"experimentId": "abc",
"variantId": "123",
"accountId": "123",
"deviceId": "123",
"date": 1664544365
}
This function works for me in Scala 3 (sorry, I didn't include all the code but it should be enough):
import zio.*
import zio.Console.printLine
import zhttp.http.*
import zhttp.service.Server
import zio.json.*
...
case class Experiments(experimentId: String,
variantId: String,
accountId: String,
deviceId: String,
date: Int)
//case class RootInterface (events: Seq[Experiments])
object Experiments:
implicit val encoder: JsonEncoder[Experiments] = DeriveJsonEncoder.gen[Experiments]
implicit val decoder: JsonDecoder[Experiments] = DeriveJsonDecoder.gen[Experiments]
implicit val codec: JsonCodec[Experiments] = DeriveJsonCodec.gen[Experiments]
val postingExperiment: Http[Any, Throwable, Request, Response] =
Http.collectZIO[Request] {
case req#(Method.POST -> !! / "zioCollector") =>
//val c = req.body.asString.map(_.fromJson[Experiments])
val experimentsZIO = req.body.asString.map(_.fromJson[Experiments])
for {
experimentsOrError <- experimentsZIO
response <- experimentsOrError match {
case Left(e) => ZIO.debug(s"Failed to parse the input: $e").as(
Response.text(e).setStatus(Status.BadRequest)
)
case Right(experiments) => ZIO.succeed(Response.json(experiments.toJson))
}
} yield response
}
I modified your code slightly (you didn't post your ExpEnvironment class), and it returns back the object posted to the url.
and the test code is:
import sttp.client3.{SimpleHttpClient, UriContext, basicRequest}
object TestExperiments:
def main(args: Array[String]): Unit =
val client = SimpleHttpClient()
//post request
val request = basicRequest
.post(uri"http://localhost:9009/zioCollector")
.body("{ \"experimentId\": \"abc\", \"variantId\": \"123\", \"accountId\": \"123\", \"deviceId\": \"123\", \"date\": 1664544365 }")
val response = client.send(request)
println(response.body)
val invalidJsonRequest = basicRequest
.post(uri"http://localhost:9009/zioCollector")
.body("{ \"experimentId\": \"abc\", \"variantId\": \"123\", \"accountId\": \"123\", \"deviceId\": \"123\", \"date\": 1664544365 ") // missing the closing bracket
val invalidJsonResponse = client.send(invalidJsonRequest)
println(invalidJsonResponse.body)
You have to add: "com.softwaremill.sttp.client3" %% "core" % "3.8.3" to your sbt file.
build.sbt:
ThisBuild / scalaVersion := "3.2.0"
ThisBuild / version := "0.1.0-SNAPSHOT"
ThisBuild / organization := "TestSpeed"
ThisBuild / organizationName := "example"
lazy val root = (project in file("."))
.settings(
name := "TestZio",
libraryDependencies ++= Seq(
"dev.zio" %% "zio" % "2.0.2",
"dev.zio" %% "zio-json" % "0.3.0-RC11",
"io.d11" %% "zhttp" % "2.0.0-RC11",
"dev.zio" %% "zio-test" % "2.0.2" % Test,
"com.softwaremill.sttp.client3" %% "core" % "3.8.3" % Test
),
testFrameworks += new TestFramework("zio.test.sbt.ZTestFramework")
)
I didn't include anything related to avro because I am not familiar with it.

How to configure Akka to expose metrics to Prometheus using Kamon?

I am trying to configure my project which is based on Akka 2.6.10 to expose metric values to Prometheus. I saw this question which uses Kamon but I couldn't figure out what I am missing on my configuration. My build.sbt file has the following configuration:
name := """explore-akka"""
version := "1.1"
scalaVersion := "2.12.7"
val akkaVersion = "2.6.10"
lazy val kamonVersion = "2.1.9"
libraryDependencies ++= Seq(
// Akka basics
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion,
// Metrics: Kamon + Prometheus
"io.kamon" %% "kamon-core" % kamonVersion,
"io.kamon" %% "kamon-akka" % kamonVersion,
"io.kamon" %% "kamon-prometheus" % kamonVersion
)
and the plugins.sbt:
resolvers += Resolver.bintrayRepo("kamon-io", "sbt-plugins")
addSbtPlugin("io.kamon" % "sbt-aspectj-runner" % "1.1.1")
I added on the application.conf:
kamon.instrumentation.akka.filters {
actors.track {
includes = [ "CounterSystem/user/Counter**" ]
}
}
then I start a MainClass which call the counterActor:
object MainClass extends App {
Kamon.registerModule("akka-test", new PrometheusReporter())
Kamon.init()
CounterActor.run()
}
import akka.actor.{Actor, ActorSystem, Props}
import kamon.Kamon
object CounterActor extends App {
run()
def run() = {
import Counter._
val actorSystem = ActorSystem("CounterSystem")
val countActor = actorSystem.actorOf(Props[Counter], "Counter")
(1 to 100).foreach { v =>
Thread.sleep(1000)
countActor ! Increment
}
(1 to 50).foreach { v =>
Thread.sleep(1000)
countActor ! Decrement
}
countActor ! Print
}
class Counter extends Actor {
import Counter._
val counter = Kamon.counter("my-counter")
var count = 0
override def receive: Receive = {
case Increment =>
count += 1
println(s"incrementing... $count")
counter.withoutTags().increment()
case Decrement =>
count -= 1
println(s"decrementing... $count")
counter.withoutTags().increment()
case Print =>
sender() ! count
println(s"[counter] current count is: $count")
}
}
object Counter {
case object Increment
case object Decrement
case object Print
}
}
I think that after this I could launch the application using sbt run and listen to the metrics on the Prometheus console (http://127.0.0.1:9090/graph), but I do not see any metrics related to my actor. My guess is that I have to configure the scrape_config at the prometheus file /etc/prometheus/prometheus.yml. Am I right? How should I configure it?
I had to configure Prometheus to scrape the Kamon web service through the config file
cat /etc/prometheus/prometheus.yml
global:
scrape_interval: 15s
scrape_configs:
- job_name: "kamon"
scrape_interval: "5s"
static_configs:
- targets: ['localhost:9095']
metrics_path: /
add these 2 Kamon libraries at build.sbt:
"io.kamon" %% "kamon-bundle" % "2.1.9",
"io.kamon" %% "kamon-prometheus" % "2.1.9",
add this configuration at application.conf:
kamon.instrumentation.akka.filters {
actors.track {
includes = [ "AkkaQuickStart/user/*" ]
# excludes = [ "AkkaQuickStart/system/**" ]
}
}
Start Kamon and call the counter:
Kamon.init()
val counterSendMsg = Kamon.counter("counter-send-msg")
counterSendMsg.withTag("whom", message.whom).increment()
Here is the full demo application from Akka quick start with Kamon configured to count messages:
import akka.actor.typed.scaladsl.Behaviors
import akka.actor.typed.{ActorRef, ActorSystem, Behavior}
import kamon.Kamon
import scala.util.Random
object Greeter {
val counterSendMsg = Kamon.counter("counter-send-msg")
def apply(): Behavior[Greet] = Behaviors.receive { (context, message) =>
context.log.info("Hello {}!", message.whom)
//#greeter-send-messages
message.replyTo ! Greeted(message.whom, context.self)
counterSendMsg.withTag("whom", message.whom).increment()
//#greeter-send-messages
Behaviors.same
}
final case class Greet(whom: String, replyTo: ActorRef[Greeted])
final case class Greeted(whom: String, from: ActorRef[Greet])
}
object GreeterBot {
def apply(max: Int): Behavior[Greeter.Greeted] = {
bot(0, max)
}
private def bot(greetingCounter: Int, max: Int): Behavior[Greeter.Greeted] =
Behaviors.receive { (context, message) =>
val n = greetingCounter + 1
context.log.info("Greeting {} for {}", n, message.whom)
if (n == max) {
Behaviors.stopped
} else {
message.from ! Greeter.Greet(message.whom, context.self)
bot(n, max)
}
}
}
object GreeterMain {
def apply(): Behavior[SayHello] =
Behaviors.setup { context =>
//#create-actors
val greeter = context.spawn(Greeter(), "greeter")
//#create-actors
Behaviors.receiveMessage { message =>
//#create-actors
val replyTo = context.spawn(GreeterBot(max = 3), message.name)
//#create-actors
greeter ! Greeter.Greet(message.name, replyTo)
Behaviors.same
}
}
final case class SayHello(name: String)
}
object AkkaQuickstart {
def main(args: Array[String]): Unit = {
run()
}
def run() = {
Kamon.init()
import GreeterMain._
val greeterMain: ActorSystem[GreeterMain.SayHello] = ActorSystem(GreeterMain(), "AkkaQuickStart")
val allPerson = List("Charles", "Bob", "Felipe", "Simone", "Fabio")
def randomPerson = allPerson(Random.nextInt(allPerson.length))
while (true) {
greeterMain ! SayHello(randomPerson)
Thread.sleep(1000)
}
}
}
and my Prometheus web console:

Simple ScalaFx TableView example not compiling

I took a TableView code from a simple ScalaFx example (simplified from ScalaFx Custom cells):
import scalafx.application.JFXApp
import scalafx.beans.property.StringProperty
import scalafx.collections.ObservableBuffer
import scalafx.scene.Scene
import scalafx.scene.control.{TableColumn, TableView}
object MyTableApp extends JFXApp {
class Person(nameStr : String) {
val name = new StringProperty(this, "firstName", nameStr)
}
val characters = ObservableBuffer[Person](
new Person("Peggy Sue"),
new Person("Rocky Raccoon"),
new Person("Bungalow Bill")
)
stage = new JFXApp.PrimaryStage {
title = "Simple TableView"
scene = new Scene {
content = new TableView[Person](characters) {
columns ++= List(
new TableColumn[Person, String] {
text = "First Name"
cellValueFactory = { _.value.name }
prefWidth = 100
}
)
}
}
}
}
When compiling it, I get a confusing error:
Error:(24, 11) type mismatch;
found : scalafx.scene.control.TableColumn[MyTableApp.Person,String]
required: javafx.scene.control.TableColumn[MyTableApp.Person, ?]
new TableColumn[Person, String] {
What am I doing wrong?
My build.sbt contains:
scalaVersion := "2.11.8"
libraryDependencies += "org.scalafx" %% "scalafx" % "8.0.60-R9"
I did not copy the example source carefully, and I was missing an import:
import scalafx.scene.control.TableColumn._

embedmongo with reactivemongo process does not exit

I am trying to do some tests using ScalaTest + embedmongo + reactivemongo but I fail. My first problem is that after a test mongod process does not shut down, I have this message in console:
INFO: stopOrDestroyProcess: process has not exited
and tests are paused till I kill the process manually. That happens even if body of my test is empty. I am running windows 8.1.
The other issue is, that when I try to connect to db inside test using reactive mongo and insert anything to db I get this exception:
reactivemongo.core.errors.ConnectionNotInitialized: MongoError['Connection is missing metadata (like protocol version, etc.) The connection pool is probably being initialized.']
I have literally no idea how to set it up. Here is my test code:
package model
import com.github.simplyscala.MongoEmbedDatabase
import org.scalatest.{OptionValues, Matchers, BeforeAndAfter, FlatSpec}
import reactivemongo.api.MongoDriver
import reactivemongo.api.collections.bson.BSONCollection
import scala.concurrent.duration._
import reactivemongo.bson.BSONDocument
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await
class MongoBookingRepositoryTest extends FlatSpec
with Matchers
with OptionValues
with MongoEmbedDatabase
with BeforeAndAfter {
"A MongoBookingRepository" should "..." in withEmbedMongoFixture(port = 12345) { mongoProps =>
val driver = new MongoDriver
val connection = driver.connection("localhost:12345" :: Nil)
val db = connection("testDatabase")
val collection = db.collection[BSONCollection]("bookings")
val future = collection.insert(BSONDocument("a" -> 5))
println(Await.result(future, 3.seconds))
driver.close()
connection.close()
}
}
In play2.4 I tried as below and it worked.
dependencies:
"org.reactivemongo" %% "play2-reactivemongo" % "0.11.7.play24",
"org.reactivemongo" %% "reactivemongo-extensions-json" % "0.11.7.play24",
// test
"org.scalatest" %% "scalatest" % "2.2.4" % Test,
"de.flapdoodle.embed" % "de.flapdoodle.embed.mongo" % "1.50.0" % Test,
"org.mockito" % "mockito-core" % "1.10.19" % Test
TestMongoSetup:
import de.flapdoodle.embed.mongo.config.{Net, MongodConfigBuilder}
import de.flapdoodle.embed.mongo.distribution.Version
import de.flapdoodle.embed.mongo.{MongodStarter, MongodProcess, MongodExecutable}
import de.flapdoodle.embed.process.runtime.Network
import org.mockito.Mockito._
import play.modules.reactivemongo.ReactiveMongoApi
import reactivemongo.api.MongoConnection.ParsedURI
import reactivemongo.api.{MongoConnectionOptions, MongoDriver, MongoConnection}
import scala.concurrent.ExecutionContext
trait TestMongoSetup {
private var port : Int = _
private var mongodExe: MongodExecutable = _
private var mongodProcess: MongodProcess = _
def start(intiAtPort: Int): Unit = {
port=intiAtPort
mongodExe = MongodStarter.getDefaultInstance.prepare(
new MongodConfigBuilder()
.version(Version.Main.V3_0)
.net(new Net(port, Network.localhostIsIPv6()))
.build()
)
mongodProcess = mongodExe.start()
}
def stop(): Unit = {
mongodProcess.stop()
mongodExe.stop()
}
def createConnection(): MongoConnection = {
val driver = new MongoDriver
driver.connection(ParsedURI(
hosts = List(("localhost", port)),
options = MongoConnectionOptions(),
ignoredOptions = List.empty[String],
db = None,
authenticate = None
))
}
def createMockedReactiveMongoApi(dbName: String)(implicit ctx: ExecutionContext): ReactiveMongoApi = {
val connection = createConnection()
val db = connection(dbName)
val api = mock(classOf[ReactiveMongoApi])
doReturn(db).when(api).db
doReturn(connection).when(api).connection
api
}
}
TestClass:
import db.TestMongoSetup
import models.dao.UserDAO
import org.scalatest._
import play.modules.reactivemongo.ReactiveMongoApi
import scala.concurrent.Await
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.duration.Duration
class UserServiceTest extends FlatSpec with ShouldMatchers with GivenWhenThen with BeforeAndAfterAll with TestMongoSetup { private var mockedAPI: ReactiveMongoApi = _
var dao: UserDAO = _
val port : Int = 12345
override def beforeAll(): Unit = {
start(port)
mockedAPI = createMockedReactiveMongoApi("testDB")
dao = new UserDAO(mockedAPI)
}
override def afterAll(): Unit = {
mockedAPI.connection.actorSystem.shutdown()
mockedAPI.connection.actorSystem.awaitTermination()
stop()
}
"Check" should "check User object into DB" in {
Given("a user info")
val email = "xyx#abc.com"
val pwd= "abcddd"
When("it fetch from DB")
val fromDBUser = Await.result(dao.fetch(email,pwd), Duration.Inf)
Then("it should be fetched")
fromDBUser.get.email should equal(email)
}
}
withEmbedMongoFixture doesn't work very well.
Prefer use the "classic" way : https://github.com/SimplyScala/scalatest-embedmongo#basic-usage-mutable-way
For Mongo 3 use 2.3_SNAPSHOT version of the library.

Scala Play Framework Slick session doesn't work

I'm trying a simple test application with Slick and the Play2 Framework, but the compiler keeps complaining the implicit session cannot be inferred.
Here is my Build.scala:
import sbt._
import Keys._
import play.Project._
object ApplicationBuild extends Build {
val appName = "dummy"
val appVersion = "1.0"
val appDependencies = Seq(
jdbc,
"mysql" % "mysql-connector-java" % "5.1.26",
"com.typesafe.slick" %% "slick" % "1.0.1"
)
val main = play.Project(appName, appVersion, appDependencies).settings(
// Add your own project settings here
)
}
And this is my Global singleton that holds my database connections:
package models
import play.api.Play.current
import play.api.db.DB
import slick.session.Session
import slick.driver.MySQLDriver.simple._
import scala.slick.session.Database.threadLocalSession
object Global {
lazy val database = Database.forDataSource(DB.getDataSource())
lazy val session = database.createSession()
}
And my controller:
package controllers
import scala.language.implicitConversions
import play.api._
import play.api.mvc._
import models.Global.session
import slick.driver.MySQLDriver.simple._
object Application extends Controller {
def index = Action {
/*slick.driver.MySQLDriver.simple.*/Query(Foo).foreach( _ => () ) // Do nothing for now
Ok(views.html.index("hola"))
}
object Foo extends Table[(Long, String, String)]("Foo") {
def * = column[Long]("id") ~ column[String]("bar1") ~ column[String]("bar2")
}
}
As you can see my Global.session val is imported, but it keeps saying no implicit session was found.
To make queries you need two things: connection to the database and a session, so your problem is how you define and use them.
With Database.threadLocalSession in scope, you can make your queries like this :
Database.forURL("jdbc:h2:mem:play", driver = "org.h2.Driver") withSession {
//create table
Foo.ddl.create
//insert data
Foo.insert((1.toLong,"foo","bar"))
//get data
val data : (Long,String,String) = (for{f<-Foo}yield(f)).first
}
or you can do it like this:
val database = Database.forDataSource(DB.getDataSource())
database.withSession{ implicit session : Session =>
Foo.ddl.create
Foo.insert((1.toLong,"foo","bar"))
val data : (Long,String,String) = (for{f<-Foo}yield(f)).first
}
I have create a test and it works fine, you can play with it:
"Foo should be creatable " in {
running(FakeApplication(additionalConfiguration = inMemoryDatabase())) {
val database = Database.forDataSource(DB.getDataSource())
database.withSession{ implicit session : Session =>
Foo.ddl.create
Foo.insert((1.toLong,"foo","bar"))
val data : (Long,String,String) = (for{f<-Foo}yield(f)).first
data._1 must equalTo(1)
}
}
}
Also you may look at here