spray Collection ToResponseMarshallable - scala

I am trying to return a List from my complete directive in spray-routing.
complete {
List("hello")
}
However, I am getting an error -
Expression of type List[String] doesn't conform to expected type ToResponseMarshallable
I am getting the same error with Seq. I see that marshallers for List and Seq are not provided by default in spray-httpx documentation. However, spray-json provides marshalling support in its DefaultJsonProtocol. I have imported spray.httpx.SprayJsonSupport._ and spray.json.DefaultJsonProtocol._ in my code, but that too hasn't helped.
Any idea how can I marshal a List/Seq using spray-json? Or will I have to write my own Marshaller?
(My scala version is 2.11.4)

Using spray 1.3.2 and spray-json 1.3.2 it should be possible.
Make sure you import (although you say you do, but double check).
import spray.httpx.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
Consider following example:
import akka.actor.{ActorSystem, Props, Actor}
import akka.io.IO
import spray.can.Http
import spray.routing.HttpService
import akka.pattern.ask
import akka.util.Timeout
import scala.concurrent.duration._
import spray.httpx.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
object Boot extends App {
implicit val system = ActorSystem("so")
val testActor = system.actorOf(Props[TestActor])
implicit val timeout = Timeout(5.seconds)
IO(Http) ? Http.Bind(testActor, interface = "0.0.0.0", port = 5000)
}
class TestActor extends Actor with HttpService {
def receive = runRoute(route)
def actorRefFactory = context
val route = get{
path("ping") {
complete(List("OK"))
}
}
}
Requesting /ping returns ["OK"] which looks okay.
Just for reference build.sbt bellow.
build.sbt
val akkaVersion = "2.3.5"
val sprayVersion = "1.3.2"
resolvers += "spray" at "http://repo.spray.io/"
scalaVersion := "2.11.5"
scalacOptions := Seq("-feature", "-unchecked", "-deprecation", "-encoding", "utf8")
libraryDependencies ++= Seq(
"com.typesafe.akka" %% "akka-actor" % akkaVersion,
"io.spray" %% "spray-can" % sprayVersion,
"io.spray" %% "spray-routing" % sprayVersion,
"io.spray" %% "spray-client" % sprayVersion,
"io.spray" %% "spray-json" % "1.3.1"
)

API for marshalling seems to have changed since akka 2.3.5. For akka-2.4.11.2, SprayJsonSupport needs import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
1) Following is working sbt,
name := "some-project"
version := "1.0"
scalaVersion := "2.11.5"
scalacOptions := Seq("-unchecked", "-deprecation", "-encoding", "utf8")
val akkaVersion = "2.4.5"
libraryDependencies ++= {
Seq(
"com.typesafe.akka" %% "akka-http-experimental" % akkaVersion,
"com.typesafe.akka" % "akka-http-spray-json-experimental_2.11" % akkaVersion,
"com.typesafe.akka" %% "akka-slf4j" % akkaVersion,
"org.mongodb" % "mongo-java-driver" % "3.4.1",
"org.apache.kafka" %% "kafka" % "0.10.1.1",
"org.apache.kafka" % "kafka-clients" % "0.10.1.1",
//test
"org.scalatest" %% "scalatest" % "2.2.5" % "test",
"com.typesafe.akka" %% "akka-http-testkit" % "10.0.9" % "test",
"de.flapdoodle.embed" % "de.flapdoodle.embed.mongo" % "2.0.0" % "test"
)
}
parallelExecution in Test := false
2) And imports are,
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._
object GetMusicResources {
val music_get_api =
path("music") {
get {
complete {
List("my data1")
}
}
}
}
3) Tests
Get("/music") ~> GetMusicResources.music_get_api ~> check {
val response1: Future[ByteString] = response.entity.toStrict(300.millis).map {_.data }
response1.map(_.utf8String).map { responseString =>
responseString shouldBe """["my data1"]"""
}
}

In my case, I was missing the second import in :
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport._
import spray.json.DefaultJsonProtocol._ // <= this one
In case anyone stumble into the same issue

Related

Simple hello world api using finagle - Error: object twitter is not a member of package com import com.twitter.finagle.Http

Im really new to this microservice world and was trying to learn from scratch. But for some reason not able to call a simple dependency.
Getting the error
object twitter is not a member of package com import
com.twitter.finagle._
SBT file, Ive tried three ways
Trial 1
name := "microservicestest"
version := "0.1"
scalaVersion := "2.12.15"
libraryDependencies ++= Seq(
"com.github.finagle" %% "finch-core" % "0.31.0",
"com.github.finagle" %% "finch-circe" % "0.31.0",
"io.circe" %% "circe-generic" % "0.9.0"
)
Trial 2 (puttting just the dependency line in here)
libraryDependencies += "com.twitter" %% "finagle-http" % "21.8.0"
Trial 3
libraryDependencies ++= Seq(
"com.github.finagle" %% "finch-core" % "0.15.1"
)
Scala file
import io.finch._
import com.twitter.finagle.Http
import com.twitter.util.Await
object HelloWorld extends App {
val api: Endpoint[String] = get("hello") { Ok("Hello, World!") }
Await.ready(Http.server.serve(":8080", api.toServiceAs[Text.Plain]))
}
SBT file
libraryDependencies += "com.twitter" %% "finagle-http" % "21.8.0"
Working simple code
import com.twitter.finagle.{Http, Service}
import com.twitter.finagle.http
import com.twitter.util.{Await, Future}
object HelloWorld extends App {
val service = new Service[http.Request, http.Response] {
def apply(req: http.Request): Future[http.Response] =
Future.value(http.Response(req.version, http.Status.Ok))
}
val server = Http.serve(":8080", service)
Await.ready(server)
}

Scala Akka Streams project fails to compile, "value ~> is not a member of akka.streams.Outlet[In]"

I am using the Akka Streams API in a Scala project, working in Intellij IDEA with the SBT plugin. I have a worker pool Flow as described here: https://doc.akka.io/docs/akka/current/scala/stream/stream-cookbook.html. Here is my code:
package streams
import akka.NotUsed
import akka.stream.scaladsl.{Balance, Flow, GraphDSL, Merge}
import akka.stream.{FlowShape, Graph}
object WorkerPoolFlow {
def apply[In, Out](
worker: Flow[In, Out, Any],
workerCount: Int):
Graph[FlowShape[In, Out], NotUsed] = {
GraphDSL.create() { implicit b =>
val balance = b.add(Balance[In](workerCount, waitForAllDownstreams = true))
val merge = b.add(Merge[Out](workerCount))
for (i <- 0 until workerCount)
balance.out(i) ~> worker.async ~> merge.in(i)
FlowShape(
balance.in,
merge.out)
}
}
}
For some reason the project is now failing to compile, giving this error: value ~> is not a member of akka.stream.Outlet[In].
It compiled fine until today. The only change I am aware of making is installing a Scala linter plugin scalafmt, and importing a few new libraries in build.sbt. Here is my build.sbt:
name := "myProject"
version := "0.1"
scalaVersion := "2.11.11"
unmanagedJars in Compile += file("localDep1.jar")
unmanagedJars in Compile += file("localDep2.jar")
libraryDependencies += "io.spray" %% "spray-json" % "1.3.3"
libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.5.6"
libraryDependencies += "com.typesafe.akka" %% "akka-stream" % "2.5.6"
libraryDependencies += "com.typesafe.akka" %% "akka-testkit" % "2.5.6" % Test
libraryDependencies += "com.typesafe.akka" %% "akka-stream-testkit" % "2.5.6" % Test
libraryDependencies += "com.47deg" %% "fetch" % "0.6.0"
libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.2.3"
I have tried reloading SBT, building from SBT outside of IDEA, removing and re-adding dependencies, and cleaning the project, with no luck.
Import GraphDSL.Implicits._:
object WorkerPoolFlow {
def apply[In, Out](
worker: Flow[In, Out, Any],
workerCount: Int):
Graph[FlowShape[In, Out], NotUsed] = {
import GraphDSL.Implicits._
GraphDSL.create() { implicit b =>
...
}
}
}

Symbol 'type <none>.scalacheck.Shrink' is missing from the classpath

I have the following ScalaCheck unit test using Mockito:
import org.scalatest.mockito.MockitoSugar
import org.mockito.Mockito.when
import org.scalatest.prop.PropertyChecks
import org.mockito.Mockito.verify
class PlayerTest extends org.scalatest.FunSuite with MockitoSugar with PropertyChecks {
test("doesn't accept anything but M") {
val mockIOHandler = mock[IOHandler]
val name = "me"
val player = new Player(name)
when(mockIOHandler.nextLine).thenReturn("m")
val apiUser = new Player("player1")
apiUser.chooseHand(mockIOHandler)
verify(mockIOHandler).write("some value")
}
}
In my build.sbt I have the following dependencies:
scalaVersion := "2.12.2"
libraryDependencies += "org.scalactic" %% "scalactic" % "3.0.1"
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.1" % "test"
// https://mvnrepository.com/artifact/org.mockito/mockito-core
libraryDependencies += "org.mockito" % "mockito-core" % "1.8.5"
For it, I am getting this error:
Error:(12, 41) Symbol 'type <none>.scalacheck.Shrink' is missing from the classpath.
This symbol is required by 'value org.scalatest.prop.GeneratorDrivenPropertyChecks.shrA'.
Make sure that type Shrink is in your classpath and check for conflicting dependencies with `-Ylog-classpath`.
A full rebuild may help if 'GeneratorDrivenPropertyChecks.class' was compiled against an incompatible version of <none>.scalacheck.
test("doesn't accept anything but M") {
Any idea what could be wrong?
Adding scalacheck did the trick for me
lazy val scalaTest = "org.scalatest" %% "scalatest" % "3.0.+"
lazy val scalaCheck = "org.scalacheck" %% "scalacheck" % "1.13.+"

Failure retries with Finagle

I'm struggling with Finagle client retries.
For some reason, client doesn't retry failed requests in test, even though I use a custom classifier, which should mark any response code other than 200 as RetryableFailure. I have tried bulding client both with ClientBuilder and with Http.client.
I narrowed down my code and test to only the necessary bit.
Please take a look:
MyApi.scala:
import io.circe._, io.circe.generic.auto._, io.circe.jawn._, io.circe.syntax._
import com.twitter.finagle.http._
import com.twitter.finagle.Service
import com.twitter.finagle.Codec
import com.twitter.finagle.builder._
import scala.concurrent.ExecutionContext.Implicits.global
import com.twitter.conversions.time._
import com.twitter.finagle.http.service.HttpResponseClassifier
import com.twitter.finagle.service._
import com.twitter.util.Return
class MyApi(val host: String, val protocol: String = "https") {
// def request: Service[Request, Response] = {
// val clientBuilder = ClientBuilder()
// .codec(Http())
// .responseClassifier(classifier)
// .noFailureAccrual
// .requestTimeout(30.seconds)
// .hostConnectionLimit(5)
// .tcpConnectTimeout(5.seconds)
// .retryBudget(budget)
// .retries(5)
// protocol match {
// case "https" => clientBuilder.hosts(s"$host:443").tls(host).build()
// case _ => clientBuilder.hosts(host).build()
// }
// }
val budget: RetryBudget = RetryBudget(
ttl = 10.seconds,
minRetriesPerSec = 5,
percentCanRetry = 0.1
)
val classifier: ResponseClassifier = {
case ReqRep(_, Return(r: Response)) if r.statusCode == 200 =>
ResponseClass.Success
case _ => ResponseClass.RetryableFailure
}
def request: Service[Request, Response] = com.twitter.finagle.Http.client
.withResponseClassifier(classifier)
.withRetryBudget(budget)
.withRetryBackoff(Backoff.exponentialJittered(2.seconds, 32.seconds))
.newService(host)
def requestUsers = request(users(1))
val users: (Int => Request) = (n) => RequestBuilder()
.url(s"$protocol://$host/api?results=$n")
.buildGet()
}
object MyApi {
def apply(host: String, protocol: String) = new MyApi(host, protocol)
}
MyApiSpec.scala:
import com.twitter.util.Return
import com.twitter.finagle.http.service.HttpResponseClassifier
import org.scalatest._
import org.scalatest.concurrent._
import org.scalamock.proxy.ProxyMockFactory
import scala.language.postfixOps
import com.twitter.util.{Await}
import org.scalamock.scalatest.MockFactory
import com.twitter.finagle.http._
import org.scalamock.proxy.ProxyMockFactory
import io.finch._
import com.twitter.finagle.Http
import scala.io.Source
import com.twitter.io.{Reader, Buf}
import com.twitter.finagle.ListeningServer
class MyApiSpec extends FlatSpec with Matchers with MockFactory with ScalaFutures with ProxyMockFactory with BeforeAndAfter with BeforeAndAfterAll {
var server: ListeningServer = _
val ru = MyApi("localhost:1490", "http")
after {
server.close()
}
var attempts = 0
val failure: Endpoint[String] = get("api") {
println(attempts)
if (attempts > 1) {
Ok("test message")
}
else {
attempts += 1
BadGateway(new Exception("try again"))
}
}
it should "avoid network issues by retrying" in {
server = Http.server
.serve(":1490", failure.toServiceAs[Text.Plain])
val users = Await.result(ru.requestUsers).contentString
assert(users == "test message")
}
}
build.sbt:
name := "myapi"
version := "1.0"
scalaVersion := "2.11.7"
libraryDependencies += "org.scalactic" %% "scalactic" % "2.2.6"
libraryDependencies += "org.scalatest" %% "scalatest" % "2.2.6" % "test"
libraryDependencies += "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.2"
val circeVersion = "0.4.1"
libraryDependencies ++= Seq(
"io.circe" %% "circe-core",
"io.circe" %% "circe-generic",
"io.circe" %% "circe-parser"
).map(_ % circeVersion)
libraryDependencies ++= Seq(
"com.github.finagle" %% "finch-core" % "0.11.0-M2",
"com.github.finagle" %% "finch-circe" % "0.11.0-M2"
)
libraryDependencies += "com.typesafe" % "config" % "1.3.0"
libraryDependencies += "com.typesafe.slick" %% "slick" % "3.1.1"
libraryDependencies += "com.typesafe.slick" %% "slick-hikaricp" % "3.1.0"
libraryDependencies += "com.typesafe.slick" %% "slick-testkit" % "3.1.1" % "test"
libraryDependencies += "org.postgresql" % "postgresql" % "9.4-1206-jdbc4"
libraryDependencies += "junit" % "junit" % "4.8.1" % "test"
libraryDependencies += "org.scalamock" %% "scalamock-scalatest-support" % "3.2.2" % "test"
val scalazVersion = "7.2.4"
libraryDependencies ++= Seq(
"org.scalaz" %% "scalaz-core" % scalazVersion,
"org.scalaz" %% "scalaz-effect" % scalazVersion,
"org.scalaz" %% "scalaz-scalacheck-binding" % scalazVersion % "test"
)
scalacOptions += "-feature"
initialCommands in console := "import scalaz._, Scalaz._"
Sorry, it's so confusing. Finagle doesn't retry application-level failures (set via ResponseClassifiers) by default so you'd need to enable retries explicitly (see Retries section in the user guide).
Try building something between the lines:
import com.twitter.conversions.time._
import com.twitter.finagle.Http
import com.twitter.finagle.http.{Request, Response}
import com.twitter.finagle.util.DefaultTimer
import com.twitter.finagle.service.{Backoff, RetryFilter}
import com.twitter.finagle.stats.NullStatsReceiver
import com.twitter.util._
import com.twitter.finagle.param.HighResTimer
val twitter = Http.client.newService("twitter.com")
val shouldRetry: PartialFunction [(Request, Try[Response]), Boolean] = {
case (_, Return(rep)) => rep.status != 200
}
implicit val t = HighResTimer.Default
// 3 retries, backoff 1 second
val retry = RetryFilter(Backoff.const(1.second).take(3))(shouldRetry)
val retryTwitter = retry.andThen(twitter)

java.lang.NoSuchMethodError with Scala actors

I have a simple Scala application (taken from here) which I want to test. The whole project is compiled successfully with SBT. However, when I launch the tests with sbt test I get the following error message:
Could not run test ConnectionTest:java.lang.NoSuchMethodError: akka.actor.Props$.apply(Lscala/Function0;)Lakka/actor/Props;
From the internet search, I get the impression that some of my versioning is not compatible but that is merely a guess. What may be wrong?
[Test Case]
import akka.actor.{Props, Actor, ActorSystem, ActorRef}
import akka.testkit.{TestKit, TestActorRef, ImplicitSender}
import org.scalatest.{WordSpecLike, BeforeAndAfterAll}
import org.scalatest.matchers.MustMatchers
class ConnectionTest extends TestKit(ActorSystem("ConnectionSpec"))
with WordSpecLike
with MustMatchers
with BeforeAndAfterAll {
override def afterAll() { system.shutdown() }
"Server" must {
"bind to port " in {
// create server
val server = system.actorOf(Props[Server], name = "server")
expectMsg("Bound to /127.0.0.1:8888")
}
}
}
[build.sbt]
name := "MyApp"
version := "0.2"
scalaVersion := "2.10.4"
mainClass := Some("akka.Main")
resolvers += "Typesafe Repository" at "http://repo.typesafe.com/typesafe/releases/"
libraryDependencies +=
"com.typesafe.akka" %% "akka-actor" % "2.3.2"
libraryDependencies +=
"com.typesafe.akka" %% "akka-testkit" % "2.1.4" % "test"
libraryDependencies +=
"org.scalatest" % "scalatest_2.10" % "2.0" % "test"
You need to use the same version of Akka for testkit.
libraryDependencies +=
"com.typesafe.akka" %% "akka-testkit" % "2.1.4" % "test"
should be
libraryDependencies +=
"com.typesafe.akka" %% "akka-testkit" % "2.3.2" % "test"