I am trying a simple example where I am using a modification of the Dispatch example from the Scalatra site to make an async http request. The code is below. I get a compilation error that says value OK is not a member of String. I put together a standalone scala test with no Scalatra dependencies and it works as expected. I suspect OK is being pulled in from some Scalatra dependency.
I am new to Scala and I am putting together a test web app using Scalatra. Any help will be appreciated.
import dispatch._
trait AppAPIStack extends AppStack {
before() {
contentType = "application/json"
}
object MyAsyncClient {
def sendReq(phrase: param): Future[String] = {
val searchURL = url("https://www.google.com/#q="+phrase)
val result = dispatch.Http(searchURL OK as.String)
for(r <- result) yield r
}
}
}
I fixed the issue by changing the line
url("https://www.google.com/#q="+phrase)
to
dispatch.url("https://www.google.com/#q="+phrase)
Dispatch and Scalatra base servlet I think have an implementation of url which was clashing. The url version from Scalatra does not return OK.
First of all inspect type of result. If you use IDE just hover your mouse on it and see what it is actually is. It must be different than Future[String], it is Future[T], find out what is T and how to convert it to String.
Related
I'm trying to read a query parameter in an REST Api I'm building with Akka-http and having trouble unmarshalling it to a domain object. This is my route definition:
override lazy val getManyRoute: Route = (pathEndOrSingleSlash & get & parameters("userId".as[UserId].?)) { maybeUserId =>
complete {
eventService.getEventsBy(maybeUserId)
}
}
Everything compiles fine, so I am assuming the necessary unmarshalling implicits are in scope. Using curl to test the endpoint, I get the following error:
➜ ~ curl "localhost:8080/events?userId=6be0f45e-084e-41db-9e5b-e6bc8d208b54"
The query parameter 'userId' was malformed:
expected whitespace or eof got 'be0f45...' (line 1, column 2)%
It seems like it's reading just the first character ('6') from the UUID, and for some reason is not including the rest of the string in the parameter. However, if I change the Route definition to read a String and manually "unmarshall" it, it works fine. Something like the following:
override lazy val getManyRoute: Route = (pathEndOrSingleSlash & get & parameters("userId".?)) { maybeUserId =>
// ^ no more unmarshalling
complete {
eventService.getEventsBy(maybeUserId.map(functionToUnmarshallStringToUserId))
}
}
Any ideas? Is there something I'm missing in the original route definition?
EDIT:
I have narrowed down the problem using the following change to the route, trying to see exactly what Unmarshaller was being used:
override lazy val getManyRoute: Route = (pathEndOrSingleSlash & get & parameters("userId".?)) { userId =>
complete {
val um = implicitly[Unmarshaller[String, UserId]]
val t = userId map {
um.apply
}
val fuid = t.fold(Future.successful[Option[UserId]](None)) { _.map { a => Some(a) } }
fuid flatMap { eventService.getEventsBy }
}
}
The result was the same, but I got a bit more context in the error:
io.circe.ParsingFailure: expected whitespace or eof got 'be0f45...' (line 1, column 2)
The problem is that my chain of implicits has eventually led to circe being responsible for unmarshalling the query parameter. Because circe is a json library, it expects strings to be in quotes, and so the following curl request works as expected:
curl "localhost:8080/events/?userId=%226be0f45e-084e-41db-9e5b-e6bc8d208b54%22"
# note the URL-escaped quotes (%22) around the id
So now the problem seems to be that I need multiple unmarshallers for the same class in scope, one to unmarshall directly from plain strings for query parameters, and one to unmarshall from json payloads from request entities. Is there a way to do that using implicits? As suggested in comments, I could use “userId”.as(uidUnmarshaller), but I would prefer to take advantage of implicits if possible.
I am writing unit tests for Play application using Scalamock and Scalatest.
My original code looks like:
// Here ws is an injected WSClient
val req = Json.toJson(someRequestObject)
val resp: Future[WSResponse] = ws.url(remoteURL).post(Json.toJson(req))
In a part I have to mock external calls to a web service, which I am trying to do using scalamock:
ws = stub[WSClient]
wsReq = stub[WSRequest]
wsResp = stub[WSResponse]
ws.url _ when(*) returns wsReq
wsReq.withRequestTimeout _ when(*) returns wsReq
(wsReq.post (_: java.io.File)).when(*) returns Future(wsResp)
I am successfully able to mock post requests using a file, but I cannot mock post requests using JSON.
I tried putting stub function references separately like:
val f: StubFunction1[java.io.File, Future[WSResponse]] = wsReq.post (_: java.io.File)
val j: StubFunction1[JsValue, Future[WSResponse]] = wsReq.post(_: JsValue)
I get the compile error for second line: Unable to resolve overloaded method post
What am I missing here? Why cannot I mock one overloaded method but not the other one?
play.api.libs.ws.WSRequest has two post methods (https://www.playframework.com/documentation/2.4.x/api/scala/index.html#play.api.libs.ws.WSRequest), taking:
File
T (where T has an implicit bounds on Writeable)
The compiler is failing because you are trying to calling post with a single parameter, which only matches version 1. However, JsValue cannot be substituted with File.
You actually want to call the 2nd version, but this is a curried method that takes two sets of parameters (albeit the 2nd are implicit). Therefore you need to explicitly provide the mock value that you expect for the implicit, i.e.
val j: StubFunction1[JsValue, Future[WSResponse]] = wsReq.post(_: JsValue)(implicitly[Writeable[JsValue]])
Therefore a working solution would be:
(wsReq.post(_)(_)).when(*) returns Future(wsResp)
Old answer:
WSRequest provides 4 overloads of post method (https://www.playframework.com/documentation/2.5.8/api/java/play/libs/ws/WSRequest.html), taking:
String
JsonNode
InputStream
File
You can mock with a File because it matches overload 4, but JsValue does not match (this is part of the Play JSON model, whereas JsonNode is part of the Jackson JSON model). If you convert to a String or JsonNode, then it will resolve the correct overload and compile.
My best guess is that your WSRequest is actually a play.libs.ws.WSRequest which is part of the Java API, instead you should use play.api.libs.ws.WSRequest which is the Scala API.
The method WSRequest.post exists and BodyWritable[JsValue] is implicitly provided by WSBodyWritables in the Scala API but not in the Java API.
Another cause could be that your JsValue is not a play.api.libs.json.JsValue but something else (e.g. spray.json.JsValue).
I'll quote an example where I have successfully achieved what you are trying to do, the main difference is that I used mock instead of stub.
The important part is:
val ws = mock[WSClient]
val responseBody = "{...}"
...
"availableBooks" should {
"retrieve available books" in {
val expectedBooks = "BTC_DASH ETH_DASH USDT_LTC BNB_LTC".split(" ").map(Book.fromString).map(_.get).toList
val request = mock[WSRequest]
val response = mock[WSResponse]
val json = Json.parse(responseBody)
when(ws.url(anyString)).thenReturn(request)
when(response.status).thenReturn(200)
when(response.json).thenReturn(json)
when(request.get()).thenReturn(Future.successful(response))
whenReady(service.availableBooks()) { books =>
books.size mustEqual expectedBooks.size
books.sortBy(_.string) mustEqual expectedBooks.sortBy(_.string)
}
}
}
An you could see the complete test at: BinanceServiceSpec
I guess it should work fine, if you mock a response that is JsValue.
when(wsReq.post(Json.parse("""{...json request...}"""))).thenReturn(Future(wsResp))
Here Json.parse returns JsValue. Yo should pass the json string that you expect in the request body.
I'm fumbling my way through akka-http; I've got a single route that compiles:
val route = get {
pathPrefix("compass") {
path("route") {
parameters(('srclat.as[Double], 'srclon.as[Double],
'destlat.as[Double], 'destlon.as[Double])) {
(srclat, srclon, destlat, destlon) =>
complete(getRoute((LatLong(srclat, srclon),
LatLong(destlat, destlon))))
}
}
}
}
And I've verified that the parameters are being extracted correctly. When I call the method (with valid lat / longs), I'm expecting to receive an array of coordinates representing a (physical) route, but instead I receive a route object with an empty list of coordinates. Here's the signature of the method being run by complete:
// the Route here is a domain object, not an akka-http Route
def getRoute(coordinates: (LatLong, LatLong)):
Future[Option[Route]] = ???
And starting the server itself looks something like this:
val bindingFuture = Http().bindAndHandle(service.route, "0.0.0.0",
port)
I'm using akka and akka-streams 2.5.4 and akka-http 10.0.9, with Circe support from hseeberger (version 1.18.0). If anyone knows what I'm doing wrong here, please let me know...any help would be appreciated!
instead I receive a route object with an empty list of coordinates.
I think the problem is not in the code shown, but somewhere inside the getRoute function.
I have a hunch that you might be making changes to an immutable case class, and returning a previous copy, rather than the updated version?
e.g. code like the following would give the bug you describe:
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
trait LatLong
case class Route(steps: List[String]) // for example
def getRoute(coordinates: (LatLong, LatLong)): Future[Option[Route]] = {
val myRoute = new Route(Nil)
val steps = List("one", "two", "three")
myRoute.copy(steps = steps) // BUG HERE, new copy discarded
Future(Some(myRoute))
}
If that doesn't explain things, please could you show more of the getRoute function?
A quick test to narrow things down might be to change getRoute temporarily to return a hardcoded non-empty Route, and check that it comes back OK over HTTP.
This was completely my fault; due to a cut-and-paste error, the source coordinates were being sent in as both source and destination, so the empty Route object was legitimate. Thanks to those who took their time looking into my screw-up!
I'm trying to test the following line of code using ScalaTest and ScalaMock.
val responseFuture = wsClient.url(url).withQueryString(params: _*).get()
wsClient type is THttpClient, which is a wrapper of play.api.libs.ws.WS.
Given that:
val mockHttpClient = mock[THttpClient]
is properly injected into my class under test, the test code is something like this:
val expectedUrl = "some url"
val mockRequestHolder = mock[WSRequestHolder]
inSequence {
(mockHttpClient.url _).expects(expectedUrl).returns(mockRequestHolder)
(mockRequestHolder.withQueryString _).expects(where {
(parameters: Seq[(String, String)]) => {
// assertions on parameters
// ...
true
}
}).returns(mockRequestHolder)
val stubResponse = stub[WSResponse]
val jsonBody = "{}"
(stubResponse.json _).when().returns(Json.parse(jsonBody))
(mockRequestHolder.get _).expects().returns(Future(stubResponse))
}
IntelliJ is highlighting mockRequestHolder.get as an error saying: cannot resolve symbol get. Nevertheless I'm able to run the test but the mock is clearly not working, and I'm getting: java.util.NoSuchElementException: JsError.get.
The mock is working when I try to mock any other method of WSRequestHolder, but not with method get.
Is this a ScalaMock bug or am I doing something wrong?
I don't know if you have solved already the issue, but I have tried to do something similar recently and I kind of got it working with the following code:
val wsClientMock = mock[WSClient]
val wsRequestMock = mock[WSRequest]
val wsResponseMock = mock[WSResponse]
(wsRequestMock.withAuth _).expects(username, password, WSAuthScheme.BASIC).returning(wsRequestMock)
(wsRequestMock.get _).expects().returning(Future[WSResponse](wsResponseMock))
(wsClientMock.url _).expects(bootstrapUrl).returning(wsRequestMock)
(wsResponseMock.status _).expects().returning(200)
"kind of" because I need to mock also the response, otherwise I get results like
ERROR[default-akka.actor.default-dispatcher-4] OneForOneStrategy - Unexpected call: json()
due to the fact that the code calling the WSClient is calling the method .json of WSResponse.
Sorry, I don't know Scala Mock but I suggest you to have a look at MockWS a library which comes with a mocked WS client: play-mockws
With MockWS you define a partial function which returns an Action for a Route. This enables you to precisely configure mocked answers and test your http client code.
My attempt so far ran into java.lang.ClassCastException: scala.runtime.BoxedUnit cannot be cast to scala.Option. In fact I even can not think how the appropriate response may look like. Is it possible?
Here could be two options: either it's a bug in your program or you didn't provide spray any way to marshall your Foo type, cause Future and Option are handled by default. E.g this route can be handled by standard spray marshallers without any problems:
val route = {
(get & path("test")) {
complete {
Future(Option("Hello Spray!"))
}
}
}
Now if you make a GET request on /test you'll get a correct response.
If you have a specific type, then you need to provide you own marshaller of type ToResponseMarshallable to sray through implicit context. I think the most common and easiest way would be to make a Json response, for this you need a spray-json (or some other supported json lib) and just provide a converter to json, like:
import spray.json.DefaultJsonProtocol._
case class Boy(name: String, age: String)
object Boy {
implicit val boyJson = jsonFormat2(Boy.apply)
}
Now the only thing left to do is place a json marshaller into the scope:
import spray.httpx.SprayJsonSupport._ // import json marshaller
val route = {
(get & path("test") {
complete {
Future(Option(Boy("Name", 0)))
}
}
}
Now you'll get a json response. IF you need some other kind of response, make a custom marshaller for a Boy type.
In short - Yes, but you need to make compiler happy.
Spray lets you return Future or plain response as long as it can be marshalled back to a response. You need to make sure that you either have implicit conversions in the scope that do the marshalling for you or transform your Foo object explicitly.
Here is how response transformation is performed: http://spray.io/documentation/1.2.0/spray-httpx/response-transformation/.
To anyone still getting issues returning a future in the complete block, make sure to have an execution context such as scala.concurrent.ExecutionContext.Implicits.global in scope! IDE tends to not understand this is the problem and can send you down a long rabbit hole.