Mocking of BlazeClientBuilder[IO] to return mock client[IO] - scala

I am using the BlazeClientBuilder[IO].resource method to get Client[IO]. Now, I want to mock the client for unit testing but cannot figure out how to do so. Is there a good way of mocking this and how would I do that?
class ExternalCall(val resource: Resource[IO, Client[IO]], externalServiceUrl: Uri) {
def retrieveData: IO[Either[Throwable, String]] = {
for {
req <- IO(Request[IO](Method.GET, uri = externalServiceUrl))
response <- resource.use(client => {
client.fetch[String](req)(httpResponse => {
if (!httpResponse.status.isSuccess)
throw new Exception(httpResponse.status.reason)
else
httpResponse.as[String]
})
})
} yield Right(response)
}
}
Caller code
new ExternalCall(BlazeClientBuilder[IO](global).resource).retrieveData

It seems you only need to do something like
val resourceMock = mock[Resource[IO, Client[IO]]]
//stub whatever is necessary
val call = new ExternalCall(resourceMock).retrieveData
//do asserts and verifications as needed
EDIT:
You can see a fully working example below, but I'd like to stress that this is a good example of why it is a good practice to avoid mocking APIs that you don't own.
A better way to test this would be to place the http4s related code witin a class you own (YourHttpClient or whatever) and write an integration test for that class that checks that the http4s client does the right thing (you can use wiremock to simulate a real http server).
Then you can pass mocks of YourHttpClient to the components that depend on it, with the advantage that you control its API so it will be simpler and if http4s ever updates its API you only have one breaking class rather than having to fix tens or hundreds of mock interactions.
BTW, the example is written using mockito-scala as using the Java version of mockito would have yielded code much harder to read.
val resourceMock = mock[Resource[IO, Client[IO]]]
val clientMock = mock[Client[IO]]
val response: Response[IO] = Response(Status.Ok,
body = Stream("Mocked!!!").through(text.utf8Encode),
headers = Headers(`Content-Type`(MediaType.text.plain, Charset.`UTF-8`)))
clientMock.fetch[String](any[Request[IO]])(*) shouldAnswer { (_: Request[IO], f: Response[IO] => IO[String]) =>
f(response)
}
resourceMock.use[String](*)(*) shouldAnswer { (f: Client[IO] => IO[String]) =>
f(clientMock)
}
val data = new ExternalCall(resourceMock, Uri.unsafeFromString("http://www.example.com")).retrieveData
data.unsafeRunSync().right.value shouldBe "Mocked!!!"

You can easly mock Client using following snippet
import fs2.Stream
import org.http4s.Response
import org.http4s.client.Client
def httpClient(body: String): Client[IO] = Client.apply[IO] { _ =>
Resource.liftF(IO(Response[IO](body = Stream.emits(body.getBytes("UTF-8")))))
}
In order to have the client as resource you need to wrap it with IO and lift to Resource
Resource.liftF(IO(httpClient("body")))

Related

How to handle asynchronous API response in scala

I have an API that I need to query in scala. API returns a code that would be equal to 1 when results are ready.
I thought about an until loop to handle as the following:
var code= -1
while(code!=1){
var response = parse(Http(URL).asString.body)
code = response.get("code").get.asInstanceOf[BigInt].toInt
}
println(response)
But this code returns:
error: not found: value response
So I thought about doing the following:
var code = -1
var res = null.asInstanceOf[Map[String, Any]]
while(code!=1){
var response = parse(Http(URL).asString.body)
code = response.get("code").get.asInstanceOf[BigInt].toInt
res = response
}
println(res)
And it works. But I would like to know if this is really the best scala-friendly way to do so ?
How can I properly use a variable that outside of an until loop ?
When you say API, do you mean you use a http api and you're using a http library in scala, or do you mean there's some class/api written up in scala? If you have to keep checking then you have to keep checking I suppose.
If you're using a Scala framework like Akka or Play, they'd have solutions to asyncrhonously poll or schedule jobs in the background as a part of their solutions which you can read about.
If you're writing a Scala script, then from a design perspective I would either run the script every 1 minute and instead of having the while loop I'd just quit until code = 1. Otherwise I'd essentially do what you've done.
Another library that could help a scala script might be fs2 or ZIO which can allow you to setup tasks that periodically poll.
This appears to be a very open question about designing apps which do polling. A specific answer is hard to give.
You can just use simple recursion:
def runUntil[A](block: => A)(cond: A => Boolean): A = {
#annotation.tailrec
def loop(current: A): A =
if (cond(current)) current
else loop(current = block)
loop(current = block)
}
// Which can be used like:
val response = runUntil {
parse(Http(URL).asString.body)
} { res =>
res.getCode == 1
}
println(response)
An, if your real code uses some kind of effect type like IO or Future
// Actually cats already provides this, is called iterateUntil
def runUntil[A](program: IO[A])(cond: A => Boolean): IO[A] = {
def loop: IO[A] =
program.flatMap { a =>
if (cond(a)) IO.pure(a)
else loop
}
loop
}
// Used like:
val request = IO {
parse(Http(URL).asString.body)
}
val response = runUntil(request) { res =>
res.getCode == 1
}
response.flatMap(IO.println)
Note, for Future you would need to use () => Future instead to be able to re-execute the operation.
You can see the code running here.

ScalaPlay > 2.6 how to access POST requests while faking a trivial server in tests

I'm trying to setup a fake server with Play2.7 and the environment suggested by https://developer.lightbend.com/guides/play-rest-api/ just echoing json from a POST request. While I was able to make GET and POST requests returning hardwired values I can't access directly the request to return or process it. NOTE: this was doable with versions < 2.6 but now Action has become deprecated, so I'm wondering which is the correct way to deal with this in Play >= 2.6
I have read the following how to mock external WS API calls in Scala Play framework and How to unit test servers in Play 2.6 now that Action singleton is deprecated which are actually doing almost all I am trying to do, but it seems I need something different to access the Request. In previous version of Play I could do something like the following:
case POST(p"/route") => Action { request => Ok(request.body.asJson.getOrElse(JsObject.empty)) }
But it seems calling the action this way is not more possible since I received the 'infamous'
object Action in package mvc is deprecated: Inject an ActionBuilder (e.g. DefaultActionBuilder) or extend BaseController/AbstractController/InjectedController
error.
my actual working code is
object FakeServer {
def withServerForStep1[T](codeBlock: WSClient => T): T =
Server.withRouterFromComponents() { cs =>
{
case POST(p"/route") =>
cs.defaultActionBuilder {
Results.Ok(Json.arr(Json.obj("full_name" -> "octocat/Hello-World")))
}
}
} { implicit port =>
WsTestClient.withClient(codeBlock)
}
}
and the unit Spec is something like
"The step 1" should {
"Just call the fakeservice" in {
setupContext()
FakeServer.withServerForStep1 ( {
ws =>
val request = ws.url("/route")
val data = Json.obj(
"key1" -> "value1",
"key2" -> "value2"
)
val response = request.post(data).futureValue
response.status mustBe 200
response.body mustBe Json.toJson(data)
})
}
}
I would like to write the FakeServer in such a way that the Spec will succeed in checking that returned body is equal to original sent json. Currently it is obviously failing with
"[{"full_name":"octocat/Hello-World"}]" was not equal to {"key1":"value1","key2":"value2"}
I eventually found how to do it, and the correct way as often happens in Scala is... trivial.
The "trick" was just to add request => in the body of cs.defaultActionBuilder as in the next example
object FakeServer {
def withServerForStep1[T](codeBlock: WSClient => T): T =
Server.withRouterFromComponents() { cs =>
{
case POST(p"/route") =>
cs.defaultActionBuilder { request =>
val bodyAsJson = request.body.asJson.getOrElse(JsObject.empty)
Results.Ok(bodyAsJson)
}
}
} { implicit port =>
WsTestClient.withClient(codeBlock)
}
}
Then the test just needed to deal with possible extra wrapping quotes and reads as
val response = request.post(data).futureValue
response.status mustBe 200
response.body mustBe Json.toJson(data).toString()

What are ways to unit test Rest API clients - i.e. testing that the right Rest requests are sent

At work we're developing a service that sits between two other services. We expose a Rest API which is called by some service -- the request is processed by some logic and then, according to the logic, HTTP requests will be sent out another service.
There're plenty of resources on the internet on how to best test API endpoints you provide. I, on the other hand want to test what API requests are sent out, without sending them to the actual service.
I guess I could always set up an entire server skeleton on localhost:8080 that just records what it receives but this feels kinda dirty. Testing the Rest API we provide for external services (we use akka-http to do that) is pretty comfortable to test with akka-http-testkit which is excellent. I was just wondering if there is any comparably comfortable to use tooling to test what Http reqeusts go out.
Functional Programming "Tooling"
The easiest way I've found to test these scenarios is to use plain-old functional programming principles in your design. You can embed your Route creation within a higher order function. This higher order function will take in the function which queries the downstream service:
type ComputedData = ???
val computeData : HttpRequest => ComputedData = ???
def intermediateRoute(downstreamService : ComputedData => Future[HttpResponse]) : Route =
extractRequest { request =>
val computedData : ComputedData = computeData(request)
complete(downstreamService(computedData))
}
This higher order function can now be used in Production:
val queryDownStreamService : ComputedData => Future[HttpResponse] = ???
val productionRoute : Route = intermediateRoute(queryDownStreamService)
Or, it can be used in unit-testing to ensure the logic is correct:
val testComputedData : ComputedData => Boolean = ???
val testResponse : HttpResponse = ???
val testService : ComputedData => Future[HttpResponse] = (computedData) => {
assert(testComputedData(computedData))
Success(testResponse)
}
val testRoute = intermediateRoute(testService)
Get("/test") ~> testRoute ~> check {
response should be testResponse
}
We do it the way you call dirty, though I don't think it's dirty.
we have a base trait that starts/shuts down the server (we use http4s and scalatest)
trait EmbeddedServer extends BeforeAndAfterAll with Http4sDsl[IO] {
self: Suite =>
private var server: Server[IO] = _
protected var lastRequest: Request[IO] = _
private def captureRequest: HttpService[IO] = Kleisli { req: Request[IO] =>
lastRequest = req
service(req)
}
override protected def beforeAll(): Unit = {
server = BlazeBuilder[IO]
.bindAny()
.mountService(captureRequest, "/")
.start
.unsafeRunSync()
super.beforeAll()
}
override protected def afterAll(): Unit = {
super.afterAll()
server.shutdownNow()
}
def address: InetSocketAddress = server.address
def rootURI: String = s"http:/$address"
def service: HttpService[IO]
}
then we mix it in our client spec
something along these lines
class SomeRequesterSpec extends WordSpec with EmbeddedServer {
override def service: HttpService[IO] = HttpService[IO] {
case GET -> Root / "failure" => ServiceUnavailable()
case GET -> Root / "success" => Ok(SuccessBody)
case GET -> Root / "partial-success" => Ok(PartialSuccessBody)
case GET -> Root / "malformed" => Ok(MalformedBody)
case GET -> Root / "empty" => Ok(EmptyResponse)
}
//... you specs go here
}
and in you specs you call your mocked server with your client
using s"$rootURI/success" or s"$rootURI/failure" endpoints and check that it handles responses correctly.
Also lastRequest var always has last request issues, so you can run assertions against it, like
lastRequest.headers should contain(`Accept-Encoding`(ContentCoding.gzip))
This approach works very well for us and we can test that our clients handle all sorts of outputs from servers as well as all manipulations with request they do

Converting WebSockets in Play framework from version 2.4 to 2.6

I'm trying to convert this code, that uses the Play version 2.4 to the current version (2.6) and I'm having some issues because I'm still a noob in Scala.
def wsWeatherIntervals = WebSocket.using[String] {
request =>
val url = "http://api.openweathermap.org/data/2.5/weather?q=Amsterdam,nl"
val outEnumerator = Enumerator.repeatM[String]({
Thread.sleep(3000)
ws.url(url).get().map(r => s"${new java.util.Date()}\n ${r.body}")
})
(Iteratee.ignore[String], outEnumerator)
}
I followed this guide, but now I'm stuck on the stuff that I should return on the method.
This is the code that I'm trying to run using the version 2.6:
import play.api.mvc._
import scala.concurrent.Future
import akka.stream.scaladsl._
def wsWeatherIntervals = WebSocket.accept[String, Future[String]] { res =>
val url = "http://api.openweathermap.org/data/2.5/weather?q=Amsterdam,nl"
val source = Source.repeat({
Thread.sleep(3000)
ws.url(url).get().map(r => s"${new java.util.Date()}\n ${r.body}")
})
Flow.fromSinkAndSource(Sink.ignore, source)
}
But I'm getting this error when running the server, that points to the first line of the method:
could not find implicit value for parameter transformer: play.api.mvc.WebSocket.MessageFlowTransformer[String,scala.concurrent.Future[String]]
Note: I also tried to call WebSocket.apply instead of WebSocket.accept and I did some search about the differences between the two but didn't find anything useful. Can someone explain the difference between the two? Thanks.
The superficial error is that Play doesn't know how to turn a Future[String] into a Websocket message, for which you'd normally use an implicit transformer. However, in this case you don't want to return a Future[String] anyway but just a plain string which can be automatically marshalled (using the provided stringMessageFlowTransformer as it happens.) Here's something that should work:
def wsWeatherIntervals = WebSocket.accept[String, String] { res =>
val url = "http://api.openweathermap.org/data/2.5/weather?q=Amsterdam,nl"
def f = ws.url(url).get().map(r => s"${new java.util.Date()}\n ${r.body}")
val source = Source.unfoldAsync(f)(last => {
Thread.sleep(3000)
f.map(next => Some((last, next)))
})
Flow.fromSinkAndSource(Sink.ignore, source)
}
The unfoldAsync source lets us repeated run a function returning a future of the next element in the stream. (Since we want the stream to go on forever we return the value wrapped as Some.)
The Websocket.apply method is basically a more complicated version of accept which allows you to reject a websocket connection for some reason by returning a response, but if you need to do this it's better to use acceptOrResult, which handles transforming whatever your flow emits into websocket messages.

Slick - What if database does not contain result

I am trying to build a simple RESTful service that performs CRUD operations on a database and returns JSON. I have a service adhering to an API like this
GET mydomain.com/predictions/some%20string
I use a DAO which contains the following method that I have created to retrieve the associated prediction:
def getPrediction(rawText: String): Prediction = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.head
val f = db.run(header)
f.onComplete{case pred => pred}
throw new Exception("Oops")
}
However, this can't be right, so I started reading about Option. I changed my code accordingly:
def getPrediction(rawText: String): Option[Prediction] = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.headOption
val f = db.run(header)
f.onSuccess{case pred => pred}
None
}
This still doesn't feel quite right. What is the best way to invoke these filters, return the results, and handle any uncertainty?
I think the best way to rewrite your code is like this:
def getPrediction(rawText: String): Future[Option[Prediction]] = {
db.run(users.filter(_.rawText === rawText).result.headOption)
}
In other words, return a Future instead of the plain result. This way, the database actions will execute asynchronously, which is the preferred way for both Play and Akka.
The client code will then work with the Future. Per instance, a Play action would be like:
def prediction = Action.async {
predictionDao.getPrediction("some string").map { pred =>
Ok(views.html.predictions.show(pred))
}.recover {
case ex =>
logger.error(ex)
BadRequest()
}
}