I am trying to build a simple RESTful service that performs CRUD operations on a database and returns JSON. I have a service adhering to an API like this
GET mydomain.com/predictions/some%20string
I use a DAO which contains the following method that I have created to retrieve the associated prediction:
def getPrediction(rawText: String): Prediction = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.head
val f = db.run(header)
f.onComplete{case pred => pred}
throw new Exception("Oops")
}
However, this can't be right, so I started reading about Option. I changed my code accordingly:
def getPrediction(rawText: String): Option[Prediction] = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.headOption
val f = db.run(header)
f.onSuccess{case pred => pred}
None
}
This still doesn't feel quite right. What is the best way to invoke these filters, return the results, and handle any uncertainty?
I think the best way to rewrite your code is like this:
def getPrediction(rawText: String): Future[Option[Prediction]] = {
db.run(users.filter(_.rawText === rawText).result.headOption)
}
In other words, return a Future instead of the plain result. This way, the database actions will execute asynchronously, which is the preferred way for both Play and Akka.
The client code will then work with the Future. Per instance, a Play action would be like:
def prediction = Action.async {
predictionDao.getPrediction("some string").map { pred =>
Ok(views.html.predictions.show(pred))
}.recover {
case ex =>
logger.error(ex)
BadRequest()
}
}
Related
I’m attempting to setup a simple graph structure that process data via invoking rest services, forwards the result of each service to an intermediary processing unit before forwarding the result. Here is a high level architecture :
Can this be defined using Akka graph streams ? Reading https://doc.akka.io/docs/akka/current/stream/stream-graphs.html I don't understand how to even implement this simple architecture.
I've tried to implement custom code to execute functions within a graph :
package com.graph
class RestG {
def flow (in : String) : String = {
return in + "extra"
}
}
object RestG {
case class Flow(in: String) {
def out : String = in+"out"
}
def main(args: Array[String]): Unit = {
List(new RestG().flow("test") , new RestG().flow("test2")).foreach(println)
}
}
I'm unsure how to send data between the functions. So I think I should be using Akka Graphs but how to implement the architecture above ?
Here's how I would approach the problem. First some types:
type Data = Int
type RestService1Response = String
type RestService2Response = String
type DisplayedResult = Boolean
Then stub functions to asynchronously call the external services:
def callRestService1(data: Data): Future[RestService1Response] = ???
def callRestService2(data: Data): Future[RestService2Response] = ???
def resultCombiner(resp1: RestService1Response, resp2: RestService2Response): DisplayedResult = ???
Now for the Akka Streams (I'm leaving out setting up an ActorSystem etc.)
import akka.Done
import akka.stream.scaladsl._
type SourceMatVal = Any
val dataSource: Source[Data, SourceMatVal] = ???
def restServiceFlow[Response](callF: Data => Future[Data, Response], maxInflight: Int) = Flow[Data].mapAsync(maxInflight)(callF)
// NB: since we're fanning out, there's no reason to have different maxInflights here...
val service1 = restServiceFlow(callRestService1, 4)
val service2 = restServiceFlow(callRestService2, 4)
val downstream = Flow[(RestService1Response, RestService2Response)]
.map((resultCombiner _).tupled)
val splitAndCombine = GraphDSL.create() { implicit b =>
import GraphDSL.Implicits._
val fanOut = b.add(Broadcast[Data](2))
val fanIn = b.add(Zip[RestService1Response, RestService2Response])
fanOut.out(0).via(service1) ~> fanIn.in0
fanOut.out(1).via(service2) ~> fanIn.in1
FlowShape(fanOut.in, fanIn.out)
}
// This future will complete with a `Done` if/when the stream completes
val future: Future[Done] = dataSource
.via(splitAndCombine)
.via(downstream)
.runForeach { displayableData =>
??? // Display the data
}
It's possible to do all the wiring within the Graph DSL, but I generally prefer to keep my graph stages as simple as possible and only use them to the extent that the standard methods on Source/Flow/Sink can't do what I want.
I am using the BlazeClientBuilder[IO].resource method to get Client[IO]. Now, I want to mock the client for unit testing but cannot figure out how to do so. Is there a good way of mocking this and how would I do that?
class ExternalCall(val resource: Resource[IO, Client[IO]], externalServiceUrl: Uri) {
def retrieveData: IO[Either[Throwable, String]] = {
for {
req <- IO(Request[IO](Method.GET, uri = externalServiceUrl))
response <- resource.use(client => {
client.fetch[String](req)(httpResponse => {
if (!httpResponse.status.isSuccess)
throw new Exception(httpResponse.status.reason)
else
httpResponse.as[String]
})
})
} yield Right(response)
}
}
Caller code
new ExternalCall(BlazeClientBuilder[IO](global).resource).retrieveData
It seems you only need to do something like
val resourceMock = mock[Resource[IO, Client[IO]]]
//stub whatever is necessary
val call = new ExternalCall(resourceMock).retrieveData
//do asserts and verifications as needed
EDIT:
You can see a fully working example below, but I'd like to stress that this is a good example of why it is a good practice to avoid mocking APIs that you don't own.
A better way to test this would be to place the http4s related code witin a class you own (YourHttpClient or whatever) and write an integration test for that class that checks that the http4s client does the right thing (you can use wiremock to simulate a real http server).
Then you can pass mocks of YourHttpClient to the components that depend on it, with the advantage that you control its API so it will be simpler and if http4s ever updates its API you only have one breaking class rather than having to fix tens or hundreds of mock interactions.
BTW, the example is written using mockito-scala as using the Java version of mockito would have yielded code much harder to read.
val resourceMock = mock[Resource[IO, Client[IO]]]
val clientMock = mock[Client[IO]]
val response: Response[IO] = Response(Status.Ok,
body = Stream("Mocked!!!").through(text.utf8Encode),
headers = Headers(`Content-Type`(MediaType.text.plain, Charset.`UTF-8`)))
clientMock.fetch[String](any[Request[IO]])(*) shouldAnswer { (_: Request[IO], f: Response[IO] => IO[String]) =>
f(response)
}
resourceMock.use[String](*)(*) shouldAnswer { (f: Client[IO] => IO[String]) =>
f(clientMock)
}
val data = new ExternalCall(resourceMock, Uri.unsafeFromString("http://www.example.com")).retrieveData
data.unsafeRunSync().right.value shouldBe "Mocked!!!"
You can easly mock Client using following snippet
import fs2.Stream
import org.http4s.Response
import org.http4s.client.Client
def httpClient(body: String): Client[IO] = Client.apply[IO] { _ =>
Resource.liftF(IO(Response[IO](body = Stream.emits(body.getBytes("UTF-8")))))
}
In order to have the client as resource you need to wrap it with IO and lift to Resource
Resource.liftF(IO(httpClient("body")))
I have a method that returns a Future[Boolean] in a Play controller and i want to evaluate that using async but i can't seem to get it to compile.
The following will work:
def health = Action {
logger.info("Endpoint method: health")
val isHealthy = healthCheckService.checkDynamo()
val b: Boolean = Await.result(isHealthy, scala.concurrent.duration.Duration(5, "seconds"))
Ok(Json.toJson(HealthCheckResponse(b.toString)))
}
But i don't think i want that Await in there. So i'm trying things like this with no success:
def health =
Action.async {
Future {
logger.info("Endpoint method: health")
healthCheckService.checkDynamo() match {
case Future.successful(true) => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
val r = healthCheckService.checkDynamo() match {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
}
I can't even get those to compile to test them out.
Any suggestions?
Try this:
def health = Action.async {
healthCheckService.checkDynamo().map {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
Let Play handle the awaiting for you under the hood. That is, Action.async accepts a Future, which checkDynamo() already returns. All you have to do is map it to the appropriate result.
With Futures you have to use combinators like map and flatMap to express the final value. For example:
Action.async {
healthCheckService.checkDynamo()
.map { result => // boolean
HealthCheckResponse(result.toString)
}
.map(Json.toJson(_))
.map(Ok(_))
}
(You can merge maps above to one map and construct the final Ok value there; it is more or less a matter of taste)
If you have, say, two async calls which you want to execute and return a result based on their results, you can use flatMap, which could be easily expressed using a for comprehension:
Action.async {
for {
result1 <- someService.someCall()
result2 <- anotherService.anotherCall(result1.someProperty)
finalResult = SomeFinalResultType(result1, result2)
} yield Ok(Json.toJson(finalResult))
}
If you are not familiar with futures, you might want to read some tutorial which explains their nature, how to combine them and how to get useful results from them, like this one: http://hello-scala.com/920-scala-futures.html
I have model repository class with byId and save metdhods
def byID(id:Long) = db.run{ query.filter(_.id === id).result }.map(_.headOption)
def save(model:User) = db.run{ query.filter(_.id===model.id).update(model) }
Now I want to use both these methods, first load user, then change something, and then save user, like this
userRepository.byID(5L).map{_.map{user =>
val newuser = user.copy(name = "John")
userRepository.save(newuser)
}}
How can I do it in one transaction?
I think that slick 3 doesn't support having a transaction that spans over different futures and when you call db.run you are passing in a DBIO[SomeType] in and getting a Future out. The good news is that you can instead compose your DBIOs structuring your API in a slightly different way:
def byID(id:Long) = query.filter(_.id === id).result }.map(_.headOption)
def save(model:User) = query.filter(_.id===model.id).update(model)
And then:
db.run(userRepository.byID(5L).flatMap { users =>
DBIO.sequence(users.map { user =>
userRepository.save(user.copy(name = "John"))
})
}.transactionally)
I'd like to keep my application as asynchronously as possible. Now I have this repository:
object LanguageRepository extends LanguageRepositoryTrait
{
private val languages = TableQuery[Languages]
private def db:Database = Database.forDataSource(DB.getDataSource())
private def filterQuery(id: Long): Query[Languages, Language, Seq] = languages.filter(_.id === id)
private def filterCode(code: String): Query[Languages, Language, Seq] = languages.filter(_.code === code)
private def all() : Query[Languages, Language, Seq] = languages
override def find(id: Long): Future[Language] =
{
try db.run(filterQuery(id).result.head)
finally db.close()
}
override def find(code: String): Future[Language] =
{
try db.run(filterCode(code).result.head)
finally db.close()
}
override def get(): Future[Seq[Language]] =
{
try db.run(all().result)
finally db.close()
}
}
When I call a url like "domain.tld/{language}" I want to check whether the given language(code) actually exists. Like, if the site isn't available in french (fr) I want to throw an exception or a 404.
Now, my problem is that this whole asynchronously thing is pretty cool and while I do think to understand the theory behind it, I'm rather baffled right now. I mean, I want this to be non-blocking (and asynchronously, which is the reason for me using Future and async ;))
In my controller I want to do something like:
def checkLanguage(language:String) = Action
{
val lang:Future[Language] = languageRepository.find(language)
lang.onComplete
{
case Success(s) = Ok("Yay")
case Failure(f) = 404("Oh no!")
}
}
Of course this can't work, but that's a schema of how I want to have things working. I want to wait or postpone the rendering of the site, until it's confirmed that the given language-code is valid or invalid.
I had a look at the Playframeworks async-documentation for 2.3.6 (https://www.playframework.com/documentation/2.3.6/ScalaAsync) but I couldn't really get this to work as intended.
Any input appreciated!
From your db query don't use .head instead use .headOption. that way ur return type will be a Future [Option [x]]
In ur controller u can do something like this
Lang.map { case Some (x) => Ok (x)
case None => 404 ( "not found ")
}
Try this,
Action.async {
val lang:Future[Option[Language]] = languageRepository.find(language)
lang.map {l => l.map{_ => Ok("Yay") }.getOrElse(NotFound("Oh no!"))
}
First of all, I am assuming that if there is a possibility that a language will not exist then languageRepository.find(language) should return an Option of Language. Change Future[Language] to Future[Result] and use Action.async instead of Action
Now for some explanation, Action takes a block whose result should be Result. However, what you get is Future[Option[Language]]. Play provides async method for Action which needs Future[Result] and it takes cares of completing the request.
So, you need to convert Future[Option[Language]] to Future[Result].
lang.map {l => l.map{_ => Ok("Yay") }.getOrElse(NotFound("Oh no!"))
We map over the lang, if the Option[Language] is not None then we convert it to Ok("yay") else we convert it to NotFound
Even, If you don't get Option[Language], the idea remains the same. Convert Future[Language] to Future[Result] and use Action.async instead of Action