How to write idiomatic scala code for http4s - scala

I'm struggling a bit with how to translate an imperative style to a functional style.
In a imperative web request I'm used to saying something like the following psudo code:
public Response controllerAction(Request request) {
val (req, parserErrors) = parser.parseRequest(request);
if (parserErrors.any()) {
return FourHundredError(parserErrors);
}
val businessErrors = model.validate(req);
if (businessErrors.any()){
return FourOhFour(businessErrors);
}
val (response, errorsWithOurStuff) = model.doBusinessLogicStuff(req);
if (errorsWithOurStuff.any()) {
return FiveHundredError(errorsWithOurStuff);
}
return OK(response)
}
I'm trying to translate this into a functional style using http4s.
def businessRoutes[F[_]: Sync](BL: BusinessLogic[F]): HttpRoutes[F] = {
val dsl = new Http4sDsl[F]{}
import dsl._
HttpRoutes.of[F] {
case req # POST -> Root / "sms" =>
for {
request <- req.as[BL.BuisnessRequest]
requestErrors <- BL.validateRequest(request)
response <- if (requestErrors.isEmpty) {
BL.processRequest(request) match {
case Failure(e) => InternalServerError(e)
case Success(response) => Ok(response)
}
} else {
BadRequest(requestErrors)
}
} yield response
}
}
The above code just looks... bad to me and I don't know how to make it better. My goal for this is to keep all the http style abstractions contained here, because I don't want to leak http4s or circe down into a business layer. I feel like I've got a for then an if, then a match and the responses are all jumbled together out of order. I'm writing hard to understand code here and I'm hoping some scala guru could show me how to clean this up and make this readable.

IMHO, the problem is rooted in the way you are modeling the data; mainly with validateRequest
Always remember, parse, don't validate.
Additionally, I would go the untyped errors route with a main handler like this:
import cats.syntax.all._
import io.circe.{Error => CirceError}
object model {
final case class RawRequest(...)
final case class BuisnessRequest(...)
final case class BuisnessResponse(...)
}
object errors {
// Depending on how you end up using those,
// it may be good to use scala.util.control.NoStackTrace with these.
// They may also be case classes to hold some context.
final case object ValidationError extends Throwable
final case object BusinessError extends Throwable
}
trait BusinessLogic {
def validateRequest(rawRequest: RawRequest): IO[BusinessRequets]
def processRequest(request: BusinessRequets): IO[BuisnessResponse]
}
final class HttpLayer(bl: BusinessLogic) extends Http4sDsl[IO] {
private final val errorHanlder: PartialFunction[Throwable, IO[Response[IO]] = {
case circeError: CirceError =>
BadRequest(...)
case ValidationError =>
NotFound(...)
case BusinessError =>
InternalServerError(...)
}
val routes: HttpRoutes[IO] = HttpRoutes[F] {
case req # POST -> Root / "sms" =>
req
.as[RawRequest] // This may fail with CirceError.
.flatMap(bl.validateRequest) // This may fail with ValidationError.
.flatMap(bl.processRequest) // This may fail with BusinessError.
.redeemWith(recover = errorHandler, response => Ok(response))
}
}
I used concrete IO here for simplicity, you may use F[_] if you please.

Related

Recursive HTTP-requests in Scala

I need to do recursive requests and then collect all models into one List, but not understand how to do it. Please tell me am I thinking right way?
package kindSir.main
import dispatch.Defaults._
import dispatch._
import kindSir.models._
import org.json4s._
import org.json4s.jackson.JsonMethods._
object ApplicationMain extends App {
def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
Http(requestsUrl).map { res =>
(parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
case (list#JArray(_), nextPage: Int) =>
val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
// And how to merge these two lists?
case (list#JArray(_), _) => MergeRequest.parseList(list).get
case _ => throw new RuntimeException(s"No merge requests for project found")
}
}
}
}
The main problem you're dealing with here is that you're trying to combine data you already have (List[MergeRequest]) with the data you'll retrieve in future (Future[List[MergeRequest]]). There are a few things you need to do to handle this scenario:
Use flatMap instead of map on result of the HTTP request. This allows you to make further HTTP requests inside the recursion but map them back to a single Future.
Call map on the result of the recursion fetchMergeRequests(nextPage) to combine the data you already have with the future data from the recursion.
Wrap the other list in Future.successful() because flatMap requires all the pattern matches to return a Future — except for the exception.
I'm not familiar with the libraries you are using so I haven't tested it, but I think your code should work like this:
def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
Http(requestsUrl).flatMap { res =>
(parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
case (list#JArray(_), nextPage: Int) =>
val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
nextPageListFuture.map(nextPageList => currentList ++ nextPageList)
case (list#JArray(_), _) =>
Future.successful(MergeRequest.parseList(list).get)
case _ => throw new RuntimeException(s"No merge requests for project found")
}
}
}

akka http to use Json Support and xmlsupport

I want to print data in xml format when department is "hr"but data in json format when department is "tech" .
Can we use
spray-json Support https://doc.akka.io/docs/akka-http/current/common/json-support.html and XML support https://doc.akka.io/docs/akka-http/current/common/xml-support.html together
private[rest] def route =
(pathPrefix("employee") & get) {
path(Segment) { id =>
parameters('department ? "") { (flag) =>
extractUri { uri =>
complete {
flag match {
case "hr": => {
HttpEntity(MediaTypes.`application/xml`.withCharset(HttpCharsets.`UTF-8`),"hr department")
}
case "tech" =>{
HttpEntity(ContentType(MediaTypes.`application/json`), mapper.writeValueAsString("tech department"))
}
}
}
}
}
}
}
Solution I tried
I tried the below by using by using JsonProtocols and ScalaXmlSupport I get the compile error expected ToResponseMarshallable but found Department
case class department(name:String)
private[rest] def route =
(pathPrefix("employee") & get) {
path(Segment) { id =>
parameters('department ? "") { (flag) =>
extractUri { uri =>
complete {
flag match {
case "hr": => {
complete(department(name =flag))
}
case "tech" =>{
complete(department(name =flag))
}
}
}
}
}
}
}
I think there are several issues you have to overcome to achieve what you want:
You want to customize response type basing on the request parameters. It means standard implicit-based marshaling will not work for you, you'll have to do some explicit steps
You want to marshal into an XML-string some business objects. Unfortunately, ScalaXmlSupport that you referenced does not support this, it can marshal only an XML-tree into a response. So you'll need some library that can do XML serialization. One option would be to use jackson-dataformat-xml with jackson-module-scala. It also means you'll have to write your own custom Marshaller. Luckily it is not that hard.
So here goes some simple code that might work for you:
import akka.http.scaladsl.marshalling.{ToResponseMarshallable, Marshaller}
// json marshalling
import akka.http.scaladsl.marshallers.sprayjson.SprayJsonSupport
import spray.json._
import spray.json.DefaultJsonProtocol._
implicit val departmentFormat = DefaultJsonProtocol.jsonFormat1(department)
val departmentJsonMarshaller = SprayJsonSupport.sprayJsonMarshaller[department]
// xml marshalling, need to write a custom Marshaller
// there are several MediaTypes for XML such as `application/xml` and `text/xml`, you'll have to choose the one you need.
val departmentXmlMarshaller = Marshaller.StringMarshaller.wrap(MediaTypes.`application/xml`)((d: department) => {
import com.fasterxml.jackson.dataformat.xml.XmlMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
val mapper = new XmlMapper()
mapper.registerModule(DefaultScalaModule)
mapper.writeValueAsString(d)
})
private val route =
(pathPrefix("employee") & get) {
path(Segment) { id =>
parameters('department ? "") { (flag) =>
extractUri { uri => {
flag match {
case "hr" => {
// explicitly use the XML marshaller
complete(ToResponseMarshallable(department(name = flag))(departmentXmlMarshaller))
}
case "tech" => {
// explicitly use the JSON marshaller
complete(ToResponseMarshallable(department(name = flag))(departmentJsonMarshaller))
}
}
}
}
}
}
}
Note that for Jackson XML serializer to work correctly the department class should be a top level class or you'll get a cryptic error about bad root name.
Akka Http already has built in content type negotiation. Ideally you should just use that by having a marshaller that knows how to turn your department into either xml or json and having the client set the Accept header.
However it sounds like maybe you can't get your client to do that, so here's what you can do, assuming you have already made a ToEntityMarshaller[department] for both xml and json using ScalaXmlSupport and SprayJsonSupport.
val toXmlEntityMarshaller: ToEntityMarshaller[department] = ???
val toJsonEntityMarshaller: ToEntityMarshaller[department] = ???
implicit val departmentMarshaller = Marshaller.oneOf(toJsonEntityMarshaller, toXmlEntityMarshaller)
def route =
parameters("department") { departmentName =>
// capture the Accept header in case the client did request one
optionalHeaderValueByType[Accept] { maybeAcceptHeader =>
mapRequest ( _
.removeHeader(Accept.name)
// set the Accept header based on the department
.addHeader(maybeAcceptHeader.getOrElse(
Accept(departmentName match {
case "hr" ⇒ MediaTypes.`application/xml`
case "tech" ⇒ MediaTypes.`application/json`
})
))
) (
// none of our logic code is concerned with the response type
complete(department(departmentName))
)
}
}

Loss of type info in servlet code

I have a simple flash implementation for use with Jersey that looks like this:
#PostConstruct def before { flash.rotateIn }
#PreDestroy def after { flash.rotateOut }
object flash {
val KeyNow = "local.flash.now"
val KeyNext = "local.flash.next"
// case class Wrapper(wrapped: Map[String, Seq[String]])
case class Wrapper(wrapped: String)
def rotateIn {
for {
session <- Option(request.getSession(false))
obj <- Option(session.getAttribute(KeyNext))
} {
request.setAttribute(KeyNow, obj)
session.removeAttribute(KeyNext)
}
}
def rotateOut {
for (obj <- Option(request.getAttribute(KeyNext))) {
request.getSession.setAttribute(KeyNext, obj)
}
}
def now = Option(request.getAttribute(KeyNow)) match {
case Some(x: Wrapper) => x.wrapped
case Some(x) if x.isInstanceOf[Wrapper] => "WHAT"
case _ => "NOPE"
}
def next(value: String) {
request.setAttribute(KeyNext, Wrapper(value))
}
}
I have simplified it here somewhat, but it lets me set a value for flash with flash.next and read the current flash value with flash.now.
The weird thing is that my now value is always "WHAT". If I do something similar in my REPL, I don't have the same issues:
val req = new org.springframework.mock.web.MockHttpServletRequest
val res = req.getSession
res.setAttribute("foo", Wrapper("foo"))
req.setAttribute("foo", res.getAttribute("foo"))
// Is not None
Option(req.getAttribute("foo")).collect { case x: Wrapper => x }
Am I missing something obvious?
EDIT
I've added a minimal example webapp replicating this issue at https://github.com/kardeiz/sc-issue-20160229.
I tried your example. Check my answer for your other question for details how pattern matching works in this case.
In short, as you Wrapper is an inner class, patter matching also checks the "outer class" reference. It seems that depending on the application server implementation Router.flash can be different instance for each request, so pattern matching fails.
Simple fix for that is to make Wrapper top-level class, so it doesn't have reference to any other class.

How to transform submitted json in Play 2.0?

I'm trying to build a REST API using play 2.0. I have a User case class that contains some fields (like username & password) that shouldn't be updatable by the updateMember method.
Is there a good, functional way, of dealing with multiple Options somehow, because request.body.asJson returns an Option[JsValue], and my user lookup also returns an Option:
package controllers.api
import org.joda.time.LocalDate
import play.api.Play.current
import play.api.db.slick.{DB, Session}
import play.api.mvc._
import play.api.libs.json._
import play.api.libs.functional.syntax._
import models.{Gender, User, UserId}
import repositories.UserRepository
object Member extends Controller {
def updateMember(id: Long) = Action {
DB.withSession {
implicit session: Session =>
val json: JsValue = request.body.asJson // how to deal with this?
val repository = new UserRepository
repository.findById(new UserId(id)).map {
user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
json.transform(usernameAppender) // transform not found
Ok("updated")
}.getOrElse(NotFound)
}
}
}
I could move the map call to where I try to parse the request, but then inside there I guess I'd need another map over the user Option like I already have. So in that style, I'd need a map per Option.
Is there a better way of dealing with multiple Options like this in FP?
You're basically dealing with a nested monad, and the main tool for working with such is flatMap, particularly if both options being None has the same semantic meaning to your program:
request.body.asJson.flatMap { requestJson =>
val repository = new UserRepository
repository.findById(new UserId(id)).map { user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
requestJson.transform(usernameAppender)
Ok("updated") // EDIT: Do you not want to return the JSON?
}
}.getOrElse(NotFound)
But it might also be the case that your Nones have different meanings, in which case, you probably just want to pattern match, and handle the error cases separately:
request.body.asJson match {
case Some(requestJson) =>
val repository = new UserRepository
repository.findById(new UserId(id)).map { user =>
def usernameAppender = __.json.update(
__.read[JsObject].map { o => o ++ Json.obj("username" -> user.username) }
)
requestJson.transform(usernameAppender)
Ok("updated")
}.getOrElse(NotFound)
case None => BadRequest // Or whatever you response makes sense for this case
}

Use Future in Spray Routing

I'm new to asynchronous programming. I read this tutorial http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html and was surprised by how effortless I can incorporate Future into the program. However, when I was using Future with Routing, the return type is kind of wrong.
get {
optionalCookie("commToken") {
case Some(commCookie) =>
val response = (MTurkerProgressActor ? Register).mapTo[..].map({...})
val result = Await.result(response, 5 seconds)
setCookie(HttpCookie("commToken", content = result._2.mturker.get.commToken)) {
complete(result._1, result._2.mturker.get)
}
case None => // ...
}
}
I really don't want to use Await (what's the point of asynchronous if I just block the thread and wait for 5 seconds?). I tried to use for-comprehension or flatMap and place the setCookie and complete actions inside, but the return type is unacceptable to Spray. For-comprehension returns "Unit", and flatMap returns a Future.
Since I need to set up this cookie, I need the data inside. Is Await the solution? Or is there a smatter way?
You can use the onSuccess directive:
get {
optionalCookie("commToken") { cookie =>
//....
val response = (MTurkerProgressActor ? Register).mapTo[..].map({...})
onSuccess(response) {
case (result, mTurkerResponse) =>
setCookie(HttpCookie("commToken", content = mTurkerResponse.mturker.get.commToken)) {
complete(result, mturkerResponse.mturker.get)
}
}
}
There's also onFailure and onComplete (for which you have to match on Success and Failure) See http://spray.io/documentation/1.2.1/spray-routing/future-directives/onComplete/
Also, instead of using get directly it's much more idiomatic to use map (I assume the mturker is an Option or something similar):
case (result, mTurkerResponse) =>
mTurkerResponse.mturker.map { mt =>
setCookie(HttpCookie("commToken", content = mt.commToken)) {
complete(result, mt)
}
}
You can also make a custom directive using this code -
case class ExceptionRejection(ex: Throwable) extends Rejection
protected def futureDirective[T](x: Future[T],
exceptionHandler: (Throwable) => Rejection = ExceptionRejection(_)) =
new Directive1[T] {
override def happly(f: (::[T, HNil]) => Route): Route = { ctx =>
x
.map(t => f(t :: HNil)(ctx))
.onFailure { case ex: Exception =>
ctx.reject(exceptionHandler(ex))
}
}
}
Example usage -
protected def getLogin(account: Account) = futureDirective(
logins.findById(account.id)
)
getAccount(access_token) { account =>
getLogin(account) { login =>
// ...
}
}