How to structure a RESTful API with Spray.io? - scala

When I'm using Spray.io to develop a RESTful API, how should I structure my application?
I already saw this answer on how to split a Spray application, but I'm not satisfied with it, since it doesn't seem to use the "one actor per request" approach. Can I forward requests from the root actor to other actors in my application based on paths and, inside these actors, define the related routes?
Thanks

You can certainly forward requests from one actor to another, based on paths or whatever else. Check out my example project (which is a fork of a fork of an example project):
https://github.com/gangstead/spray-moviedb/blob/master/src/main/scala/com/example/routes/ApiRouter.scala
Relavent code from the main actor that receives all requests and routes them to other actors that handle each service:
def receive = runRoute {
compressResponseIfRequested(){
alwaysCache(simpleCache) {
pathPrefix("movies") { ctx => asb.moviesRoute ! ctx } ~
pathPrefix("people") { ctx => asb.peopleRoute ! ctx }
} ~
pathPrefix("login") { ctx => asb.loginRoute ! ctx } ~
pathPrefix("account") { ctx => asb.accountRoute ! ctx }
}
}
And for example the movies route:
def receive = runRoute {
get {
parameters('query, 'page ? 1).as(TitleSearchQuery) { query =>
val titleSearchResults = ms.getTitleSearchResults(query)
complete(titleSearchResults)
}~
path(LongNumber) { movieId =>
val movie = ms.getMovie(movieId)
complete(movie)
}~
path(LongNumber / "cast") { movieId =>
val movieCast = ms.getMovieCast(movieId)
complete(movieCast)
}~
path(LongNumber / "trailers") { movieId =>
val trailers = ms.getTrailers(movieId)
complete(trailers)
}
}
}

I was struggling a lot with creating first full REST project. The examples I've found was on hello world level... I've read few blogs, few comments and I decided to create example project. It is based on scala/akka/spray/mysql
It full working example with websocket to notify clients that data was changed etc. You can check it out on https://github.com/vixxx123/scalasprayslickexample
Here is sample code of routing from that project:
val personCreateHandler = actorRefFactory.actorOf(RoundRobinPool(2).props(Props[CreateActor]), s"${TableName}CreateRouter")
val personPutHandler = actorRefFactory.actorOf(RoundRobinPool(5).props(Props[UpdateActor]), s"${TableName}PutRouter")
val personGetHandler = actorRefFactory.actorOf(RoundRobinPool(20).props(Props[GetActor]), s"${TableName}GetRouter")
val personDeleteHandler = actorRefFactory.actorOf(RoundRobinPool(2).props(Props[DeleteActor]), s"${TableName}DeleteRouter")
val userRoute =
pathPrefix("person") {
pathEnd {
get {
ctx => personGetHandler ! GetMessage(ctx, None)
} ~
post {
entity(as[Person]) {
entity =>
ctx => personCreateHandler ! CreateMessage(ctx, entity)
}
}
} ~
pathPrefix (IntNumber){
entityId => {
pathEnd {
get {
ctx => personGetHandler ! GetMessage(ctx, Some(entityId))
} ~ put {
entity(as[Person]) { entity =>
ctx => personPutHandler ! PutMessage(ctx, entity.copy(id = Some(entityId)))
}
} ~ delete {
ctx => personDeleteHandler ! DeleteMessage(ctx, entityId)
} ~ patch {
ctx => personPutHandler ! PatchMessage(ctx, entityId)
}
}
}
}
}
And sample from create actor handler:
override def receive: Receive = {
case CreateMessage(ctx, person) =>
val localCtx = ctx
connectionPool withSession {
implicit session =>
try {
val resId = PersonsIdReturning += person
val addedPerson = person.copy(id = Some(resId.asInstanceOf[Int]))
localCtx.complete(addedPerson)
publishAll(CreatePublishMessage(TableName, localCtx.request.uri + "/" + addedPerson.id.get, addedPerson))
L.debug(s"Person create success")
} catch {
case e: Exception =>
L.error(s"Ups cannot create person: ${e.getMessage}", e)
localCtx.complete(e)
}
}
}
There are still two important things missing: oauth2 and push notifications to specific user/connection via websocket

Related

Play Framework - Respond with JSON after uploading a file (multipartFormData)

I am using this code to upload an image on server , that i get from this link play upload
def upload = Action(parse.multipartFormData) { request =>
request.body
.file("picture")
.map { picture =>
val dataParts = request.body.dataParts;
val filename = Paths.get(picture.filename).getFileName
val fileSize = picture.fileSize
val contentType = picture.contentType
val picturePaths =
picture.ref.copyTo(
Paths.get(
s"/opt/docker/images/$filename"
),
replace = true
)
if (dataParts.get("firstPoint") == None) {
val pointlocation = new Point_LocationModel(
dataParts.get("step").get(0),
dataParts.get("backgroundTargetName").get(0),
dataParts.get("x").get(0),
dataParts.get("y").get(0),
dataParts.get("z").get(0),
dataParts.get("rotation").get(0),
dataParts.get("note").get(0),
dataParts.get("tag").get(0),
dataParts.get("work_session_id").get(0),
(picturePaths).toString
)
point_LocationRepository.create(pointlocation).map { data =>
Created(Json.toJson(data._2))
}
} else {
val jValuefirstPoint =
Json.parse(dataParts.get("firstPoint").get(0)).as[PointModel]
val jValuesecondPoint =
Json.parse(dataParts.get("secondPoint").get(0)).as[PointModel]
val pointlocation = new Point_LocationModel(
dataParts.get("step").get(0),
dataParts.get("backgroundTargetName").get(0),
Some(jValuefirstPoint),
Some(jValuesecondPoint),
dataParts.get("rotation").get(0),
dataParts.get("note").get(0),
dataParts.get("tag").get(0),
dataParts.get("work_session_id").get(0),
(picturePaths).toString
)
point_LocationRepository.create(pointlocation).map { data =>
logger.info(s"repoResponse: ${data}");
Created(Json.toJson(data._2))
}
}
Ok(s"picturePaths ${picturePaths}")
}
.getOrElse(Ok("Invalid Format"))
}
This code works very well, but on the response I want to get the response from the repository. How can i await for the response of the repository to return this?
Can you give me any idea how can i do it?
Thanks in advance.
If we simplify your code to the essential bits, you have:
def upload = Action(parse.multipartFormData) { request =>
request.body
.file("picture")
.map { picture =>
if (someConditionA) {
someBusinessLogicA().map { data =>
Created(Json.toJson(data._2))
}
} else {
someBusinessLogicB().map { data =>
Created(Json.toJson(data._2))
}
}
Ok(s"picturePaths ${picturePaths}")
}
.getOrElse(Ok("Invalid Format"))
}
There are 2 problems:
the return of your if/else is swallowed by the Ok(s"picturePaths ${picturePaths}")
assuming your business logic returns Future[_] you need to "propagate" the Future everywhere
Something like this should work:
def upload = Action.async(parse.multipartFormData) { request =>
request.body
.file("picture")
.map { picture =>
if (someConditionA) {
someBusinessLogicA().map { data =>
Created(Json.toJson(data._2))
}
} else {
someBusinessLogicB().map { data =>
Created(Json.toJson(data._2))
}
}
}
.getOrElse(Future.successful(Ok("Invalid Format")))
}
Notice the Action.async and the Future in the getOrElse.

Spray Routing Doesn't match anything

I have tried many things, but no matter what I do in my tests (which simply sends a PUT request to "create a user") the logs do not enter any of the pathPrefix and just go to the end and fail to match anything.
Can anyone offer insight? Below is the class I have written as well as the simple test (which doesn't even check anything yet)
I know overall it is rather rudimentary and I am not doing things great, but it's just something I threw together to the point where I felt i could do a few quick tests on the routing
package system
import akka.actor.{Props, ActorRef, ActorLogging}
import akka.util.Timeout
import spray.http.HttpHeader
import spray.routing.directives.OnCompleteFutureMagnet
import spray.routing.{RequestContext, HttpServiceActor, Route}
import spray.http.StatusCodes._
import scala.concurrent.duration._
import akka.pattern.ask
import system.F_BackBone._
import scala.util.{Failure, Success}
import language.postfixOps
class F_Listener(backbone: ActorRef) extends HttpServiceActor with ActorLogging {
import context.dispatcher
implicit val timeout = Timeout(5 seconds)
log.debug("beginning user log\n")
//TODO write the main listener spawner that acts as the main spray server and binds these listeners to connections
//TODO must watch the spray side http server actor and die with it
val route: Route = { uri =>
log.debug("request received " + uri.unmatchedPath)
pathPrefix("data") { req =>
log.debug("data path detected " + req.unmatchedPath + "\n")
get {
genericGet(req, GetImage) //URIs for actual data like pictures
} ~
path("uploadimage") {
put { req =>
genericPut(PutImage(req.request.entity))
}
}
} ~
pathPrefix("user") { req =>
log.debug("user path detected" + req.unmatchedPath + "\n")
get {
genericGet(req, GetUserInfo)
} ~
post {
genericPost(req, req.request.headers, UpdateUserData)
} ~
delete {
genericDelete(req, DeleteUser)
}
pathPrefix("newuser") {
put { req =>
genericPut(CreateUser(req.request))
}
}
} ~
pathPrefix("page") { req =>
log.debug("page path detected" + "\n")
get {
genericGet(req, GetPageInfo)
} ~
post {
genericPost(req, req.request.headers, UpdatePageData)
} ~
delete {
genericDelete(req, DeletePage)
}
path("newpage") {
put { req =>
genericPut(CreatePage(req.request))
}
}
} ~
pathPrefix("profile") { req =>
log.debug("profile path detected" + "\n")
get {
genericGet(req, GetProfileInfo)
} ~ //no need for "createprofile" they are created with the user and can be accessed through the JSON returned with that creation
post {
genericPost(req, req.request.headers, UpdateProfileData)
} //no need to delete profile, deleted with user
} ~
pathPrefix("picture") { req =>
log.debug("picture path detected" + "\n")
get {
genericGet(req, GetPictureInfo)
} ~ //same as for profile, when you upload an image the picture JSON is created
post {
genericPost(req, req.request.headers, UpdateImageData)
} ~
delete {
genericDelete(req, DeletePicture)
}
} ~
pathPrefix("album") { req =>
log.debug("album path detected" + "\n")
get {
genericGet(req, GetAlbumInfo)
} ~
post {
genericPost(req, req.request.headers, UpdateAlbumData)
} ~
delete {
genericDelete(req, DeleteAlbum)
}
path("createalbum") {
put { req =>
genericPut(CreateAlbum(req.request))
}
}
}
log.error("no path matched" + "\n")
complete(NotFound, "That resource does not exist")
}
/**
* goes inside of a spray routing "get" and completes the passed message to the backbone given the id
* it internally converts the remaining request path to a bigint, if this fails it completes with a failure
* #param req the reques who contains the string to be converted to bigint as an ID
* #param messageConstructor the case class message (constructor but can be passed with sugar) to compose the bigint into
* #return returns a route for thed spray routing to go through and side effects the complete needed
*/
def genericGet(req: RequestContext, messageConstructor: (BigInt) => GetInfo): Route = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) { //if this is the last element only
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig)).mapTo[String])) {
case Success(entityJson) =>
log.info("get completed successfully: " + messageConstructor + " " + "for " + idBig)
complete(entityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
/**
* same as above but for posts, I treid to write a more generic function to repeat rewriting code but it ended up just not being worth the thought
* #param req request who contains id to parse to bigint
* #param args arguments to change
* #param messageConstructor the message to send
* #return
*/
def genericPost(req: RequestContext, args: List[HttpHeader], messageConstructor: (BigInt, List[HttpHeader]) => PostInfo) = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) {
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig, args)).mapTo[String])) {
case Success(newEntityJson) =>
log.info("post completed successfully: " + messageConstructor + " for " + idBig)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
/**
* same as above except no need to parse a special message since the path tells all and this is for putting, so the fully constructed message gets passed here
* #param message constructed message to send to the backbone for handling
* #return
*/
def genericPut(message: PutInfo) = {
pathEnd {
onComplete(OnCompleteFutureMagnet((backbone ? message).mapTo[String])) {
case Success(newEntityJson) =>
log.info("put completed successfully: " + message)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "put failed: " + message)
complete(InternalServerError, "Error putting entity: " + ex.getMessage)
}
}
}
/**
* almost identical to get, should probobly only be one function
* #param req identical
* #param messageConstructor identical
* #return route
*/
def genericDelete(req: RequestContext, messageConstructor: (BigInt) => DeleteInfo) = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) {
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig)).mapTo[String])) {
case Success(newEntityJson) =>
log.info("delete completed successfully: " + messageConstructor + " for " + idBig)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
override def receive = runRoute(route)
}
object F_Listener {
def props(backbone: ActorRef) = Props(new F_Listener(backbone))
}
/*ideas for speed improvements:
parse out arguments before passing to backbone (might help with scaling to distributed system)
genericDelete, Post, Put, and Get are all pretty similar, some functional composition is probobly possible, esp for delete and get
*/
import akka.testkit.TestActorRef
import org.scalatest.{WordSpec}
import spray.http.HttpHeaders
import spray.testkit.ScalatestRouteTest
import system.F_Listener
class FakebookRoutingTests extends WordSpec with ScalatestRouteTest {
val route = TestActorRef(new F_Listener(null)).underlyingActor.route
//implicit val host = DefaultHostInfo(HttpHeaders.Host("fake.fakebook.com"), false)
"the route" should {
"succeed on path to create a user" in {
Put("/user/newuser") ~> route ~> check {
}
}
}
}
Removing req => helps. If I remember correctly, put, get, post etc. don't provide any arguments for directives. Same goes for path. Removing uri => may also be needed.
Update:
If req => is to stay, then all calls finalizing the request should be called on that object. So, for example, instead of complete(entityJson) there should be req.complete(entityJson) on the top-most context object.
For given code top-most block with context is the first one containing uri so somewhere in the code there should be a call to uri.complete.

return/break a function from the derived base class

In my REST API service layer, I have a class ProductService.
The following logic exists in all my functions: Do Validate, if validation fails i throw invalid exception, if passes, i continue to the next try-catch and throw general-error in case of failure:
def addProduct(request:AddProductRequest): BaseResponse[String] =
{
try
{
request.validate
}
catch
{
case ex: Exception => {
Logger.error("Failed to add product, Validation failed", ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return errorResponse
}
}
try
{
val addedProductId = productRepository.addProduct(request.language, request.tenantId, request.product)
DTOResponse(addedProductId)
}
catch
{
case ex: Exception => {
Logger.error("Failed to add product to tenant Id="+request.tenantId+" language="+request.language, ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().GeneralError
errorResponse.addMessage(ex.getMessage())
return errorResponse
}
}
}
Now, instead of repeating the request.validate with the same try and catch for all functions, i created a base class with the following function:
abstract class ServiceBase {
def validate[T](request:BaseRequest)
{
try
{
request.validate
}
catch
{
case ex: Exception => {
Logger.error("Validation failed", ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return errorResponse
}
}
}
So now, my addProduct(..) will look like:
validate(request)
..the rest of the code - the 2nd try-catch
This saves alot of lines.
The problem is that if validation fails, it will never return. I get the following errors in ServiceBase:
Multiple markers at this line
- enclosing method validate has result type Unit: return value discarded
- enclosing method validate has result type Unit: return value discarded
- a pure expression does nothing in statement position; you may be omitting necessary
parentheses
validate has no return type (and thus defaults to returning Unit), in ServiceBase your signature for validate should look like this:
def validate[T](request:BaseRequest): BaseResponse[String] =
(assuming you want to return a BaseResponse[String])
this may be useful to someone, someday, functional programming.. Did we say ^_^
Changed the ServiceBase validate to:
def validate[T](request:BaseRequest):Option[BaseResponse[T]] =
{
try
{
request.validate
None
}
catch
{
case ex: Exception => {
Logger.error("Validation failed", ex);
val errorResponse:ErrorResponse[T] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return Some(errorResponse)
}
}
}
And now i do:
def getProducts(request:GetProductsRequest) :BaseResponse[ProductSearchResults] =
{
validate[ProductSearchResults](request).getOrElse(
{
try
{
val products = productRepository.getProducts(request.language,request.tenantId,request.productIds)
val foundProducts = for (product <- products) yield (product.id)
val notFoundProducts = request.productIds filterNot (foundProducts.toSet)
val responseWrapper = new ProductSearchResults(products, notFoundProducts)
DTOResponse(responseWrapper)
}
catch
{
case ex: Exception => {
Logger.error("Failed to get products from tenant Id="+request.tenantId+" language="+request.language, ex);
val errorResponse:ErrorResponse[ProductSearchResults] = new ErrorResponseList().GeneralError
errorResponse.addMessage(ex.getMessage())
return errorResponse
}
}})
}

How to create custom directive to reuse routes?

I have a route snippet that I want to reuse in multiple scenarios:
val dirSegment = "licenses"
path( dirSegment ~ PathEnd ) {
redirect( dirSegment + "/", StatusCodes.MovedPermanently )
} ~
pathPrefix(dirSegment) {
path("") {
/* do something */
}
}
I'd like to turn this in to a directive (or parameterizable route?) where I can specify the value of the dirSegment val and arbitrary further routing/code in place of path("") { /* do something */ } white retaining the redirect behavior, looking something like the following:
directoryPath("licenses") {
path("") {
/* do something */
}
} ~
directoryPath("about") {
path("") {
/* do somthing else */
}
}
Whereas that would have equivalent behavior to the following without all the repetition:
val dirSegment = "licenses"
val anotherDir = "About"
path( dirSegment ~ PathEnd ) {
redirect(dirSegment + "/", StatusCodes.MovedPermanently )
} ~
pathPrefix(dirSegment) {
path("") {
/* do something */
}
} ~
path( anotherDir ~ PathEnd ) {
redirect(anotherDir + "/", StatusCodes.MovedPermanently )
} ~
pathPrefix(anotherDir) {
path("") {
/* do something else */
}
}
Note that this question was inspired by some of the discussion in How do I automatically add slash to end of a url in spray routing?
You'll need to write a custom directive for this.
// additional imports you may need
import shapeless.HNil
import spray.http.StatusCodes
import spray.routing.Directive0
import spray.routing.PathMatcher
Now that that's out of the way:
/**
* Spray's PathEnd matches trailing optional slashes... we can't have that
* otherwise it will cause a redirect loop.
*/
object PathEndNoSlash extends PathMatcher[HNil] {
def apply(path: Path) = path match {
case Path.Empty ⇒ PathMatcher.Matched.Empty
case _ ⇒ PathMatcher.Unmatched
}
}
/**
* Custom directive that uses a redirect to add a trailing slashe to segment
* if the slash isn't present.
*/
def directoryPath(segment: String) = new Directive0 {
def happly(f: HNil ⇒ Route) =
// first, the redirect
pathPrefix(segment ~ PathEndNoSlash) {
redirect("/" + segment + "/", StatusCodes.MovedPermanently) } ~
// delegate actual work to nested directives
pathPrefix(segment).happly(f)
}
Usage:
directoryPath("about") {
path("a") {
complete {
"this is /about/a"
}
} ~ path("b") {
complete {
"this is /about/b"
}
} ~ path(PathEnd) {
complete {
"this is /about/"
}
}
}
If a user accesses /about, they will be forwarded to /about/ and see "this is /about/". The nested paths a and b work as expected (that is, without redirects of their own).
Note: This solution is for Spray 1.2.

Subsequent Call to Actor Freezes Program

I put together the code below; the intent was to have a non-blocking server accept a connection and then pass off this connection to an actor for further processing. This works the first time through, but on the subsequent request the program freezes at conServ ! servSoc.accept. Any ideas why this is happening?
import java.net._
import java.io._
import java.nio._
import java.nio.channels._
import java.util._
import scala.actors.Actor
import scala.actors.Actor._
def test() = {
var channel: ServerSocketChannel = null
val isa: InetSocketAddress = new InetSocketAddress(23)
val conServ = actor {
react {
case conn: Socket => {
try {
var pw: PrintWriter = new PrintWriter(conn.getOutputStream(), true)
pw.println("Current time: " + new Date)
pw.close
conn.close
} catch {
case ioe: IOException => println("IOException: " + ioe.getMessage)
case e: Exception => println("Exception: " + e.getMessage)
}
}
}
}
try {
channel = ServerSocketChannel.open
channel.configureBlocking(false)
channel.socket().bind(isa)
var selector: Selector = Selector.open
channel.register(selector, SelectionKey.OP_ACCEPT)
println("** Server ready for requests **")
while (true) {
if (selector.select > 0) {
var selKeys: Set[SelectionKey] = selector.selectedKeys
var selIt: Iterator[SelectionKey] = selKeys.iterator
while (selIt.hasNext) {
var key: SelectionKey = selIt.next.asInstanceOf[SelectionKey]
selIt.remove
if (key.isAcceptable) {
var ssc: ServerSocketChannel = key.channel.asInstanceOf[ServerSocketChannel]
var servSoc: ServerSocket = ssc.socket
try {
conServ ! servSoc.accept
} catch {
case ioe: IOException => println(ioe.printStackTrace)
}
}
}
} else {
continue
}
}
} catch {
case ioe: IOException => println("Could not listen to port 23. " + ioe.printStackTrace)
case e: Exception => println("Error: " + e.printStackTrace)
}
}
test
Enclose your react in a loop block like this:
val conServ = actor {
loop {
react {
// ...
}
}
}
What happens now, is that your actor is started, processes the first message and is not "reacting" again to process additional message from its queue.
See An actor's act method that uses loop.
This is what an actor do, treating one message at the time. What you want is a separate thread to handle each request. For this you can try using Futures.