I put together the code below; the intent was to have a non-blocking server accept a connection and then pass off this connection to an actor for further processing. This works the first time through, but on the subsequent request the program freezes at conServ ! servSoc.accept. Any ideas why this is happening?
import java.net._
import java.io._
import java.nio._
import java.nio.channels._
import java.util._
import scala.actors.Actor
import scala.actors.Actor._
def test() = {
var channel: ServerSocketChannel = null
val isa: InetSocketAddress = new InetSocketAddress(23)
val conServ = actor {
react {
case conn: Socket => {
try {
var pw: PrintWriter = new PrintWriter(conn.getOutputStream(), true)
pw.println("Current time: " + new Date)
pw.close
conn.close
} catch {
case ioe: IOException => println("IOException: " + ioe.getMessage)
case e: Exception => println("Exception: " + e.getMessage)
}
}
}
}
try {
channel = ServerSocketChannel.open
channel.configureBlocking(false)
channel.socket().bind(isa)
var selector: Selector = Selector.open
channel.register(selector, SelectionKey.OP_ACCEPT)
println("** Server ready for requests **")
while (true) {
if (selector.select > 0) {
var selKeys: Set[SelectionKey] = selector.selectedKeys
var selIt: Iterator[SelectionKey] = selKeys.iterator
while (selIt.hasNext) {
var key: SelectionKey = selIt.next.asInstanceOf[SelectionKey]
selIt.remove
if (key.isAcceptable) {
var ssc: ServerSocketChannel = key.channel.asInstanceOf[ServerSocketChannel]
var servSoc: ServerSocket = ssc.socket
try {
conServ ! servSoc.accept
} catch {
case ioe: IOException => println(ioe.printStackTrace)
}
}
}
} else {
continue
}
}
} catch {
case ioe: IOException => println("Could not listen to port 23. " + ioe.printStackTrace)
case e: Exception => println("Error: " + e.printStackTrace)
}
}
test
Enclose your react in a loop block like this:
val conServ = actor {
loop {
react {
// ...
}
}
}
What happens now, is that your actor is started, processes the first message and is not "reacting" again to process additional message from its queue.
See An actor's act method that uses loop.
This is what an actor do, treating one message at the time. What you want is a separate thread to handle each request. For this you can try using Futures.
Related
When using a PublishSubject in conjunction with blockingGet(), there seems to be a race condition where a subscriber does not receive the events.
I attached a basic JUnit test in Kotlin, which has 2 methods.
rxTestBroken() shows the broken behavior with a PublishSubject.
rxTestOk() shows that everything works fine with a BehaviorSubject, because the latter replays the last event in case the subscriber is not subscribed in time.
Where does this race condition come from and is using a BehaviorSubject the correct fix?
import io.reactivex.Single
import io.reactivex.subjects.BehaviorSubject
import io.reactivex.subjects.PublishSubject
import io.reactivex.subjects.Subject
import org.assertj.core.api.Assertions.assertThat
import org.junit.Test
import java.util.concurrent.TimeUnit
class StateMachine(val stateSubject: Subject<Int>) {
companion object {
val STATE_IDLE = 1
val STATE_READY = 2
}
val output = 10L
var currentState = STATE_IDLE
fun scheduleNextState(nextState: Int) {
Thread(Runnable {
currentState = nextState
stateSubject.onNext(currentState)
}).start()
}
fun start() = scheduleNextState(STATE_READY)
fun stop() = scheduleNextState(STATE_IDLE)
}
class RxTest {
fun stateOutput(stateSubject: Subject<Int>): Single<Int> {
val stateMachine = StateMachine(stateSubject)
val waitForIdle = stateSubject
.startWith(stateMachine.currentState)
.doOnNext {
if (it != StateMachine.STATE_IDLE) { stateMachine.stop() }
}
.filter { it == StateMachine.STATE_IDLE }
.firstOrError()
val scanFile = stateSubject
.doOnSubscribe { stateMachine.start() }
.filter {
when (it) {
StateMachine.STATE_READY -> true
StateMachine.STATE_IDLE -> false
else -> throw RuntimeException("Wrong state $it")
}
}
.firstOrError()
.map { stateMachine.output.toInt() }
.doFinally { stateMachine.stop() }
return waitForIdle.flatMap { scanFile }.timeout(1, TimeUnit.SECONDS).onErrorReturnItem(-1)
}
#Test
fun rxTestBroken() {
for (i in 1..10000) {
assertThat(stateOutput(PublishSubject.create<Int>()).blockingGet())
.withFailMessage("worked $i times")
.isEqualTo(10)
}
}
#Test
fun rxTestOk() {
for (i in 1..10000) {
assertThat(stateOutput(BehaviorSubject.createDefault(StateMachine.STATE_IDLE)).blockingGet())
.withFailMessage("worked $i times")
.isEqualTo(10)
}
}
}
how to catch a exception and do something in catch block when I save data into hdfs fail. like this:
try {
item.map(
r => doSome(r).saveAsTextFiles(outputPath + "/data")
} catch {
case e: Exception => {
val failMessage = "Exception from output part" + e.getClass + "\t" + e.getMessage
println("The exception is executed")
update(aaa)
}
} finally {
mc.close()
}
I want to update some status when the save action throw exception. how to do it ?
Use NonFatal hope it will solve your issue
import scala.util.control.NonFatal
try {
item.map(
r => doSome(r).saveAsTextFiles(outputPath + "/data")
} catch {
case NonFatal(error) => {
val failMessage = s"Exception from output part $error"
println("The exception is executed")
update(aaa)
}
} finally {
mc.close()
}
I have tried many things, but no matter what I do in my tests (which simply sends a PUT request to "create a user") the logs do not enter any of the pathPrefix and just go to the end and fail to match anything.
Can anyone offer insight? Below is the class I have written as well as the simple test (which doesn't even check anything yet)
I know overall it is rather rudimentary and I am not doing things great, but it's just something I threw together to the point where I felt i could do a few quick tests on the routing
package system
import akka.actor.{Props, ActorRef, ActorLogging}
import akka.util.Timeout
import spray.http.HttpHeader
import spray.routing.directives.OnCompleteFutureMagnet
import spray.routing.{RequestContext, HttpServiceActor, Route}
import spray.http.StatusCodes._
import scala.concurrent.duration._
import akka.pattern.ask
import system.F_BackBone._
import scala.util.{Failure, Success}
import language.postfixOps
class F_Listener(backbone: ActorRef) extends HttpServiceActor with ActorLogging {
import context.dispatcher
implicit val timeout = Timeout(5 seconds)
log.debug("beginning user log\n")
//TODO write the main listener spawner that acts as the main spray server and binds these listeners to connections
//TODO must watch the spray side http server actor and die with it
val route: Route = { uri =>
log.debug("request received " + uri.unmatchedPath)
pathPrefix("data") { req =>
log.debug("data path detected " + req.unmatchedPath + "\n")
get {
genericGet(req, GetImage) //URIs for actual data like pictures
} ~
path("uploadimage") {
put { req =>
genericPut(PutImage(req.request.entity))
}
}
} ~
pathPrefix("user") { req =>
log.debug("user path detected" + req.unmatchedPath + "\n")
get {
genericGet(req, GetUserInfo)
} ~
post {
genericPost(req, req.request.headers, UpdateUserData)
} ~
delete {
genericDelete(req, DeleteUser)
}
pathPrefix("newuser") {
put { req =>
genericPut(CreateUser(req.request))
}
}
} ~
pathPrefix("page") { req =>
log.debug("page path detected" + "\n")
get {
genericGet(req, GetPageInfo)
} ~
post {
genericPost(req, req.request.headers, UpdatePageData)
} ~
delete {
genericDelete(req, DeletePage)
}
path("newpage") {
put { req =>
genericPut(CreatePage(req.request))
}
}
} ~
pathPrefix("profile") { req =>
log.debug("profile path detected" + "\n")
get {
genericGet(req, GetProfileInfo)
} ~ //no need for "createprofile" they are created with the user and can be accessed through the JSON returned with that creation
post {
genericPost(req, req.request.headers, UpdateProfileData)
} //no need to delete profile, deleted with user
} ~
pathPrefix("picture") { req =>
log.debug("picture path detected" + "\n")
get {
genericGet(req, GetPictureInfo)
} ~ //same as for profile, when you upload an image the picture JSON is created
post {
genericPost(req, req.request.headers, UpdateImageData)
} ~
delete {
genericDelete(req, DeletePicture)
}
} ~
pathPrefix("album") { req =>
log.debug("album path detected" + "\n")
get {
genericGet(req, GetAlbumInfo)
} ~
post {
genericPost(req, req.request.headers, UpdateAlbumData)
} ~
delete {
genericDelete(req, DeleteAlbum)
}
path("createalbum") {
put { req =>
genericPut(CreateAlbum(req.request))
}
}
}
log.error("no path matched" + "\n")
complete(NotFound, "That resource does not exist")
}
/**
* goes inside of a spray routing "get" and completes the passed message to the backbone given the id
* it internally converts the remaining request path to a bigint, if this fails it completes with a failure
* #param req the reques who contains the string to be converted to bigint as an ID
* #param messageConstructor the case class message (constructor but can be passed with sugar) to compose the bigint into
* #return returns a route for thed spray routing to go through and side effects the complete needed
*/
def genericGet(req: RequestContext, messageConstructor: (BigInt) => GetInfo): Route = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) { //if this is the last element only
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig)).mapTo[String])) {
case Success(entityJson) =>
log.info("get completed successfully: " + messageConstructor + " " + "for " + idBig)
complete(entityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
/**
* same as above but for posts, I treid to write a more generic function to repeat rewriting code but it ended up just not being worth the thought
* #param req request who contains id to parse to bigint
* #param args arguments to change
* #param messageConstructor the message to send
* #return
*/
def genericPost(req: RequestContext, args: List[HttpHeader], messageConstructor: (BigInt, List[HttpHeader]) => PostInfo) = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) {
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig, args)).mapTo[String])) {
case Success(newEntityJson) =>
log.info("post completed successfully: " + messageConstructor + " for " + idBig)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
/**
* same as above except no need to parse a special message since the path tells all and this is for putting, so the fully constructed message gets passed here
* #param message constructed message to send to the backbone for handling
* #return
*/
def genericPut(message: PutInfo) = {
pathEnd {
onComplete(OnCompleteFutureMagnet((backbone ? message).mapTo[String])) {
case Success(newEntityJson) =>
log.info("put completed successfully: " + message)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "put failed: " + message)
complete(InternalServerError, "Error putting entity: " + ex.getMessage)
}
}
}
/**
* almost identical to get, should probobly only be one function
* #param req identical
* #param messageConstructor identical
* #return route
*/
def genericDelete(req: RequestContext, messageConstructor: (BigInt) => DeleteInfo) = {
val id = req.unmatchedPath.toString
if (!id.contains("/")) {
try {
val idBig = BigInt(id)
onComplete(OnCompleteFutureMagnet((backbone ? messageConstructor.apply(idBig)).mapTo[String])) {
case Success(newEntityJson) =>
log.info("delete completed successfully: " + messageConstructor + " for " + idBig)
complete(newEntityJson)
case Failure(ex) =>
log.error(ex, "get failed: " + messageConstructor + " for " + idBig)
complete(InternalServerError, "Request could not be completed: \n" + ex.getMessage)
}
} catch {
case e: NumberFormatException =>
log.info("illegally formatted id requested: " + id)
complete(BadRequest, "Numbers are formatted incorrectly")
}
}
else reject
}
override def receive = runRoute(route)
}
object F_Listener {
def props(backbone: ActorRef) = Props(new F_Listener(backbone))
}
/*ideas for speed improvements:
parse out arguments before passing to backbone (might help with scaling to distributed system)
genericDelete, Post, Put, and Get are all pretty similar, some functional composition is probobly possible, esp for delete and get
*/
import akka.testkit.TestActorRef
import org.scalatest.{WordSpec}
import spray.http.HttpHeaders
import spray.testkit.ScalatestRouteTest
import system.F_Listener
class FakebookRoutingTests extends WordSpec with ScalatestRouteTest {
val route = TestActorRef(new F_Listener(null)).underlyingActor.route
//implicit val host = DefaultHostInfo(HttpHeaders.Host("fake.fakebook.com"), false)
"the route" should {
"succeed on path to create a user" in {
Put("/user/newuser") ~> route ~> check {
}
}
}
}
Removing req => helps. If I remember correctly, put, get, post etc. don't provide any arguments for directives. Same goes for path. Removing uri => may also be needed.
Update:
If req => is to stay, then all calls finalizing the request should be called on that object. So, for example, instead of complete(entityJson) there should be req.complete(entityJson) on the top-most context object.
For given code top-most block with context is the first one containing uri so somewhere in the code there should be a call to uri.complete.
When I'm using Spray.io to develop a RESTful API, how should I structure my application?
I already saw this answer on how to split a Spray application, but I'm not satisfied with it, since it doesn't seem to use the "one actor per request" approach. Can I forward requests from the root actor to other actors in my application based on paths and, inside these actors, define the related routes?
Thanks
You can certainly forward requests from one actor to another, based on paths or whatever else. Check out my example project (which is a fork of a fork of an example project):
https://github.com/gangstead/spray-moviedb/blob/master/src/main/scala/com/example/routes/ApiRouter.scala
Relavent code from the main actor that receives all requests and routes them to other actors that handle each service:
def receive = runRoute {
compressResponseIfRequested(){
alwaysCache(simpleCache) {
pathPrefix("movies") { ctx => asb.moviesRoute ! ctx } ~
pathPrefix("people") { ctx => asb.peopleRoute ! ctx }
} ~
pathPrefix("login") { ctx => asb.loginRoute ! ctx } ~
pathPrefix("account") { ctx => asb.accountRoute ! ctx }
}
}
And for example the movies route:
def receive = runRoute {
get {
parameters('query, 'page ? 1).as(TitleSearchQuery) { query =>
val titleSearchResults = ms.getTitleSearchResults(query)
complete(titleSearchResults)
}~
path(LongNumber) { movieId =>
val movie = ms.getMovie(movieId)
complete(movie)
}~
path(LongNumber / "cast") { movieId =>
val movieCast = ms.getMovieCast(movieId)
complete(movieCast)
}~
path(LongNumber / "trailers") { movieId =>
val trailers = ms.getTrailers(movieId)
complete(trailers)
}
}
}
I was struggling a lot with creating first full REST project. The examples I've found was on hello world level... I've read few blogs, few comments and I decided to create example project. It is based on scala/akka/spray/mysql
It full working example with websocket to notify clients that data was changed etc. You can check it out on https://github.com/vixxx123/scalasprayslickexample
Here is sample code of routing from that project:
val personCreateHandler = actorRefFactory.actorOf(RoundRobinPool(2).props(Props[CreateActor]), s"${TableName}CreateRouter")
val personPutHandler = actorRefFactory.actorOf(RoundRobinPool(5).props(Props[UpdateActor]), s"${TableName}PutRouter")
val personGetHandler = actorRefFactory.actorOf(RoundRobinPool(20).props(Props[GetActor]), s"${TableName}GetRouter")
val personDeleteHandler = actorRefFactory.actorOf(RoundRobinPool(2).props(Props[DeleteActor]), s"${TableName}DeleteRouter")
val userRoute =
pathPrefix("person") {
pathEnd {
get {
ctx => personGetHandler ! GetMessage(ctx, None)
} ~
post {
entity(as[Person]) {
entity =>
ctx => personCreateHandler ! CreateMessage(ctx, entity)
}
}
} ~
pathPrefix (IntNumber){
entityId => {
pathEnd {
get {
ctx => personGetHandler ! GetMessage(ctx, Some(entityId))
} ~ put {
entity(as[Person]) { entity =>
ctx => personPutHandler ! PutMessage(ctx, entity.copy(id = Some(entityId)))
}
} ~ delete {
ctx => personDeleteHandler ! DeleteMessage(ctx, entityId)
} ~ patch {
ctx => personPutHandler ! PatchMessage(ctx, entityId)
}
}
}
}
}
And sample from create actor handler:
override def receive: Receive = {
case CreateMessage(ctx, person) =>
val localCtx = ctx
connectionPool withSession {
implicit session =>
try {
val resId = PersonsIdReturning += person
val addedPerson = person.copy(id = Some(resId.asInstanceOf[Int]))
localCtx.complete(addedPerson)
publishAll(CreatePublishMessage(TableName, localCtx.request.uri + "/" + addedPerson.id.get, addedPerson))
L.debug(s"Person create success")
} catch {
case e: Exception =>
L.error(s"Ups cannot create person: ${e.getMessage}", e)
localCtx.complete(e)
}
}
}
There are still two important things missing: oauth2 and push notifications to specific user/connection via websocket
In my REST API service layer, I have a class ProductService.
The following logic exists in all my functions: Do Validate, if validation fails i throw invalid exception, if passes, i continue to the next try-catch and throw general-error in case of failure:
def addProduct(request:AddProductRequest): BaseResponse[String] =
{
try
{
request.validate
}
catch
{
case ex: Exception => {
Logger.error("Failed to add product, Validation failed", ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return errorResponse
}
}
try
{
val addedProductId = productRepository.addProduct(request.language, request.tenantId, request.product)
DTOResponse(addedProductId)
}
catch
{
case ex: Exception => {
Logger.error("Failed to add product to tenant Id="+request.tenantId+" language="+request.language, ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().GeneralError
errorResponse.addMessage(ex.getMessage())
return errorResponse
}
}
}
Now, instead of repeating the request.validate with the same try and catch for all functions, i created a base class with the following function:
abstract class ServiceBase {
def validate[T](request:BaseRequest)
{
try
{
request.validate
}
catch
{
case ex: Exception => {
Logger.error("Validation failed", ex);
val errorResponse:ErrorResponse[String] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return errorResponse
}
}
}
So now, my addProduct(..) will look like:
validate(request)
..the rest of the code - the 2nd try-catch
This saves alot of lines.
The problem is that if validation fails, it will never return. I get the following errors in ServiceBase:
Multiple markers at this line
- enclosing method validate has result type Unit: return value discarded
- enclosing method validate has result type Unit: return value discarded
- a pure expression does nothing in statement position; you may be omitting necessary
parentheses
validate has no return type (and thus defaults to returning Unit), in ServiceBase your signature for validate should look like this:
def validate[T](request:BaseRequest): BaseResponse[String] =
(assuming you want to return a BaseResponse[String])
this may be useful to someone, someday, functional programming.. Did we say ^_^
Changed the ServiceBase validate to:
def validate[T](request:BaseRequest):Option[BaseResponse[T]] =
{
try
{
request.validate
None
}
catch
{
case ex: Exception => {
Logger.error("Validation failed", ex);
val errorResponse:ErrorResponse[T] = new ErrorResponseList().InvalidParameters
errorResponse.addMessage(ex.getMessage)
return Some(errorResponse)
}
}
}
And now i do:
def getProducts(request:GetProductsRequest) :BaseResponse[ProductSearchResults] =
{
validate[ProductSearchResults](request).getOrElse(
{
try
{
val products = productRepository.getProducts(request.language,request.tenantId,request.productIds)
val foundProducts = for (product <- products) yield (product.id)
val notFoundProducts = request.productIds filterNot (foundProducts.toSet)
val responseWrapper = new ProductSearchResults(products, notFoundProducts)
DTOResponse(responseWrapper)
}
catch
{
case ex: Exception => {
Logger.error("Failed to get products from tenant Id="+request.tenantId+" language="+request.language, ex);
val errorResponse:ErrorResponse[ProductSearchResults] = new ErrorResponseList().GeneralError
errorResponse.addMessage(ex.getMessage())
return errorResponse
}
}})
}