I have a akka cluster of microservices running on nodes. I'm trying to use the Jackson serializer for application specific messages exchanged between the nodes (microservices).
My code is written in scala, not Java
I'm using scala 'case class' as akka messages exchanged between actors and each message (case class) have a val LOG:org.slf4j.Logger built in to log specific business logic processing information.
When I send messages between nodes I'm getting the exception
WARN akka.remote.artery.Deserializer - Failed to deserialize message from [akka://MyCluster#127.0.0.1:25251] with serializer id [33] and manifest [com...MyCaseClassMessage]. com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of org.slf4j.Logger (no Creators, like default constructor, exist): abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
at [Source: ..."[truncated 'n' bytes]; line: 'n', column: 'n'] (through reference chain: com...MyCaseClassMessage["LOG"])
My case class essentially is something like:
case class MyCaseClassMessage()
extends CborSerializable {
override val LOG:Logger = LoggerFactory.getLogger(classOf[MyCaseClassMessage])
val businessLogic:Array[Byte] = ...
def apply():Array[Byte] = ...
}
I have no idea of how to specify to Jackson how to serialize and (or) deserialize a "val LOG:Logger" in my case class. I just know that if I remove my Logger, substituing it to (for example) println("message") I don't have any problem with serialization and deserialization.
Any help?
Because Jackson relies on reflection and does not understand the convention in Scala case classes that only the constructor parameters are required for defining the message, it will attempt to serialize every field of the object.
The LOG field can be ignored by Jackson by annotating it with an #JsonIgnore annotation (com.fasterxml.jackson.annotation.JsonIgnore):
#JsonIgnore
override val LOG: Logger = LoggerFactory.getLogger(classOf[MyCaseClassMessage])
Related
Let's say I have DAO Actors (CassDaoActor, VerticaDaoActor, etc) that respond to message 'Read'.
First of all, is there a way to express a interface or abstract class that defines the message 'Read' that extending actors should implement?
Now assume it's only at runtime that I could get to know which Actor needs to be created based on the configured db. For example, if configured db is cassandra, I need to create CassDaoActor, etc. This apparently is a typical use case for Factory Method Pattern as we know. I want to understand how can we implement such a thing? Evidently we can't pass "context" around since it looses content outside the scope of the actor.
Please suggest.
What I have tried so far is that I am returning respective props based on the configured db to the actor within which I need to create these actors.
object `package` {
val CASS = "cass"
val VERTICA = "vertica"
def getDAOProps(db: String): Props = db match {
case CASS => CassDaoActor.props
case VERTICA => VerticaDaoActor.props
}
}
// SupervisorActor
val db = configuredDb()
context.actorOf(getDAOProps(db), db)
Currently I am having a problem of recognizing a receiving serialized object which is sent through network using UDP.
I have an abstract class called MsgType:
sealed abstract class MsgType
case class Msg(message : String) extends MsgType
case class End() extends MsgType
in that, Msg means a normal message whilst End means a termination request at client side.
===========================================================================
At server side, I have a function call isMessage to detect whether it's a normal message or the termination request:
def isMessage(message: AnyRef): Boolean = {
message match{
case End => false
case Msg(message) => true
}
}
===========================================================================
Here is the code using Kryo for receiving the message sent from client:
val inputString = kyro.readObject(input, classOf[MsgType])
println("incoming Message: " + isMessage(inputString))
However, when I run the code, there is an exception named:
Exception in thread "main" com.esotericsoftware.kryo.KryoException: Error
constructing instance of class: MsgType
I know it's because MsgType is an abstract class....
Could anyone suggest me a better solution to deal with this problem of recognizing the type of received serialized object?
Thanks and Best Regards,
Long.
Unfortunately Kryo doesn't work this way. It should know the type of object being deserialized before deserialization is started. It means you need to save this information in your serialized data as well. Conviently Kryo provides writeClassAndObject and readClassAndObject that do exactly that.
Also I expect that you'll have another issue related to the usage of the case classes. Unless you provided custom deserializators, Kryo will fail because your Msg case class doesn't have a default constructor (i.e. one with no parameters). You may consider using twitter chill - a Scala wrapper for Kryo. See alao Handling case classes in twitter chill (Scala interface to Kryo)?
I have the following scala object that I would like to serialize with protobuf:
case class Worker(id: String, ref: ActorRef)
My understanding is that Akka comes with a ProtobufSerializer that I can use to serialize from/to ActorRef. Therefore I defined the following message in a proto file:
message Worker {
string id = 1;
bytes ref = 2;
}
However the static methods ProtobufSerializer take an ExtentendActorSystem that I do not have at hand when I need to serialize those messages (only an ActorSystem through the context.system attribute of my actor). I am not sure how to actually serialize my object to protobuf and the other way around.
I am not sure in which cases it is true, but I was able to simply cast my ActorSystem to an ExtendedActorSystem. In my specific case (I am using Akka Persistence with Akka Singleton) it seems to work fine.
I am building a market simulator using Scala/Akka/Play. I have an Akka actor with two children. The children need to have specific types which I would like to specify as parameters.
Suppose that I have the following class definition...
case class SecuritiesMarket[A <: AuctionMechanismLike, C <: ClearingMechanismLike](instrument: Security) extends Actor
with ActorLogging {
val auctionMechanism: ActorRef = context.actorOf(Props[A], "auction-mechanism")
val clearingMechanism: ActorRef = context.actorOf(Props[C], "clearing-mechanism")
def receive: Receive = {
case order: OrderLike => auctionMechanism forward order
case fill: FillLike => clearingMechanism forward fill
}
}
Instances of this class can be created as follows...
val stockMarket = SecuritiesMarket[DoubleAuctionMechanism, CCPClearingMechanism](Security("GOOG"))
val derivativesMarket = SecuritiesMarket[BatchAuctionMechanism, BilateralClearingMechanism](Security("SomeDerivative"))
There are many possible combinations of auction mechanism types and clearing mechanism types that I might use when creating SecuritiesMarket instance for a particular model/simulation.
Can I specify the type parameters that I wish to use in a given simulation in the application.conf file?
I see two questions here.
Can I get a Class instance from a String?
Yes.
val cls: Class[DoubleAuctionMechanism] = Class.forName("your.app.DoubleAuctionMechanism").asInstanceOf[Class[DoubleAuctionMechanism]]
You would still need the cast, as forName returns Class[_].
Can I instantiate a type with type parameters are not known compile time?
Well sort of, but not really.
object SecuritiesMarket {
def apply[A, C](clsAuc: Class[A], clsClr: Class[C])(security: Security): SecuritiesMarket[A, C] = {
SecuritiesMarket[A, C](security)
}
}
I think auction mechanisms and clearing mechanisms are dependencies for SecurityMarket. I'm guessing you instantiate them in its constructor somehow (how?). If that's the case why not just pass them in as a constructor parameter?
Edit:
I don't see how I could create the child actors inside SecurityMarket
Answering this in the comments; Props[T] can also be written as Props[T](classOfT), which can be simplified as Props(classOfT). Those three are the same. So the following code:
val auctionMechanism: ActorRef = context.actorOf(Props[A], "auction-mechanism")
Can be replaced with:
val classOfA = Class.forName("path.to.A")
val auctionMechanism: ActorRef = context.actorOf(Props(classOfA), "auction-mechanism")
First, application.conf is a runtime artifact and its contents are as far as I know not normally parsed at compile time. When the file is parsed at runtime, the parser creates an instance of the class Config which then controls the Akka setup.
The Typesafe Config library project readme is quite nice and the linked documentation has all of the details:
https://github.com/typesafehub/config/blob/master/README.md.
Second, since template parameters are not available at runtime because of type erasure, you can't normally use application.conf to control templating. You could create a custom build step to parse application.conf and modify your code before compilation, but this is maybe not what you want. (And if you do want a custom build step, perhaps a different .conf would be appropriate.)
Instead you might try simply eliminating the type parameters for the securities market class. Then create a single, simple implementation of the auction and clearing actors. Implement these actors by reading the names of the respective mechanisms from application.conf, instantiating the configured mechanism reflectively, and delegating to the instantiated mechanism. The mechanism classes could be independent of Akka, which is perhaps nice if that's where you keep most of your logic?
Neo4j server provides a REST api dealing with Json format.
I use spring-data-neo4j to map a domain object (in Scala) to a neo4j node easily.
Here's an example of my User node:
#NodeEntity
class User(#Indexed #JsonProperty var id: UserId)
UserId being a value object:
final case class UserId(value: String) {
override def toString = value
}
object UserId {
def validation(userId: String): ValidationNel[IllegalUserFailure, UserId] =
Option(userId).map(_.trim).filter(!_.isEmpty).map(userId => new UserId(userId)).toSuccess(NonEmptyList[IllegalUserFailure](EmptyId))
}
At runtime, I got this error:
Execution exception[[RuntimeException: org.codehaus.jackson.map.JsonMappingException: No serializer found for class com.myApp.domain.model.user.UserId and no properties discovered to create BeanSerializer (to avoid exception, disable SerializationConfig.Feature.FAIL_ON_EMPTY_BEANS) ) (through reference chain: java.util.HashMap["value"])]]
Then, I came across this little article on the web, explaining a solution.
I ended up with this User class:
#NodeEntity
#JsonAutoDetect(Array(JsonMethod.NONE))
class User (#Indexed #JsonProperty var id: UserId)
I also tried to put the #JsonProperty on the UserId value object itself like this:
JsonAutoDetect(Array(JsonMethod.NONE))
final case class UserId(#JsonProperty value: String) {
override def toString = value
}
but I still get exactly the same error.
Did someone using Scala already had this issue?
I think your problem is that case classes don't generate the JavaBean boilerplate (or member fields annotated appropriately) Jackson expects. For example, I believe Scala generates this method in UserId:
public java.lang.String value();
Jackson doesn't know what to do with that. It isn't a recognizable field or a JavaBean-style method (i.e. getValue() or setValue()).
I haven't yet used it, but you might want to try jackson-module-scala as a more Scala-aware wrapper around Jackson. Another option is spray-json.
The reason for the error is that the version of Jackson that you appear to be using (1.x) is not matching up the "value" property to the constructor argument. When applied to constructors, #JsonProperty usually requires a name parameter to match up parameters to properties; with your current setup, I believe the following would work:
case class UserId #JsonCreator() (#JsonProperty("value") value: String)
The Jackson Scala Module also provides more support for Scala-isms, and might possibly handle the UserId class without any Jackson-specific annotations. That said, your version of Jackson is quite old (the current latest version is 2.3.1), and upgrading might not be trivial for your configuration.