We are getting random Serialization Exceptions when calling a Reliable Actor service like this:
Type: System.Runtime.Serialization.SerializationException
Message: Element 'http://schemas.microsoft.com/2003/10/Serialization/:anyType' contains data from a type that maps to the name 'urn:ServiceFabric.Communication:ServiceResponseMessageHeaders'. The deserializer has no knowledge of any type that maps to this name. Consider using a DataContractResolver if you are using DataContractSerializer or add the type corresponding to 'ServiceResponseMessageHeaders' to the list of known types - for example, by using the KnownTypeAttribute attribute or by adding it to the list of known types passed to the serializer.
ToString: System.Runtime.Serialization.SerializationException: Element 'http://schemas.microsoft.com/2003/10/Serialization/:anyType' contains data from a type that maps to the name
After research, many say this is remoting library v1, v2 mismatch problem. However, we have the v2 remoting assembly attribute defined in all actor.interface assemblies.
[assembly: FabricTransportActorRemotingProvider(RemotingListenerVersion = RemotingListenerVersion.V2, RemotingClientVersion = RemotingClientVersion.V2)]
Also the exception doesn't happen 100% of the time; it happens completely randomly. Usually in a pattern like this:
Failed to call actor service. Throw serialization exception.
After a few retries, we see an "Actor activated" message.
Calling the actor works now.
It seems happen before the actor is activated. Why does it happen like that?
Related
I’ve been trying to name a KStream using Kafka Streams DSL in Scala but I cannot find a way to name a processor in org.apache.kafka.streams.scala.kstream.Consumed.
Although there is a java method org.apache.kafka.streams.kstream.Consumed#as but it throws an exception. Does anyone knows what can be done?
ClassCastException invoking Processor. Do the Processor's input types match the deserialized types? Check the Serde setup and change the default Serdes in StreamConfig or provide correct Serdes via method parameters. Make sure the Processor can accept the deserialized input of type key: [B, and value: [B.
Note that although incorrect Serdes are a common cause of error, the cast exception might have another cause (in user code, for example). For example, if a processor wires in a store, but casts the generics incorrectly, a class cast exception could be raised during processing, but the cause would not be wrong Serdes.
org.apache.kafka.streams.errors.StreamsException: ClassCastException invoking Processor. Do the Processor's input types match the deserialized types? Check the Serde setup and change the default Serdes in StreamConfig or provide correct Serdes via method parameters. Make sure the Processor can accept the deserialized input of type key: [B, and value: [B.
Note that although incorrect Serdes are a common cause of error, the cast exception might have another cause (in user code, for example). For example, if a processor wires in a store, but casts the generics incorrectly, a class cast exception could be raised during processing, but the cause would not be wrong Serdes.
at org.apache.kafka.streams.processor.internals.ProcessorNode.process(ProcessorNode.java:146)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:236)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:216)
at org.apache.kafka.streams.processor.internals.ProcessorContextImpl.forward(ProcessorContextImpl.java:168)
at org.apache.kafka.streams.processor.internals.SourceNode.process(SourceNode.java:85)
at org.apache.kafka.streams.processor.internals.StreamTask.lambda$process$1(StreamTask.java:678)
at ...
EDIT:
The code that I used:
val someEvents = builder
.stream[String, String]("some_events")
(org.apache.kafka.streams.kstream.Consumed.as("some_event_stream"))
The code I should've used:
val someEvents = builder
.stream[String, String]("some_events")
(org.apache.kafka.streams.kstream.Consumed.as("some_event_stream")
.withKeySerde(Serdes.String())
.withValueSerde(Serdes.String()))
Initial code:
val someEvents = builder
.stream[String, String]("some_events")
(org.apache.kafka.streams.kstream.Consumed.as("some_event_stream"))
I made two mistakes:
Put Consumed parameter in a separate line. (todo needs explanation why it doesn't work in Scala)
Used Java method org.apache.kafka.streams.kstream.Consumed.as and didn't provide SerDes.
This works:
val someEvents = builder.stream[String, String]("some_events")(org.apache.kafka.streams.kstream.Consumed.as("some_event_stream").withKeySerde(Serdes.String()).withValueSerde(Serdes.String()))
We just upgraded our Spring to:
Spring boot: 2.1.0.RELEASE
Spring cloud: Greenwich.SR1
Spring integration kafka: 3.1.0.RELEASE
Spring kafka: 2.2.7.RELEASE
and we're using
Kafka 2.1.1
We have a topic that more than one type of class instances can be sent to, all extend from the same abstract class. Let's the abstract class named AbstractMessage and there are to subclasses MessageImpl1 and MessageImpl2.
We used to receive it in the consumer as an object (in order to write a log if a mistaken class has been received somehow) and then cast it to the relevant MessageImpl by using if(message instanceof MessageImpl){}
After the upgrade, all the messages were bound to String instead to their classes.
I read here that content-type=application/json binds to a pojo, but even though I added in both input and output it was bound to a string:
spring.cloud.stream.bindings.input.contentType=application/json
spring.cloud.stream.bindings.output.contentType=application/json
Trying to receive the MessageImpl directly got this error:
Caused by: com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot
construct instance of MessageImpl1 (no Creators, like default
construct, exist): cannot deserialize from Object value (no delegate-
or property-based Creator)
at [Source: (byte[])
Any idea how to fix it?
What version did you upgrade from? Show your code and configuration properties. Previous versions used Kryo serialization by default, which is now deprecated in favor of JSON, but your POJOs need to be json-friendly.
The Kryo serialization is deprecated, but you can add the converter.
See the documentation.
Provided MessageConverters
As mentioned earlier, the framework already provides a stack of MessageConverters to handle most common use cases. The following list describes the provided MessageConverters, in order of precedence (the first MessageConverter that works is used):
ApplicationJsonMessageMarshallingConverter: Variation of the org.springframework.messaging.converter.MappingJackson2MessageConverter. Supports conversion of the payload of the Message to/from POJO for cases when contentType is application/json (DEFAULT).
TupleJsonMessageConverter: DEPRECATED Supports conversion of the payload of the Message to/from org.springframework.tuple.Tuple.
ByteArrayMessageConverter: Supports conversion of the payload of the Message from byte[] to byte[] for cases when contentType is application/octet-stream. It is essentially a pass through and exists primarily for backward compatibility.
ObjectStringMessageConverter: Supports conversion of any type to a String when contentType is text/plain. It invokes Object’s toString() method or, if the payload is byte[], a new String(byte[]).
JavaSerializationMessageConverter: DEPRECATED Supports conversion based on java serialization when contentType is application/x-java-serialized-object.
KryoMessageConverter: DEPRECATED Supports conversion based on Kryo serialization when contentType is application/x-java-object.
JsonUnmarshallingConverter: Similar to the ApplicationJsonMessageMarshallingConverter. It supports conversion of any type when contentType is application/x-java-object. It expects the actual type information to be embedded in the contentType as an attribute (for example, application/x-java-object;type=foo.bar.Cat).
When no appropriate converter is found, the framework throws an exception. When that happens, you should check your code and configuration and ensure you did not miss anything (that is, ensure that you provided a contentType by using a binding or a header). However, most likely, you found some uncommon case (such as a custom contentType perhaps) and the current stack of provided MessageConverters does not know how to convert. If that is the case, you can add custom MessageConverter. See User-defined Message Converters.
I want to use sttp library with guice(with scalaguice wrapper) in my app. But seems it is not so easy to correctly bind things like SttpBackend[Try, Nothing]
SttpBackend.scala
Try[_] and Try[AnyRef] show some other errors, but still have no idea how it should be done correctly
the error I got:
kinds of the type arguments (scala.util.Try) do not conform to the expected kinds of the type parameters (type T).
[error] scala.util.Try's type parameters do not match type T's expected parameters:
[error] class Try has one type parameter, but type T has none
[error] bind[SttpBackend[Try, Nothing]].toProvider[SttpBackendProvider]
[error] ` ^
SttpBackendProvider looks like:
def get: SttpBackend[Try, Nothing] = TryHttpURLConnectionBackend(opts)
complete example in scastie
interesting that version scalaguice 4.1.0 show this error, but latest 4.2.2 shows error inside it with converting Nothing to JavaType
I believe you hit two different bugs in the Scala-Guice one of which is not fixed yet (and probably even not submitted yet).
To describe those issues I need a fast intro into how Guice and Scala-Guice work. Essentially what Guice do is have a mapping from type onto the factory method for an object of that type. To support some advanced features types are mapped onto some internal "keys" representation and then for each "key" Guice builds a way to construct a corresponding object. Also it is important that generics in Java are implemented using type erasure. That's why when you write something like:
bind(classOf[SttpBackend[Try, Nothing]]).toProvider(classOf[SttpBackendProvider])
in raw-Guice, the "key" actually becomes something like "com.softwaremill.sttp.SttpBackend". Luckily Guice developers have thought about this issue with generics and introduced TypeLiteral[T] so you can convey the information about generics.
Scala type system is more reach than in Java and it has some better reflection support from the compiler. Scala-Guice exploits it to map Scala-types on those more detailed keys automatically. Unfortunately it doesn't always work perfectly.
The first issue is the result of the facts that the type SttpBackend is defined as
trait SttpBackend[R[_], -S]
so it uses it expects its first parameter to be a type constructor; and that originally Scala-Guice used the scala.reflect.Manifest infrastructure. AFAIU such higher-kind types are not representable as Manifest and this is what the error in your question really says.
Luckily Scala has added a new scala.reflect.runtime.universe.TypeTag infrastructure to tackle this issue in a better and more consistent way and the Scala-Guice migrated to its usage. That's why with the newer version of Scala-Guice the compiler error goes away. Unfortunately there is another bug in the Scala-Guice that makes the code fail in runtime and it is a lack of handling of the Nothing Scala type. You see, the Nothing type is a kind of fake one on the JVM. It is one of the things where the Scala type system is more reach than the Java one. There is no direct mapping for Nothing in the JVM world. Luckily there is no way to create any value of the type Nothing. Unfortunately you still can create a classOf[Nothing]. The Scala-to-JVM compiler handles it by using an artificial scala.runtime.Nothing$. It is not a part of the public API, it is implementation details of specifically Scala over JVM. Anyway this means that the Nothing type needs additional handling when converting into the Guice TypeLiteral and there is none. There is for Any the cousin of Nothing but not for Nothing (see the usage of the anyType in TypeConversions.scala).
So there are really two workarounds:
Use raw Java-based syntax for Guice instead of the nice Scala-Guice one:
bind(new TypeLiteral[SttpBackend[Try, Nothing]]() {})
.toInstance(sttpBackend) // or to whatever
See online demo based on your example.
Patch the TypeConversions.scala in the Scala-Guice as in:
private[scalaguice] object TypeConversions {
private val mirror = runtimeMirror(getClass.getClassLoader)
private val anyType = typeOf[Any]
private val nothingType = typeOf[Nothing] // added
...
def scalaTypeToJavaType(scalaType: ScalaType): JavaType = {
scalaType.dealias match {
case `anyType` => classOf[java.lang.Object]
case `nothingType` => classOf[scala.runtime.Nothing$] //added
...
I tried it locally and it seems to fix your example. I didn't do any extensive tests so it might have broken something else.
Here's my code:
class Eapproximator
var step : F64
new create(step' :F64) =>
step = step'
fun evaluate() :F64 =>
var total = F64(0)
var value = F64(1)
while total < 1 do
total = total + step
value = value + (value * step)
end
value
actor Main
new create(env: Env) =>
var e_approx = Eapproximator(0.00001)
var e_val = e_approx.evaluate()
env.out.print(e_val.string())
It works well and prints (as expected) 2.7183. However, if I replace class with actor in Eapproximator definition I get a bunch of errors:
Error:
/src/main/main.pony:18:34: receiver type is not a subtype of target type
var e_val = e_approx.evaluate()
^
Info:
/src/main/main.pony:18:17: receiver type: Eapproximator tag
var e_val = e_approx.evaluate()
^
/src/main/main.pony:6:3: target type: Eapproximator box
fun evaluate() :F64 =>
^
/src/main/main.pony:3:3: Eapproximator tag is not a subtype of Eapproxim
ator box: tag is not a subcap of box
new create(step' :F64) =>
^
Error:
/src/main/main.pony:19:19: cannot infer type of e_val
env.out.print(e_val.string())
What can I do to fix this?
The actor is the unit of concurrency in Pony. This means that many different actors in the same program can run at the same time, including your Main and Eapproximator actors. Now what would happen if the fields of an actor were modified by multiple actors at the same time? You'd most likely get some garbage value in the end because of the way concurrent programs work on modern hardware. This is called a data race and it is the source of many, many bugs in concurrent programming. One of the goals of Pony is to detect data races at compile time, and this error message is the compiler telling you that what you're trying to do is potentially unsafe.
Let's walk through that error message.
receiver type is not a subtype of target type
The receiver type is the type of the called object, e_approx here. The target type is the type of this inside of the method, Eapproximator.evaluate here. Subtyping means that an object of the subtype can be used as if it was an object of the supertype. So that part is telling you that evaluate cannot be called on e_approx because of a type mismatch.
receiver type: Eapproximator tag
e_approx is an Eapproximator tag. A tag object can neither be read nor written. I'll detail why e_approx is tag in a minute.
target type: Eapproximator box
this inside of evaluate is an Eapproximator box. A box object can be read, but not written. this is box because evaluate is declared as fun evaluate, which implicitly means fun box evaluate (which means that by default, methods cannot modify their receiver.)
Eapproximator tag is not a subtype of Eapproxim
ator box: tag is not a subcap of box
According to this error message, a tag object isn't a subtype of a box object, which means that a tag cannot be used as if it was a box. This is logical if we look at what tag and box allow. box allows more things than tag: it can be read while tag cannot. A type can only be a subtype of another type if it allows less (or as much) things than the supertype.
So why does replacing class with actor make the object tag? This has to do with the data race problems I talked about earlier. An actor has free reign over its own fields. It can read from them and write to them. Since actors can run concurrently, they must be denied access to each other's fields in order to avoid data races with the fields' owner. And there is something in the type system that does exactly that: tag. An actor can only see other actors as tag, because it would be unsafe to read from or write to them. The main useful thing it can do with those tag references is send asynchronous messages (by calling the be methods, or behaviours), because that's neither reading nor writing.
Of course, since you're not doing any mutation of Eapproximatorin your program, your specific case would be safe. But it is much easier to try to forbid every unsafe program than to try to allow every safe program in addition to that.
To sum it up, there isn't really a fix for your program, except keeping Eapproximator as a class. Not anything needs to be an actor in a Pony program. The actor is the unit of concurrency, but that means it is also the unit of sequentiality. Computations that need to be sequential and synchronous must live in a single actor. You can then break down those computations into various classes for good code hygiene.
I have a class that inherits the Actor trait. In my code, I have a method that creates x numbers of this actor using a loop and another method that simply sends the Finish message to all of them to tell them to terminate. I made the kill method just take an array of Actor since I want to be able to use it with an array of any type of Actor. For some reason, however, when I pass a value of type Array[Producer], where Producer extends Actor, to a method that accepts the type Array[Actor], I get a type error. Shouldn't Scala see that Producer is a type of Actor and automatically cast this?
What you are describing is called covariance, and it is a property of most of the collection classes in Scala--a collection of a subtype is a subtype of the collection of the supertype. However, since Array is a Java primitive array, it is not covariant--a collection of a subtype is simply different. (The situation is more complicated in 2.7 where it's almost a Java primitive array; in 2.8 Array is just a plain Java primitive array, since the 2.7 complications turned out to have unfortunate corner cases.)
If you try the same thing with an ArrayBuffer (from collection.mutable <- edit: this part is wrong, see comments) or a List (<- edit: this is true) or a Set (<- edit: no, Set is also invariant), you'll get the behavior you want. You could also create an Array[Actor] to begin with but always feed it Producer values.
If for some reason you really must use Array[Producer], you can still cast it using .asInstanceOf[Array[Actor]]. But I suggest using something other than primitive arrays--anything you could possibly be doing with actors will be far slower than the tiny overhead of using a more full-featured collection class.