I'm using Akka Persistence, with LevelDB as storage plugin, in an application written in Scala. On the query-side, the current implementation uses PersistentView, which polls messages from a PersistentActor's journal by just knowing the identifier of the actor.
Now I've learned that PersistentView is deprecated, and one is encouraged to use Persistent Query instead. However, I haven't found any thorough description on how to adapt the code from using PersistentView to support the preferred Persistence Query implementation.
Any help would be appreciated!
From the 2.4.x-to-2.5.x migration guide:
Removal of PersistentView
After being deprecated for a long time, and replaced by Persistence Query PersistentView has now been removed.
The corresponding query type is EventsByPersistenceId. There are several alternatives for connecting the Source to an actor corresponding to a previous PersistentView actor which are documented in Integration.
The consuming actor may be a plain Actor or an PersistentActor if it needs to store its own state (e.g. fromSequenceNr offset).
Please note that Persistence Query is not experimental/may-change anymore in Akka 2.5.0, so you can safely upgrade to it.
Related
We are considering a serialization approach for our scala-based Akka Persistence app. We consider it likely that our persisted events will "evolve" over time, so we want to support schema evolution, and are considering Avro first.
We'd like to avoid including the full schema with every message. However, for the foreseeable future, this Akka Persistence app is the only app that will be serializing and deserializing these messages, so we don't see a need for a separate schema registry.
Checking the docs for avro and the various scala libs, I see ways to include the schema with messages, and also how to use it "schema-less" by using a schema registry, but what about the in-between case? What's the correct approach for going schema-less, but somehow including an identifier to be able to look up the correct schema (available in the local deployed codebase) for the deserialized object? Would I literally just create a schema that represents my case class, but with an additional "identifier" field for schema version, and then have some sort of in-memory map of identifier->schema at runtime?
Also, is the correct approach to have one serializer/deserialize class for each version of the schema, so it knows how to translate every version to/from the most recent version?
Finally, are there recommendations on how to unit-test schema evolutions? For instance, store a message in akka-persistence, then actually change the definition of the case class, and then kill the actor and make sure it properly evolves. (I don't see how to change the definition of the case class at runtime.)
After spending more time on this, here are the answers I came up with.
Using avro4s, you can use the default data output stream to include the schema with every serialized message. Or, you can use the binary output stream, which simply omits the schema when serializing each message. ('binary' is a bit of a misnomer here since all it does is omit the schema. In either case it is still an Array[Byte].)
Akka itself supplies a Serializer trait or a SerializerWithStringManifest trait, which will automatically include a field for a "schema identifier" in the object of whatever you serialize.
So when you create your custom serializer, you can extend the appropriate trait, define your schema identifier, and use the binary output stream. When those techniques are combined, you'll successfully be using schema-less serialization while including a schema identifier.
One common technique is to "fingerprint" your schema - treat it as a string and then calculate its digest (MD5, SHA-256, whatever). If you construct an in-memory map of fingerprint to schema, that can serve as your application's in-memory schema registry.
So then when deserializing, your incoming object will have the schema identifier of the schema that was used to serialize it (the "writer"). While deserializing, you should know the identifier of the schema to use to deserialize it (the "reader"). Avro4s supports a way for you to specify both using a builder pattern, so avro can translate the object from the old format to the new. That's how you support "schema evolution". Because of how that works, you don't need a separate serializer for each schema version. Your custom serializer will know how to evolve your objects, because that's the part that Avro gives you for free.
As for unit testing, your best bet is exploratory testing. Actually define multiple versions of a case class in your test, and multiple accompanying versions of its schema, and then explore how Avro works by writing tests that will evolve an object between different versions of that schema.
Unfortunately that won't be directly relevant to the code you are writing, because it's hard to simulate actually changing the code you are testing as you test it.
I developed a prototype that demonstrates several of these answers, and it's available on github. It uses avro, avro4s, and akka persistence. For this one, I demonstrated a changing codebase by actually changing it across commits - you'd check out commit #1, run the code, then move to commit #2, etc. It runs against cassandra so it will demonstrate replaying events that need to be evolved using new schema, all without using an external schema registry.
Hi I create my own Connector of Cassandra using the datastax drivers. But I´m facing some Memory leaks issues, so I start considering another solutions like Alpakka de lightbend which has a Cassandra connector.
But after check the poor documentation I´m changing my mind, since it´s just using the connector with CQLSH queries, and in my case I manage DTO objects.
Anybody knows any documentation where I can see if Alpakka cassandra manage the save of DTO´s with consistency level?.
This code is from my current connector. I would like to achieve something similar.
private void updateCreateEntry(DTO originalDto, Mapper cassandraMapper) {
ConsistencyLevel consistencyLevel = ((DTOCassandra) originalDto).getConsistencyLevel();
//.- For writing we set the consistency level to quorum
cassandraMapper.save(originalDto, Option.consistencyLevel(consistencyLevel != null ? consistencyLevel : DEFAULT_CONSISTENCY_LEVEL));
}
As you've noticed, presently the Cassandra connector within Alpakka is quite thin. If you need a richer support for your DTO, you could choose a richer client like Phantom.
There are excellent examples on how to use Phantom - check this one out for instance. Once you have created you model, Phantom will give you a def store[T](t: T): Future[ResultSet] function to insert data.
You can feed calls to these function to a mapAsync(n) combinator to make use of them in your Akka Stream.
I am became in charge EJB 3.1/JPA part of our project running on Glassfish 4.0. I am quite new to EJB and so not very confident in Session beans(and/or their methods) transaction attributes. I am dealing with stateless session beans.
I read that NOT_SUPPORTED, NEVER, SUPPORTS transaction attributes should be used with caution because their behavior varies app server vendor. Actually, I could not find statements like that in other sources. Are they really vendor specific? Also, is that correct that not annotating method or bean defaults to REQUIRED transaction attribute for it?
And also here the situation. Let's I have transaction T and two stateless beans A and B with methods mA and mB. mA calls mB. What are the possible combinations for the transaction attributes of these methods so that the transaction T will go successfully? I know that if mA has REQUIRED and mB has NEVER the exception throw. Is SUPPORTS on mB good for any kind for incoming transaction - like a safe option to make sure any transaction will go through this method without error?
Thank you
GlassFish 4.0 is the reference implementation of Java EE 7 and according to the release notes it supports Enterprise JavaBeans 3.2 (JSR-345).
I read that NOT_SUPPORTED, NEVER, SUPPORTS transaction attributes should be used with caution because their behavior varies app server vendor. I could not find alerts in other sources rather than the one i mentioned. Please advice me If I should concerned about that.
Basically EJB specification says what to implement not how to implement thus there still may be some rare corner cases since we are not living in a perfect world. I suppose this is why you have been alerted. From the other hand I wouldn't concern about that as GlassFish is widely used across the world and it surely conforms the JSR specification.
Also, is that correct that not annotating method or bean defaults to REQUIRED transaction attribute?
Yes, this is correct. According to EJB 3.2 Specification, chapter 8.3.7 Specification of the Transaction Attributes for a Bean’s Methods:
By default, the value of the transaction attribute for a method of a
bean with container-managed transaction demarcation is the REQUIRED
transaction attribute, and the transaction attribute does not need to
be explicitly specified in this case.
Can you please advice a me source where I can find correct or in incorrect combinations of the transaction attributes on different methods of different EJBs in one call stack for one given transaction.
The ultimate source of knowledge is the mentioned EJB Specification (chapter 8.6 to be more specific) but you will find a lot of useful posts around.
In general take a closer look at transaction propagation and transaction demarcation related topics.
Will SUPPORTS attribute adapt method/bean to a calling method/bean with whatever transaction attribute, so that no error will occur?
Not really. I would say SUPPORTS propagates a transaction context (if any) of a calling method/bean, so you may safely query data within but you should avoid operations that change the persistence context.
I'm using Akka Persistence with Cluster Sharding. What is the proper way to provide dependencies into such PersistentActor-s?
As far as I understand, passing them as constructor arguments is not possible, as Cluster Sharding is creating these actors.
Using Spring/Guice/etc. is not idiomatic Scala (and possibly has other issues (?)).
Using an object to implement a singleton makes for cumbersome testing and seems bad style.
What is the proper way?
P.S. If you plan to suggest the Cake pattern, please provide sample code in this specific Akka Persistence Cluster Sharding context.
UPDATED VERSION:
the solution I offered earlier did not allow to mock services of the actor under test in unit test cases.
I am using instead one solution offered on that article http://letitcrash.com/post/55958814293/akka-dependency-injection that is called "aspect weaving" and that consists of injecting the dependencies in the actor using aspect oriented programming.
This solution can be used to inject Spring dependencies on any bean not controlled by Spring container (potentially useful for legacy code).
A full example is provided by the above article: https://github.com/huntc/akka-spring/blob/f137c98b621517301f636e6ea03519388fcd5fff/src/main/scala/org/typesafe/Akkaspring.scala
And to enable aspect weaving in a spring based application you should check the documentation on Spring doc
In my case, on a jetty application server, it consists of using the spring agent and setting it in the jvm arguments.
As far as tests are concerned, I :
created setters for the injected services
created basic configuration for my actors with null beans referenced for my dependencies
instantiated the actor in my test case
replace the actor's services with mocks
run the actor's inner methods and check the results, actor's state or calls to dependencies
ORIGINAL:
I am using Akka in a Spring Application to enable clustering. At first
it raises the following issue: you cannot inject spring managed
dependencies in the actor constructor, as you said. (it tries to
serialize the application context and fails)
So I created a class that holds the application context and provides a
static method to retrieve beans I need. I retrieve the bean only if I
need it, this way:
public void onReceive{
if (message instanceof HandledMessage) {
(MyService) SpringApplicationContext.getBean("myService");
...
}
}
It's not conventional but it does the job, what do you think? Hope
otherwise it might help another one.
I'm trying to find the 'right' actor implementation. I realized there is a bunch of them and it's a bit confusing to pick one. Personally I'm especially interested in remote actors, but I guess a complete overview would be helpful to many others. This is a pretty general question, so feel free to answer just for the implementation you know about.
I know about the following Scala Actor implementations (SAI). Please add the missing ones.
Scala 2.7 (difference to)
Scala 2.8
Akka (http://www.akkasource.org/)
Lift (http://liftweb.net/)
Scalaz (http://code.google.com/p/scalaz/)
What are the target use-cases for these SAIs (lightweight vs. "heavy" enterprise framework)?
do they support remote actors? What shortcomings do remote actors have in the SAIs?
How is their performace?
How active is there community?
How easy are they to get started? How good is the documentation?
How easy are they to extend?
How stable are they? Which projects are using them?
What are their shortcomings?
What are their design principles?
Are they thread based or event based (receive/ react) or both?
Nested receiveS
hotswapping the Actor’s message loop
This is the most comprehensive comparison I have seen so far:
http://doc.akka.io/docs/misc/Comparison_between_4_actor_frameworks.pdf via http://klangism.tumblr.com/post/2497057136/all-actors-in-scala-compared
As of Scala 2.10, scala actors is now deprecated and Akka Actors is now part of standard distribution
Scala 2.7.7. vs 2.8 after The Scala 2.8.0 RC3 distribution:
New Reactors provide more lightweight, purely event-based actors with optional, implicit sender identification. Support for actors with daemon-style semantics was added. Actors can be configured to use the efficient JSR166y fork/join pool, resulting in significant performance improvements on 1.6 JVMs. Schedulers are now pluggable and easier to customize.
There's also a design document of Haller: Scala Actors: Unifying Thread-based and Event-based Programming
As far as I know, only Scala and Akka support remote actors.
Akka is backed up by scalablesolutions, which offer commerical support and plug ins for akka.
Akka seems like a heavyweight solution, which targets integration with existing frameworks (camel, AMQP, JTA, Comet, Spring, Redis) and additionally STMs and persistence.
Akka compared to Scala doesn't support nested receives, but supports hotswapping the actors message loop and has both, thread based and event based actors and so called "Event-based single-threaded" ones.
I realized that akka enforces exhaustive matches. So even if technically receive expects a partial function, the function must not be partial. This means you have to handle every message immediately.