I'm writing a websocket client using the nv-websocket-client Java library. In order to process incoming data I have to declare a class extending WebSocketAdapter and override several methods (most importantly OnMessage)
class Client extends WebSocketAdapter {
override def onTextMessage(websocket: WebSocket, text: String): Unit = {
processData(text)
}
def processData(text: String): Unit = ???
}
The problem is I have to save some data in between the calls and I don't see how to do it without declaring a var variable in Client. Is there any way to do it in a strict functional style?
PS: I know that akka-http also has a websockets functionality, but I'm stuck with this library, unfortunately.
Related
I have an actor (actually a persistent actor) that, in response to a message (command), needs to make an HTTP call. The question now is how do I deal with this HTTP call in unit test?
Normally I would use the combination of DI and mocking to inject a mock implementation of the HTTP service while testing. But I am not sure if this is how to approach the problem in Akka? Even if this is the how to approach it in Akka, I am not sure how to go about doing the injecting and mocking in testing.
Any thoughts on this? What is the idiomatic way for testing Actors that perform IO operations (HTTP calls, writing to DB etc).
PS: I am using Akka Typed.
My personal belief is that you should avoid IO operations in Actors if at all possible (see this presentation for more details).
That being said, I’m sure there are people who would disagree and you probably shouldn’t listen to me :)
Here is how I would go about mocking it for a test.
Create a trait that represents your http call.
trait Client {
def makeHttpCall(input: Int): Future[Output]
}
(You will want to create a class of some kind that implements this trait and test it separately from your actor.)
Pass an instance of that trait into your actor using its constructor/apply method.
def apply(client: Client): Behavior[...] = {
// use your client inside of your behavior
}
In your test create a mock instance of Client and pass that in to your actor. You can mock your Client using a mocking library (ScalaMock or Mockito) or you can actually just create a test implementation of your trait with relative ease:
class TestClient extends Client {
def makeHttpCall(input: Int): Future[Output] =
Future.successful(Output(...))
}
Note: all of the names for classes and methods that I chose are just placeholders. You should, of course, choose more specific names based on the context you are working in.
In particular situations, you need to have some utility methods that are required across different classes. To solve this situation, you create an Util object wherein you place all these methods
object AggregatorUtil {
def aggregateValues(list : List[BigDecimal]) = //some logic...
}
// Import everything in the Utilities object
import AggregatorUtil._
and then import whichever members of util are required in your class. However, the downside to this is that, as all your methods are inside the singleton object and it becomes tricky to mock the object and unit test methods of the class that use utility methods.
To solve this problem again, the only solution that came to mind was Extracting the functionality out to a trait and then mocking the trait.
Please let me know if there is any other approach for handling and testing of util methods and which one is a rather cleaner approach.
Thanks in advance !!!
Note: -I am using scalatest and mockito in my project.
If you need to mock, putting this all in a mocked-out trait is the way forward. If mocking is unnecessary though, avoid it. Mocking unnecessarily is... unnecessary. You'll just be wasting time and effort for something which provides no additional value.
Mocking is best used when you have complex functionality or functionality in other files which you want to treat as a black box and just assume it works as expected (you'd then typically unit test this stuff separately). If you can avoid it and use the functions' actual functionality though, you'll get a much more realistic view of what your application does and will spot new bugs/breaking changes quicker (if you have mocked out functionality and forget to update your mocks, you might not any spot new bugs you introduce).
A good example of when mocking is necessary is when you're mocking calls to a database in a MVC application (e.g. a Scala Play microservice). You obviously don't want to have to run an actual database when testing your code, so you'd typically mock out your connector layer and return dummy/mocked data from your connector functions.
An example of something you wouldn't mock is something like:
trait MyTrait {
def toInt(str: String): Int
}
val mockedTrait = mock[MyTrait]
when(mockedTrait.toInt(eq("3")).thenReturn(3)
It's a bit of a silly example, but I think it explains the point clearly - doing something like this would be ridiculous. Mocking isn't always the answer.
I mostly mock with Test-Implementation which I find more readable and you don't have to learn a mocking framework.
Here an example:
The Interface:
trait DataRepo {
def persist(data: DataObject): Future[DataObject]
def idents(): Future[List[String]]
def insertData(dataCont: DataObject): Future[Int]
...
}
The Mocked Interface:
object DataRepoMock extends DataRepo {
def persist(data: DataObject): Future[DataObject] = ??? // only implement when needed
def idents(): Future[List[String]] = Future.successful((0 to 10).map(_=>Random.nextInt(100)))
def insertData(dataCont: DataObject): Future[Int] = Future.successful(Random.nextInt(100))
...
}
You can also use all the Scala goodies, like Pattern Matching, to make your Mock react differently to input.
Here is an example that this is not just used by myself;):
EPFLx: scala-reactiveX see Lecture 2.5 Testing Actor Systems:
def fakeGetter(url:String, depth: Int):Props =
Props(new Getter(url, depth){
override def webClient: WebClient = FakeWebClient
})
I am writing a library that provides a distributed algorithm. The idea being that existing applications can add the library to use the algorithm. The algorithm is in a library module and it abstracts the actual transmission of data over the network behind a trait. An application that uses the algorithm has to provide the actual network transport code. In code it looks something like the following:
// library is really a separate project not a single object
object Library {
// handle to a remote server
trait RemoteProcess
// concrete server need to know how to actually send to a real remote process
trait Server {
def send(client: RemoteProcess, msg: String)
}
}
// the application uses the library and provides the concrete transport
object AkkaDemoApplication {
// concreate ref is a m wrapper to an actor ref in demo app
case class ConcreteRemoteProcess(ref: akka.actor.ActorRef) extends Library.RemoteProcess
class AkkaServer extends Library.Server {
// **WARNING** this wont compile its here to make my question
override def send(client: ConcreteRemoteProcess, msg: String): Unit = client.ref ! msg
}
}
A couple of options I have considered:
Have the signature of the AkkaServer method overload the library trait method then perform an unsafe cast to ConcreteRemoteProcess. Yuk!
Have the signature of the AkkaServer method overload the library trait method then pattern match on the RemoteProcesss argument give a ConcreteRemoteProcess. This is no better than an unsafe cast in terms of blowing up at runtime if the wrong thing is passed.
Make the library server generic in terms of the RemoteProcess.
An example of option 3 looks like:
object Library {
trait Server[RemoteProcess] {
def send(client: RemoteProcess, msg: String)
}
}
object Application {
class AkkaServer extends Library.Server[ActorRef] {
override def send(client: ActorRef, msg: String): Unit = client ! msg
}
}
I tried option 3 and it worked but the generic type ended up be stamped on just about every type throughout the entire library module. There was then a lot of covariant and contravariant hassles to get the algorithmic code to compile. Simply to get compile time certainty at the one integration point the cognitive overhead was very large. Visually the library code is dominated by the generic signatures as though understanding that is critical to understanding the library when in fact it's a total distraction to understanding the library logic.
So using the genetic works and gave me compile time certainly but now I wished I had gone with the option 2 (the pattern match) with the excuse "it would fail fast at startup if someone got it wrong lets keep it simple".
Am I missing some Scala feature or idiom here to get compile time certainty without the cognitive overhead of a "high touch" generic that all the library code touches but ignores?
Edit I have considered that perhaps my code library is badly factored such that a refactor could move the generic to the boundaries. Yet the library has already been refactored for testability and that breakup into testable responsibilities is part of the problem of the generic getting smeared around the codebase. Hence my question is: in general is there another technique I don't know about to avoid a generic to provide a concrete implementation to an abstract API?
I think you are coupling your algorithm and Akka too closely. Further more, I assume the server to send data to the remote client that performs some operation and sends back the result
Answer
Why not
object Library {
// handle to a remote server
trait RemoteProcessInput
trait RemoteProcessResult
// concrete server need to know how to actually send to a real remote process and how to deal with the result
trait Server {
def handle(clientData: RemoteProcessInput) : Future[RemoteProcessResult]
}
}
A concrete implementation provides the implementation with Akka
object Application {
class AkkaServerImpl(system: ActorSystem)
extends Library.Server {
override def handle(clientData: RemoteProcessInput)
: ActorRef, msg: String): Future[RemoteProcessResult] = {
// send data to client and expect result
// you can distinguish target and msg from the concrete input
val ref : ActorRef = ??? // (resolve client actor)
val msg = ??? // (create your message based on concrete impl)
val result = ref ? msg // using ask pattern here
// alternatively have an actor living on server side that sends msgs and receives the clients results, triggered by handle method
result
}
}
}
I'm trying to serialize an instance of a Scala class using Kryo via Twitter's chill-scala library. It's from a library (external jar) and thus, I think, needs to be registered with Kryo.
How do I register a class to (de)serialize using chill-scala?
Here's the core of my code, based mostly on examining the chill-scala test suite.
// This is from the chill-scala test suite
def serialize[T](t: T): Array[Byte] = ScalaKryoInstantiator.defaultPool.toBytesWithClass(t)
def deserialize[T](bytes: Array[Byte]): T =
ScalaKryoInstantiator.defaultPool.fromBytes(bytes).asInstanceOf[T]
/**
* Save a value in cache.
*/
def save[T](key: String, value: T, expiration: Int = 0): Future[T] = {
cache.put(key, serialize[T](value), expiration, TimeUnit.SECONDS)
Future.successful(value)
}
/**
* Finds a value in the cache.
*/
def find[T: ClassTag](key: String): Future[Option[T]] = Future {
val result = deserialize[T](cache.get(key).asInstanceOf[Array[Byte]])
Option(result)
}
When I run it, it throws
com.esotericsoftware.kryo.KryoException: Unable to find class: <name_of_external_class>
More generally, is there documentation someplace on how to use chill-scala? The authors of that package have obviously done a substantial amount of work and I've seen a number of positive references to it -- but no documentation.
Thanks for any pointers,
Byron
I believe that chill is just an extension to Kryo, which provides serializers for supported scala classes better than the Kryo default FieldSerializer.
So, if you do not know how to use chill, you should try to read the docs of Kryo.
And, the exception of
Unable to find class: <name_of_external_class>
is probably because the class is not in your class path. It may not relate to Kryo.
Note that, even not registered in Kryo, the class can still be serialized by Kryo.
If you need good example of using chill, source code of Apache Spark is a good choice. Also, the small example in this repo is acceptable.
I was searching a way of doing dependency injection in Scala kind of like Spring or Unity in C# and I found nothing really interesting.
MacWire: I don't understand the benefit as we have to give the class in wire[CASS]. So what's the point if you give the implementation when you call wire? I can do new CASS it will be the same.
Cake pattern with self type: Seems to not answer what I'm searching for.
So I decided to make my implementation and ask you what do you think because it's surprising me that nothing like this has been done before. Maybe my implementation have lot's of issues in real life also.
So here is an example:
trait Messenger {
def send
}
class SkypeMessenger extends Messenger {
def send = println("Skype")
}
class ViberMessenger extends Messenger {
def send = println("Viber")
}
I want here to inject everywhere in my app the implementation configured in only one place:
object App {
val messenger = Inject[Messenger]
def main(args: Array[String]) {
messenger.send
}
}
Note the Inject[Messenger] that I define like below with the config I want (prod or dev):
object Inject extends Injector with DevConfig
trait ProdConfig {
this: Injector =>
register[Messager](new SkypeMessager)
register[Messager](new ViberMessager, "viber")
}
trait DevConfig {
this: Injector =>
register[Messager](new ViberMessager)
register[Messager](new ViberMessager, "viber")
}
And finally here is the Injector which contains all methods apply and register:
class Injector {
var map = Map[String, Any]()
def apply[T: ClassTag] =
map(classTag[T].toString).asInstanceOf[T]
def apply[T: ClassTag](id: String) =
map(classTag[T].toString + id).asInstanceOf[T]
def register[T: ClassTag](instance: T, id: String = "") = {
map += (classTag[T].toString + id -> instance)
instance
}
}
To summaries:
I have a class Injector which is a Map between interfaces/traits (eventually also an id) and an instance of the implementation.
We define a trait for each config (dev, prod...) which contains the registers. It also have a self reference to Injector.
And we create an instance of the Injector with the Config we want
The usage is to call the apply method giving the Interface type (eventually also an id) and it will return the implementation's instance.
What do you think?
You code looks a lot like dependency injection in Lift web framework. You can consult Lift source code to see how it's implemented or just use the framework. You don't have to run a Lift app to use its libraries. Here is a small intro doc. Basically you should be looking at this code in Lift:
package net.liftweb.http
/**
* A base trait for a Factory. A Factory is both an Injector and
* a collection of FactorMaker instances. The FactoryMaker instances auto-register
* with the Injector. This provides both concrete Maker/Vender functionality as
* well as Injector functionality.
*/
trait Factory extends SimpleInjector
You can also check this related question: Scala - write unit tests for objects/singletons that extends a trait/class with DB connection where I show how Lift injector is used.
Thanks guys,
So I make my answer but the one from Aleksey was very good.
I understand better the Cake Pattern with this sample:
https://github.com/freekh/play-slick/tree/master/samples/play-slick-cake-sample
Take a look also to the other implementations without DI and compare:
https://github.com/freekh/play-slick/tree/master/samples/
And so the cake pattern doesn't have a centralized config like we can have with my shown lift style DI. I will anyway use the Cake pattern as it fits well with Slick.
What I didn't like with Subcut is the implicits everywhere. I know there is a way to avoid them but it looks like a fix to me.
Thanks
To comment on MacWire, you are right that you could just use new - and that's the whole point :). MacWire is there only to let you remove some boilerplate from your code, by not having to enumerate all the dependencies again (which is already done in the constructor).
The main idea is that you do the wiring at "the end of the world", where you assemble your application (or you could divide that into trait-modules, but that's optional). Otherwise you just use constructors to express dependencies. No magic, no frameworks.