[Scala] I have a class someClass with a function A in which I use log4j2 for logging.
class someClass {
def A ={
//smt
logger.warn("oups")
}
}
In my unit tests someClassTest, I want to collect what A logs and assert the message content.
class someClassTest {
it("should log warning") {
//trigger A()
//collect logs
val logs //contain logs
assert(logs(0).contains("oups"))
}
}
Theatrically, I should create a custom Appender (extending the AbstractAppender) and collect the logs in a list for example, but I cannot do that, or more like idk how to do that.
Is there other ways to assert the logs ? or how to create that custom Appender?
Thank yall
Related
I have a set of input cases stored in a file.
I would like each case to be a specific scalatest "test", i.e., reported in the console as an individual test and failed individually.
Unfortunately, experimentation and Google suggest that this capability might not be present?
E.g., this seems to be the common case (eliding for simplicity)
class MyTestingGoop extends FunSuite {
val input : Seq[SpecificTestCase] = ...
test("input data test") {
forAll(input) { case => ... }
}
//...
}
Ideally, each case presents as a separate test. How can this be done with ScalaTest?
You can do this:
class MyTestingGoop extends FunSuite {
val input : Seq[SpecificTestCase] = ...
forAll(input) {
test("testing input" + input) {
// do something with the test
}
}
}
The only limit is that input has a unique toString.
Basically calling test in Funsuite registers the test and later runs it so as long as your test creation is done as part of the class construction and each test has a unique string, you should be fine.
I tried copying the logging trait from the spark to my project.
Previously I was using print instead of log. The below code was working fine with print statement. But after adding the log,
I am getting task not serializable error.
This is due to the log is referred by outer class and the closure is trying to serialize the entire class object.
The issue can be solved by making the whole class as serializable. But my questions are
1) Is it worth to serialize the whole class? what are the pros/cons of it?
2) I tried adding another object and use that logger inside the parserFunction it works but Is there any other efficient way to get the log inside the function ?
class SampleClass(interval: String) extends Logging {
// other stuffs
def getParserFunction():
(CassandraRow) => List[String] = {
(row: CassandraRow) =>
try {
// doSomeThing
}
catch {
case NonFatal(ex) => {
logWarning("invalid data:")
}
}
}
}
Now, I am asked to add logging function in akka's actor.
When a message is received, before it is handled, this message should be written into log.
And before a message is sent out, this message should be logged first.
I think I should override the receive and send functions in Actor. Suppose I create a trait actorlog which extends Actor. And class myActor extends actorlog. But in myActor, I need to override receive function (it seems it causes problems here). So I am confused what I should do.
PS. I know akka provides logging. But now I need implement this function by myself.
There is a utility to get logging of the received messages in Akka. It's briefly mentioned in the logging documentation. Basically you wrap your receive function in it like this:
def receive = LoggingReceive {
// your normal receive here
}
And then you enable it in your config with:
akka.actor.debug.receive=on
The information will be logged at debug level.
Besides the other answers here, another approach is to use orElse to prepend a partial function to your receive. In that partial function, put the logging in isDefinedAt so it gets called on every message.
For example:
trait ReceiveLogger {
this: Actor with ActorLogging =>
def logMessage: Receive = new Receive {
def isDefinedAt(x: Any) = {
log.debug(s"Got a $x")
false
}
def apply(x: Any) = throw new UnsupportedOperationException
}
}
class MyActor extends Actor with ActorLogging with ReceiveLogger {
def receive: Receive = logMessage orElse {
case ...
}
}
Using orElse is a general approach for composing receive behavior. In most cases I am composing things like this:
def otherBehavior: Receive = {
case OtherMessage => ...
}
class MyActor extends Actor {
def receive = otherBehavior orElse {
case ...
}
}
A good example of the stackable traits approach can be seen in this presentation: http://www.slideshare.net/EvanChan2/akka-inproductionpnw-scala2013
use stackable modificator abstract override for stackable call of receive method.
use new feature of Akka: event-sourced actor system (docs here).
I've had a situation come up and bite me a few times where I'm testing an Actor and the Actor throws an exception unexpectedly (due to a bug), but the test still passes. Now most of the time the exception in the Actor means that whatever the test is verifying won't come out properly so it the test fails, but in rare cases that's not true. The exception occurs in a different thread than the test runner so the test runner knows nothing about it.
One example is when I'm using a mock to verify some dependency gets called, and due to a mistake in the Actor code I call an unexpected method in the mock. That causes the mock to throw an exception which blows up the actor but not the test. Sometimes this can even cause downstream tests to fail mysteriously because of how the Actor blew up. For example:
// using scala 2.10, akka 2.1.1, scalatest 1.9.1, easymock 3.1
// (FunSpec and TestKit)
class SomeAPI {
def foo(x: String) = println(x)
def bar(y: String) = println(y)
}
class SomeActor(someApi: SomeAPI) extends Actor {
def receive = {
case x:String =>
someApi.foo(x)
someApi.bar(x)
}
}
describe("problem example") {
it("calls foo only when it receives a message") {
val mockAPI = mock[SomeAPI]
val ref = TestActorRef(new SomeActor(mockAPI))
expecting {
mockAPI.foo("Hi").once()
}
whenExecuting(mockAPI) {
ref.tell("Hi", testActor)
}
}
it("ok actor") {
val ref = TestActorRef(new Actor {
def receive = {
case "Hi" => sender ! "Hello"
}
})
ref.tell("Hi", testActor)
expectMsg("Hello")
}
}
"problemExample" passes, but then downstream "ok actor" fails for some reason I don't really understand... with this exception:
cannot reserve actor name '$$b': already terminated
java.lang.IllegalStateException: cannot reserve actor name '$$b': already terminated
at akka.actor.dungeon.ChildrenContainer$TerminatedChildrenContainer$.reserve(ChildrenContainer.scala:86)
at akka.actor.dungeon.Children$class.reserveChild(Children.scala:78)
at akka.actor.ActorCell.reserveChild(ActorCell.scala:306)
at akka.testkit.TestActorRef.<init>(TestActorRef.scala:29)
So, I can see ways of catching this sort of thing by examining the logger output in afterEach handlers. Definitely doable, although a little complicated in cases where I actually expect an exception and that's what I'm trying to test. But is there any more direct way of handling this and making the test fail?
Addendum: I have looked at the TestEventListener and suspect there's maybe something there that would help, but I can't see it. The only documentation I could find was about using it to check for expected exceptions, not unexpected ones.
Thinking in Actors there is also another solution: failures travel to the supervisor, so that is the perfect place to catch them and feed them into the test procedure:
val failures = TestProbe()
val props = ... // description for the actor under test
val failureParent = system.actorOf(Props(new Actor {
val child = context.actorOf(props, "child")
override val supervisorStrategy = OneForOneStrategy() {
case f => failures.ref ! f; Stop // or whichever directive is appropriate
}
def receive = {
case msg => child forward msg
}
}))
You can send to the actor under test by sending to failureParent and all failures—expected or not—go to the failures probe for inspection.
Other than examining the logs, I can think of two ways to fail tests when an actor crashes:
Ensure that no Terminated message is received
Check the TestActorRef.isTerminated property
The latter option is deprecated, so I'll ignore it.
Watching Other Actors from Probes describes how to setup a TestProbe. In this case it might look something like:
val probe = TestProbe()
probe watch ref
// Actual test goes here ...
probe.expectNoMessage()
If the actor dies due to an exception it will generate the Terminated message. If that happens during the test and you expect something else, the test will fail. If it happens after your last message expectation, then the expectNoMessage() should fail when Terminated is received.
Okay, I've had a little time to play with this. I've got a nice solution that uses an event listener and filter to catch errors. (Checking isTerminated or using TestProbes is probably good in more focused cases but seems awkward when trying to make something to mix into any old test.)
import akka.actor.{Props, Actor, ActorSystem}
import akka.event.Logging.Error
import akka.testkit._
import com.typesafe.config.Config
import org.scalatest._
import org.scalatest.matchers.ShouldMatchers
import org.scalatest.mock.EasyMockSugar
import scala.collection.mutable
trait AkkaErrorChecking extends ShouldMatchers {
val system:ActorSystem
val errors:mutable.MutableList[Error] = new mutable.MutableList[Error]
val errorCaptureFilter = EventFilter.custom {
case e: Error =>
errors += e
false // don't actually filter out this event - it's nice to see the full output in console.
}
lazy val testListener = system.actorOf(Props(new akka.testkit.TestEventListener {
addFilter(errorCaptureFilter)
}))
def withErrorChecking[T](block: => T) = {
try {
system.eventStream.subscribe(testListener, classOf[Error])
filterEvents(errorCaptureFilter)(block)(system)
withClue(errors.mkString("Akka error(s):\n", "\n", ""))(errors should be('empty))
} finally {
system.eventStream.unsubscribe(testListener)
errors.clear()
}
}
}
You can just use withErrorChecking inline at specific spots, or mix it into a Suite and use withFixture to do it globally across all tests, like this:
trait AkkaErrorCheckingSuite extends AkkaErrorChecking with FunSpec {
override protected def withFixture(test: NoArgTest) {
withErrorChecking(test())
}
}
If you use this in my original example, then you will get the first test "calls foo only when it receives a message" to fail, which is nice because that's where the real failure is. But the downstream test will still fail as well due to the system blowing up. To fix that, I went a step further and used a fixture.Suite to instance a separate TestKit for each test. That solves lots of other potential test isolation issues when you have noisy actors. It requires a little more ceremony declaring each test but I think it's well worth it. Using this trait with my original example I get the first test failing and the second one passing which is just what I want!
trait IsolatedTestKit extends ShouldMatchers { this: fixture.Suite =>
type FixtureParam = TestKit
// override this if you want to pass a Config to the actor system instead of using default reference configuration
val actorSystemConfig: Option[Config] = None
private val systemNameRegex = "[^a-zA-Z0-9]".r
override protected def withFixture(test: OneArgTest) {
val fixtureSystem = actorSystemConfig.map(config => ActorSystem(systemNameRegex.replaceAllIn(test.name, "-"), config))
.getOrElse (ActorSystem (systemNameRegex.replaceAllIn(test.name, "-")))
try {
val errorCheck = new AkkaErrorChecking {
val system = fixtureSystem
}
errorCheck.withErrorChecking {
test(new TestKit(fixtureSystem))
}
}
finally {
fixtureSystem.shutdown()
}
}
}
I am using the breeze.optimize package of Scala Breeze, and it looks like Breeze ahs implemented its own logging library.
Is there a way to configure Breeze to use standard logging like log4j or slf4j so I can configure logging for optimizations the same way I do for everything else in my application?
Alternatively, how do I just turn off the logging messages. They are on by defaults and logging every iteration of a function minimization for me, which is generating a lot of log noise for me.
Wrapper for Apache Commons based ont he answer below:
import breeze.util.logging.Logger
import breeze.util.logging.Logger.Level
import org.apache.commons.logging.LogFactory
class BreezeCommonsLogger[T: ClassManifest] extends Logger {
private val log = LogFactory.getLog(classManifest[T].erasure)
def trace(f: => Any) { if (log.isTraceEnabled()) log.trace(f.toString) }
def debug(f: => Any) { if (log.isDebugEnabled()) log.debug(f.toString) }
def info(f: => Any) { if (log.isInfoEnabled()) log.info(f.toString) }
def warn(f: => Any) { if (log.isWarnEnabled()) log.warn(f.toString) }
def error(f: => Any) { if (log.isErrorEnabled()) log.error(f.toString) }
def fatal(f: => Any) { if (log.isFatalEnabled()) log.fatal(f.toString) }
def level_=(level: Level) {}
def level = Logger.NEVER
}
I was able to use this like:
val lbfgs = new LBFGS[Mat](maxIters, 5, tolerance) {
override val log = new BreezeCommonsLogger[LBFGS[Mat]]
}
Notice how the optimization classes have a log member variable that is of type breeze.util.logging.Logger. To turn off logging messages, there is a provided class called breeze.util.logging.NullLogger. To override the default log, you can do something like this:
val opt = new breeze.optimize.OWLQN {
override val log = breeze.util.logging.NullLogger
}
This will create an anonymous class that keeps everything the same for the OWLQN optimizer, sans the logging.
In terms of using an alternative logging source like log4j or slf4j, try using implicit conversions to convert from log4j loggers to Breeze loggers, or look into wrapping a logger in a class that inherits from Logger.