I am using the breeze.optimize package of Scala Breeze, and it looks like Breeze ahs implemented its own logging library.
Is there a way to configure Breeze to use standard logging like log4j or slf4j so I can configure logging for optimizations the same way I do for everything else in my application?
Alternatively, how do I just turn off the logging messages. They are on by defaults and logging every iteration of a function minimization for me, which is generating a lot of log noise for me.
Wrapper for Apache Commons based ont he answer below:
import breeze.util.logging.Logger
import breeze.util.logging.Logger.Level
import org.apache.commons.logging.LogFactory
class BreezeCommonsLogger[T: ClassManifest] extends Logger {
private val log = LogFactory.getLog(classManifest[T].erasure)
def trace(f: => Any) { if (log.isTraceEnabled()) log.trace(f.toString) }
def debug(f: => Any) { if (log.isDebugEnabled()) log.debug(f.toString) }
def info(f: => Any) { if (log.isInfoEnabled()) log.info(f.toString) }
def warn(f: => Any) { if (log.isWarnEnabled()) log.warn(f.toString) }
def error(f: => Any) { if (log.isErrorEnabled()) log.error(f.toString) }
def fatal(f: => Any) { if (log.isFatalEnabled()) log.fatal(f.toString) }
def level_=(level: Level) {}
def level = Logger.NEVER
}
I was able to use this like:
val lbfgs = new LBFGS[Mat](maxIters, 5, tolerance) {
override val log = new BreezeCommonsLogger[LBFGS[Mat]]
}
Notice how the optimization classes have a log member variable that is of type breeze.util.logging.Logger. To turn off logging messages, there is a provided class called breeze.util.logging.NullLogger. To override the default log, you can do something like this:
val opt = new breeze.optimize.OWLQN {
override val log = breeze.util.logging.NullLogger
}
This will create an anonymous class that keeps everything the same for the OWLQN optimizer, sans the logging.
In terms of using an alternative logging source like log4j or slf4j, try using implicit conversions to convert from log4j loggers to Breeze loggers, or look into wrapping a logger in a class that inherits from Logger.
Related
I started experimenting with ZIO, and was trying to run the following highly sophisticated program:
import logger.{Logger, _}
import zio.console._
import zio.{system, _}
object MyApp extends App {
override def run(args: List[String]): ZIO[ZEnv, Nothing, Int] = {
app
.provideSome[Logger](_ => Slf4jLogger.create) //1
.fold(_ => 1, _ => 0)
}
val app: ZIO[Console with system.System with Logger, SecurityException, Unit] =
for {
_ <- info("This message from the logger") //2
maybeUser <- system.env("USER")
_ <- maybeUser match {
case Some(userName) => putStrLn(s"Hello ${userName}!")
case None => putStrLn("I don't know your name")
}
} yield ()
}
(Slf4jLogger is https://github.com/NeQuissimus/zio-slf4j)
If I comment lines //1 and //2 out everything works fine. But in the above form I get a type mismatch error:
Error:(13, 45) type mismatch;
found : logger.Slf4jLogger
required: zio.console.Console with zio.system.System with logger.Logger
.provideSome[Logger](_ => Slf4jLogger.create)
I don't understand the following:
The program required a Console and a System instance in the environment, yet I did not have to .provide it before, when I was not logging. Why? And why do I suddenly need it?
According to the scaladoc, .provideSome Provides *some* of the environment required to run this effect, leaving the remainder R0. For me that means that I don't have to provide the full environment type, I could add the required modules one by one - yet it seems to expect the full ZEnv type, just like .provide - so what's the point?
I cannot instantiate an anonymous class by new Logger with Console.Live with system.System.Live, because the Slf4jLogger has a factory method in the companion object. How can I use this factory method with the other traits?
Since creating a logger instance in Java is kind of effectful, is .provide even the right function to call?
PS: I was able to get this working in a kind of ugly way:
app
.provideSome[ZEnv](_ =>
new Console with Logger with system.System {
override val console = Console.Live.console
override val system = System.Live.system
override val logger = Slf4jLogger.create.logger
}
) //1
.fold(_ => 1, _ => 0)
This compiles and runs, but overriding the mixed in traits' inner members does not seem right...
And it works in exactly the same way with .provide:
app
.provide(
new Console with Logger with system.System {
override val console = Console.Live.console
override val system = System.Live.system
override val logger = Slf4jLogger.create.logger
}
) //1
.fold(_ => 1, _ => 0)
What is going on?
I am afraid you do have to return an instance that implements all the traits you want to provide.
If you look at the example in ZIO's documentation for provideSome, you see that they are doing the exact same thing: They are taking a Console and turn it into a Console with Logging.
/**
* Provides some of the environment required to run this effect,
* leaving the remainder `R0`.
*
* {{{
* val effect: ZIO[Console with Logging, Nothing, Unit] = ???
*
* effect.provideSome[Console](console =>
* new Console with Logging {
* val console = console
* val logging = new Logging {
* def log(line: String) = console.putStrLn(line)
* }
* }
* )
* }}}
*/
For me that means that I don't have to provide the full environment type, I could add the required modules one by one
No. You have to provide an instance of the full environment type to the effect you are wrapping (because it needs that). I think this is because we want to have this checked at compile-time and there is no way (without macros) to generate a "mash-up" stacked environment without spelling out how to do that. There will probably be some macros for that down the road.
just like .provide - so what's the point?
The difference is that with .provide you have to make up that instance of the required environment from scratch without any input. So that as a result, the effect (with your wrapping) no longer requires anything.
Whereas with .provideSome you yourself can request an environment to help you construct the instance you are providing. That environment will be passed into your function. In the example above, they require a Console and augment it to Console with Logging (by using the console instance they are given). As a result, effect.provideSome[Console](...) still requires a Console (whereas written with .provide it would require Nothing).
.provideSome[ZEnv](_ =>
new Console with Logger with system.System {
override val console = Console.Live.console
Since you are using provideSome[ZEnv], you should probably not access the global singletons Console.Live, but take that from the env being passed in:
.provideSome[ZEnv](env =>
new Console with Logger with system.System {
override val console = env.console
The program required a Console and a System instance in the environment, yet I did not have to .provide it before
That is because ZEnv, the default environment already provides these two.
Since creating a logger instance in Java is kind of effectful, is .provide even the right function to call?
Personally, I would ignore that constructing loggers is effectful. It does not seem like something worth tracking. I consider it some basic facility that it is almost a part of class-loading.
If you did want to track the effect, you could do
app
.provideSomeM(env => for {
logger <- UIO(Slf4jLogger.create)
} yield new MyEnvironment(env, logger)
)
Let me try to explain.
If you have a look at what ZEnv is it's an alias for Clock with Console with System with Random with Blocking. So, when you started implementing def run(args: List[String]): ZIO[ZEnv, Nothing, Int] zio already provided you with these capabilities.
Your app variable signature has a return type of zio with environment Console with System with Logger - one thing that stands out is a Logger capability. It's not standard so ZIO can't provide it for you. You have to provide it yourself.
That's what you did using .provideSome[Logger] - you eliminated one of the capabilities from environment requirements making it type compatible with the standard:
type ZEnv = Clock with Console with System with Random with Blocking
type YourEnv = Console with System
type Proof = ZEnv <:< YourEnv
This is a way that I use:
import logger.{Logger, _}
import zio.console._
import zio.{system, _}
object MyApp extends App {
override def run(args: List[String]): ZIO[ZEnv, Nothing, Int] = {
app
.fold(_ => 1, _ => 0)
}
val app: ZIO[Environment, SecurityException, Unit] =
for {
_ <- info("This message from the logger").provide( Slf4jLogger.create) // 1
maybeUser <- system.env("USER")
_ <- maybeUser match {
case Some(userName) => putStrLn(s"Hello ${userName}!")
case None => putStrLn("I don't know your name")
}
} yield ()
}
I provide it directly in the for-comprehension of the app. All the rest you have from the zio.App.
I have a service class, and the service have one method getSomethingFromApi , now , I want to have play Configuration instance so I can pull stuff from the application.conf, and a play WSClient so I can perform http calls.
this is how I want my service to look:
class MyApiService {
def getSomethingFromApi(whichApi: String): Future[ApiRes] = {
wsClient.url(configuration.getString(whichApi)).withHttpHeaders(("Content-Type", "application/json")).get.map { res =>
response.status match {
case Status.OK => // do something
case _ => throw new Exception
}
}
}
}
and this is the ServicesModule that is wiring my services:
import com.softwaremill.macwire._
trait ServicesModule {
lazy val myService: MyApiService = wire[MyApiService]
}
my question now is what is the right way of using wiring play Configuration and WSClient instances..? cause currently i need those instances in my service but i dont have them, how should i do this the right way?
thanks
With macwire it'll probably look like this
// MyApiService.scala
class MyApiService(wsClient: WSClient) { ... }
// ServicesModule.scala
trait ServicesModule with NingWSComponents {
lazy val wsClient = wire[WSClient]
lazy val apiService = wire[MyApiService]
}
I haven't tried using macwire with play myself, so I have relatively low confidence that it'll work on the first try, but macwire play example suggests mixing in certain Play modules to provide values needed for WSClient. Most likely not all of them are needed, but some might be - soo I'd suggest starting with just NingWSComponents and gradually adding more until it works.
For the configuration I suggest using something like PureConfig and load the configuration as follows
import pureconfig._
import pureconfig.error.ConfigReaderFailures
case class YourConfClass(name: String, quantity: Int)
val config: Either[pureconfig.error.ConfigReaderFailures,YourConfClass] = loadConfig[YourConfClass]
This then can be passed on to any component of your app using macwire.
As of Play 2.6.X one should use AhcWSComponents that are provided by the ws dependency as follows:
In your build.sbt file add the ws dependency to your project
libraryDependencies += ws
In your module trait mix-in the AhcWSComponents trait and wire the WSClient
trait ServicesModule with AhcWSComponents {
lazy val wsClient = wire[WSClient]
lazy val apiService = wire[MyApiService]
}
In your MyApiService add the WSClient as a param. to the constructor
class MyApiService(wsClient: WSClient) { ... }
And now you're done. This general rule applies to all provided dependencies.
I have been trying to Log things within my scalaTest as such:
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup {
val log = Logging(system, this)
Basically let's just say that ChangesetActorTest inherit from TestKit(system)
Unfortunately Logging(system, this) does not work with the this.
I get the following error:
[error]
/Users/maatary/Dev/IdeaProjects/PoolpartyConnector/src/test/scala/org/iadb/poolpartyconnector/changepropagation/ChangeSetActorTest.scala:22:
Cannot find LogSource for
org.iadb.poolpartyconnector.changepropagation.ChangeSetActorTest
please see ScalaDoc for LogSource for how to obtain or construct one.
[error] val log = Logging(system, this)
I believe in the Akka Logging Doc this is the following point:
and in all other cases a compile error occurs unless and implicit LogSource[T] is in scope for the type in question.
In other words there is no LogSource[TestKit]
I would like the simplest solution to deal with that issue, with minimal additional configuration. So far what i did is the following and everything works as expected:
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup {
val log = system.log
From there I just go and do things like
val received = chgtFetcher.receiveWhile((requestInterval + ProcessingLag).*(3)) {
case msg:FetchNewChangeSet => log.info(s"received: ${msg}" ) ; chgtFetcher.reply(NoAvailableChangeSet); msg
}
My question, is this recommended approach. So far the order of the message coming from my actor and the one from the test are well ordered.
What is the recommended approach to log in a unified way:
From the Test class (e.g. above) and the Actor at the same time ?
If one uses a system where external class needs to log as well and we need one unified logging (asynchronous) going on.
Have a look at this comment:
https://github.com/akka/akka/blob/master/akka-actor/src/main/scala/akka/event/Logging.scala#L196-L237
I believe a more straight forward approach would be to define your implicit LogSource[ChangeSetActorTest] locally.
I.E:
val log = {
implicit val logSource = new LogSource[ChangeSetActorTest] {
override def genString(t: ChangeSetActorTest) = "ChangeSetActorTest"
}
Logging(system, this)
}
Simplest way to log in a TestKit is either:
Get the logger from underlyingActor:
val mockActor = TestActorRef(new XXXActor)
val log = mockActor.underlyingActor.log
Use FeatureSpecLike
http://doc.scalatest.org/3.0.1-2.12/org/scalatest/FeatureSpecLike.html
class ChangeSetActorTest extends PersistenceSpec(ActorSystem("Persistent-test-System")) with PersistenceCleanup with FeatureSpecLike {
//...
alert("Something like warning")
info("Infos")
note("Green infos")
markup("documents")
}
I'm new to play framework and have limited experience with scala. Using play framework 2.4 I'm trying to write full integrations tests where I want to call controller, and using in-memory db retrieve data from it. I'm using scalatest-plus together with scalatest. I have read about play unit tests in play documentation and now trying to use play.api.test.FakeApplication but i cannot find a way to inject my guice module to this fake application. I'm ussing OneAppPerSuite trait I can override FakeApplication but cannot pass any guice module for injection. Here is FakeApplication source from:
case class FakeApplication(
override val path: java.io.File = new java.io.File("."),
override val classloader: ClassLoader = classOf[FakeApplication].getClassLoader,
additionalPlugins: Seq[String] = Nil,
withoutPlugins: Seq[String] = Nil,
additionalConfiguration: Map[String, _ <: Any] = Map.empty,
withGlobal: Option[play.api.GlobalSettings] = None,
withRoutes: PartialFunction[(String, String), Handler] = PartialFunction.empty) extends Application {
private val app: Application = new GuiceApplicationBuilder()
.in(Environment(path, classloader, Mode.Test))
.global(withGlobal.orNull)
.configure(additionalConfiguration)
.bindings(
bind[FakePluginsConfig] to FakePluginsConfig(additionalPlugins, withoutPlugins),
bind[FakeRouterConfig] to FakeRouterConfig(withRoutes))
.overrides(
bind[Plugins].toProvider[FakePluginsProvider],
bind[Router].toProvider[FakeRouterProvider])
.build
....
So there is no way for me to pass custom module here.
Here is how my test looks like:
class UsersControllerTest extends PlaySpec with OneAppPerSuite {
override implicit lazy val app = FakeApplication(additionalConfiguration = inMemoryDatabase())
"Application " should {
"work" in {
val resp = route(FakeRequest(GET, "/api/users")).get
}
}
}
And here is my controller I want to test:
class UserController #Inject()
(userService: UserService)
(implicit ec: ExecutionContext) extends Controller {
def get = Action.async { implicit request => {
// implementation
}
}
}
Obviously my test now fail since it cannot find UserService instance.
So my question is:
How to properly write end2end integration tests with play using guice context
Is it common to write such tests in play application, since I cannot find any decent examples or documentation of doing something like that. I used to write such tests in java web applications where integration tests load spring context, but can't find such examples in play.
Thanks in advance.
If you want to test your default configuration, starting FakeApplication, is enough. It loads modules specified in application.conf automatically. So, If it can't find UserService, that means you don't have proper module configurations in your application.conf and there for your server can't serve those requests.
https://www.playframework.com/documentation/2.4.x/ScalaDependencyInjection
If you want to use some Mockups, and override default configurations:
https://www.playframework.com/documentation/2.4.x/ScalaTestingWithGuice
I'm configuring my Akka application to use the SLF4J logger as specified here:
http://doc.akka.io/docs/akka/2.3.4/scala/logging.html
Underneath the hood, I'm depending on Logback to do the logging. I'm developing a common module for logging purposes that users can use in their actor systems. Mainly, I'm creating a trait they can mixin.
I have a trait that does this:
I have something as such:
trait ActorLogger {
val log: DiagnosticLoggingAdapter = Logging(this);
}
I have some extra logic which will add MDC values to the DiagnosticLoggingAdapter's MDC.
The problem is now this: I expose a different logger entirely if users want to mixin to their non-actor classes. So I might have something like this:
trait ClassLogger {
val log = LoggerFactory getLogger getClass.getName
}
I want the MDC values to carry over to this logger. So for example, if I put MDC values into my DiagnosticAdapterLogger, I should expect to be able to get those values from org.slf4j.MDC
How can this be achieved in a clean way?
Thanks!
If all your code outside the actor system is single-threaded (i.e. you don't spawn any additional futures or threads), there's a simpler solution than the one #jasop references.
I have this mixin that takes care of populating the MDC both inside and outside actors:
import akka.actor.DiagnosticActorLogging
import akka.contrib.pattern.ReceivePipeline
import org.slf4j.MDC
import scala.collection.JavaConverters.mapAsJavaMapConverter
trait MdcActorLogging extends DiagnosticActorLogging {
this: ReceivePipeline =>
/**
* This is for logging in Akka actors.
*/
override def mdc(message: Any): akka.event.Logging.MDC = {
message match {
case MyMessage(requestId) => Map("requestId" -> requestId)
case _ => Map()
}
}
/**
* This makes the MDC accessible for logging outside of Akka actors by wrapping the actor's
* `receive` method.
* Implements the [[http://doc.akka.io/docs/akka/2.4/contrib/receive-pipeline.html ReceivePipeline]]
* pattern.
*/
pipelineOuter {
case e # MyMessage(requestId) =>
val origContext = MDC.getCopyOfContextMap
val mdcWithPath = Map("requestId" -> requestId,
// inside actors this is already provided, but outside we have to add this manually
"akkaSource" -> self.path.toString)
MDC.setContextMap(mdcWithPath.asJava)
ReceivePipeline.Inner(evt) // invoke actual actor logic
.andAfter {
if (origContext != null)
MDC.setContextMap(origContext)
else
MDC.clear()
}
case e => ReceivePipeline.Inner(e) // pass through
}
}
The non-actor code can use any logger, e.g. mix in the com.typesafe.scalalogging.LazyLogging trait.