Play 2.6x Scala app does not log despite logback configuration - scala

I'm trying to enable logging in my Play 2.6 Scala app.
I've basically implemented this logging filter to log each request processed by the server: https://www.playframework.com/documentation/2.6.x/ScalaHttpFilters.
I cannot for the life of me get the logs to show in either the console OR the specific log/ directory. I've tried everything from playing around with my logback.xml file and my app config.
Does anyone have any ideas where I'm going wrong? The filter I wrote is being processed on request (I set a breakpoint to ensure).
logback.xml:
<!-- https://www.playframework.com/documentation/latest/SettingsLogger -->
<configuration>
<conversionRule conversionWord="coloredLevel" converterClass="play.api.libs.logback.ColoredLevel" />
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>${application.home:-.}/logs/application.log</file>
<encoder>
<pattern>%date [%level] from %logger in %thread - %message%n%xException</pattern>
</encoder>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%coloredLevel %logger{15} - %message%n%xException{10}</pattern>
</encoder>
</appender>
<appender name="ASYNCFILE" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
</appender>
<appender name="ASYNCSTDOUT" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="STDOUT" />
</appender>
<logger name="play" level="INFO" />
<logger name="application" level="DEBUG" />
<!-- Off these ones as they are annoying, and anyway we manage configuration ourselves -->
<logger name="com.avaje.ebean.config.PropertyMapLoader" level="OFF" />
<logger name="com.avaje.ebeaninternal.server.core.XmlConfigLoader" level="OFF" />
<logger name="com.avaje.ebeaninternal.server.lib.BackgroundThread" level="OFF" />
<!--<logger name="org.jdbcdslog.StatementLogger" level="DEBUG" /> <!– Will log all statements –>-->
<!--<logger name="slick.jdbc.JdbcBackend.statement" level="DEBUG" />-->
<logger name="com.gargoylesoftware.htmlunit.javascript" level="OFF" />
<root level="INFO">
<appender-ref ref="ASYNCFILE" />
<appender-ref ref="ASYNCSTDOUT" />
</root>
</configuration>
LoggingFilter.scala:
package filters
import javax.inject.Inject
import akka.stream.Materializer
import play.api.Logger
import play.api.mvc._
import scala.concurrent.{ExecutionContext, Future}
class LoggingFilter #Inject() (implicit val mat: Materializer, ec: ExecutionContext) extends Filter {
def apply(nextFilter: RequestHeader => Future[Result])
(requestHeader: RequestHeader): Future[Result] = {
Logger.debug("Hi!") // Why you no work!!
val startTime = System.currentTimeMillis
nextFilter(requestHeader).map { result =>
val endTime = System.currentTimeMillis
val requestTime = endTime - startTime
Logger.info(s"${requestHeader.method} ${requestHeader.uri} took ${requestTime}ms and returned ${result.header.status}")
result.withHeaders("Request-Time" -> requestTime.toString)
}
}
}

Well, after several hours of debugging, I finally found the cause (could be helpful for others so posting here).
Cause:
The root cause of the issue is a conflicting logging dependency bundled in play-silhoutte. My app was using play-silhoutte version 5.0.3 which apparently contains a bug causing the Play logger to be overwritten by a logging configuration in a sub-dependency of play-silhoutte(com.typesafe.play:play-openid_2.12:2.6.9).
Diagnosis:
Unfortunately this took a long time to debug - combination of reading Play source code + debugging the stack frames as the Logger gets initialized. I noticed that the Logger was being initialized twice - once for the actual Play logger (with correct configuration as specified in my logback.xml then again as a result of play-silhoutte (effectively overwriting the good logger).
Looking at the initialization params, it was reading its logback.xml from /Users/.../.ivy2/cache/com.typesafe.play/play-openid_2.12/jars/play-openid_2.12-2.6.9.jar!logback.xml. I expanded the jar and found the following logging config:
<!--
~ Copyright (C) 2009-2017 Lightbend Inc. <https://www.lightbend.com>
-->
<configuration>
<!-- Suppress logback complaining about multiple logback-test.xml files -->
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<!-- We use short exception stack trace logging to limit output for travis. -->
<!-- Change to full if you need to do further debugging, but never commit that. -->
<pattern>%level %logger{15} - %message%n%ex{short}</pattern>
</encoder>
</appender>
<root level="WARN">
<appender-ref ref="STDOUT" />
</root>
</configuration>
At this point I knew it was a conflicting depedency issue.
Solution:
After doing some digging, I actually found a bug open on Github describing my issue. I was running play-silhoutte version 5.0.3 so bumping it to 5.0.4 fixed the issue.

Related

Debug logs are not printing for application level logs in root logback.xml configuration

I'm facing an issue with logback for slf4j-api 2.0+ versions, earlier it used to work when I set root logger level to "DEBUG"
Later on, post upgradation to 2.0+ (with logback-classic-1.4.+) I'm unable to print debug statements using root logger.
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration debug="true" scan="true" scanPeriod="60 seconds">
<property scope="system" name="ff_logs" value="${firefly.logging.directory}" />
<include optional="true" file="/etc/firefly.temp/service-logback.xml"/>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>[%d{yyyy-MM-dd'T'HH:mm:ss.SSSZ}] [%thread] %-5level %logger - %msg%n</Pattern>
</layout>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${ff_logs}/${app.name}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${ff_logs}/${app.name}.log.%i.gz</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>20</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>20MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>[%d{yyyy-MM-dd'T'HH:mm:ss.SSSZ}] [%thread] %-5level %logger - %msg%n</pattern>
</encoder>
</appender>
<logger name="com.tran.firefly" level="INFO"/>
<logger name="org.apache.activemq.transport" level="WARN"/>
<logger name="com.org.firefly.sessionmanager" level="INFO"/>
<logger name="com.org.apps" level="INFO"/>
<logger name="com.datastax.driver.core.Cluster" level="INFO"/>
<logger name="com.datastax.driver.core.Connection" level="INFO"/>
<logger name="com.datastax.driver.core.Session" level="INFO"/>
<logger name="com.datastax.driver.core.RequestHandler" level="INFO"/>
<logger name="org.apache.kafka.clients.consumer.ConsumerConfig" level="WARN"/>
<logger name="org.apache.kafka.clients.consumer.internals.Fetcher" level="WARN"/>
<logger name="org.apache.kafka.clients.NetworkClient" level="WARN"/>
<logger name="org.apache.kafka.clients.FetchSessionHandler" level="WARN"/>
<logger name="com.org.firefly.crud.initialization.module.PrepareModuleTask" level="WARN"/>
<logger name="io.netty" level="WARN"/>
<root level="DEBUG">
<!-- <appender-ref ref="STDOUT" /> Remove comment markers to log to STDOUT inside the container -->
<appender-ref ref="FILE" />
</root>
</configuration>
Surprisingly getLogger with package name is working, but not with getClass. I got some answer to similar problem, but in play logs. Someone had similar issue with play logs
val log: Logger = LoggerFactory.getLogger(getClass)
my question is: do I have any work around to get application level logs when I set to Debug? Or is there any official announcement of deprecation?
val log: Logger = LoggerFactory.getLogger("com.package.SomeQualified.CanonicalNameHere")
is working, I could see prints. but it is not a viable solution for me. Suggest on this.
Im testing under com.org.firefly.aaa.authorization package
I have tried to upgrade /downgrade slf4j versions. tried with setting static strings in place of getClass.

How to enable logging DEBUG-level only for the Application Controllers'?

I'm on Scala 2.12.x Play 2.7.x and as part of my Application Controllers I'm using the logger like this (or equivalently by doing with play.api.Logging):
val logger = play.api.Logger(this.getClass)
and the configuration conf/logback.xml is shown below. The issue is that Application-Controllers' logger.debug(...) statements are not being output in the logs. I assumed that the application was set to DEBUG see one of the lasts entries <logger name="application" level="DEBUG" /> but there is obviously something wrong as the DEBUG statements are not being output?
<!-- https://www.playframework.com/documentation/latest/SettingsLogger -->
<configuration>
<conversionRule conversionWord="coloredLevel" converterClass="play.api.libs.logback.ColoredLevel" />
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>${application.home:-.}/logs/application.log</file>
<encoder>
<pattern>%date [%level] from %logger in %thread - %message%n%xException</pattern>
</encoder>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%coloredLevel %logger{15} - %message%n%xException{10}</pattern>
</encoder>
</appender>
<appender name="ASYNCFILE" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
</appender>
<appender name="ASYNCSTDOUT" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="STDOUT" />
</appender>
<logger name="play" level="INFO" />
<logger name="application" level="DEBUG" />
<logger name="slick" level="INFO" />
<logger name="slick.jdbc" level="DEBUG" />
<root level="WARN">
<appender-ref ref="ASYNCFILE" />
<appender-ref ref="ASYNCSTDOUT" />
</root>
</configuration>
PS: I know I can enable DEBUG for everything but then it is too much information I don't need, I'd like to see what's happening with my Controllers' logging.
The logger instantiated with play.api.Logger(this.getClass) inside a class like so
package controller
class HomeController extends ... {
val logger = play.api.Logger(this.getClass)
logger.debug("msg")
}
is not the application logger, but a distinct logger named
controller.HomeController
Thus to specify its log level add the following config to logback.xml
<logger name="controllers" level="DEBUG" />
This makes DEBUG the log level of all classes under controllers package.
Note application logger seems to be deprecated according to docs:
There is also a play.api.Logger singleton object that allows you to
access a logger named application, but its use is deprecated in Play
2.7.0 and above. You should declare your own logger instances...
If the messages are still not logging try changing from async appender to synchronous ones like so:
<root level="WARN">
<appender-ref ref="FILE" />
<appender-ref ref="STDOUT" />
</root>

Change log level for phantom embedded Cassandra

In my Scala project I'am using the phantom-sbt plugin in order to start embedded Cassandra. The problem is, this plugin is pretty verbose - all cassandra logs will be written to stdout.
I've seen on phantom github page, they are using log4j to configure all loggers. But it seems not to work (at least for me). I've set all loggers in the log4j.xml on 'ERROR', but it has no effect.
How should I change log level for all cassandra loggers?
You need a logback-test.xml inside /src/test/resources wherever you are running the embedded Cassandra. Then you can easily turn off individual loggers or set them to the appropriate levels.
Take this as an example configuration:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type
ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
</root>
<logger name="com.datastax.driver.core" level="ERROR"/>
<logger name="io.netty" level="ERROR"/>
<logger name="org.cassandraunit" level="ERROR"/>

Gatling - Log body of request in simulation.log or console

I would like to see what is in the body of the post that I am sending in my script. In fact, I would like to see the request, request body and response. From looking at the docs and the forums, I see that I can uncomment a line in logback-test.xml which I did as shown below
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%-5level] %logger{15} - %msg%n%rEx</pattern>
<immediateFlush>false</immediateFlush>
</encoder>
</appender>
<!-- Uncomment for logging ALL HTTP request and responses -->
<logger name="io.gatling.http" level="TRACE" />
<!-- Uncomment for logging ONLY FAILED HTTP request and responses -->
<!--<logger name="io.gatling.http" level="DEBUG" /> -->
<root level="DEBUG">
<appender-ref ref="CONSOLE" />
</root>
The simulation.log file nor the console shows me the request, response etc. After a bit of googling and reading documentation, I saw that I could do this -
.extraInfoExtractor(extraInfo => List(extraInfo.request, extraInfo.response,extraInfo.session))
This provides me with pretty much everything except the request body. How do I get the request body? I am trying to debug an issue where I am sure the body that is getting sent is not what I actually want.
Add this to your logback.xml
<logger name="io.gatling.http.ahc" level="DEBUG" />
This will print following details for each failure -
Request url
Request header
Request body
Response header
Gatling session data
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%-5level] %logger{15} - %msg%n%rEx</pattern>
<immediateFlush>false</immediateFlush>
</encoder>
</appender>
<timestamp key="timestamp" datePattern="yyyy-MM-dd'T'HH:mm:ss"/>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>logs/test_${timestamp}.log</file>
<append>true</append>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%-5level] %logger{15} - %msg%n%rEx</pattern>
</encoder>
</appender>
<!-- TRACE logs all HTTP requests/response, DEBUG logs only failed HTTP requests/response-->
<logger name="io.gatling.http.engine.response" level="TRACE" />
<root level="INFO">
<appender-ref ref="FILE" />
<appender-ref ref="CONSOLE"/>
</root>
Uncommenting only TRACE and leaving DEBUG commented helped.
An update with Gatling 3.x, the comment in logback.xml is quite self-explain
<!-- uncomment and set to DEBUG to log all failing HTTP requests -->
<!-- uncomment and set to TRACE to log all HTTP requests -->
<!-- <logger name="io.gatling.http.engine.response" level="TRACE" />-->
After uncomment the above config we can get the request url, request header, response etc in the log or IDE'S console. That's very helpful to verify the dynamic feeder value used in request url or body, and it's the way to debug the failed requests.
Just choose the root level as DEBUG
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%-5level] %logger{15} - %msg%n%rEx</pattern>
</encoder>
<immediateFlush>false</immediateFlush>
</appender>
<logger name="io.gatling.http.engine.response" level="DEBUG" />
<root level="DEBUG">
<appender-ref ref="CONSOLE" />
</root>
If you want to see logging of all HTTP requests and responses plus Session contents in your console output you can do the following:
Add or uncomment the following lines in your logger file:
<logger name="io.gatling.http.ahc" level="TRACE" />
<logger name="io.gatling.http.response" level="TRACE" />
Set root level as TRACE:
<root level="TRACE">
<appender-ref ref="CONSOLE" />
</root>

Changing the log level for a running Actor system ?

My resources/application.conf looks like the following.
akka {
loggers = ["akka.event.slf4j.Slf4jLogger"]
loglevel = "DEBUG"
}
I'm creating an fat jar using $sbt assembly that contains this application.conf and deploying this jar to run my actors. Now can I change the log level (from DEBUG to INFO) of my program at runtime without brining down my actor system? If yes, how ?
My logback.xml looks like the following:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<target>System.out</target>
<encoder>
<pattern>%X{akkaTimestamp} %-5level[%thread] %logger{0} - %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logs/myjobs.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>logs/myjobs.%i.log.zip</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>5</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%date{yyyy-MM-dd} %X{akkaTimestamp} %-5level[%thread] %logger{1} - %msg%n</pattern>
</encoder>
</appender>
<logger name="akka" level="INFO" />
<root level="DEBUG">
<appender-ref ref="CONSOLE"/>
<appender-ref ref="FILE"/>
</root>
</configuration>
You can set the Akka loglevel using system.eventStream.setLogLevel() (within an Actor you would use context.system as the starting point).