Problem with project dependencies and logs in scala - scala

I have two scala maven projects, we will call them "A" and "B".
"A" project is a dependency of "B". I use "A" like a library; "A" is included in dependencies in "B" pom.xml file.
And I use org.apache.logging.log4j.scala.Logging library to see the logs with configuration files (xml extension) like this in the src/main/resources folder in the both projects:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} %L - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="group.id.projectname" level="trace" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
<Root level="warn">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
When I run my tests I see perfectly the result of the logs in my IDE console.
But, when I run my project in a cluster in AWS I can see error and warn logs of the project "B" only.
In project "B":
logger.error("message") -> yes
logger.warn("message") -> yes
logger.info("message") -> no
logger.trace("message") -> no
In project "A": I can see all in IDE but nothing in a cluster
I have tried some things:
Remove:
<Logger name="group.id.projectA" level="trace" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
Add dependency exclusion in pom file :
<dependency>
<groupId>group.id</groupId>
<artifactId>projectB</artifactId>
<version>${projectB.version}</version>
<exclusions>
<exclusion>
<artifactId>log4j-api-scala_${scala.tools.version}</artifactId>
<groupId>org.apache.logging.log4j</groupId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
</exclusions>
</dependency>
Change additivity="false" to additivity="true" in log4j2.xml file in "B" project
Change additivity="false" to additivity="true" in log4j2.xml file in "B" project and "A" project
But no test has worked.
Any ideas? Thanks in advance!!

Related

Hello, I was wondering is there any way for use Tinylog with app which is requiring Log4J2 implementation. (adapter or something)

I was wondering is there any way for use Tinylog with app which is requiring Log4J2 implementation. (adapter or something)
If I understand your question correctly, you want to use the tinylog 2 API with an Apache Log4j 2 logging back-end. You can do that by using JBoss logging as an adapter between both.
Required dependencies:
<dependency>
<groupId>org.tinylog</groupId>
<artifactId>tinylog-api</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.tinylog</groupId>
<artifactId>tinylog-jboss</artifactId>
<version>2.2.1</version>
</dependency>
<dependency>
<groupId>org.jboss.logging</groupId>
<artifactId>jboss-logging</artifactId>
<version>3.4.1.Final</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.14.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.14.0</version>
</dependency>
Example Application:
import org.apache.logging.log4j.LogManager;
import org.tinylog.Logger;
public class Application {
public static void main(String[] args) {
Logger.info("Greetings from tinylog!");
LogManager.getLogger().info("Greetings from Apache Log4j 2!");
}
}
Example Log4j configuration (nothing special):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
<Appenders>
<Console name="ConsoleAppender" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %logger %level: %msg%n" />
</Console>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="ConsoleAppender" />
</Root>
</Loggers>
</Configuration>
Output on my computer:
21:01:27.367 [main] Application INFO: Greetings from tinylog!
21:01:27.368 [main] Application INFO: Greetings from Apache Log4j 2!

Getting ClassCastException when trying to cast Logger object to my own wrapper logger

I am trying write a wrapper logger class by extending Logger, which will log class name , method name, message and error/exception details.
Getting exception in below line
ILogger logger =
(CustomLogger)org.apache.logging.log4j.LogManager.getLogger(name);
Using below XML, sometimes it's taking SLF4J logger too and getting SLF4j binding exceptions too.
My pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.Logger</groupId>
<artifactId>WrapperLogger</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>WrapperLogger</name>
<description>Demo project for Spring Boot</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.2.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
===================================================================
log4j.xml
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<!-- Console Appender -->
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yyyy-MMM-dd HH:mm:ss a} [%t] %M,%C,
%-5level %logger{36} - %msg%n" />
</Console>
<!-- File Appender -->
<File name="File" fileName="d:/app.log">
<PatternLayout pattern="%d{yyyy-MMM-dd HH:mm:ss a} [%t] %-5level
%logger{36} - %msg%n" />
</File>
</Appenders>
<Loggers>
<!-- Log everything in custom package -->
<Logger name="com.boraji.tutorial.springboot" level="debug" additivity="false">
<AppenderRef ref="Console" />
<AppenderRef ref="File" />
</Logger>
<!-- Log everything in Spring Boot -->
<Logger name="org.springframework.boot" level="debug" additivity="false">
<AppenderRef ref="Console" />
<AppenderRef ref="File" />
</Logger>
<!-- Log everything in Spring Core -->
<Logger name="org.springframework.core" level="debug" additivity="false">
<AppenderRef ref="Console" />
<AppenderRef ref="File" />
</Logger>
<Root level="info">
<AppenderRef ref="Console" />
<AppenderRef ref="File" />
</Root>
To remove slf4j binding exceptions, you should probably remove spring-boot-starter-logging dependency from your pom.xml.
Actually, this dependency is already added by spring-boot-starter artificat so you need to exclude it like below -
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
<exclusions>
<exclusion>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-logging</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-log4j2</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>

Changing logging level of spark in Scala using log4j2

I am using Scala to do some Spark job (my spark version is 2.1 running on Amazon AWS)
To be able to use log4j2 I have added this to my POM.XML
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
<!--log4j-api and log4j-core versions should be 2.8.2 it's not working for higher version at this moment-->
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.8.2</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.8.2</version>
</dependency>
And I have this log4j2.xml in my resources
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss.SSS}t [%t] %-5level %logger{36} - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Root level="INFO">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
I want to change the level of the log for Spark to WARN because of this issue, but since spark using log4j (version 1 ) I can not control it in my log4j2.xml file. Any workaround?
I have also followed other solution by adding following code but still, it didn't work
import org.apache.log4j.{Level, LogManager, Logger}
val ss = SparkSession.builder().getOrCreate()
ss.sparkContext.setLogLevel("WARN")
LogManager.getLogger("org").setLevel(Level.WARN)
LogManager.getLogger("akka").setLevel(Level.WARN)
Logger.getLogger("org").setLevel(Level.WARN)
Logger.getLogger("akka").setLevel(Level.WARN)
Logger.getRootLogger().setLevel(Level.WARN)

Setting up log4j2 with Scala project

I am using Maven for my Scala project and I am trying to use log4j2.
At First, I just added
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
To list of my dependency and add the following XML file to my resources folder. I have created the resources folder alongside scala folder both in main and test.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level %logger{0} %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="com.simpleproject" level="debug" additivity="false">
<AppenderRef ref="console"/>
</Logger>
<Root level="error">
<AppenderRef ref="console"/>
</Root>
</Loggers>
</Configuration>
Yet it didn't work a produce this error :
ERROR StatusLogger Log4j2 could not find a logging implementation.
Please add log4j-core to the classpath. Using SimpleLogger to log to
the console...
Even I tried properties file instead of XML but still no luck:
name=PropertiesConfig
property.filename = logs
appenders = console, file
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
appender.file.type = File
appender.file.name = LOGFILE
appender.file.fileName=${filename}/full-scala.log
appender.file.layout.type=PatternLayout
appender.file.layout.pattern=[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
loggers=file
logger.file.name=guru.springframework.blog.log4j2properties
logger.file.level = debug
logger.file.appenderRefs = file
logger.file.appenderRef.file.ref = LOGFILE
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
Then I find this post, so I update my pom.xml to have this 3 dependancy:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.10.0</version>
</dependency>
But Still I get the same error and it doesn't show the log the way I want or the level of the log I want (Only shows level above Error)
According to Apache page of this project log4j-api and log4j-core versions should be 2.8.2 (Version 2.8.2 worked for this case)
According to GitHub page of this project log4j-api and log4j-core versions should be 2.11.0

How to configure tranquility to show debug log

I want to use tranquility and its Storm Bolt to send data to druid.So I wrote a storm topology(with dependency of tranquility) and compile it to a jar file, then use run storm with that jar file in local mode. Yet I came across problems: the storm bolt show's nothing wrong and the druid's overlord node log nothing. I feel like the overlord had not received data send by storm bolt.
I want to find out the problem. I did enable storm config's debug
conf.setDebug(true);
but it just show detailed information of every spout and bolt, it does not show tranquility's debug log info. I tried change storm's logback/cluster.xml to
<root level="DEBUG">
<appender-ref ref="A1"/>
</root>
But it seems do not work. I try add a log4j.xml in my project's top directory:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="true"
xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="console" />
</root>
</log4j:configuration>
It does not work too.
I make some digging in tranquility's source code, and see tranquility depend on ladylog library which depend on log4j, so I thought adding a log4j.xml in project would enable the debug mode and can see the debug output of this BeamBot
I am totally confused, because now I wrote a jar that use tranquiity library which use loglady library which use log4j and my jar that run by storm which use logback. Could anyone give me some suggestions?
then use run storm with that jar file in local mode
Since you run the jar file in local mode, it makes no sense to modify logback/cluster.xml. Generally, Storm will read default configurations from the specific dependency like storm-core-0.9.4.jar under local mode, unless you change configurations manually in your topology. You should run you topology in cluster mode to have a try.
On the other hand, since there are many conflicting problems with log4j, we use slf4j as the default library for log4j like this
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class MyBolt {
private static final Logger LOG = LoggerFactory
.getLogger(MyBolt.class);
}
and excludes all other log4j libs in all dependencies like this
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.1</version>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
Hope this helps.