How to configure tranquility to show debug log - scala

I want to use tranquility and its Storm Bolt to send data to druid.So I wrote a storm topology(with dependency of tranquility) and compile it to a jar file, then use run storm with that jar file in local mode. Yet I came across problems: the storm bolt show's nothing wrong and the druid's overlord node log nothing. I feel like the overlord had not received data send by storm bolt.
I want to find out the problem. I did enable storm config's debug
conf.setDebug(true);
but it just show detailed information of every spout and bolt, it does not show tranquility's debug log info. I tried change storm's logback/cluster.xml to
<root level="DEBUG">
<appender-ref ref="A1"/>
</root>
But it seems do not work. I try add a log4j.xml in my project's top directory:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug="true"
xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern"
value="%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L - %m%n" />
</layout>
</appender>
<root>
<level value="DEBUG" />
<appender-ref ref="console" />
</root>
</log4j:configuration>
It does not work too.
I make some digging in tranquility's source code, and see tranquility depend on ladylog library which depend on log4j, so I thought adding a log4j.xml in project would enable the debug mode and can see the debug output of this BeamBot
I am totally confused, because now I wrote a jar that use tranquiity library which use loglady library which use log4j and my jar that run by storm which use logback. Could anyone give me some suggestions?

then use run storm with that jar file in local mode
Since you run the jar file in local mode, it makes no sense to modify logback/cluster.xml. Generally, Storm will read default configurations from the specific dependency like storm-core-0.9.4.jar under local mode, unless you change configurations manually in your topology. You should run you topology in cluster mode to have a try.
On the other hand, since there are many conflicting problems with log4j, we use slf4j as the default library for log4j like this
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class MyBolt {
private static final Logger LOG = LoggerFactory
.getLogger(MyBolt.class);
}
and excludes all other log4j libs in all dependencies like this
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.1</version>
<scope>provided</scope>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
</exclusions>
</dependency>
Hope this helps.

Related

Problem with project dependencies and logs in scala

I have two scala maven projects, we will call them "A" and "B".
"A" project is a dependency of "B". I use "A" like a library; "A" is included in dependencies in "B" pom.xml file.
And I use org.apache.logging.log4j.scala.Logging library to see the logs with configuration files (xml extension) like this in the src/main/resources folder in the both projects:
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} %L - %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="group.id.projectname" level="trace" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
<Root level="warn">
<AppenderRef ref="Console"/>
</Root>
</Loggers>
</Configuration>
When I run my tests I see perfectly the result of the logs in my IDE console.
But, when I run my project in a cluster in AWS I can see error and warn logs of the project "B" only.
In project "B":
logger.error("message") -> yes
logger.warn("message") -> yes
logger.info("message") -> no
logger.trace("message") -> no
In project "A": I can see all in IDE but nothing in a cluster
I have tried some things:
Remove:
<Logger name="group.id.projectA" level="trace" additivity="false">
<AppenderRef ref="Console"/>
</Logger>
Add dependency exclusion in pom file :
<dependency>
<groupId>group.id</groupId>
<artifactId>projectB</artifactId>
<version>${projectB.version}</version>
<exclusions>
<exclusion>
<artifactId>log4j-api-scala_${scala.tools.version}</artifactId>
<groupId>org.apache.logging.log4j</groupId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
</exclusion>
</exclusions>
</dependency>
Change additivity="false" to additivity="true" in log4j2.xml file in "B" project
Change additivity="false" to additivity="true" in log4j2.xml file in "B" project and "A" project
But no test has worked.
Any ideas? Thanks in advance!!

Quartz 1.8.5 + OpenLiberty 18.0.0.4/ Websphere Liberty 17.0.0.4 java.lang.NoClassDefFoundError: oracle/sql/BLOB

we get the following exception when using quartz 1.8.5 and the liberty server. While using a tomcat-server (7.0.81) the exception doesn't occur.
java.lang.NoClassDefFoundError: oracle/sql/BLOB
at org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.writeDataToBlob(OracleDelegate.java:642)
at org.quartz.impl.jdbcjobstore.oracle.OracleDelegate.insertJobDetail(OracleDelegate.java:207)
pom.xml
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz</artifactId>
<version>1.8.5</version>
</dependency>
<dependency>
<!-- ASYNC-METHOD-INVOCATION -->
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-oracle</artifactId>
<version>1.8.5</version>
</dependency>
<dependency>
<groupId>com.oracle</groupId>
<artifactId>ojdbc6</artifactId>
<version>12.1.0.2.0</version>
<scope>provided</scope>
</dependency>
dataSource
<server>
<library id="oracleDriver">
<fileset dir="..\sw\oracle" includes="*.jar" scanInterval="120s" />
</library>
<!-- xa datasource -->
<dataSource id="ORACLE_DS_XA" jndiName="jdbc/xxx/xxx" pool-name="xxx">
<jdbcDriver libraryRef="oracleDriver" javax.sql.ConnectionPoolDataSource="oracle.jdbc.xa.client.OracleXADataSource" />
<properties.oracle URL="jdbc:oracle:thin:#localhost:1521:sid" password="user" user="password" />
<connectionManager minPoolSize="1" maxPoolSize="10" />
</dataSource>
<keyStore id="defaultKeyStore" password="password" />
</server>
What could be the reason for that exception? And how can we solve the problem?
Tell me if i should provide more information about our configuration.
TIA
I'm not really familiar with either, but if quartz is bundled in your application and needs access to the classes from your Oracle driver, then you need to expose the shared library to your application.
You'd do this by adding a <classloader> section to your <application> or <webapplication> block in your server.xml
E.g.
<application ...rest of your app configuration...>
<classloader commonLibraryRef="oracleDriver"/>
</application>
If you're currently deploying your app by putting it in the dropins directory, you'll have to change that to deploy your application to the apps directory instead and create an <application> or <webApplication> block in your server.xml.
Documentation links:
Deploying an app and adding the server.xml configuration
Reference for <application> element (includes classloader as a sub-element)
Reference for <webApplication> element (includes classloader as a sub-element)

Setting up log4j2 with Scala project

I am using Maven for my Scala project and I am trying to use log4j2.
At First, I just added
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
To list of my dependency and add the following XML file to my resources folder. I have created the resources folder alongside scala folder both in main and test.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
<Appenders>
<Console name="console" target="SYSTEM_OUT">
<PatternLayout pattern="%d{HH:mm:ss.SSS} %-5level %logger{0} %msg%n"/>
</Console>
</Appenders>
<Loggers>
<Logger name="com.simpleproject" level="debug" additivity="false">
<AppenderRef ref="console"/>
</Logger>
<Root level="error">
<AppenderRef ref="console"/>
</Root>
</Loggers>
</Configuration>
Yet it didn't work a produce this error :
ERROR StatusLogger Log4j2 could not find a logging implementation.
Please add log4j-core to the classpath. Using SimpleLogger to log to
the console...
Even I tried properties file instead of XML but still no luck:
name=PropertiesConfig
property.filename = logs
appenders = console, file
appender.console.type = Console
appender.console.name = STDOUT
appender.console.layout.type = PatternLayout
appender.console.layout.pattern = [%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
appender.file.type = File
appender.file.name = LOGFILE
appender.file.fileName=${filename}/full-scala.log
appender.file.layout.type=PatternLayout
appender.file.layout.pattern=[%-5level] %d{yyyy-MM-dd HH:mm:ss.SSS} [%t] %c{1} - %msg%n
loggers=file
logger.file.name=guru.springframework.blog.log4j2properties
logger.file.level = debug
logger.file.appenderRefs = file
logger.file.appenderRef.file.ref = LOGFILE
rootLogger.level = debug
rootLogger.appenderRefs = stdout
rootLogger.appenderRef.stdout.ref = STDOUT
Then I find this post, so I update my pom.xml to have this 3 dependancy:
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api-scala_2.11</artifactId>
<version>11.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-api</artifactId>
<version>2.10.0</version>
</dependency>
<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-core</artifactId>
<version>2.10.0</version>
</dependency>
But Still I get the same error and it doesn't show the log the way I want or the level of the log I want (Only shows level above Error)
According to Apache page of this project log4j-api and log4j-core versions should be 2.8.2 (Version 2.8.2 worked for this case)
According to GitHub page of this project log4j-api and log4j-core versions should be 2.11.0

Wildfly Swarm and cxf-rt-transports-http

I'm trying to convert my project from EAP 6.3 to Wildfly Swarm...
I'm having some trouble with CXF.
I've got the fraction dependency:
<dependency>
<groupId>org.wildfly.swarm</groupId>
<artifactId>camel-cxf</artifactId>
</dependency>
And then later, I include this, because otherwise I get a "package org.apache.cxf.message does not exist" error in my Camel Processor code
<dependency>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-rt-transports-http</artifactId>
<version>3.1.9</version>
</dependency>
So if I include it, I get the error:
Apache CXF library (cxf-core-3.1.9.jar) detected in ws endpoint
deployment; either provide a proper deployment replacing embedded
libraries with container module dependencies or disable the
webservices subsystem for the current deployment adding a proper
jboss-deployment-structure.xml descriptor to it. The former approach
is recommended, as the latter approach causes most of the webservices
Java EE and any JBossWS specific functionality to be disabled.
Ok, so maybe there's a lib conflict between the camel-cxf modules and my jar... or it's not picking up my jboss-deployment-structure.xml file?
I've tried a few things as suggested by other posts.
I tried adding this to jboss-deployment-structure.xml under ${basedir}/src/main/webapp/WEB-INF and adding the xml file to the webResources of the maven-war-plugin.
<deployment>
<exclude-subsystems>
<subsystem name="webservices" />
</exclude-subsystems>
</deployment>
I've also tried adding this to jboss-deployment-structure.xml:
<dependencies>
<module name="org.apache.cxf" />
<module name="org.apache.cxf.impl" />
</dependencies>
But I keep getting that error. So I'm not sure how to resolve this conflict. Any idea?
One Approach is to use Jboss provided CXF libraries and set <scope>provided</scope> for cxf-rt-transports-http, so that it compiles and but not package the CXF dependencies .
You find more on Jboss specific configuration

Spring boot and JBoss 8 Wildfly log configuration application

I have a Spring boot application prepare to be a WAR. It deploys without problems on Tomcat 8 (embedded or standalone) as well as on JBoss 8 Wildfly.
But while on Tomcat we have had configured working logback configuration on JBoos it does not work any more.
I have tried few different suggested solutions:
https://stackoverflow.com/a/21887529/3997870
https://stackoverflow.com/a/23080264/3997870
The best which I have found was to add to my project WEB-INF/jboss-deployment-structure.xml with
<jboss-deployment-structure>
<deployment>
<!-- exclude-subsystem prevents a subsystems deployment unit processors running on a deployment -->
<!-- which gives basically the same effect as removing the subsystem, but it only affects single deployment -->
<exclude-subsystems>
<subsystem name="logging" />
</exclude-subsystems>
</deployment>
</jboss-deployment-structure>
but still it does not solve problem completely. In the logs I have same line twice (not because of logback configuration because on Tomcat worked properly) and also double info about time, level, thread was printed in first record.
[2014-11-26 15:28:42,605] [INFO ] [MSC service thread 1-3 ] [stdout] [NONE ] [2014-11-26 15:28:42.605 INFO 8228 --- [vice thread 1-3] o.s.boot.SpringApplication : Starting application on LCJLT306 with PID 8228 (started by Piotr.Konczak in D:\servers\wildfly-8.2.0.Final\bin)
]
[2014-11-26 15:28:42,605] [INFO ] [MSC service thread 1-3 ] [o.s.boot.SpringApplication] [NONE ] [Starting application on LCJLT306 with PID 8228 (started by Piotr.Konczak in D:\servers\wildfly-8.2.0.Final\bin)]
As you can see in above example first record contains somehow additional timestamp, level and thread (I guess add by Wildfly during some redirect) while the second line is proper and expected.
My logback config has 2 parts - 1st inside app and 2nd outside app to make it possible to reconfigure in environments).
Inside the classpath:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/base.xml" />
<include file="/opt/appName/config/appNameLogConfig.xml" />
</configuration>
Outside the app (included one):
<?xml version="1.0" encoding="UTF-8"?>
<included>
<property name="DESTINATION_FOLDER" value="/opt/appName/logs" />
<property name="FILE_NAME" value="AppName" />
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DESTINATION_FOLDER}/${FILE_NAME}.log</file>
<append>true</append>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!--daily rollover-->
<fileNamePattern>${DESTINATION_FOLDER}/${FILE_NAME}.%d{yyyy-MM-dd}.log</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>%-5level %date %-30thread %-30logger{30} [%-10mdc{requestId:-NONE}] %msg%n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="INFO"/>
<logger name="org.hibernate" level="INFO"/>
<logger name="com.wordnik" level="INFO"/>
<logger name="com.mangofactory" level="INFO"/>
<logger name="com.company.appName" level="INFO"/>
<root level="INFO">
<appender-ref ref="FILE" />
</root>
</included>
Does anyone see possible reason or misconfiguration?
I know it is a little bit late but, if some of you face this problem, an alternative is: Don't disable the whole logging subsystem, instead just exclude slf4j libraries provided by JBoss/Wildfly, to use the one used by spring-boot.
<jboss-deployment-structure>
<deployment>
<exclusions>
<module name='org.slf4j'/>
<module name='org.slf4j.impl'/>
</exclusions>
</deployment>
</jboss-deployment-structure>
Hope helps somebody.
I am using my own logging configuration, log4j2 xml instead of spring's logging and faced the same issue with the Wildfly. I commented the whole subsytem system for over-riding that.