How do I configure how the format of the logging output?
E.g. the time format, thread name level etc.
How do I get the following?
17:15:00.154 Main INFO : Here is a log...
log4s is merely a wrapper for slf4j. So its purpose is to provide a logging facade, not an implementation. You'll have to decide which actual logger implementation to use. A popular choice is logback, which you would configure using a logback.xml file. See http://www.slf4j.org/manual.html#swapping and http://logback.qos.ch/manual/configuration.html.
Related
In my workflows and activities, I’d like to log some messages for debugging purposes.
I saw the cadence.GetLogger(ctx).Info() function, but don’t know where to find the logs.
Go Client:
You can use the following in the workflow code:
cadence.GetLogger(ctx).Info(...)
In the activity code, you should use the following:
cadence.GetActivityLogger(ctx).Info(...)
By default, the logger will write to console, which may be sufficient for development purposes. However, you should log to a file if you need the logs in production, too. Here is how to setup your cadence worker to do it:
workerOptions := cadence.WorkerOptions{
Logger: myLogger,
}
worker := cadence.NewWorker(service, domain, taskList, workerOptions)
The Cadence client uses zap as the logging framework. You can create the zap logger and specify the log file path per your needs. Check out the zap documentation to learn more about configuring the logs.
Java Client
The Java client uses slf4j for logging. You can get the logger instance by calling Workflow.getLogger() and configure it in logback.xml as usual.
Our log server consumes our log messages through kubernetes pods sysout formatted in json and indexes json fields.
We need to specify some predefined fields in messages, so that we can track transactions across pods.
For one of our pod we use Liberty profile and have issue to configure logging for these needs.
One idea was to use log4j to send customized json message in console. But all message are corrupted by Liberty log system that handles and modifies all logs done in console. I failed to configure Liberty logging parameters (copySystemStreams = false, console log level = NO) for my needs and always have liberty modify my outputs and interleaved non json messages.
To workaround all that I used liberty consoleFormat="json" logging parameter, but this introduced unnecessary fields and also do not allow me to specify my custom fields.
Is it possible to control liberty logging and console ?
What is the best way to do my use case with Liberty (and if possible Log4j)
As you mentioned, Liberty has the ability to log to console in JSON format [1]. The two problems you mentioned with that, for your use case, are 1) unnecessary fields, and 2) did not allow you to specify your custom fields.
Regarding unnecessary fields, Liberty has a fixed set of fields in its JSON schema, which you cannot customize. If you find you don't want some of the fields I can think of a few options:
use Logstash.
Some log handling tools, like Logstash, allow you to remove [2] or mutate [3] fields. If you are sending your logs to Logstash you could adjust the JSON to your needs that way.
change the JSON format Liberty sends to stdout using jq.
The default CMD (from the websphere-liberty:kernel Dockerfile) is:
CMD ["/opt/ibm/wlp/bin/server", "run", "defaultServer"]
You can add your own CMD to your Dockerfile to override that as follows (adjust jq command as needed):
CMD /opt/ibm/wlp/bin/server run defaultServer | grep --line-buffered "}" | jq -c '{ibm_datetime, message}'
If your use case also requires sending log4J output to stdout, I would suggest changing the Dockerfile CMD to run a script you add to the image. In that script you would need to tail your log4J log file as follows (this could be combined with the above advice on how to change the CMD to use jq as well)
`tail -F myLog.json &`
`/opt/ibm/wlp/bin/server run defaultServer`
[1] https://www.ibm.com/support/knowledgecenter/en/SSEQTP_liberty/com.ibm.websphere.wlp.doc/ae/rwlp_logging.html
[2] https://www.elastic.co/guide/en/logstash/current/plugins-filters-prune.html
[3] https://www.elastic.co/guide/en/logstash/current/plugins-filters-mutate.html
Just in case it helps, I ran into the same issue and the best solution I found was:
Convert app to use java.util.Logging (JUL)
In server.xml add <logging consoleSource="message,trace" consoleFormat="json" traceSpecification="{package}={level}"/> (swap package and level as required).
Add a bootstrap.properties that contains com.ibm.ws.logging.console.format=json.
This will give you consistent server and application logging in JSON. A couple of lines at the boot of the server are not json but that was one empty line and a "Launching defaultServer..." line.
I too wanted the JSON structure to be consistent with other containers using Log4j2 so, I followed the advice from dbourne above and add jq to my CMD in my dockerfile to reformat the JSON:
CMD /opt/ol/wlp/bin/server run defaultServer | stdbuf -o0 -i0 -e0 jq -crR '. as $line | try (fromjson | {level: .loglevel, message: .message, loggerName: .module, thread: .ext_thread}) catch $line'
The stdbuf -o0 -i0 -e0 stops pipe ("|") from buffering its output.
This strips out the liberty specific json attributes, which is either good or bad depending on your perspective. I don't need to new values so I don't have a good recommendation for that.
Although the JUL API is not quite as nice as Log4j2 or SLF4j, it's very little code to wrap the JUL API in something closer to Log4j2 E.g. to have varargs rather than an Object[].
OpenLiberty will also dynamically change logging if you edit the server.xml so, it pretty much has all the necessary bits; IMHO.
The Play framework requires (by default) that you configure logging through a logback.xml file. I'd like to build my log appenders through code so I can fetch parameters at runtime (e.g. the graylog destination for the logs is fetched from the deployment environment, rather than baking it in statically through an XML file).
This sort of thing is fairly easy to achieve in Java (by overriding logging factories and the like), I wondered if the same were possible in Play.
Yes, you can configure logback programmatically, see: https://akhikhl.wordpress.com/2013/07/11/programmatic-configuration-of-slf4jlogback/
But I wouldn't recommend it. For starters it's a verbose API that isn't pleasant to work with. Beyond that, it generally nice for configuration to be declarative (even if it is in XML in this case).
For your usecase, Logback's XML does support variables which can come from System properties or Environment variables: https://logback.qos.ch/manual/configuration.html#definingProps
However, you probably want a different config across environments (no greylog locally). I think many projects do that by specifying the logback XML location as a system property at startup: https://logback.qos.ch/manual/configuration.html#configFileProperty
Alternatively, I suspect greylog has some method of watching a file to pickup your logging. That's what we do for picking up logs in Splunk in my team. We don't want to make a change to our code when someone reconfigures Splunk/Greylog.
The solution I used in the end was to use a logback contextlistener to populate the context with the parameters pulled from the environment. The listener can be added as follows to the logback.xml:
<contextListener class="LoggerStartup"/>
The LoggerStartup can then populate the context, which I achieved through AWS SSM (see the simplified code below).
class LoggerStartup extends ContextAwareBase with LoggerContextListener with LifeCycle {
override def start() = {
val context = getContext()
val graylogUrl = ... // Go get value from remote store
context.putProperty("GRAYLOG_URL", graylogUrl)
}
}
And then referenced this context variable in the logback file:
<appender name="GELF UDP APPENDER" class="me.moocar.logbackgelf.GelfUDPAppender">
<remoteHost>${GRAYLOG_URL}</remoteHost>
...
</appender>
I have an issue on the script, basically I don't use any log4net or whatever and im not planning, but some resource which i access during my script i suppose has some references to this log4net, so i get this messages:
log4net:ERROR XmlConfigurator: Failed to find configuration section
'log4net' in the application's .config file. Check your .config file
for the and elements. The configuration
section should look like:
I don't really care about this, as this is also not a real error, i would prefere to somehow hide this messages from the propmpt window, is this possible?
How can I ignore this information, without too much hassle?
This message comes from the log4net internal debugging, and means that not log4net configuration information is found in the config file. What I find strange is that this kind of info is usually opt-in:
There are 2 different ways to enable internal debugging in log4net.
These are listed below. The preferred method is to specify the
log4net.Internal.Debug option in the application's config file.
Internal debugging can also be enabled by setting a value in the application's configuration file (not the log4net configuration file,
unless the log4net config data is embedded in the application's config
file). The log4net.Internal.Debug application setting must be set to
the value true. For example:
This setting is read immediately on startup an will cause all internal debugging messages to be emitted.
To enable log4net's internal debug programmatically you need to set the log4net.Util.LogLog.InternalDebugging property to true.
Obviously the sooner this is set the more debug will be produced.
So either the code of one component uses the code approach, or there is a configuration value set to true. Your options are:
look through the configuration files for a reference to the log4net.Internal.Debug config key; if you find one set to true, set it to false.
add an empty log4net section in the configuration file to satisfy the configurator and prevent it from complaining
if the internal debugging is set through code, you may be able to redirect console out and the trace appenders (see link for where the internal debugging writes to) but this really depends on your environment so you'll need to dig a bit more to find how to catch all outputs. Not really simple
How can JBoss requests/responses ONLY be logged using log4j?
For my 3-tiered application (client, web-service and database), I'm trying to gather request/response times.
For instance, timestamps before/after:
Client sends request
WS receives request
WS sends query to database
Currently, my log displays several thousand lines of text (DEBUG mode). But, I'm looking only for request/response information.
I suppose I could choose a different log level, but I'm not able to find my log4j.xml that most solutions are referring to (server/xxx/conf/jboss-log4j.xml). The log4j.properties file in my Eclipse for some reason is not allowing edits.
I'm new to JBoss; in fact inherited the current setup from somebody else, so I'm a little clueless about the entire JBoss thing.
Edit 1
Examples of log4j.properties can be found here.
Edit 2
My log4j.properties:
log4j.rootLogger=TRACE, file
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.File=C\:log4j.log
log4j.appender.file.MaxFileSize=1MB
log4j.appender.file.MaxBackupIndex=1
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=%d{ABSOLUTE} %5p %c{1}:%L - %m%n
log4j.category.org.springframework.beans.factory=DEBUG
It's probably best to configure logging through the logging subsystem. If you're looking to use your a log4j configuration file, see the instructions here.