I have configured logstash in a way we can dynamically configure the alert keyword which will send an email when it appears in message.
Logstash fails to send emails when i use a variable as an email trigger keyword.
My old configuration worked: I got emails when there is ERROR keyword in message
if "ERROR" in [message] {
email {
address=>"mailsrv.unix.gsm1900.org"
port=>25
from => "logstash_alert#t-mobile.com"
subject => "(${SPRING_PROFILES_ACTIVE}) Logstash Alert from ${APPLICATION_NAME}"
via => "smtp"
to => "${CLIENT_MAIL}"
body => "In host ${HOST_IP:HOST_NOT_SET} the event line that occurred: %{message}"
}
New config: It is not sending any emails. I have setup the ERROR key word for this variable in /etc/default/logstash file
if "${EXCEPTION_STRING}" in [message] {
email {
address=>"mailsrv.unix.gsm1900.org"
port=>25
from => "logstash_alert#t-mobile.com"
subject => "(${SPRING_PROFILES_ACTIVE}) Logstash Alert from ${APPLICATION_NAME}"
via => "smtp"
to => "${CLIENT_MAIL}"
body => "In host ${HOST_IP:HOST_NOT_SET} the event line that occurred: %{message}"
}
Please help here. Thank you
Jump in your wayback machine to 2016 to see that variables are not supported in conditionals. That post provides a workaround of setting the variable into metadata, which can then be used in the conditional:
mutate {
add_field => { "[#metadata][EXCEPTION_STRING]" => "${EXCEPTION_STRING}" }
}
if [#metadata][EXCEPTION_STRING] in [message] {
...
}
Related
I'm trying to use ELK pipeline to read an email (IMAP), extract generic attachments (mainly PDF, eventually doc or ppt) and put them on ElasticSearch.
This is what I was able to do:
Loading directly to ElasticSearch from file some base64 data using Logstash, using the Ingest Attachment Processor on ElasticSearch to read the base64 content.
Loading data from IMAP (exchange email) I can correctly load all email information on ElasticSearch except the attachment (what I need).
The first solution works fine and does what I am looking for, except that it doesn't extract attachments directly from the email and that I have hardcoded base64 data inside the files.
With the second solution I have a field x-ms-has-attach: yes on Kibana, but there isn't anywhere the attachment itself. The imap plugin is intended to load only the content of the email without the attachment?
What am I missing? Could you suggest me a pipeline to achieve what I am looking for?
This is my logstash configuration for the first example:
input {
file {
path => "/my/path/to/data/*"
start_position => "beginning"
# sincedb_path => "/my/path/to/sincedb"
sincedb_path => "/dev/null"
close_older => 0
tags => ["attachment"]
}
}
output {
elasticsearch {
index => "email-attachment"
hosts => [ "localhost:9200" ]
}
}
This is the pipeline:
PUT _ingest/pipeline/email-attachment
{
"description": "Pipeline to parse an email and its attachments",
"processors": [
{
"attachment" : {
"field" : "message"
}
},
{
"remove" : {
"field" : "message"
}
},
{
"date_index_name" : {
"field" : "#timestamp",
"index_name_prefix" : "email-attachment-",
"index_name_format": "yyyy-MM",
"date_rounding" : "M"
}
}
]
}
This is my logstash configuration for the second example:
input {
imap {
host => "my.domain.it"
password => "mypassword"
user => "myuser"
port => 12345
type => "imap"
secure => true
strip_attachment => true
}
}
output {
elasticsearch {
index => "email-attachment"
hosts => [ "localhost:9200" ]
}
}
UPDATE
I'm using version 5.2.2
In the end I defined a totally different pipeline.
I read emails using a Ruby application with the mail library (you can find it on github), where it's quite easy to extract attachments.
Then I put the base64 encoding of those attachments directly on ElasticSearch, using Ingest Attachment Processor.
I filter on content_type just to be sure to load only "real" attachments, as the multiparts emails treat any multimedial content in the body (ie: images) as attachment.
P.S.
Using the mail library, you should do something like:
Mail.defaults do
retriever_method :imap, { :address => address,
:port => port,
:user_name => user_name,
:password => password,
:enable_ssl => enable_ssl,
:openssl_verify_mode => openssl_verify_mode }
and new_messages = Mail.find(keys: ['NOT','SEEN']) to retrieve unseen messages.
Then iterate over new_messages. After, you can encode a message simply using encoded = Base64.strict_encode64(attachment.body.to_s). Please inspect new_messages to check the exact field names to use.
Your problem might come from strip_attachment => true in the imap input plugin.
Does anybody know a way to include multiple messages in the same email from Logstash?
Currently this is the configuration that I am using:
if [LOGLEVEL] == "ERROR" AND [type] == "application" {
email {
from => "logstash#example.com"
subject => "Application error on %{host}"
to => "foo#example.com"
via => "smtp"
body => "%{message}"
replyto => "bar#example.com"
}
}
and it is sending emails, however what I'd like to be able to do is to send, say, the previous 20 messages from the same logfile, so that there is more information in the emails. Is it possible to use a query as the body of the email?
If that's not possible has anyone been able to get the emails to send a link to a page or location in the Logstash server where more details can be found?
I'm using Logstash version 1.4.2 and have checked the documentation at http://logstash.net/docs/1.4.2/outputs/email but I can't see anything that might allow me to do what I'm trying to do. I've also tried searching for examples of what I want on Google, but I can't find anything where people are including more information than what is in the current event.
Thanks,
Bill
message_format would be help you
if [LOGLEVEL] == "ERROR" AND [type] == "application" {
email {
from => "logstash#example.com"
subject => "Application error on %{host}"
to => "foo#example.com"
via => "smtp"
message_format => "%{mesage} yourlink, etc..."
body => "%{message}"
replyto => "bar#example.com"
}
}
Here's my controller action.
public function actionContact()
{
$model=new ContactForm;
if(isset($_POST['ContactForm']))
{
$model->attributes=$_POST['ContactForm'];
if($model->validate())
{
$headers="From: {$model->email}\r\nReply-To: {$model->email}";
mail(Yii::app()->params['adminEmail'],$model->subject,$model->body,$headers)`enter code here`;
Yii::app()->user->setFlash('contact','Thank you for contacting us. We will respond to you as soon as possible.');
$this->refresh();
}
}
$this->render('contact',array('model'=>$model));
}
It validates the data and shows up the success message. but email isn;t sent to the adminEmail address which is mine.
Thanks.
If it is validating and showing flash messages, then it has to do with mail server, nothing to do with yii. I advise you look into mail server and see if it is running properly.
if you are doing in a localhost you must configure your mail server.you can use smtp for sending mails from localhost.If it is in mail server no need to configure.In yout config/main.php you can do like this for localhost
'mail' => array(
'class' => 'application.extensions.yii-mail.YiiMail',
'transportType' => 'php',
'transportOptions'=>array(
'host'=>'yourhostname',
'port'=>'your port no'
),
'viewPath' => 'application.views.mail',
'logging' => true,
'dryRun' => false
),
Note: i am using YiiMail Extension
So I have a script that I'm trying to get VERP running correctly on. It's using MIME::Lite and postfix as the mail server. Here is the code:
use strict;
use MIME::Lite;
use LWP::Simple;
use Mail::Verp;
my $email = 'someuser#somesite.com';
Mail::Verp->separator('+');
my $verp_email = Mail::Verp->encode('root#somesite.net', $email);
my $content = '<html><body>Hi!</body></html>';
my $msg = MIME::Lite->new(
Subject => 'Hi',
From => 'root#somesite.net',
To => $email,
'Return-Path' => $verp_email,
Type => 'text/html',
Data => $content
);
$msg->send('smtp', 'XXX.XXX.XXX.XXX');
When the message is bounced postfix isn't routing it to the root#somesite.net email inbox. How do I route the message so that the sender of the bounce is the $verp_email value?
I'm trying to create a log of all bounced emails with the email addresses included so that it can then be sent to a file or a database.
If anyone can point me in the right direction with this I would be extremely appreciative.
Thanks.
Return-Path is not the correct place for the VERP address, and will be ignored and/or overridden. You need to put it as the actual, honest to $dmr, real SMTP envelope sender (MAIL FROM:<>) address.
The question is a bit old, but hopefully my answer will contribute to someone who find this while googling.
I had the same problem, and the root cause is that you must use "MAIL FROM: " during the smtp exchange with the target server.
Setting the return-path in the MIME::Header gets overwriten by the smtp server itself precisely based on the MAIL FROM smtp command.
So you can have a Mail envelope containing From: root#somesite.net but make sure the smtp MAIL FROM uses $verp_email
For example, this is what I have done:
my $msg = MIME::Entity->build(
'Return-Path' => 'bounce+user=user-domain.com#my-server.com',
'From' => 'admin#my-server.com',
'To' => 'user#user-domain.com',
'Subject' => $subject,
'Errors-To' => 'bounce+user=user-domain.com#my-server.com'
);
## Then some more handling with MIME::Entity
## and finally send it over smtp
my #rcpt = $msg->smtpsend(
## Make it verbose for debugging
'Debug' => DEBUG,
'Hello' => 'mx1.my-server.com',
'Host' => 'mx.user-domain.com,
'MailFrom' => 'bounce+user=user-domain.com#my-server.com',
'To' => 'user#user-domain.com',
'Port' => 25,
);
I'm building a CakePHP website that sends an e-mail like this:
$email = new CakeEmail('default');
$email->template('test');
$email->emailFormat('html');
$email->to(array('john_doe#example.com' => 'John Doe'));
$email->subject('Test E-mail');
$email->helpers(array('Html', 'Text'));
$email->viewVars(
array(
...
)
);
if ($email->send()) {
$this->Session->setFlash('The e-mail was sent!', 'default', array('class' => 'alert alert-success'));
}
else {
$this->Session->setFlash('An unexpected error occurred while sending the e-mail.', 'default', array('class' => 'alert alert-error'));
}
I'd like to be able to capture the HTML rendered by the e-mail in a variable in addition to actually sending the e-mail. This way, I can record in the database the exact content of the e-mail's body. Is this doable?
Per line 50 of the MailTransport class, it appears the actual send() function returns the message and the header. So instead of:
if($email->send()) {
Try:
$mySend = $email->send();
if($mySend) {
//...
Then, $mySend should be an array:
array('headers' => $headers, 'message' => $message);
Thats what I do in my EmailLib:
https://github.com/dereuromark/tools/blob/2.0/Lib/EmailLib.php
it logs email attempts and captures the email output into a log file (email_trace.log) in /tmp/logs/ - if you are in debug mode it will only log (no emails sent - this has been proven quite useful for local delopment).
you can write a similar wrapper for your case.
but if you want to write it back into the DB Dave's approach seems to fit better.