this question is related to HTML image not showing in Gmail , but the answers there do not (or no longer) work.
the problem is that my perl program (below) fails when my html-formatted messages that I want to send off as soon as an img tag is included. it does not matter whether the image itself is served from the google drive or not. I am guessing that I am running into a novel gmail restriction (my script used to work), but I am not sure.
(of course, this problem is not that my recipients do not see the image; it is that the sending perl script aborts with an error---unfortunately, not with more information explaining to me why it is an error. of course, I understand that my recipients need to agree to view images to prevent tracking.)
so here are my questions:
is this a gmail or a perl module problem?
is it possible to send images, so that if my recipients want to see
images from my website (preferably not just images from my google drive as in my example below), they can agree to see this?
is it possible to get a better error message from google about why it fails?
here is the [almost] working code:
#!/usr/bin/perl -w
use strict;
use warnings;
use Email::MIME::CreateHTML;
use Email::Send;
use Email::Send::Gmail;
my $toemail = 'ivo.welch#gmail.com';
my $subject = 'testing image mailing';
my $bodytext= '<html> <body> fails: <img src="https://drive.google.com/open?id=1K4psrWWolTSqx_f6MQP-T1-FMFpegT1Trg" alt="photo" /> </body> </html>\n';
use readcredentials;
my $gmailaccount= readcredentials( 'account' );
my $gmailuserlogin= readcredentials( 'userlogin');
my $gmailpasswd= readcredentials( 'gmailpassword');
eval {
my $message = Email::MIME->create_html(
header => [
From => $gmailaccount,
To => $toemail,
Subject => $subject,
],
body => $bodytext,
);
my $sender = Email::Send->new(
{ mailer => 'Gmail',
mailer_args => [
username => $gmailuserlogin,
password => $gmailpasswd,
]
}
);
$sender->send($message);
};
warn "Error sending email: $#" if $#;
print STDERR "emailed!\n";
I ran your script, with some modifications to remove the dependency on readcredentials to make it run in my environment, and the email was delivered with no problem. Problems could be:
You could have some problems with your gmail credentials.
The script downloads and attaches the image, so perhaps your local environment is preventing the image from being downloaded.
But without your specific error message, it's hard to diagnose any further.
Related
I want to extract the body of an email from a gmail url: https://mail.google.com/mail/u/0/#inbox/FMfBlahwDrHlsBlahlzHWzQXHFKhjpTp1
I am using the Perl module: Net::IMAP::Simple::Gmail which gets me sequential access to the email inboxes and their imap ids. It also has a search method for searching emails. It also provides imap thread ids of each email.
But I haven't found an easy way to link the thread's URL in the browser to a particular imap email id so I can extract the body.
Here is a solution, though it's a bit convoluted:
Navigate to the gmail message thread of interest in your browser
Run the following JS in the developer console (or better, programmatically with Applescript, if using Safari, or some other browser automation program like WWW::Mechanize::Chrome) to get the "legacy" thread id:
document.querySelector('[data-legacy-thread-id]').getAttribute('data-legacy-thread-id')
Once you get the thread id, you then need to convert it to a decimal number. In Perl:
my $dec_num = sprintf("%d", hex($thread_id));
Now that you have obtained the thread id, you can use the Perl Mail::IMAPClient cpan module to obtain the messages in the thread by searching on the thread id:
use Mail::IMAPClient;
my $imap = Mail::IMAPClient->new(
Server => 'imap.gmail.com',
User => 'me',
Password => 'blah',
Ssl => 1,
Uid => 1,
);
my $folders = $imap->select('INBOX');
my $msgs = $imap->search("X-GM-THRID", $decimal_thread_id);
foreach my $msg (#$msgs) {
my $msg_st = $imap->message_string($msg);
# slice and dice messages with modules listed below
}
Now you can use cpan modules like Email::MIME and Email::MIME::Attachment::Stripper and Email::MIME::Encodings to parse the emails in $msgs and decode them as necessary.
So I'm trying to create a perl script that logs in to SAP BusinessObjects Central Management Console (CMC) page but it doesn't even look like it's finding the right form or finding the right field or even clicking Submit.
Here's my code:
use strict;
use warnings;
use WWW::Mechanize;
use HTTP::Cookies;
my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());
$mech->get("http://myserver:8080/BOE/CMC");
$mech->form_name("_id2");
$mech->field("_id2:logon:CMS", "MYSERVER:6400");
$mech->field("_id2:logon:SAP_SYSTEM", "");
$mech->field("_id2:logon:SAP_CLIENT", "");
$mech->field("_id2:logon:USERNAME", "MYUSER");
$mech->field("_id2:logon:PASSWORD", "MYPWD");
$mech->field("_id2:logon:AUTH_TYPE", "secEnterprise");
$mech->click;
print $mech->content();
When I run it, I don't get any errors but the output I get is the login page again. Even more puzzling, it doesn't seem to be accepting the field values I send it (the output would display default values instead of the values I assign it). Putting in a wrong user or password doesn't change anything - no error but I just get the login page back with default values
I think the script itself is fine since I changed the necessary fields and I was able to log in to our Nagios page (the output page definitely shows Nagios details). I think the CMC page is not so simple, but I need help in figuring out how to make it work.
What I've tried:
1
use Data::Dumper;
print $mech->forms;
print Dumper($mech->forms());
What that gave me is:
Current form is: WWW::Mechanize=HASH(0x243d828)
Part of the Dumper output is:
'attr' => {
'target' => 'servletBridgeIframe',
'style' => 'display:none;',
'method' => 'post'
},
'inputs' => []
I'm showing just that part of the Dumper output because it seems that's the relevant part. When I tried the same thing with our Nagios page, the 'attr' section had a 'name' field which the above doesn't. The Nagios page also had entries for 'inputs' such as 'useralias' and 'password' but the above doesn't have any entries.
2
$mech->form_number(1);
Since I wasn't sure I was referencing the form correctly, I just had it try using the first form it finds (the page only has one form anyway). My result was the same - no error and the output is the login page with default values.
3
I messed around with escaping (with '\') the underscore (_) and colon (:) in the field names.
I've searched and didn't find anything that said I had to escape any characters but it was worth a shot. All I know is, the Nagios page field names only contained letters and it worked.
I got field names from Chrome's developer tool. For example, the User Name form field showed:
<input type="text" id="_id2:logon:USERNAME" name="_id2:logon:USERNAME" value="Administrator">
I don't know if Mechanize has a problem with names starting with underscore or names containing colons.
4
$mech->click("_id2:logon:logonButton");
Since I wasn't sure the "Log On" button was being clicked I tried to specify it but it gave me an error:
No clickable input with name _id2:logon:logonButton at /usr/share/perl5/WWW/Mechanize.pm line 1676
That's probably because there is no name defined on the button (I used the id instead) but I thought it was worth a shot. Here's the code of the button:
<input type="submit" id="_id2:logon:logonButton" value="Log On" class="logonButtonNoHover logon_button_no_hover" onmouseover="this.className = 'logonButtonHover logon_button_hover';" onmouseout="this.className = 'logonButtonNoHover logon_button_no_hover';">
There's only one button on the form anyway so I shouldn't have needed to specify it (I didn't need to for the Nagios page)
5
The interactive shell of Mechanize
Here's the output when I tried to retrieve all forms on the page:
$ perl -MWWW::Mechanize::Shell -eshell
(no url)>get http://myserver:8080/BOE/CMC
Retrieving http://myserver:8080/BOE/CMC(200)
http://myserver:8080/BOE/CMC>forms
Form [1]
POST http://myserver:8080/BOE/CMC/1412201223/admin/logon.faces
Help!
I don't really know perl so I don't know how to troubleshoot this further - especially since I'm not seeing errors. If someone can direct me to other things to try, it would be helpful.
In this age of DOM and Javascript, there's lots of things that can go wrong with Web automation. From your results, it looks like maybe the form is built in browser space, which can be really hard to deal with programmatically.
The way to be sure is to dump the original response and look at the form code it contains.
If that turns out to be your problem, your simplest recourse is something like Mozilla::Mechanize.
When dealing with forms, it can sometimes be easier to replicate the request the form generates than to try to work with the form through Mechanize.
Try using your browser's developer tools to monitor what happens when you log into the site manually (in Firefox or Chrome it'll be under the Network tab), and then generate the same request with Mechanize.
For example, the resulting code MIGHT look something like:
my $post_data => {
'_id2:logon:CMS' => "MYSERVER:6400",
'_id2:logon:SAP_SYSTEM' => "",
'_id2:logon:SAP_CLIENT' => "",
'_id2:logon:USERNAME' => "MYUSER",
'_id2:logon:PASSWORD' => "MYPWD",
'_id2:logon:AUTH_TYPE' => "secEnterprise",
};
$mech->post($url, $post_data);
unless ($mech->success()){
warn "Failed to post to $url: " . $mech->response()->status_line() . "\n";
}
print $mech->content();
Where %post_data should match exactly the data that's passed in the manual post to the site and not just what's in the HTML--the keys or data could be transformed by javascript before the actual post is made.
I had someone more knowledgeable than me give me help. The main hurdle was how the page was constructed in frames and how it operated. Here are the details:
The URL of the frame that contained the login page is "http://myserver:8080/BOE/CMC/0000000000/myuser/logon.faces". The main frame of the page had a form in it, but it wasn't the logon form, which explains why the form from my original code didn't have the logon fields I was expecting.
The other "gotcha" that I ran into was that after a successful logon, the site redirects you to a different URL: "http://myserver:8080/BOE/CMC/0000000000/myuser/App/home.faces?service=%2Fmyuser%2FApp%2F". So to check a successful login, I had to get this URL and check for whatever text I decided to look for.
I also had to refer to the logon form by id and not by name (since the form did not have a name).
Here's the working code:
use strict;
use warnings;
use WWW::Mechanize;
use HTTP::Cookies;
my $mech = WWW::Mechanize->new();
$mech->cookie_jar(HTTP::Cookies->new());
$mech->get("http://myserver:8080/BOE/CMC/0000000000/myuser/logon.faces");
$mech->form_id("_id2");
$mech->field("_id2:logon:CMS", "MYSERVER:6400");
$mech->field("_id2:logon:SAP_SYSTEM", "");
$mech->field("_id2:logon:SAP_CLIENT", "");
$mech->field("_id2:logon:USERNAME", "MyUser");
$mech->field("_id2:logon:PASSWORD", "MyPwd");
$mech->field("_id2:logon:AUTH_TYPE", "secEnterprise");
$mech->click;
$mech->get("http://myserver:8080/BOE/CMC/0000000000/myuser/App/home.faces?service=%2Fmyuser%2FApp%2FappService.jsp&appKind=CMC");
$output_page = $mech->content();
if (index($output_page, "Welcome:") != -1)
{
print "\n\n+++++ Successful login! ++++++\n\n";
}
else
{
print "\n\n----- Login failed!-----\n\n";
}
For validating that I had successfully logged in, I kept it very simple and just searched for the "Welcome:" text (as in "Welcome: MyUser").
I am trying to access an internal webpage to start and stop application using WWW::Mechanize. So far I am able to log in to the application successfully. My next action item is to identify a particular service from list of services and stop them.
The problem I am facing is I am unable to follow the link on the webpage. After looking at the HTML and link object, it is evident that there isn't a URL but an on click event.
Here is snippet of HTML
<ul>
<li>
servicename
</li>
</ul>
The link object dump is
$VAR1 = \bless( [
'#',
'servicename',
'j_id_id1:j_id_id9:2:j_id_id10',
'a',
bless( do{\(my $o = 'http://blah.services.jsf')}, 'URI::http' ),
{
'href' => '#',
'style' => 'color:#3BB9FF;',
'name' => 'j_id_id1:j_id_id9:2:j_id_id10',
'onclick' => 'A4J.AJAX.Submit(\'j_id_id1\',event,{\'similarityGroupingId\':\'j_id_id1:j_id_id9:2:j_id_id10\',\'parameters\':{\'j_id_id1:j_id_id9:2:j_id_id10\':\'j_id_id1:j_id_id9:2:j_id_id10\',\'ajaxSingle\':\'j_id_id1:j_id_id9:2:j_id_id10\'} ,\'containerId\':\'j_id_id0\',\'actionUrl\':\'/pages/services.jsf;jsessionid=NghBSoEJZKXbWcK0uVzcHvyebl8G_zSpf_Zu4uqrLI7xosHAnheK!1108773228\'} );return false;',
'id' => 'j_id_id1:j_id_id9:2:j_id_id10'
}
], 'WWW::Mechanize::Link' );
Here is my code so far:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
use WWW::Mechanize;
my $username = 'myuser';
my $password = 'mypass';
my $url = 'myinternalurl';
my $mech = WWW::Mechanize->new();
$mech->credentials($username,$password);
$mech->get($url);
my $link = $mech->find_link( text => 'servicename' );
#print Dumper \$link;
#$mech->follow_link( url => $link->url_abs() );
$mech->get($link->url_abs());
print $mech->text();
If I use follow_link, I get Link not found at log_in.pl line 16.. If I use get then I get back the same page. The problem is all these services appear to be hyperlinks but have the same url as my main url.
Here is a pic of the webpage:
When I manually click a service the Operations and Properties section change which allows the user to view Operation and Properties of the service they just clicked. Every service has different set of Operations and Properties.
How should I go about do this using perl? Is WWW::Mechanize the wrong tool for this one? Can anyone please suggest a solution or an alternate perl module that could help. Installing any CPAN module is not an issue. Working with latest version of perl is not an issue either. I have just started automating with perl and currently unaware of all the modules that could get the job done.
Looking forward to your guidance and help.
Note: If you feel there is any pertinent information, I may have missed, please leave a comment and I will update the question to add more details. I have modified proprietary information.
That button contains a Javascript onclick event, which will not work when using WWW::Mechanize.
Per the docs:
Please note that Mech does NOT support JavaScript, you need additional software for that. Please check "JavaScript" in WWW::Mechanize::FAQ for more.
One alternative that does support Javascript in a forms is WWW::Mechanize::Firefox.
I currently have a working Perl code that fires off an email to the correct email address with my generic subject and content.
However, currently whenever a new user starts, we fire off a premade .oft template that says everything that they need to know. The .oft is stored on our server. I was wondering is there a way to alter the perl code to make it so it takes the to: but uses the .oft template to make the rest of the email??
so basically
$smtp = Net::SMTP->new('smtp.blah');
$smtp->('no-replay#blah.com');
$smtp->to('$personEmail');
$smtp->data();
$smtp->datasend($locationOfOftTemplate);
$smtp->dataend();
$smtp->quit;
AFAIK there is no module for handling this proprietary format. Instead, translate it to a standard templating system:
my $template = <<'TEMPLATE';
Good morning, [% name %]!
Today is your first day at work. You have already received your universal login
credentials:
[% ldap_user %]
[% ldap_pass %]
It works for authenticating at the Web proxy, mail system, Jabber and internal
services. Change the password ASAP: <[% ldap_web %]>
--
Yours sincerely, the greeting daemon
TEMPLATE
use Text::Xslate qw();
my $text = Text::Xslate->new(syntax => 'TTerse')->render_string($template, {
name => 'New Bee',
ldap_user => 'nbee',
ldap_pass => 'CON-GLOM-O',
ldap_web => 'http://192.168.0.1:8080/',
});
Use a MIME toolkit to create emails. HTML/multipart/attachments are easy with
Courriel::Builder:
use Courriel::Builder;
my $email = build_email(
subject('Welcome'),
from('no-reply#example.com'),
to('…'),
plain_body($text),
);
Finally, send it using the high-level library Email::Sender that gives you nice error checking and allows you to easily switch out transports - run local delivery for testing.
use Email::Sender::Simple qw(sendmail);
use Email::Sender::Transport::SMTP qw();
use Try::Tiny;
try {
sendmail(
$email,
{
transport => Email::Sender::Transport::SMTP->new({
host => 'smtp.example.invalid',
})
}
);
} catch {
warn "sending failed: $_";
};
I’m writing a small Perl page that receives a POST method submit. I want to be able to prevent from a single person/computer to submit the form multiple times (to avoid flooding with repetitive submits). But I can’t find any examples or explanations on how to do this in Perl CGI. Could you advise or direct me to some examples?
I understand I can use some data from the HTTP header (token?) and/or plant a cookie after the first submit, but I’m not sure how.
Any help will be appreciated.
Best Regards,
-Arseny
The most simple way of avoiding users clicking the button several times would be to add some Javascript to your page. That would ofc not work for scripts or for i.e. pressing F5.
<input type="submit" name="go" id="go" value="go" onclick="this.disabled='disabled'"/>
You could also write a log file/database on the server that holds the IP address of the user and the timestamp, and check whether he as already submitted. Doing that in addition to setting and checking a cookie is probably the way to go.
For cookies, see cookies in the CGI doc. Simple example:
use strict; use warnings;
use CGI;
my $q = new CGI;
my $submitted = 0;
if ($q->cookie('submitted ')) {
$submitted = 1;
}
# Here you could place the file/db check to also set $voted
if ($submitted) {
print $q->header('text/plain');
print "You have already submitted!";
} else {
# Do stuff with the form, like $q->param('foo')...
# Once you're done, place the cookie
print $q->header(
-type => 'text/plain',
-cookie => $q->cookie(
-name => 'submitted',
-value => 1,
-expires => '+1y',
));
}