i want bulid faster webmail
i've built small webmail script based on ( php imap functions ( imap port connection ) )
but it take a long time to connect and get the mail ..
So, i decided to read the mail manually without connect ( by my own functions ) ..
i've built my own functions, that go to the ( user mails ) path, and then i use ( scandir function )
to get all mails in the folder, and then read/get them manually!
i'll show you an example code
<?
$current_folder = 'new';
$virtual_user = 'someone';
$path_to_mails = '/home/user/mail/' . $virtual_user . '/' . $current_folder;
$all_emails = scandir( $path_to_mails );
foreach ( $all_emails as $mail_file ) {
$file = file_get_contents ( $mail_file ) ;
//Now i've the mail file ..
//i'll explode it and extract the important information from it
}
?>
Now i got emails without connect to any port
i think it faster than the ( php imap functions ) ...
but it also take a long time to get and read the file!!
why gmail and yahoo is soooooooooooooooooooo faster??? may be they using database to store their webmail files?
NOW MY QUESTIONS IS
1 - is my own functions really faster than the php imap functions theoretically? ( may be i am wrong )
2 - ( Gmail , Yahoo , Hotmail ) where they storing their mail files? database or hard disk? they are so faster and
in the same time they allow you to connect to their server via imap and get your mails via php, that mean they using hard disk to store email files!!
or may be they use database and they customized their webmail softwares
3 - is there any way to customize the postfix, store the mails to database instant of the hard disk??
4 - tell me the best idea to build a faster and strong webmail system
PLEASE DO NOT IGNORE ANY OF THIS QUESTIONS
i am working on this project 3 months ago.. i've tired!
1 - Yes.
2 - Depends on the provider. I assume Yahoo and Hotmail might be using actual IMAP servers but I don't think they disclose their infrastructure.
3 - This does not relate to postfix. Postfix is just the MTA after all. It doesn't store the mails it just transfers them. So you can of course code your own database driven service. Daunting task ;)
4 - Build on existing tools. The easiest choice is to build on top of the Horde Webmail
Webmail is a daunting task. The small snippet of PHP code you showed is really light years away from reality if you consider the complexity of modern webmailers. If you really want something working you need to start with existing building blocks. Horde is the best option there because it is a development framework, provides efficient IMAP caching capabilities, a decent AJAX backend and so on. Nevertheless: Your own webmail service will remain a daunting task nevertheless.
Related
I'm looking for a way to do in Sieve something that I've been doing in Procmail for years, which is to insert an unambiguous date header in incoming messages that makes it clear to me -- independent of buried "received" headers from possibly multiple servers and however my mail client interprets the date the message was sent -- when my server received the message. This is how I did it in Procmail:
# First create the "date_received" variable for my time zone:
date_received=`/bin/date -ud '+2 hour' +'%A %Y-%m-%d %H:%M:%S +0200'`
# Second, insert the header containing the date_received variable:
:0 fh w
| formail -i "X-Local-Date-Received: $date_received"
I found "addheader" (RFC 5293) which will, obviously, add a header, but due to something else I read (sorry, don't remember where) I believe that Sieve won't run the "date" command in the shell due to either a limitation or an intended (and understandable) preference not to run shell commands for security reasons.
Other possibly useful information: I'm doing this through Roundcube 1.3.6, but I have a feeling (also due to something I read) that Roundcube might overwrite a custom Sieve filter set if I edit the raw code within Roundcube. If necessary I'm quite happy to edit or create a Sieve configuration file on the server directly to achieve this for all users on the server, but having run Sendmail and Procmail for years I'm unsure of the best place to do this.
EDIT:
As a test in Roundcube I added this at the top of my Sieve filter set:
require ["fileinto","editheader"];
# rule:[test editheader]
if true
{
addheader "X-Test-Header" "This is a test header.";
}
I didn't actually add the line "require ["fileinto","editheader"];"; I just added "editheader" to the existing line at the top of the filter set, like so:
require ["copy","fileinto","regex","editheader"];
I expect this to add ...
X-Test-Header: This is a test header.
... to every incoming message, but Roundcube won't let me save it:
An error occurred.
Unable to save filter. Server error occurred.
A search for this error returns one related result, with no solution posted.
I'm not intending to focus on Roundcube, however. Like I said earlier, I'll add this Sieve filter from the command line if necessary.
The Pigeonhole Sieve Editheader extension isn't available by default. Per its documentation, you need to ensure it's added in your list of sieve extensions on the server:
plugin {
# Use editheader
sieve_extensions = +editheader
}
If you want to run arbitrary scripts using sieve on Dovecot like you can with procmail, then you can use its external programs plugins, configure in Dovecot which external programs you want to allow users to use, and then the users can use the "vnd.dovecot.execute" extension to run those programs. You might be able to use this to port over whatever scripts you used with procmail.
In the general case, the purpose of sieve is for users to be able to configure their own mail filtering, while it seems like you're trying to actually do something globally for the server. Dovecot should add its own Received header when it processes the mail, which is the standard method for marking when a mail system gets a message, so it's not clear to me why you're not just using that, or what changes you want to make to its default behavior. It may be that what you're looking to do may be better handled in your mail transport agent rather than in your mail delivery agent.
Here is my sieve script that converts Received to Date:
require "editheader";
require "regex";
require "variables";
if not exists "Date" {
if header :regex "Received" "^from[[:space:]]+.*[[:space:]]+by[[:space:]]+mail.mydomain.com[[:space:]]+with[[:space:]]+.*[[:space:]]+for[[:space:]]+.*;(.*)$" {
addheader :last "Date" "${1}";
}
}
Note that mail.mydomain.com is a stand-in for the actual mail server address, which means it only matches the header when the message was received on a specific mail server. I made this work with dovecot-2.3.5.1
You can use date plugin. See: rfc5260:
require "date";
require "editheader";
if currentdate :matches "std11" "*" {
addheader :last "X-Local-Date-Received" "${1}";
}
I need tips to build an infrastructe to send 1000 simultaneous voice calls (automated IVR calls with voicexml). Up to now i used asterisk with voiceglue but now i have performance issues.
The infrasturcture was like this:
the asterisk pulls request from queue
the queue consumer create a call file
when the call ends, call file is read and status is sent to the application server
To be honest, i am asking for tips to implement an infrastructure like callfire[1] or voxeo[2]?
[1]https://www.callfire.com/
[2]http://voxeo.com/
you can go with voxeo prophecy (http://voxeo.com/prophecy/) one of the good server which have the capability of making simultaneous voice calls
Note: The requirement which your are expecting to do will not only possible with voxeo prophecy it should also depend the web server like Tomcat, IIS e.t.c in case if you dealing with databases like Sql , Oracle e.t.c
Please do refer to know the architecture
http://www.alpensoftware.com/define_VoiceOverview.html
CallFire's API has a CreateBroadcast method where you can throw up an IVR using their XML in seconds. You can read up on the documentation here:
https://www.callfire.com/api-documentation/rest/version/1.1#!/broadcast
CallFire also offers a PHP-SDK, hosted on Github, with examples of how to do this. The SDK is minimal setup and allows you to easily tap into the APIs robust functionality. Version 1.1 can be found here, with instructions on how to get started: https://github.com/CallFire/CallFire-PHP-SDK
The method call might look something like this. Note the required dependencies.
<?php
use CallFire\Api\Rest\Request;
use CallFire\Api\Rest\Response;
require 'vendor/autoload.php';
$dialplan = <<<DIALPLAN
<dialplan><play type="tts">Congratulations! You have successfully configured a CallFire I V R.</play></dialplan>
DIALPLAN;
$client = CallFire\Api\Client::Rest("<api-login>", "<api-password>", "Broadcast");
$request = new Request\CreateBroadcast;
$request->setName('My CallFire Broadcast');
$request->setType('IVR');
$request->setFrom('15551231234'); // A valid Caller ID number
$request->setDialplanXml($dialplan);
$response = $client->CreateBroadcast($request);
$result = $client::response($response);
if($result instanceof Response\ResourceReference) {
// Success
}
You can read this:
http://www.voip-info.org/wiki/view/Asterisk+auto-dial+out
Main tip: you WILL have ALOT of issues. If you are not expert with at least 5 years development experience with asterisk, you have use already developed dialling cores or hire guru. There are no opensource core that can do more then 300 calls on single server.
You can't do 1000 calls on single asterisk in app developed by "just nice developer". It will just not work.
Task of create dialling core for 1000 calls is "rocket science" type task. It require very special dialling core, very special server/server tunning and very specialized dialler with pre-planning.
1000 calls will result 23Mbit to 80Mbit bandwidth usage with SMALL packets, even that single fact can result you be banned on your hosting and require linux network stack be tunned.
You can use ICTBroadcast REST API to integerate your application with reknown autodialer , please visit following link for more detail
http://www.ictbroadcast.com/news/using-rest-api-integerate-ictbroadcast--third-party-application-autodialer
ICTBroadcast is based on asterisk communication engine
I've already done this for phone validation and for phone message broadcasting using Asterisk and Freeswitch. I would go with Freeswitch and xmlrpc:
https://wiki.freeswitch.org/wiki/Freeswitch_XML-RPC
The email addresses are stored in a database and the number of people to be emailed each day is variable. I'm not sure yet whether the emails would need to be sent individually or as a mass email. I want recommendations as to what language to use to do this and any other components necessary in a solution.
thanks
In this context, 1,000 people is a pretty small number. I probably wouldn't bother with a database, and I would do the whole thing with the scripting language of my choice (ksh or Lua, in either case piping output to sendmail. This is a very Unix-specific sort of solution.
One thing you may have to watch out for is to throttle the outgoing email—depending on your service provider, if you inject messages into the server at too high a rate, your IP address may be temporarily blacklisted. At home I tell postfix not to deliver more than 1 message per second to Verizon's server.
If I had to write platform-independent code, I would use the LuaSocket library to make a TCP connection directly with a SMTP server. They have a reasonably useful setup for building and sending RFC-compliant messages.
Here is a C# implementation for this:
System.Net.Mail.MailMessage message = new System.Net.Mail.MailMessage("from#address.com", "to#address.com", "subject", "body");
System.Net.Mail.SmtpClient client = new System.Net.Mail.SmtpClient("host.address.com", 1234);
client.Send(message);
Just about any modern language can do this. Java, C#, VB.NET, PHP, PERL, Python and many many more.
Sending emails is such a common requirement that most languages and frameworks support it natively.
As for the requirement of up to 1000 emails a day - that's not that many emails and the limiting factor will be limits imposed by an ISP most likely.
In short - use the language and platform you are most comfortable with and find out how email works in that.
As others have mentioned, it's easy to do this in just about any modern language. I'm a fan of Python, which features great scripting capabilities as well as a solid base for building applications. Python's library is well documented, and includes a number of sophisticated features (including the ability to do multipart MIME encoding).
This is from the examples:
# Import smtplib for the actual sending function
import smtplib
# Import the email modules we'll need
from email.mime.text import MIMEText
# Open a plain text file for reading. For this example, assume that
# the text file contains only ASCII characters.
fp = open(textfile, 'rb')
# Create a text/plain message
msg = MIMEText(fp.read())
fp.close()
# me == the sender's email address
# you == the recipient's email address
msg['Subject'] = 'The contents of %s' % textfile
msg['From'] = me
msg['To'] = you
# Send the message via our own SMTP server, but don't include the
# envelope header.
s = smtplib.SMTP()
s.sendmail(me, [you], msg.as_string())
s.quit()
I want recommendations as to what language to use to do this and any other components necessary in a solution
You can do this in whatever language you feel comfortable with. .NET has some nice stuff built in, and you can probably do it in less than 20 lines of code.
I have a simple email sender for user account activation. Depending on which email address I use, I get significantly different response times: University email - 1 minute, Gmail - 3-4 hours, Yahoo - 1 or 2 days -- which seems bizarre. Has anyone else seen this phenomenon?
EDIT:
There weren't many responses (even for a bounty), but I'll try to explain my problem more clearly.
This probably isn't greylsting -- If I so a simple:
php mail ($to, $subject, $body) // this delivers instantly.
My cakephp code:
function __sendActivationEmail($id) {
$User = $this->User->read ( null, $id );
$this->set ( 'suffix_url', $User ['User'] ['id'] . '/' . $this->User->getActivationHash () );
$this->set ( 'username', $User ['User'] ['username'] );
$this->Email->to = $User ['User'] ['email'];
$this->Email->subject = 'Test.com - ' . __ ( 'please confirm your email address', true );
$this->Email->from = 'noreply#test.com';
$this->Email->template = 'user_confirm';
$this->Email->sendAs = 'text';
$this->Email->delivery = 'mail';
$this->Email->send ();
}
Causes delays from 13 minutes (ok; we'll deal with it) to 5-6 hours (less okay, since this is an activation email). For some of my users, it works instantly, but for other users (of the same service provider, i.e., gmail, it sees these delays).
Any clues?
The code looks fine, but it of course doesn't tell anything about the mail server's configuration.
3-4 hours I would put down to Greylisting, but 1-2 days is definitely too much. Is this reproducible? How many addresses have you tried this with?
What do the full headers of the (received) mails look like? The "received from: .... "path should tell you at which point it took 1-2 days to deliver.
Maybe you can install PHPMailer as a Vendor and create a Component called "Mail"...
And don't forget to authenticate with your SMTP server! :)
Ignore the whole PHP element of it for a moment.
If its a linux server for example, send a mail from the command line e.g. mail myemail#me.com
see if the same thing is happening that way. Its quite likely its a server configuration issue not a php or cakePHP issue.
Look up a few basics like having a FQDN and maybe look into setting up SPF records for your email. Make sure the emails are coming from your domain name not someone elses e.g. not the users email.
Also check if you have email spam software set up that could be grey listing you email on the way out (unlikely but possible). the mostly like thing is the destination spam filter is delaying it. Try send to a gmail account and see if it gets through fine or goes into spam.
Do all this without touching PHP, if all is going fine there then set up a basic php script to do a basic email not using CakePHP, if that works fine then you know its CakePHP etc but I doubt it.
So after further digging, I realized that it was our server host's problem. We use Slicehost, and it just so happens that a range of ips that had been blacklisted included our own ip. We got our name off the list, and we're good to go.
I have a bot that replies to users. But sometimes when my bot sends its reply, the user or their email provider will auto-respond (vacation message, bounce message, error from mailer-daemon, etc). That is then a new message from the user (so my bot thinks) that it in turn replies to. Mail loop!
I'd like my bot to only reply to real emails from real humans. I'm currently filtering out email that admits to being bulk precedence or from a mailing list or has the Auto-Submitted header equal to "auto-replied" or "auto-generated" (see code below). But I imagine there's a more comprehensive or standard way to deal with this. (I'm happy to see solutions in other languages besides Perl.)
NB: Remember to have your own bot declare that it is autoresponding! Include
Auto-Submitted: auto-reply
in the header of your bot's email.
My original code for avoiding mail loops follows. Only reply if realmail returns true.
sub realmail {
my($email) = #_;
$email =~ /\nSubject\:\s*([^\n]*)\n/s;
my $subject = $1;
$email =~ /\nPrecedence\:\s*([^\n]*)\n/s;
my $precedence = $1;
$email =~ /\nAuto-Submitted\:\s*([^\n]*)\n/s;
my $autosub = $1;
return !($precedence =~ /bulk|list|junk/i ||
$autosub =~ /(auto\-replied|auto\-generated)/i ||
$subject =~ /^undelivered mail returned to sender$/i
);
}
(The Subject check is surely unnecessary; I just added these checks one at a time as problems arose and the above now seems to work so I don't want to touch it unless there's something definitively better.)
RFC 3834 provides some guidance for what you should do, but here are some concrete guidelines:
Set your envelope sender to a different email address than your auto-responder so bounces don't feed back into the system.
I always store in a database a key of when an email response was sent from a specific address to another address. Under no circumstance will I ever respond to the same address more than once in a 10 minute period. This alone stopped all loops, but doesn't ensure nice behavior (auto-responses to mailing lists are annoying).
Make sure you add any permutation of header that other people are matching on to stop loops. Here's the list I use:
X-Loop: autoresponder
Auto-Submitted: auto-replied
Precedence: bulk (autoreply)
Here are some header regex's I use to avoid loops and to try to play nice:
/^precedence:\s+(?:bulk|list|junk)/i
/^X-(?:Loop|Mailing-List|BeenThere|Mailman)/i
/^List-/i
/^Auto-Submitted:/i
/^Resent-/i
I also avoid responding if any of these are the envelop senders:
if ($sender eq ""
|| $sender =~ /^(?:request|owner|admin|bounce|bounces)-|-(?:request|owner|admin|bounce|bounces)\#|^(?:mailer-daemon|postmaster|daemon|majordomo|ma
ilman|bounce)\#|(?:listserv|listsrv)/i) {
That really sounds like something that's probably available as a module from CPAN, but I didn't find anything clearly relevant in five minutes of searching. Mail::Lite::Mbox::Processor looks like it might do what you want:
Mail::Lite::Message::Matcher is a
framework for automated mail
processing. For example you have a
mail server and you have a need to
process some types of incoming mail
messages automatically. For example,
you can extract automated
notifications, invoices, alerts etc.
from your mail flow and perform some
tasks based on content of those
messages.
but its docs are sparse enough that it isn't immediately obvious whether it provides those example functions itself or if you have to provide the code to drive them.
In any case, though, if you haven't already checked CPAN, that's where I would start if I wanted to do something like this.
My answer here only deals with bounces which is more straightforward.
Using DSN (Delivery Status Notification) identifier will help you detect a DSN/bounced message. It should go to Return-Path and not Reply-To.
Here's a sample of a typical DSN message. The header information includes the message id, content type has specific values (delivery-status) etc.
Not able to provide you any codes in perl, just my 2 cents of idea.
PS: Do note that not all mail servers or MTA conforms to this, but I guess most do.
There should be a standard way of dealing with this, but the problem is that you'd have to assume that systems that send auto-replies comply to that standard, when most the time, they just don't.
How do you get the address that you reply to? I hope you aren't using the From: header. Check the Reply-to: header first and if that doesn't exist, use the Return-path:.
But whatever you do, you will simply have to keep a log of what you sent to whom and throttle your bot to some sensible value of messages per time.