I need to check a list of about 40 000 mails if they are valid. I want to use this guide: http://www.labnol.org/software/verify-email-address/18220/
It works perfectly one by one but I am afraid that if I made a program that would check them all I could get banned mainly because it would be basically a dos attack.
What do you think? |Is there another way? I cant use any online service for that as I dont own the list of emails.
Thanks
I cant tell you if you will get banned but the techniques I would use to avoid getting banned are
Use a public IP address, proxy or vpn (eg mobile internet, wifi hotspot, TOR) as if you do get banned it wont effect you
start slow, process 5-10/s at first and then speed up, if you get blocked by one server go back to your last known good speed and don't connect to the blocked server for a while, check if you are still blocked manually
depending on the connection setup speed, only do a few emails per domain at a time. ie do 1 #gmail.com, 1 #Hotmail.com and 1 #yahoo.com per batch this stops you flooding one email server with thousands of requests at a time.
Hope this helps
Related
Our IP address recently got listed on UCEPROTECT-1 as a potential spam address, and we aren't able to figure out how to stop this. According to their website, UCEPROTECT-1 listing happens when: IP's get listed in Level 1 automatically if they either try to deliver e-mails to spamtraps or if they are involved in port scans or probes or any kind of attacks against our servers
Some research online suggests that the only way to ensure it doesn't happen again is to find out what is triggering these spam traps and plug them.
Any idea how we can go about looking for what is triggering these automatic listings? Any help would be appreciated!
Some background:
We use GSuite for our email servers, wix.com for our website, and namecheap.com for our DNS.
We'd originally paid to not be listed in the UCEPROTECT-2 and 3 listings but were automatically removed as soon as we got listed under UCEPROTECT-1.
I don't know how G-Suite works but in general check
logfiles of the outgoing e-mail servers for days with "strange" recipients patterns or for more e-mails than on other days
if your domain is listed on other blacklists, maybe that is giving you other hints
The problem is: if you are using the outgoing e-mail servers from Google and some of them are listed on UCEProtect (because other G-Suite customers are sending spam, mostly without knowing it cause they are hacked), you have little chance of fixing this yourselve. This is not really uncommon, me.com/icloud.com (17.58.63.0/24) is listed at UCEProtect right now too.
I looked online and can't seem to find too many similar stories, which is surprising.
I rented a server from a provider, and I run my server software on there. My iOS app connects to it. It's been up for over a month with no issues, and gets about dozens of connections a day.
Starting about 30 hours ago, someone began to connect to it every 2 seconds. Always same IP. I doubt he leaved his phone on for 30 hours? And my app has only <100 daily users, so I have no competitors who would gain from this.
I finally blocked him from my server using iptable. From the software engineering side, what is the common practice for preventing these kinds of things? Am I supposed to keep track from the server side and refuse to accept repeated connections? Do I use some sort of login/handshake, or what do I do?
To clarify, I do not use http or apache. I wrote a server based on BSD sockets using a custom protocol over TCP. I also have a crypto hash and would terminate a connection if the message doesn't hash correctly.
EDIT: I did a count in my connection log. 3 IPs connected totaling 10,000 times over the last 30 hours. 2 of them have since stopped. All from a tiny country in Asia which I won't name.
You need to implement some sort of antirobot in your application:
Drop incoming connection if there are already >= N established connections from the same IP address.
Drop incoming connection if total number of connections from this IP address for last 1, 5, and 30 minutes exceeds predefined thresholds (yes, you will need to keep track of number of connections made from each IP address).
More sophisticated antirobots used in high-load environments would also have means of detecting and blocking not only single IP addresses, but also whole ranges of IP addresses (networks). And also it would be better not to drop connections blindly but to redirect clients to captcha.
Anyway, there is no perfect solution to distinguish between robot and normal client: either you drop some good clients, or you allow some robots to sift through your antirobot.
Ok lets say I want to create a connection between my iPhone app and my server (i'd like to try and use GoDaddy servers for this) to server real time location data to users.
I've seen plenty of good stuff online about using sockets, streams, ASIHttpmessage, CFHTTPMessageRef, etc., but what I'm unclear about is how to set up a server that continuously servers real time data to users (I believe you'd need a stream of data going to the user for this, not just a single http request and response). How does one take a host like GoDaddy and run server code on it. I know you can set up a server like this using terminal, but I don't have access to command line or the ability to run this "server program" from my web host as far as I know. Is there software I can download on my cpanel for this? Do I need a virtual private server and different hosting via GoDaddy maybe?
Does anyone know how I can do this or if my understanding of this whole thing is wrong. Please keep in mind I need this real time (or close to). Please, educate me. I really just need a better understanding of how this works.
So we have command line scripts (written in Python) that sit on customer machines and send us data in CSV after every 24 hours. Now we are at a point that we actually want to be able to tell the clients to send us data any time. Almost all of the customers are on MS Windows machines and requirement is that we can install very little software on the customer machines (and most people cannot even log on to customer machines, only few people can).
I'm not actually sure as to how to best solve this problem. May be following are three possible ways (but looking for better)
We make a daemon in Python and install it on customer machine.
Daemon talks to our servers and we send back configuration
information. In that configuration information we send back the
"sleep duration". So daemon sends us the data and then goes to sleep
for number of seconds defined in "sleep duration" variable. Once the
limit is over, daemon pings us and again we send back the
configuration information. Rinse and repeat.
We install a script on customer machine and it runs every hour. At
our end, we've stored how often a customer should send us data (24
hours, 12 hours, and etc) and when script talks to us we determine
how much time has passed and if it is time that script should be
sending us data? If it's time, then we tell the script to send us
data.
We install a very small server-side (Django or Flask) application
and it runs on customer machines. Whenever we want data we send a
request to customer machine and our small server-side application
serves us. For that may be we will ask our customers to reserve a
port for us (not sure how many customers will actually allow this)
I'm sure there are better ways possible. Can you kindly let me which of the above methods are most suitable? Or please let me know if there exists a better way.
I really appreciate all insights, thanks for all help in advance.
Option 3 may not work. Most people have their machines behind a
fire-wall or router which does NAT. In such a scenario, a server that is listening for a request to come in would not typically be accessible from the public internet.
If they have static IP addresses and if the server is accessible from the public internet, then port scanners would detect it and potentially attempt to do undesirable things. You really do not someone hacking into your customer systems and wrecking havoc on them. Please avoid this option if possible.
However, it is safe to have a server on a customer system as long as it is the one logging into your server and sending data.
A better solution would be to have an app that is continuously
feeding data to your server as it is generated. Is is relatively
easy to do an equivalent of
tail -f csv_file | send_data_home
where send_data_home is program running on your customer's system.
This way there is minimal impact. The csv file creation is not
affected. The send_data_home logs into your server and sends
data as it is generated.
We run a large online community in the Netherlands. Because of that we send a lot of mail to the hotmail email addresses of our members.
Recently we have noticed that not all mail is reaching our members, because we have hit a certain limit or so it seems.
Google doesn't give a solution (yet) but we see a lot of others having the same problem.
Note, We have added (since long) SPF records for our domain, in TXT and SPF types.
What else can we do to tackle this problem?
// Ryan
To add to what bzlm said, hotmail probably isn't rejecting your mails, so much as trying to use rate-limiting to prevent spam. That said, there are a few potential solutions you could use here. You could contact hotmail and see about getting your mailserver exemption status from their rate-limiting. Depending on the size of your community, they may or may not respond to you or be willing to work with you. I suggest this only as the "diplomatic" solution.
Or, you could set up two mail servers; one for hotmail users, and one for everybody else. I know from some of the sites that I run that a lot of people register with hotmail accounts, mostly because everybody has one which they use as a "spam dump" for online services where they don't want to use their real email address. So, as you no doubt realize, the number of hotmail users in your database represents a fairly substantial percentage. Therefore, when you need to send an email, you could determine whether to send it to your normal SMTP server, or your hotmail-designated one. On the hotmail SMTP service, you'd need to add some type of waiting mechanism to sleep a certain amount of time after receiving a 421 response.
The problem with this idea is that the number of hotmail users you have, plus the delay you'll encounter in sending, means that the queue length might very well exceed the number of mails you must send. You could alleviate this problem by setting up secondary/tertiary servers, preferably on other networks... but I'm getting ahead of myself here. At any rate, I did a bit of googling around (as you probably have, too), and this isn't such an uncommon problem, but there is no obvious solution to it.
So likely, you'll either have to create some type of slightly-unorthodox network workaround, or try the "diplomatic" route and contact an organization unlikely to care about your problem. I'd suggest doing both in parallel. :)
421 means that the service is not currently available, and that the client should try again. This could be for any number of reasons, including trying to discourage you from too frequent mailing if Hotmail thinks you might be a spambot.
Why not simply let your outbound smtpd queue the mails and try again?
Why is "not all mail reaching your members"? Don't you try again if you get a 421 response?
EDIT: Do what sqook says.
The only real way to "get around" this is to become a good e-mail citizen. Make it easy for people to unsubscribe from your notifications, establish complaint feedback loops with the major mail providers, remove bouncing e-mails from your list automatically, don't send people e-mails they don't want to receive. Failing to adhere to these simple requirements makes you look like a spammer, and providers like Hotmail will treat you like one.
The mail server IP connecting to Outlook.com server has exceeded the rate limit allowed. Reason for rate limitation is related to IP/domain reputation. If you are not an email/network admin please contact your Email/Internet Service Provider for help.
https://mail.live.com/mail/troubleshooting.aspx
I advise you too wait some times