My current honey pot on my registration form is an entry for phone number which is placed off screen using position: fixed;.
It tricked a standard breed form filler, but doesn't work as well as I though. Also, the spambots immediately responded to changes in the form, meaning that this is automatic, what can I do.
Note: I hate captchs since the make they users solve my problems which is never good.
Build a really smart honeypot
That may seem obvious, but here are a few tricks(Details later):
Think Like a spam bot
Assume that they are able to know what is on screen or behind other elements
have multiple traps.
Time Trap
Honey pot
1. Think like a spam bot:
Start going through your page like a spam bot, You can even write your own which can waist time but is quite fun :).
Most spam bots will crawl through the markup looking for a <form> element. Then they will look at your inputs and fill them in appropriately, which is the catch: how do they know what to fill in. They will prbably look at the Id, class, placeholder, and label. which brings us to our first method
Method #1:
Mis label inputs in your form code. Bascily your username input should have the Id of #Form_Email boom! spam bot fills out form incorrectly. Also hide and mislabel your inputs labels, use divs instead.*
Method #2 starts here
You've probably noticed that if you simply ignore hidden stuff, based on location what is in front of it and even the good old display: none;,visibility: hidden;,opacity: 0; or type='hidden'. This gives us a powerful weapon. I discovered this by accident while testing a time trap. I used a basic form filler to fill the form. On my site(I'm not talking about GiantCowFilms.com), the register form is in a dialog that opens when a user clicks a register button. By default it is hidden. This gave me an idea for
Method #2
Default: form is hidden. Basically, your form is hidden on page load, but is uncovered by some mouse based action(I don't think bots have mouses). If you wan't your form to be visible on page load, add a I identical decoy one which is above the real one in the markup.If the bot fills in and submits it, block its Ip for a few minuets.** For really users, simply when the mouse hovers over the decoy form, switch them around.
2. Assume that they know what your page looks like
Assuming that hiding honeypot with CSS is perfect is a grave mistake. Their are a lot of super smart screen readers like JAWS that could be repurposed for spaming. That is why you have multiple lines of defense.
3. Have multiple traps
Time Traps:
Going back to thinking like a bot, would you wan't to wait on a site instead of attacking others?
Method #3:Create a time trap.
The best way is to print a time in a hidden input when the page loads. when you submit the form, it tells you how long it took. Fill the form as fast as you can. That should be the minimum amount of time to fill your for.Note: encrypt your time stamp so bots cannot change it. If you wan't to get really fancy, measure the WPM of the bot typing. This is done on stack exchange( try copy and pasting then submitting and question/answer). Also if the rate of typing is very consistent, that is a red flag.
Honeypots (Method #4):
Use all of the above at once for best results. Make sure to trick dumb bots as well as smart bots (don't assume the bot is always trying hard.).
Now, in order to spam us, bots will have to have cursors, render the page, wait, type at a variable realist speed. If they make a bot like that, Then I guess it'll be Captcha time :(.
*People using screen readers will trigger or be confused by these defenses, and depending on your country you could get into trouble for discriminating against blind to semi-blind people. Therefor, when a user triggers the bot test, take them to a non loaded form with a disability friendly captcha like reCaptcha.
**People often share Ips and you can chase away valid users.
P.S. Use simple honey pots like you already have. Some bots are just too dumb to get tricked by what we have here.
For years, I've used my own 'form spammer trap' that uses a variety of techniques to block spambots. The most effective is to obscure fields (especially the form's 'action') for a short time period. Since the bot scrapes your form for the fields, and the 'action' page, and uses that info to "CURL" your form data, putting false info in those areas will fool just about all spambots.
After a timeout (JS-based), the obscured values are replaced with their actual values. By the time that happens, the spambot has moved onto another target, and using the bogus values will fail.
I've developed this whole process into a free solution available at my "Form Spammer Trap" site https://www.FormSpammerTrap.com . That site has been around for years, and the contact form there has yet to be successfully 'spam-botted'. (You bots are welcome to try.)
To implement in your site, you add one 'include' and three functions. And you can customize the form with some advanced techniques.
For those that are interested in how a spambot works, and why common 'tricks' such as hidden fields don't work, see here: https://www.securitydawg.com/how-bots-spam-contact-forms/ .
My 'FormSpammerTrap' solution is free (PHP-based), and is available via the Contact form (just check the box to get the free code). No obligations; I don't even save your email address.
But it is quite effective. IMHO.
Added 21 Jun 2020
The "FormSpammerTrap" solution is still around, and has had significant updates over the years. The latest version allows a lot more (and easier) customization of the form, although the basic use only requires one customization entry. The documentation is greatly improved.
And I am working on version 9, which will add even more customization. It should be released in a couple of weeks.
The https://www.FormSpammerTrap.com site has full details of the latest version, how it work, and a link to a blog entry I wrote about how form spammers work (and how they can be blocked).
And it is still free - just fill out the contact form on the site and check the box and get it immediately, with full documentation and samples. It's fully guaranteed or your money back! (well, it is free...)
(And thanks for the upvote. )
Added July 2022
The program is now up to version 15, with version 16 in the works. There are tons of new features, but the process still works - it blocks all bot attempts on a contact form. Take a look at the changelog on the web site.
And, it's still free.
Related
I'm using Flask-Mail to send email notifications for events and currently I'm encountering a problem where if two emails of the same subject end with the same thing (which is a rather niche circumstance, but definitely necessary to consider nonetheless), the ending part of the second email will be collapsed by certain mail clients (gmail, for example) that think it's a sign-off or signature or whatever. How do I prevent this?
Currently what I'm doing is generating a random sequence of letters and putting it in a <span style="display:none">...</span> and appending that to the email so the client thinks it's a different footer, but that feels really hacky and I want to know if there's a proper way to do it.
After doing some research, it does not seem as though gmail gives you an option to turn this off. All of the articles written on this topic suggest making your footer unique, which is exactly what you are doing.
Furthermore, this option means that the content of the email does not change at all. This seems to be the best solution.
There is an option in Gmail which allows you to turn off conversation view.
How the grouping occurs is:
The subject must be similar
The sender must be a part of the thread OR in-reply-to must be used
Now to save space, Gmail condenses texts that are same, which is what you see. If you want to turn off conversation view in desktops just navigate to setting -- > generals (tab) for desktop and use the radio button select and press save.
If you are after other devices check this link out.
This is a recurring question on the website, but after spending 20 minutes browsing through old questions I was unable to find a modern day solution.
I've previously employed this JS-based method to protect addresses. Before the JS-method I was using image and flash-based solutions. Below is my old way.
Animated example codepen: http://codepen.io/anon/pen/kIjKe/
HTML:
<span class="reverse eml">moc.niamod#tset</span><br>
CSS:
.reverse {
unicode-bidi: bidi-override;
direction: rtl;
}
.eml {
display: inline;
}
JS:
function reverseEmails() {
if (jQuery(".eml.reverse").length > 0) {
jQuery(".eml.reverse").each(function() {
var that = jQuery(this);
var email = that.text().split("").reverse().join("");
that.removeClass("reverse");
that.html("<a href='mailto:" + email + "'>" + email + "</a>");
});
}
}
None of these methods seem to work nowadays, since Node.js based scrapers are able to generate an image of the page they are scraping, then reading any human-readable data from said image - you can guess the rest.
Is there any method that works nowadays, in which users are still able to easily read / click / copy paste e-mail adresses, but JS-enabled bots could not?
This is personally my favorite method, which I have found to work so far, it's not bullet proof, in theory a bot that can parse CSS3 and will preform a text search can still find it or a spambot that triggered events in order to harvest email addresses would have to feed the page into basically a headless browser, somehow determine what might be JS-obfuscated email content these scenarios are enormous amount of work for possibly no benefit and no spammer would ever consider doing it, the fact is I have had no spam to date and it works great for humans, both to read or click on:
<style>
.email:after{ content:'#mydomain.com'; }
</style>
Contact me at:<div class="email">myemail</div>
<script>
$('.email').click(function(){
window.location.href='mailto:'+$(this).html()+'#mydomain.com';
});
</script>
The thing is that the email is not a link so bots never trigger the click event as they don't even know it will do anything.
Put the email address on a separate page which is only reachable by solving a CAPTCHA.
Granted, then the security is only as good as the security of the CAPTCHA.
Using your own obfuscations may be a serious alternative if you only have a limited number of addresses you want to protect. Some ideas I have used in the past;
Crossword puzzle. Make it really easy, with cues like famous song titles with one word missing (easy to google and no debate about possible second interpretations). You can fill in many letters to make it even easier.
Audio recording with background noise. I didn't want to use my own voice so I used a speech synthesizer, with a German accent (-: AT&T web demo IIRC) and mixed in a couple of seconds of music in the background (Frank Zappa's Peaches en regalia worked very well for me, but tastes differ).
Hand drawn image. I like to draw letter outlines but I doubt they are regular enough to pass any OCR.
The real beef here is not the stellar brilliance of these solutions, but the different approaches which I hope can inspire you to think in new directions. In the end, you will always be safer if you come up with your own unique solution; anything resembling a "new de facto standard" will be the lowest-hanging fruit that the scrapers will spend time trying to pluck.
Incidentally, I tried to think about usability for people with disabilities, so I actually deployed the audio version as a fallback for people who have issues with interacting with the other two, which are based on visual layout.
By the by, very few people want to send me email these days anyway (or maybe they do, but end up being rejected as spam?) which is frankly a relief. Those who do typically use the whois registration info for my domain name (which uses an anonymized address provided by the whois registrar) or are good guessers.
I suspect your intuition is correct, if an email address is displayed on a page then a bot can scrape it.
Your best bet is to involve the server-side in some way.
For example, if you just want visitors to be able to reach you, then you can add a "contact us" form like this one: https://store.theonion.com/t-contact.aspx
If you want visitors to be able to reach eachother, then you might need to build an anonymization system like Craigslist uses.
I've always assumed that it's risky to identify users in urls within emails. For example, let's say my app is something like eventBrite. I'm inviting a set of users to an upcoming event. I create unique urls for each user's email which allows them to simply click those url's in the email to accept or decline. Ie, they will not have to authenticate with the website.
If they view the email on a mobile device or a public computer through webmail, then clicking the links will fully accept/decline.
Is this approach too risky? I had assumed you should avoid this as something could see those urls and make requests on them which would trigger false accepts/declines.
It'a an opinion but I would assume the link itself can be more secure than the email actually. You can make the accept link valid only through certain period of time (it would not make much sense otherwise anyways).
Moreover, you can make it pretty much arbitrary long. So it's basically arbitrarily hard to guess.
That would leave two options to "see" the link, that I can think of. Physically seeing it by eaves dropping. But you could generate a mail in the html form, which would allow you to hide the full link behind a hyper-ref text. Like Accept / Decline.
There are several parts to this answer:
Is it secure? Absolutely not. It's security through obscurity. You're betting somebody can't guess the link which, as long as it's a finite string then they totally can and as soon as they do, they can RSVP to your event.
Follow up Does it matter? Probably not. I imagine the chances of somebody trying to spoof an RSVP to an event are pretty slim. I absolutely wouldn't protect anything critical this way but if you're just doing something like event RSVP etc (no money changing hands) I don't see anything wrong with this approach. As luk32 said, you can also make the links valid for limited amounts of time etc.
The real issue here, (unless there's something you're not telling us and this is somehow a high value target) is how likely is somebody to accidentally stumble on one of these links and RSVP to an event they aren't going to? You can make the chances of that exceedingly unlikely by generating the links in a sufficiently random manner so that no two links are a like. In this case, I don't think security is the big concern so much as data integrity. That is, is the data you're receiving valid.
I'm doing a small newsletter software for my business, and I'm wondering what metrics should I collect. Obviously, bounces and clicks should be tracked, but I'm wondering should I track email opens (via an image or a bgsound element)?
Do popular webmail services and ISPs check for tracking images and possibly increase my spam score? I guess if it increases the chances of ending up in a spam filter, I'd rather not collect that metric.
Thanks.
It's generally bad form to try to track your users in that way. Email opens are a private thing.
If you have an image with a unique URL per message, yes you can track it, but IMO, you really shouldn't. Including unnecessary images in an email is bad for a number of reasons:
Images can increase your spam score. There's a time and place for images. They can improve a message, but used inappropriately, they can look spammy.
It is obvious what you are doing. Sooner or later, one of your customers is likely to get wise to it. Some people won't care; others will feel violated.
It's REALLY unreliable. Most email clients and webmails feature an option to block images by default. You will get massively understated results.
Also remember, some people open an email immediately before they click the "delete" button. You are much better off tracking clicks.
There may be some merit in tracking the images you want to include anyway, but I'd not treat it as anything more than a very basic indicator.
As always it depends on the individual ISP and Webmail services. However, I can share some anecdotal evidence: I periodically use mailchimp to send out mass email notifications, and email opens are tracked in mailchimp using the same approach you mentioned (See following link for reference: http://kb.mailchimp.com/reports/about-open-tracking). I never experienced any issues with ending up in the spam folder, I have only had challenges with bounce backs and ending up in the Gmail promotions tab.
So based on the fact that some companies are already doing this, I have to say it probably won't increase your spam score.
today just a general question about preventing scripts to register on my site.
I've got five different registration form for different services and recognized a significant growth of fake registrations. These registration are obviously performed by scripts. Thats why I'm searching for mechanism for detecting scripts in my form and than preventing them to register to my site.
All the forms are written in Java, so I use JSPs as frontend. To use captchas isn't an option, while there are good OCR-technics to hack them. My form basicly consists of 5 input-fields with validation for valid address and email. But if a script uses a standard address (e.g. Springfield, 10 Mean St) and an emailaddress like yxz123#gmail.com it is possible to succeed.
Known technics:
time to fill out all field: every human being needs at least 30 sec. to fill out, everything with less than 30 sec. is a script
hidden form field: a standard script scans a website for form field and posts some garbage into them, a human being never would fill in values in hidden form fields, a script would do this
More possibilities for script detection?
These technics arn't that save and only mark another obstacle for scripts. But are there further technics to detect and avoid scripts?
I really look forward to your experienced and creative answers. :)
Thanks ahead.
I use a capacha that makes users click in a circle that is broken. while there are other ciomplete circles on that page.
i'ts a one click validation.
You can also let them click in the bigest circle that has a certain color
Annoying the users with some form of captcha is the only way, imo.
time to fill out all field: every
human being needs at least 30 sec
So what's to stop the script from slowing down ?
hidden form field: a standard script
scans a website for form field and
posts some garbage into them, a human
being never would fill in values in
hidden form fields, a script would do
this
If the script writer has the time and skill to use OCR, I think he can manage to avoid populating hidden fields.
Use a captcha. Don't use a standard one, use something funny like: 3 * 7 = ?? or stuff like that.