I would like to know if and how it is possible to create a clickable email-link for websites, that are "encrypted" in a way emailspiders can't collect them and it is still possible for living users to click it to open in email-clients or even copy it.
I saw some links that were done in javascript but I on't know how they did this and how "safe" they are.
thank you in advance for any reply
Most approaches to this are splitting the address across multiple elements and inserting extra formatting; then for JS-enabled browsers, they use JavaScript to turn it back into an e-mail address.
The poster example for this is SpamSpan, which even has several "levels" of obfuscation - each level progressively less and less resembles an e-mail in the source code, yet it still manages to piece it back together by JS. Although some spambots today are supposedly capable of executing JavaScript, te vast majority doesn't - and the e-mails are still human-readable with JS off. An advantage of JS-assisted de/obfuscation is that it doesn't rely on external servers, you just need to (simply) integrate the JS library.
Another approach is taken by reCAPTCHA Mailhide - the e-mail is revealed only after solving a CAPTCHA (same type as for normal reCAPTCHA). This is less convenient for the user, but practically safe against robots. A disadvantage of this is that it depends on reCAPTCHA's servers (in essence, on Google) - some people are dead-set against any external dependencies.
This would be a very simple and effective way:
Scramble email addresses
All it does is convert it into ASCII, and all you need to do is insert it where your email address would go!
Although there are more (crazily) secure ways you can choose, this would be the simply option. You can also try this solution, it uses JavaScript to protect your email.
Hope this helps!
Related
I finally moved my old Outlook Express stuff to Thunderbird, and am downloading all emails stored on POP3 server.
But I don't want Thunderbird to download them all at once. I want 20 at a time, so I can see them and delete junk ones and move remaining to their proper folders before downloading more.
I can't find a button to stop downloading. Once I make Thunderbird download emails, it just keeps downloading and the only way I found to stop it is closing it.
Is there a better way?
NOTE: while the specific counts are not possible, you can switch back and forth between offline/not offline, but it's a pain, requires confirm click.
Otherwise:
Download all emails. Use spam emails to train your thunderbird spam filters once all the emails have downloaded. Start at the beginning of the list. It's a one time operation, don't overthink it. As you assign 'junk' status to emails, you can rerun the spam filters on the inbox until it's properly trained, and most of the subsequent emails will get moved to the Junk folder automatically after you've created a decent set of spam samples. If you give the junk filter a few hundred samples it will be pretty accurate. Check junk folder for false positives, mark not junk, repeat until it's good. Much less work than doing it 20 emails at a time.
Thunderbird's builtin spam filters are quite good once properly trained.
When viewing (including preview mode) spam emails, make sure that third party image view is disabled, otherwise you will be validating your email address in the spammer email databases (they use often images or links with unique id's in them that when loaded or clicked simply validate your email address as current). Ideally disable all html viewing and view in text mode only, no html.
As with training your spam filters, the same applies to email rules/filters, which are even easier to create and run.
Create filters that move the emails to the desired folders after you make the folders, and after you create each filter, run the filters on the inbox again.
Because the user is fairly stubborn in this case, here's the basics when interacting with a new piece of software:
Check to see if the desired feature that another similar program has is supported. If it isn't, and you're sure you didn't just miss it, the developers don't care about that feature, or the feature is not technically possible due to how the software is constructed. The users in general also don't care about that feature, otherwise someone would have filed an issue, and others would have piled on over time, until the devs got tired of that issue and made it happen. In this case, we can assume none of that happened.
Check to see if the program supports extensions of some kind. Thunderbird does, so look for an extension that extends the features of the software to do what you want. If nobody wrote this extension, this means that no person capable of programming cares about that feature, in most cases, or, that the software itself doesn't support it internally.
check on a site like here to make sure you didn't miss anything, and that actually that feature does exist somewhere somehow. If your results are negative again, that's about the end of the road in the quest for that personal favorite feature.
Check to see if there is another way that sort of emulates what you want. In this case, for example, you can go offline/online though it takes a few steps each time, and certainly isn't a feature intended to do what the OP asked, but it can be used in this way crudely.
If neither 1 nor 2 show support for the desired feature, decide how important that feature was to you. If it is very important, research the core codebase and see if you can extend it via extension to support the feature, or hire someone to write the extension for you, and maintain it over years, or don't use the software.
Now, if after these checks, you find that not one single person in the world cared enough to add support for the feature, it means there's not a whole lot of demand for it, and certainly no demand from the free software developers who in the case of thunderbird, wrote it, and its extensions, and probably not from the users either, which is a lot of people globally in this case.
Once you determine that there was and is no demand for the feature, move on to something more productive, and stop complaining that the feature that apparently nobody ever was willing to take the time to implement isn't implemented, it means nobody cared enough to do it, that's all. Complaining here about that won't change that fact.
If you have decided to use the new software anyway, then adjust your workflow to the fact that the desired feature is not going to be part of your life anymore. This is also known as being an adult in some societies.
Other similar things that don't support everything that another similar software does:
gimp does not support everything photoshop does
windows does not support everything gnu/linux does
gnu/linux does not support everything that windows does
claws mail does not support everything thunderbird does
thunderbird and outlook express are different pieces of software that are similar in function but internally quite unrelated, they don't even use the same mail storage format, for example.
Mac OSX is different from GNU/Linux, FreeBSD (although internally it's got some similarities), and Windows.
Windows 98 is different from Windows 2000 which is different from Windows XP which is different from Vista which is different from Windows 7 which is different from 8 and 8.1 which is different from Windows 10.
Outlook is different from every email client known to man and woman alike. As is its mail storage format. Be very glad you are not trying to find software that is similar, you'd be out of luck, since nobody else is crazy enough to produce such a mess.
And so on. The software I write does similar things to other software, but is different, quite different, from all of them. This is why people use it and like it. If it were the same, there would be no reason to use it. If someone posts an issue asking for a feature a similar but different piece of software has, I'd think: Ok, is it a good idea? Does it meet a valid need? If so, is it hard? Is it possible at all (the other software may not even be right for example)? Is it possible within my software's framework? Did I receive a patch (always a great way to motive a developer)? If the first condition, it being a good idea, was never met, then you can forget about the rest, they are irrelevant.
I agree this isn't a great stackoverflow question.
HTML e-mails are a complex beast. Deciding what to send (as the sender) and what to display (as the recipient) is tricky and potentially dangerous.
On the recipient side of things, we have webmail and we have regular e-mail clients. For my purposes, I consider 'webmail' anything that displays the HTML e-mail as part of something that in itself is HTML, and regular e-mail clients anything that displays the HTML e-mail in a different context (e.g. OS- and program-specific GUI).
What should webmail do with HTML headers (<head>, <title>, <meta>, ...) in an e-mail?
Is there a spec somewhere, be that as an actual standard or de-facto-standard?
My motivation for asking is that we use HTML Purifier to sanitise our HTML and if its Core.CollectErrors feature reports changes, they're reported. This 'reported' is both necessary... and frustrating. We strip out some of the reported errors as insignificant for our purposes, but HTML headers mark a massive hurdle:
Someone could potentially use <link> in their e-mail, which we would strip out. (HTML Purifier is intended for HTML fragments, not full documents)
The desire to use things like <link> in HTML e-mails certainly seems to exist, and there are plenty of e-mail clients that send <meta>-tags in an HTML header (e.g. Outlook), but how are things handled in the wild? Is it safe to strip them out silently (which for our purposes denotes a 'non-breaking change') and lay proverbial blame on the sending party if it does break? Is that reasonable? Has someone ever decided this in the one or other way? My google-fu is weak. :(
I seriously doubt there is a spec anywhere which specifies how HTML emails should be embedded into webmail clients. It's mostly a question of achieving parity with existing webmail providers which provide the ability to view HTML email. I suspect stylesheets are a notable exception, but I also suspect most HTML mailers that support that heavy styling are fairly constrained in what they can and cannot do, given how webmail things may handle them. I'd suggest doing some experiments, and consulting the source code of open-source webmail systems like SquirrelMail.
If you are worried about information loss, one thing that many clients allow you to do is download the original HTML, for offline viewing. Of course, it tends to be pretty atrocious, so I don't know why anyone would do that.
You should look at:
http://htmlemailboilerplate.com/
You will find a boilerplate code for HTML emails. There is also a good practice slideshow.
My approach to HTML email is to to write the sort of basic HTML we were doing in the 1990's - table layouts, minimal inline CSS (for colours only) and thats pretty much it. I don't know how modern clients deal with CSS positioning but people are still using Outlook 2003 which I believe is based on the hateful rendering engine that runs IE6 - so it pays to go for lowest common denominator.
I've never seen anything that looked like a standard for it, i've seen some email clients (GMail) strip out various things - including CSS, and others just ignore certain things (Outlook & background images).
Rationally, I can't think of what use any meta information would be in an email anyway - its hard enough to get people to read your mail anyway, I suspect even less will view the source! I've always included a title tag in case anything wants to use it as a subject - but even thats a stab in the dark.
Whenever i've looked at how mails are requested server side - admittedly a while ago, but I never noticed that anything was cached. You open the mail, the requests are made again. I'm sure things have moved on since I last checked, but personally - i'd still be inclined keep HTML email as simple and stripped back as possible.
I'm writing a web application for public consumption...How do you get over/ deal with the fear of User Input? As a web developer, you know the tricks and holes that exist that can be exploited particularly on the web which are made all the more easier with add-ons like Firebug etc
Sometimes it's so overwhelming you just want to forget the whole deal (does make you appreciate Intranet Development though!)
Sorry if this isn't a question that can be answered simply, but perhaps ideas or strategies that are helpful...Thanks!
One word: server-side validation (ok, that may have been three words).
There's lots of sound advice in other answers, but I'll add a less "programming" answer:
Have a plan for dealing with it.
Be ready for the contingency that malicious users do manage to sneak something past you. Have plans in place to mitigate damage, restore clean and complete data, and communicate with users (and potentially other interested parties such as the issuers of any credit card details you hold) to tell them what's going on. Know how you will detect the breach and close it. Know that key operational and development personnel are reachable, so that a bad guy striking at 5:01pm on the Friday before a public holiday won't get 72+ clear hours before you can go offline let alone start fixing things.
Having plans in place won't help you stop bad user input, but it should help a bit with overcoming your fears.
If its "security" related concerns you need to just push through it, security and exploits are a fact of life in software, and they need to be addressed head-on as part of the development process.
Here are some suggestions:
Keep it in perspective - Security, Exploits and compromises are going to happen to any application which is popular or useful, be prepared for them and expect them to occur
Test it, then test it again - QA, Acceptance testing and sign off should be first class parts of your design and production process, even if you are a one-man shop. Enlist users to test as a dedicated (and vocal) user will be your most useful tool in finding problems
Know your platform - Make sure you know the technology, and hardware you are deploying on. Ensure that relevant patches and security updates are applied
research - look at applications similar to your own and see what issues they experience, surf their forums, read their bug logs etc.
Be realistic - You are not going to be able to fix every bug and close every hole. Pick the most impactful ones and address those
Lots of eyes - Enlist as many people to review your designs and code as possible. This should be in addition to your QA resources
You don't get over it.
Check everything at server side - validate input again, check permissions, etc.
Sanitize all data.
That's very easy to write in bold letter and a little harder to do in practice.
Something I always did was wrap all user strings in an object, something like StringWrapper which forces you to call an encoding method to get the string. In other words, just provide access to s.htmlEncode() s.urlEncode().htmlEncode() etc. Of course you need to get the raw string so you can have a s.rawString() method, but now you have something you can grep for to review all uses of raw strings.
So when you come to 'echo userString' you will get a type error, and you are then reminded to encode/escape the string through the public methods.
Some other general things:
Prefer white-lists over black lists
Don't go overboard with stripping out bad input. I want to be able to use the < character in posts/comments/etc! Just make sure you encode data correctly
Use parameterized SQL queries. If you are SQL escaping user input yourself, you are doing it wrong.
First, I'll try to comfort you a bit by pointing out that it's good to be paranoid. Just as it's good to be a little scared while driving, it's good to be afraid of user input. Assume the worst as much as you can, and you won't be disappointed.
Second, program defensively. Assume any communication you have with the outside world is entirely compromised. Take in only parameters that the user should be able to control. Expose only that data that the user should be able to see.
Sanitize input. Sanitize sanitize sanitize. If it's input that will be displayed on the site (nicknames for a leaderboard, messages on a forum, anything), sanitize it appropriately. If it's input that might be sent to SQL, sanitize that too. In fact, don't even write SQL directly, use an intermediary of some sort.
There's really only one thing you can't defend from if you're using HTTP. If you use a cookie to identify somebody's identity, there's nothing you can do from preventing somebody else in a coffeehouse from sniffing the cookie of somebody else in that coffee house if they're both using the same wireless connection. As long as they're not using a secure connection, nothing can save you from that. Even Gmail isn't safe from that attack. The only thing you can do is make sure an authorization cookie can't last forever, and consider making them re-login before they do something big like change password or buy something.
But don't sweat it. A lot of the security details have been taken care of by whatever system you're building on top of (you ARE building on top of SOMETHING, aren't you? Spring MVC? Rails? Struts? ). It's really not that tough. If there's big money at stake, you can pay a security auditing company to try and break it. If there's not, just try to think of everything reasonable and fix holes when they're found.
But don't stop being paranoid. They're always out to get you. That's just part of being popular.
P.S. One more hint. If you have javascript like this:
if( document.forms["myForm"]["payment"].value < 0 ) {
alert("You must enter a positive number!");
return false;
}
Then you'd sure as hell have code in the backend that goes:
verify( input.payment >= 0 )
"Quote" everything so that it can not have any meaning in the 'target' language: SQL, HTML, JavaScript, etc.
This will get in the way of course, so you have to be careful to identify when this needs special handling, like through administrative privileges to deal with some if the data.
There are multiple types of injection and cross-site scripting (see this earlier answer), but there are defenses against all of them. You'll clearly want to look at stored procedures, white-listing (e.g. for HTML input), and validation, to start.
Beyond that, it's hard to give general advice. Other people have given some good tips, such as always doing server-side validation and researching past attacks.
Be vigilant, but not afraid.
No validation in web-application layer.
All validations and security checks should be done by the domain layer or business layer.
Throw exceptions with valid error messages and let these execptions be caught and processed at presentation layer or web-application.
You can use validation framework
to automate validations with the help
of custom validation attributes.
http://imar.spaanjaars.com/QuickDocId.aspx?quickdoc=477
There should be some documentation of known exploits for the language/system you're using. I know the Zend PHP Certification covers that issue a bit and you can read the study guide.
Why not hire an expert to audit your applications from time to time? It's a worthwhile investment considering your level of concern.
Our client always say: "Deal with my users as they dont differentiate between the date and text fields!!"
I code in Java, and my code is full of asserts i assume everything is wrong from the client and i check it all at server.
#1 thing for me is to always construct static SQL queries and pass your data as parameters. This limits the quoting issues you have to deal with enormously. See also http://xkcd.com/327/
This also has performance benefits, as you can re-use the prepared queries.
There are actually only 2 things you need to take care with:
Avoid SQL injection. Use parameterized queries to save user-controlled input in database. In Java terms: use PreparedStatement. In PHP terms: use mysql_real_escape_string() or PDO.
Avoid XSS. Escape user-controlled input during display. In Java/JSP terms: use JSTL <c:out>. In PHP terms: use htmlspecialchars().
That's all. You don't need to worry about the format of the data. Just about the way how you handle it.
I'm nearing the completion of migrating our existing website to a CMS and I've just finished creating all the various contact forms. The CMS I'm using has CAPTCHA built into it's form builder, which is great, but the only method available is the "decipher-the-noisy-image" method.
This approach works well, but it limits access for people who might have reading or sight disabilities. I've worked around this by having a "help" page which allows those with disabilities to contact us by telephone and I'm considering having a single-field form which says "Send us your email address and we'll contact you". Accessibility is of particular importance to me as a web developer, but from an organisational perspective; so is reducing the amount of form spam we receive.
So what I'd like to know is, has anyone in the community had any experience with other CAPTCHA methods and how have you managed to make them accessible to people with disabilities?
As a blind person I find that recaptcha is one of the better CAPTCHA services out there as far as an audio option. The issue with using sms as the only alternative is the fact that many visually impaired users don't have cell phones that allow them to read text messages.
A good captcha, like reCAPTCHA, usually includes an audio CAPTCHA. Also I have seen a site that will
send a SMS message and you enter the code in the sms (Google-gmail will do this).
I am very interested in this because I am implementing a CAPTCHA in jQuery right now.
Many sites, including this one I believe, have an option to play noisy audio with embedded spoken numbers, as an audio equivalent to the traditional CAPTCHA image.
I find the result pretty spooky, actually. Reminds me of numbers stations.
As Michael said, audio with each character of the CAPTCHA text spoken for better or worse is a common option provided. If your CMS is PHP-based or if PHP is available on the hosting infrastructure you are using anyway, here's an open source CAPTCHA application with an audio download option:
http://www.phpcaptcha.org/
I've implemented a production site with phpcaptcha, and it works as advertised.
Yes, I realize this question was asked and answered, but I have specific questions about this that I feel were not clear on that thread and I'd prefer not to get lost in the shuffle on another thread as well.
Previous threads said that rendering the email address to an image the way Facebook does is overkill and unprofessional user experience for business/professional websites. And it seems that the general consensus is to use a JavaScript document.write solution using html entities or some other method that breaks up and/or makes the string unreadable by a simple bot. The application I'm building doesn't even need the "mailto:" functionality, I just need to display the email address. Also, this is a business web application, so it needs to look/act as professional as possible. Here are my questions:
If I go the document.write route and pass the html entity version of each character, are there no web crawlers sophisticated enough to execute the javascript and pull the rendered text anyway? Or is this considered best practice and completely (or almost completely) spammer proof?
What's so unprofessional about the image solution? If Facebook is one of the highest trafficked applications in the world and not at all run by amateurs, why is their method completely dismissed in the other thread about this subject?
If your answer (as in the other thread) is to not bother myself with this issue and let the users' spam filters do all the work, please explain why you feel this way. We are displaying our users' email addresses that they have given us, and I feel responsible to protect them as much as I can. If you feel this is unnecessary, please explain why.
Thanks.
It is not spammer proof. If someone looks at the code for your site and determines the pattern that you are using for your email addresses, then specific code can be written to try and decipher that.
I don't know that I would say it is unprofessional, but it prevents copy-and-paste functionality, which is quite a big deal. With images, you simply don't get that functionality. What if you want to copy a relatively complex email address to your address book in Outlook? You have to resort to typing it out which is prone to error.
Moving the responsibility to the users spam filters is really a poor response. While I believe that users should be diligent in guarding against spam, that doesn't absolve the person publishing the address from responsibility.
To that end, trying to do this in an absolutely secure manner is nearly impossible. The only way to do that is to have a shared secret which the code uses to decipher the encoded email address. The problem with this is that because the javascript is interpreted on the client side, there isn't anything that you can keep a secret from scrapers.
Encoders for email addresses nowadays generally work because most email bot harvesters aren't going to concern themselves with coding specifically for every site. They are going to try and have a minimal algorithm which will get maximum results (the payoff isn't worth it otherwise). Because of this, simple encoders will defeat most bots. But if someone REALLY wants to get at the emails on your site, then they can and probably easily as well, since the code that writes the addresses is publically available.
Taking all this into consideration, it makes sense that Facebook went the image route. Because they can alter the image to make OCR all but impossible, they can virtually guarantee that email addresses won't be harvested. Given that they are probably one of the largest email address repositories in the world, it could be argued that they carry a heavier burden than any of us, and while inconvenient, are forced down that route to ensure security and privacy for their vast user base.
Quite a few reasons Javascript is a good solution for now (that may change as the landscape evolves).
Javascript obfuscation is a better mouse trap for now
You just need to outrun the others. As long as there are low hanging fruit, spammers will go for those. So unless everyone starts moving to javascript, you're okay for now at least
most spammers use http based scripts which GET and parse using regex. using a javascript engine to parse is certainly possible but will slow things down
Regarding the facebook solution, I don't consider it unprofessional but I can clearly see why purists may disagree.
It breaks accessibility standards (cannot be parsed by browsers, voice readers or be clicked.
It breaks semantic construct (it's an image, not a mailto link anymore)
It breaks the presentational layer. If you increase browser default font size or use high contrast custom CSS, it won't apply to the email.
Here is a nice blog post comparing a few methods, with benchmarks.
http://techblog.tilllate.com/2008/07/20/ten-methods-to-obfuscate-e-mail-addresses-compared/