I'm involved in building a donation form for non-profits. We recently got hit by a fast round of low dollar submissions. Many were invalid cards, but a few went through. Obviously someone wrote a script to check a bunch of card numbers for validity, possibly so they can sell them later.
Any ideas on how to prevent or limit the impact of this in the future?
We have control over all aspects of the system (code, webserver, etc). Yes the form runs over https.
When a flood of invalid transactions from a single IP address or small range of addresses is detected, block that address / network.
If a botnet is in use, this will not help. You can still detect floods of low dollar amount submissions and so deduce when you are under attack; during these times, stall low dollar amount submissions to make them take longer; introduce CAPTCHAs for low dollar amount donations; consult your bank's fraud prevention department in case they can make use of your server logs to catch the perpetrators.
Force donors to create accounts in order to make donations; protect account creation with a CAPTCHA, and rate limit donations from any one account.
Raise the minimum permissible donation to a point where it no longer makes financial sense for the scammers to use you in this way.
Instead of CAPTCHAs, which will annoy users, you might want to take advantage of the fact that most people have javascript enabled while bots don't. Simply create a small piece of javascript that when run inserts a particular value in a hidden field.
For those that have Javascript disabled you can show the CAPTCHA (use the <noscript> tag), and you can then accept a submission only if either of these measures check out.
For maximum annoyance to evildoers you could make the difference between the succeed message and the failure message computationally hard to distinguish (say everything is the same, except for one picture that displays the message) but easy to understand for humans.
limit submissions from the same IP address to one per minute, or whatever reasonable period of time it would take for a real person to fill out the form
Raising the minimum donation to a point where it no longer makes financial sense for the scammers to use you in this way will help in general.
This. How many legitimate donations do you get for under 5 bucks, anyway?
Related
Over the past few months random email addresses, some of which are on known spam lists, have been added at the rate of 2 or 3 a day to my website.
I know they aren't real humans - for a start the website is in a very narrow geographical area, and many of these emails are clearly from a different country, others are info# addresses that appear to have been harvested from a website, rather than something a human would use to sign up to a site.
What I can't work out is, what are reasons for somebody doing this? I can't see any benefit to an external party beyond being vaguely destructive. (I don't want to link to the site here, it's just a textbox where you enter email and press join).
These emails are never verified - my question isn't about how to prevent this, but what are some valid reasons why somebody might do this. I think it's important to understand why malicious users do what they do.
This is probably a list bombing attack, which is definitely not valid. The only valid use I can think of is for security research, and that's a corner case.
List bomb
I suspect this is part of a list bombing attack, which is when somebody uses a tool or service to maliciously sign up a victim for as much junk email as possible. I work in anti-spam and have seen victims' perspectives on this: it's nearly all opt-in verifications, meaning the damage is only one per service. It sounds like you're in the Confirmed Opt-In (COI) camp, so congratulations, it could be worse.
We don't have good solutions for list bombing. There are too many problems to entertain a global database of hashed emails that have recently opted into lists (so list maintainers could look up an address, conclude it's being bombed, and refuse to invite). A global database of hashed emails opting out of bulk mail (like the US Do Not Call list or the now-defunct Blue Frog's Do Not Intrude registry but without the controversial DDoS-the-spammers portion) could theoretically work in this capacity, though there'd still be a lot of hurdles to clear.
At the moment, the best thing you can do is to rate-limit (which this attacker is savvy enough to avoid) and use captchas. You can measure your success based on the click rate of the links in your COI emails; if it's still low, you still have a problem.
In your particular case, asking the user to identify a region via drop-down, with no default, may give you an easy way to reject subscriptions or trigger more complex captchas.
If you're interested in a more research-driven approach, you could try to fingerprint the subscription requests and see if you can identify the tool (if it's client-run, and I believe most are) or the service (if it's cloud-run, in which case you can hopefully just blacklist a few CIDR ranges instead). Pay attention to requesters' HTTP headers, especially the referer. Browser fingerprinting it its own arms race; take a gander at the EFF's Panopticlick or Brian Kreb's piece on AntiDetect.
Security research
The only valid case I can consider, whose validity is debatable, is that of security research (which is my field). When I'm given a possible phishing link, I'm going to anonymize it. This means I'll enter fake data rather than reveal my source. I'd never intentionally go after a subscription mechanism (at least with an email I don't control), but I suppose automation could accidentally stumble into such a thing.
You can avoid that by requiring POST requests to subscribe. No (well-designed) subscription mechanism should accept GET requests or action links without parameters (though there are plenty that do). No (well-designed) web crawler, for search or archiving or security, should generate POST requests, at least without several controls to ensure it's acceptable (such as already concluding that it's a bad actor's site). I'm going to be generous and not call out any security vendors that I know do this.
Say I have several accounts with Generic Bank. One account is the Master account and all the others are specific budgetary purpose accounts - like one for eating out or for gas. For any given time period theres a certain amount of money I have budgeted for each purpose - say $7 a day for eating out and $20 every three days for gas. After the given time period expires, I want the account refilled from whatever value it currently sits at, back up to the specified amount.
The idea is to passively manage my spending by limiting access and to keep me from thinking that I have more money to spend on a given purpose than I really have. For an example of the last case, say I put the whole months allotment into the gas account all at once. Anytime I checked that account's balance it would return a much larger number than what my actual time-period based allotment would be. Because of the association with the larger number and my inability to keep track of purchases or math in my head, it's very likely that I'd err on the side of making the purchase when in actuality I'm spending the money too fast.
Are there any services, software, or other utilities that can do bank transfers like this natively? Failing that, are there any money moving services, like Paypal or Google Wallet, that can accept add-on programs built to do this? I took a cursory look at the Paypal and Wallet APIs but, in addition to reminding of how out of my depth I was (I've never done anything involving APIs or banking), everything I saw was about person to person payments and not necessarily about account to account transfers. But then again, I'm not sure what the practical difference between the two is.
There are several banks who provide these types of services. I, personally, would advise against doing it with Tasker unless your bank specifically supports it, since it opens you up to glitches and security risks.
So, like any reasonably competent web development shop, we wear cotton gloves when we touch credit cards, and we use Braintree SecureVault to store them so we are clear of PCI Compliance issues.
However now we want to offer a free trial for our service, which pretty much relies on being able to guarantee that a given credit card is only used once for a free trial. Ideally we would be able to hash the credit card number itself to guarantee uniqueness. The problem there is that set of valid credit card numbers is small, so it's going to be easy to brute force the credit card numbers. Salting tactics are useless as far as I can see, because if someone has access to the database of hashes, they will most likely have the code as well, and thus the salting algorithm.
The best two ideas so far are:
A) Keeping the hashes isolated in a set, with no relation to their billing information. Therefore if the hashes are brute-forced, all they have is a list of credit card numbers that were used at some point in time, with no personal information or knowledge of whether it's even still valid. The main weakness here is that we do have record of the last-4 which could potentially be used to match them up to some extent.
B) Hash without the full number and deal with the false positives and negatives. Hashing on name, last-4 and expiration ought to be fairly unique. False positive is like winning the lottery, we can deal with it at customer support. False negative could be induced by modifying the name, we are not clear on what assurances we have about the precision of name matching (potentially affected both by the gateway and merchant account is my understanding), so this could open a loophole.
Thoughts? Suggestions? Battle-tested Wisdom?
High-level: Use Existing Payment Systems
I think that this approach -- using credit card numbers to determine if a user already has taken advantage of a free trial and should be ineligible for a subsequent free trial -- is misguided. Firstly, you will drive away potential customers by requiring a credit card upfront (which many users don't give out unless they are actually ready to buy), instead of requiring it only after the trial period has ended.
Secondly, you are reinventing the wheel. There are a plethora of "app stores" (the Chrome webstore, the Android marketplace, the iTunes app store, etc.) which provide builtin mechanisms for payment and trial periods. Using these systems will give your product increased visiblity to consumers, will offer your potential customers multiple different payment methods (making them more inclined to buy), and will also save you the hassle of implementing this mechanism yourself. Plus, users generally prefer to give out their credit card to the least number of companies possible; not only would you have to implement this complex mechanism yourself, but you would also have to get users to trust you enough to use it.
Lower-level: Implementation Details
Any hash mechanism can have collisions, hence you would still need to deal with this problem. You should obviously use full disk encryption and other best security practices with your servers. The risk of having both the database and the salting algorithm compromised at the same time can be reduced by hosting the backend database service on a separate machine from the one that hosts this code. The main vulnerability of hashing is brute force attacks. And the best way to deal with them is to just make brute forcing expensive enough that it isn't worth the attacker's while. Using a separate salt for each entry (e.g. the customer's name, the customer's zip code, etc. as part of the salt) will make using rainbow tables ineffective. Of course making the data, itself, less valuable to attackers (e.g. not including the full credit card number) is also a good way to discourage these kinds of attacks. In any case, I again advise you to take advantage of the many app stores instead of implementing this yourself.
Forgive me if I missed something, but why can't you just have a table of "UsedCreditCards" that just has a single column, which is a SHA512 hash of the credit card number and maybe the expiration date. This could not be reversed, and by keeping it in another table and not storing any other data about the code, you could easily check to see if a credit card number has been used before.
I am not sure if this would violate PCI or anything (I don't think so, but I could be wrong)
Does anyone know a good tool to test whether your emails are going into spam folders?
My web app generates emails to users, and I've been getting a lot of reports back from people saying "hey, no one ever responded to my message".
I have SPF rules in place and functioning correctly (email header shows an spf pass). I've also run my message through spam assassin and it scores very low.
Any other ideas?
To know if your email goes in the inbox, you need to get a metric called "Inbox Placement Rate". This indicator can be provided by Return Path, but it's quite expensive. If you're not sending huge volumes it might not worth it. The only way to measure the IPR is actually to have a certain number of test inboxes... In other words: the only way to chech that your email is not in the spam folder is to make the test and see what happen. There is not other magic solution and that's what Return Path is doing.
This means that when you hear about people claiming they have a 99% deliverability / delivery, it might be true be it just means that the email was "accepted" or "delivered" by the ISP. It's a lot, but it's not everything!
What you should do is the following: use an ESP focusing on deliverability. Personally I work for Mailjet. I believe it's the best value you can get: personalized DKIM and SPF are provided for free, you get the antispam scorings, the analytics, Ip reputation monitoring, throttling, etc. It's an all in one tool to avoid the headaches of optimizing yourself. It's more expensive that Amazon SES because you get a lot of added value services, but it has much lower prices than a lot of traditional ESPs!
Bottom line is: optimizing everything yourself is a full time job. Knowing exactly if an email is in the inbox or not will cost you a lot. The best way to proceed is to:
respect the best practices (opt in, not too much images, no red, etc.)
get some metrics such as open rates, click rates, delivery, etc. and watch their evolution over time. Any change from one sending to the other might be a signal for a problem you want to investigate.
Use a tool that takes care all the deliverability optimizations
Mailjet is cool because no matter which plan you pick, you get to use all the options. But if you want a full overview of what is existing, check out this comparison table:
http://socialcompare.com/en/comparison/transactional-emailing-providers-mailjet-sendgrid-critsend
If you're a perfectionist who wants to finetune the layout, how the emails are displayed etc. Check out Litmus, it's also a quite powerful tool!
http://litmus.com/
Simple answer: Use Mailgun!!!!
http://mailgun.net/
They will do all of your email deliverability and setup for you and give you a powerful API to build on! They are amazing. You'll never have to worry about domain keys or SPAM filtering again!
You should also check that your IP is not on any of major blacklists. dnsbl.info
This will at least give you an idea if you actually are getting flagged as spam.
For the past two years, we've used the service DeliveryMonitor.com. However, they've stopped accepting new applications which is a big red flag...
I'm currently evaluating the service from emailreach.com using their free trial
... We are now using DeliveryWatch.com with pertty good results thus far...
I have landed a side project where a company wants me to develop a database / smart client to track participants in their various programs. This app will be written in C# and MS SQL Server.
I was wondering if there are any tips on how to actually bid out a price for the job?
Should I charge a flat rate (if so how do you charge it?)
Should I charge by the hour (if so what is a going rate?)
Should I enter into some sort of contract?
Any comments or tips would be very helpful, as I am completely new to doing "side gigs" outside of my normal programming job.
Thanks in advance
Yes you should enter into a contract, you need to spell out the terms and conditions of the work the expected deliverables and payment. You will also want to indicate who owns the source code.
I prefer Time & Material contracts, that is they pay an hourly rate. This allows them to change the scope without having to renegotiate the terms. If you go with a fixed bid project then as the users starts to learn what they want and change the specification, which they will, you will have to make a choice. Either you do it for free or you negotiate a scope change.
Your goal is to get the gig, and once you have it, make enough money for it to be worth your while, right? With that in mind, you can do flat rate if the client demands it, but otherwise try and get it hourly, because there will always be snags, and the client will always throw changes at you from the first day to the last minute that will suck down your time if you let it. Either way, get a contract that specifies what you are going to deliver and how to handle change requests.
I did a succession of fixed price contracts for a client once, and it worked great. Then they said they were having a little problem getting a signature on this contract from head-office, but if I were to just do the work they'd make sure I got paid for the work eventually. So I spent two months working on it, only to be told that they had been turned down by head office and so I wasn't going to get paid. I'm 90% sure they'd made a copy of my work before I could delete it from their servers, but nothing I could prove in court. So I'll never work without a signed contract again. If they don't sign a contract on day one, get the hell out of there.
If you're a beginner I'd advise against a flat rate ... unless you are prepared to shoulder the risk of more work than is paid for. It is inherently hard to estimate the effort a project takes.
The rate you can take depends on your market and how you value your time. It might be interesting accepting a low rate gig to get started in a certain market when you have free times at your hand or if you're just interested at the subject.
Well there are a few things you need to worry about.
Contract, yes, this is 100% needed, otherwise you might not get paid. The details of the contract are what might by challenging, especially about how to charge, which is what you're asking.
If you charge an hourly rate, you have to worry about being fair when you charge, and I'm sure the client will worry about this. If you're on the clock, it's not fair to check your email, YouTube, or even Stack Overflow (gasp!). Then there is also the question about whether you should charge if you need to learn some new skill. For example, if you have to learn library A, is this charged to the client or should it be done on your own time?
With a flat rate, you have to worry about keeping yourself productive so that you don't end up with a lower hourly rate. On the other hand, if the job turns out to be easy, you can finish quickly, and end up with a good hourly rate. This probably would be a lot more fair to both parties since no one would be monitoring you so that you're not goofing off and you wouldn't be doing something unethical if you decided to surf the internet while you're working.
If this is a side project and not a long-term job, I'd highly recommend a flat rate. Make sure you estimate it properly though. Take a guess of how many hours you think it will take you, add 20%, and then multiply this by some reasonable hourly rate. The other party will most likely be much more open to a flat rate since then it will be like they are paying for a finished product, rather than a developer.
Good luck!
Bidding is a tight rope walking game. Neither too low nor too much. The contract or the hourly rate always drives bidders mad. Be just in your bidding price and do not undercut too much else you might end up working for nothing. Also bid in such sites that hold the project price in escrow once the winning bidder is chosen. One good resource can be found at :- http://www.freelancing-on-net.com/writing_a_successful.php#
Even if you are bidding hourly, customers always want at least an estimate. Don't forget to account for all the activities that you don't think of when you're thinking of creating software.
Specifically, think of project management, meeting and call times with the client, end user meetings, source code management, testing, and bug fixing. In short, everything that you do for this client that prevents you from making money with another client needs to be billed to this client. You can account for that with the number of hours bid or with the hourly rate (usually a combination of the two).
I used to hear, "Prepare your best estimate. Then double it." Then I heard, "Prepare your best estimate, then times it by Pi." Now I hear, "Prepare your best estimate, then times it by Pi, then double it." :)
Unfortunately, the only way you'll get good at accurately bidding projects is to bid some, complete the project, and see where you were over or under.
You're always motivated to bid low to ensure that you get the project. But if you lose money doing a project for half your cost, getting the project might not have been a good thing.