I'm a client side developer with little experience of server side, and I'm struggling to understand how to make a database-backed website without requiring users to login.
The usecase is fairly straightforward. The user lands on a website, uploads an image, and performs some processing to that image. Clicking 'share' POSTs JSON to my endpoint, stores it in a DB, and returns a unique URL in a textbox (eg, https://example.com/art/12345) which allows the user to share their artwork with others, or just to come back and do more editing later on.
What stops somebody from doing, POST <data> https://example.com/art 100 million times and filling my pay-as-you-go database?
I've seen examples of this link based method of sharing between users on plenty of sites but I don't understand how to stop abuse, or whether it is safe to just open up an API which allows writes to a database. I do not want users to have to login.
I believe the simplest method is having a quota, either by username for logged in users or by IP, if you don't require logins or only want to allow free usage to a certain point. Perhaps you could have a smaller quota for non-logged in users than for logged in users and even larger for paying users.
Your server side code that handles the POSTS and storing data into the database would have to take care of that. I'd add it to a user_data table on mine, making an additional column that tracks total space used. makes a todo
Then, when the user adds new data, increase the total space used. When they delete old data (I have versioned web pages so that eventually, the user will be able to rollback to previous versions) then the space used decreases. Having another page to look at to see where they're using space makes deciding what to delete to stay under a quota of X MB's/GB's/TB's/etc easier or maybe just an /api/delete_old_pages or notes or comments or all of the above.
Related
I have read a few questions on here about e-mail clients prefetching URLs in e-mails. An answer to this seems to be to add a new confirmation page, where the user has to click a button to confirm the desired action.
But, this answer states the following:
As of Feb 2017 Outlook (https://outlook.live.com/) scans emails
arriving in your inbox and it sends all found URLs to Bing, to be
indexed by Bing crawler.
This effectively makes all one-time use links like
login/pass-reset/etc useless.
(Users of my service were complaining that one-time login links don't
work for some of them and it appeared that BingPreview/1.0b is hitting
the URL before the user even opens the inbox)
Drupal seems to be experiencing the same problem:
https://www.drupal.org/node/2828034
My major concern is with this statement:
As of Feb 2017 Outlook (https://outlook.live.com/) scans emails
arriving in your inbox and it sends all found URLs to Bing, to be
indexed by Bing crawler.
If this is the case, any URL in an e-mail meant to confirm an action, e.g. confirming a login, subscription, or unsubscription, can end up searchable in a search engine, if that's whats meant by indexed in the quote above. In this case, it's Bing. Not even a dedicated confirmation page where the user confirms the desired action truly mitigates this.
Scenario #1
If I email the user a login link with a one-time token in the URL, that URL will end up in Bing. This token will have a short lifetime, lets say 5 minutes, so I doubt anyone will manage to search on Bing and find the URL before the user clicks it or it expires.
Scenario #2
The user gets an e-mail with a link to confirm a subscription. This link is perhaps valid for 24 hours. This might(?) be long enough for someone else to stumble over the link on a search engine and accidentally (or on purpose) confirm the subscription on behalf of the user.
Scenario #2 is not uncommon, it's even best practice to use double opt-in as far as I am aware.
Scenario #3
Unsubscribe URLs in the bottom of newsletters. Maybe valid for forever? You don't want this publicly searchable in an search engine.
Assume all the one-time confirmation links land on a confirmation page where the user confirms the desired action.
Is it truly the issue that URLs in e-mails are indexed by search engines, at least Bing? And will they actually end up publicly searchable? If not, what is meant by indexed in the quote above?
I'll add for the sake of completion that I don't think I've had much of a problem with this in my own use of the web, so my gut feeling is that this is unlikely the case.
Is it truly the issue that URLs in e-mails are indexed by search engines, at least Bing?
I can't definitely say if they are being indexed or not, only Bing could answer this question, but they are surely being visited, at least with a simple GET request. I just tested this sending myself a link to a page on my website that logs the requests that are made against it, and indeed I'm seeing a GET coming from 207.46.13.181 (reverse DNS says msnbot-207-46-13-181.search.msn.com), which suggests that an automated program from search.msn.com is crawling the link. This leads me to believe that yes, they are trying to index the link's content somehow, but it's only my opinion really.
And will they actually end up publicly searchable? If not, what is meant by "indexed" in the quote above?
Well, again, impossible to say unless you work for Bing. In any case, "indexing" means exactly what you think it does: parsing the content of a page to potentially include it in search results.
The real question here is: does this somehow represent a security problem or will it compromise my website's functionality?
It surely has the potential to: if your confirmation/reset/subscription/whatever process only relies on a single GET request with the appropriate GET parameter, then you should definitely revisit the strategy, as it obviously allows anyone to perform the action (even maliciously for example enumerating possible IDs for your GET parameters).
If the link you are trying to send contains sensible information or can be used to alter important data for an user of your website, then you should at least put it behind a login page only giving access to the interested user. This way, anyone who wants to access it (including search engines) will be redirected to a login page if not already logged in.
If the link you are trying to send is just some kind of harmless confirmation link (e.g. subscribe/unsubscribe from a newsletter), then at least use a form inside the web page to do the actual confirmation through a POST request (possibly also using a CSRF token), otherwise you will unequivocally end up with false positives.
I have a very old ASP.net Application with a Web Form with 1 Dropdown Box and 2 Text Boxes and a Submit Button.
All 3 are mandatory fields. Based on the data entered, once the user clicks Submit Button additional details are shown on the next page from the database.
On Submit data is posted via Query String that looks like
http://myserver/myapp/search.aspx?f1=1&f2=tom&f3=sales
Though the application is doing what is supposed to do, off late we came across lot of issues:
As couple of entities that are interested in our data wrote programs to programatically build the querystrings and hitting our server.
This is slowing down the server and regular users who manually search records are facing lot of slowness.
Due to come legal restrictions we couldn't implement CAPTCHA or have users get authenticated.
I would appreciate if you can let me know if any of you have come across this kinda situation and how you have dealt with it.
Thanks in advance.
You could implement source-based rate limiting. I.e. per IP address only allow so many requests per minute. If the requester makes too many requests you simply reject the requests. You could also blacklist the IP addresses that are hitting your app too aggressively. Both of these policies can be enforced by a load balancer like HAProxy or nginx.
I would like to build a simple beta request signup page where the user is rewarded with an earlier reward when he is sharing the link to the application as much as possible.
A solution like this is seeable on
trenvy.com
User enters email
User gets unique link with his unique code
User shares this link on every signup its a +1 count on him
An admin method throws out the users emails which have shared the link and brought the most people in. I want to use this email list to use in CampaignMonitor.
Anyone knows what could be a good way to achieve this? Or someone wrote such a thing or knows a git repo that has this feature implemented as is to take a look at and learn?
I have already coded a unique code generator for the app that makes unique codes of 10 chars.
Now only this social media sharing is a bit unclear to me on how to approach this in rails, any ideas on that? thx!
Something like this can be achieved pretty easily in any framework, so I think I'll provide a general answer first, and if any specific gems occur to me, I'll mention them:
1) The unique code part is easy, it's just a parameter in a controller that checks the validity of the code — this would be a unique code that's added to the user model for ease of verification and created when the user first enters his email address.
2) Every time the link is visited, it's parsed by the controller and saves an event (don't just increment a field if you want maximum data out of the interaction, you could save IP for country of origin, time of the page hit, etc), just count the click events for that user for his +1s
3) Just write a quick admin site (i used twitter bootstrap for this recently) that lets you see the user's who've interacted with the system and sort by shares, and you can use the createsend gem to add them to whatever list you like.
There are no specific gems I can think of that'll speed this process up, Devise is overkill, you don't really need an activity monitor gem since you're not storing much info, definitely twitter_bootstrap for speeding up building the admin interface. Heroku lets you add an Sendgrid as a plugin, so you're covered there for mail sending.
Am i missing anything in your requirements? Seriously though this should be a 2-4 day dev effort, nothing fancy here.
I would like to capture the following parameters:
lastAccessedTime - The time when the user visited the site the last time (usually shown during the login process)
isOnline - A boolean to represent if a user is online or not.
a. Would it make sense to have these variables as part of the User table itself or should this be handled via a separate user audit table?
b. If certain SOAP / REST API's expose the functionality via API calls, how do you track the above parameters (e.g. Would you modify the lastAccessedTime in such cases - this might confuse the user if he logs into the portal, isOnline bit also will not make sense if the user does API calls).
I would create a session table that links back to the user. Instead of an isOnline field, I would just run a query for sessions that have been active within the last x amount of time. I would also update that session field with each request, even if that request is coming through an API.
This does create some overhead in pruning the session table, but you also don't clutter up your user table with non-user information, which can't be pruned.
Make the lastTimeActive a field in the user table, and update it with each page access. Your "Users Online" list is all users whose lastTimeActive is within 5 minutes.
I would create another table (userid, lastTimeActive), and frequently update & check the table.
// update
update onlineusers set lastTimeActive = getdate() where userid=1234
// check
delete from onlineusers where lastTimeActive < dateadd(minute,-5,getdate())
The biggest problem with tracking user presence (onine/offline) over HTTP is how to determine when the user has gone offline.
It's easy to determine when the user has come online - the mere presence of an authenticated request assumes that the user is active. However, since HTTP is stateless, the lack of a subsequent request can mean either that the user is gone offline, or that the user is online, but just hasn't done anything specific with your app recently.
Thus the best guess you can make is to have a timeout and if the user has not made a request during that timeout, to switch to offline state.
The simplest implementation would be to have a lastTimeActive, as Jonathan Sampson suggested. However, this won't give you the length of the user session, only an approximation of who's online at this moment.
More complex approach would be to have lastTimeActive and lastTimeLoggedIn. LastTimeLoggedIn is set at the time of first auth request that is more than 5 minutes from a previous auth request. A user is considered online, if there was an authenticated request in the last five minutes. The session length for the user is the time difference between lastTimeActive and lastTimeLoggedIn.
If your app also offers the choice of logging out to the user, you chouls consider that action also as going offline. However, unless your app is a banking app, chances are the users will just close their browser.
Also, avoid any background threads for updating the offline/online status of your users. You should be running the logic above only when there's an explicit request about the status of particular user and you should be updating only the users you were asked for.
I hope this is allowed but I have a number of questions regarding Facebook Connect, I'm quite unsure on how I should approach implementing it.
I am working on a live music type service and currently have user registration, etc. If I were to implement Facebook Connect alongside this, would I still be able to email the Facebook Connect users as if they were on my database?
Also, would it instead be possible to let users who have Facebook "link" their accounts once registered so I am able to give them the benefits of sharing via Facebook and inviting friends while still having an actual registered user on my system.
I have tried to read up answers to the above questions but what I've found is quite ambiguous.
Thanks, look forward to your views.
Facebook's documentation process is very poor, so don't feel bad about having a hard time getting started. Their wiki-style approach to documentation without any real official documents tends to leave the "process flow" tough to grasp, and requires piecing together parts of a bunch of randomly scattered docs.
Facebook has an obligation to protect privacy, so they never make a user's actual email address available to application developers, through Connect or normal applications. They do have a proxied email system in place that you can use, however, you must get explicit permission from a user in order to email them. There's a decent document on proxied email here. You can get permission by prompting for it; there's several methods for doing so linked in that document.
In regards to linking Facebook and local accounts, this would definitely be the way to go. Once a Connect user logs in, you want to store that fact for that user so you can provide the Facebook-specific functionality. I would simply create a normal user account in the database for every new Connect user that came by, with it's own local id, so that you don't have to do special handling of two different types of user accounts all over the site. That being said, the account would obviously have to be marked as a Facebook user's account (I use an externalId column in my users table), and any part of the site that relied on information you might otherwise have locally would have to handle the Facebook aspect properly (such as using proxied email instead of normal email).
For existing users, you could arrange an "account link" by having a process whereby they log into FB Connect after they've logged into the site already, and you could detect that and simply add their FB id to your users table. After that, they could log in through Connect in the future, or through your normal process. I've never done this, but it should be possible.
If you write the account handling code generically enough, your site will be able to function well no matter what kind of user you throw at it.