I have an HTTP API that fetches a bunch of reviews. This API is cached for 5 minutes i.e. I receive Cache-Control: max-age=300,public in response headers.
A user can take some actions on the reviews like Delete, Edit, Post a new review etc.
I want to invalidate the cache from client side when any of this action is performed so that when user refreshes the page I can see the recently updated data instead of the stale data which is cached for 5 minutes.
Is it possible to do that from client?
Related
The frontend/client shows a list of items. Make a request to the backend/ API to change some items. For example, add an item or delete one. How to reflect these changes on the client side after the backend has processed this request successfully?
Examples:
POST request to the backend that adds a new item to the list. The response body contains the added item. Http Status Code 201 CREATED
DELETE request to the backend that removed an item from the list. The response body contains nothing. Http Status Code 204 NO CONTENT
Solutions?
After the request has been processed successfully (client receives 2xx status code) the complete list is going to be fetched again from the backend. Downside: This means we have two requests. First the POST, then a GET.
Return the complete list in the response body of the POST request. This seems strange, because how the client consumes the API affects the behavior of the API.
The client handles the addition or removal of the item itself after it has received a 2xx from the backend. Pro: Only one request. Downside: Problematic if multiple users operate on the data. How to keep everything in sync?
Is there a common pattern of how to handle this in regards to clean API design? I noticed, that a lot of tools make only one request if you change data. Think of Trello or something similar.
How to keep everything in sync?
In REST, you don't. Each client (potentially) has their own local cache. The server includes standardized caching metadata in its responses, to provide the clients with a hint of how often the information may change.
Cache invalidation is standardized, but only the caches that an HTTP request passes through are going to see the triggering requests. So unless your clients are sharing a cache (not likely in the world of HTTPS), somebody is going to have stale data.
The server still has the authoritative copy, of course, and we have standardized conditional requests that give us options when the clients data is too stale.
How to reflect these changes on the client side after the backend has processed this request successfully?
There is a section of the HTTP specification that describes how to identify the representation in an HTTP message. It includes this passage
If the response has a Content-Location header field and its field-value is a reference to the same URI as the effective request URI, the payload is a representation of the resource identified by the effective request URI.
So POST /foo PUT /foo PATCH /foo have a standardized way of announcing that the representation enclosed in the response is a new representation of /foo
There isn't, as far as I can see, any standardized way to communicate side effects; which is to say changes to other resources. We have to fall back on cache invalidation semantics.
For the most part, REST is a bunch of machines pretending to be web browsers talking to a machine that pretends to be a web server.
The REST interface is designed to be efficient for large-grain hypermedia data transfer, optimizing for the common case of the Web, but resulting in an interface that is not optimal for other forms of architectural interaction. -- Fielding, 2000
I have a WordPress webpage with posts retrieving from a public Facebook page. The FB page is not mine. However the problem is that I have millions of visitors on my Web page and every time a user visits the web page it make an API call to FB Page. Since facebook allows only a limited number of API calls in a time frame, My limit reaches instantly with such a huge number of visitors. Is there any solution to this problem. an idea in my mind is:
1. retrieve posts from Facebook and store them locally and display them every time a user visits. Is it possible? If Yes where to start ?
Or can we get more API calls by paying facebook or things like that. I am ready to pay as far as my API calls can be made sufficient for my needs.
I am open to any solution and would be very thankful for any help to resolve the problem.
There are several possible solutions to this problem
Storing responses in database
You can add a middlepoint to your requests to Facebook API using your application. This would mean that you would have a database table which stores Facebook-related information, possibly along with a lifecycle time, like:
facebook_data(user_id, lifecycle_time, ...)
Whenever you would theoretically need to send a request to Facebook API, you can check the database table to see whether the user already has a record in that table and whether it is still valid. If so, give this data to the user. If not, send an API request to Facebook and store the response in this table.
Storing responses in localStorage/memory/file
You can also store Facebook-related data in the localStorage of the web browser of the memory of an app or a file, or even a local database specific for each user. This would prevent a lot of communication and server load your app is issuing on your server(s).
Queueing user requests to be sent
If the Facebook-related data is not very urgent to your users, you can queue requests to be sent and send a single request instead of a request for each user's each visit. You can do this via a cron job.
Periodically sending requests to Facebook
You can group your users into batches and periodically update their values via Facebook and storing in your database.
Combination
Naturally, you can combine the approaches, for instance, you can store in local memory, file, or localStorage values and in the database in the same time, so first locally stored information is searched for, not needing even a request if it exists and is still valid. If not, then checking the database record and using that if it exists and is still valid. And if the data is not found in local resources, nor your database, then you can send an API request.
I am developing a feature, where a user requests some data by filling a form and posting it. The form contains user details (name, email, mobile etc.) and these details need to be saved in the database as a lead. Once the lead data is saved successfully then only the requested data will have to be retrieved from the database and shown to the user.
I started looking into this problem by dividing it into below two APIs and calling it sequentially
A POST API to create a lead resource.
A GET API to fetch the requested resource on the success of POST.
But then I realized that the solution will affect the speed of data retrieval as I will be doing two separate network request sequentially and I cannot compromise on speed.
How can I solve this problem without compromising on speed? Should I use POST request and return the requested data in its response?
P.S. I cannot call both the APIs in parallel
Yes, that's exactly what you should do - the POST request should respond with the newly created resource, that the client can subsequently use.
You have been thinking about the problem from the web developer's perspective. You have this byte stream and to send it to clients doesn't need any form submission, a GET request ought to suffice. But from the business' perspective, retrieving the file does require a form submission, namely, the user's personal data. Even from the user's perspective it is a single operation. The user sends in the requisite data in order to get out the desired response (e.g. a file download). The saving of the input data (user's private details) is a side effect, not a user-intended outcome.
As such, a single POST request which submits the user data, saves it, then responds with the requested resource, is sufficient to cover the situation.
The documentation says that the app is responsible for deleting it's own requests. From a application design perspective, when is the best time to delete a request? Are there cases where the request should never be deleted?
btw, this is what I mean by deleting requests...
$facebook->api("/$full_request_id",'DELETE');
According to the documentation the most appropriate time to delete a request is immediately after the user is taken to your canvas application from clicking on a request. This removes the request from the various notification channels and updates the request counter. I would recommend a global hook that is configured to monitor the inbound Facebook request_ids GET parameter, look up the appropriate Facebook requests, optionally update your internal request/invite tracking, and delete the request once processed.
I don't believe there are any official policies regarding deleting your own application requests, and it would make sense to delete pending requests early, e.g. if the request is for a time-sensitive action that has past.
Can anyone explain the difference between a HTTP-GET and HTTP-POST? And why do people say that a HTTP-POST is weaker in terms of security?
In an HTTP GET request, key/value pairs are specified in the URL:
http://server/something?value1=foo&value2=bar.
In an HTTP POST request, key/value pairs are sent as part of the HTTP request after the headers. For example:
POST /something HTTP/1.1
Host: server
Content-Length: 21
Content-Type: application/x-www-form-urlencoded
value1=foo&value2=bar
It's hard to really describe one as being more or less secure than the other, but HTTP POST data is not visible in the URL, and when submitting data to a website, an HTTP POST can usually only be performed as a result of user interaction (for example clicking on a "Submit" button).
This means a user can't be tricked into visiting a URL like http://server/update_profile?name=I_suck and sensitive data is not exposed in the URL.
You can also use nonces and other anti-forgery tokens with html forms (which use POST) to prevent other forms of cross-site request forgeries.
In general, POST should be used for requests that potentially modify state on the server, and GET should be used for read-only operations.
The HTTP specification differentiates POST and GET in terms of their intent:
GET is idempotent: it is for obtaining a resource, without changing anything on the server. As a consequence it should be perfectly safe to resubmit a GET request.
POST is not: it is for updating information on the server. It can therefore not be assumed that it is safe to re-submit the request which is why most browsers ask for confirmation when you hit refresh on a POST request.
In terms of security, no difference. POST is more obscure, perhaps, but that's a very different thing. Security needs to be added at another layer, for example SSL.
Some notes on GET requests:
GET requests can be cached
GET requests remain in the browser history
GET requests can be bookmarked
GET requests should never be used when dealing with sensitive data
GET requests have length restrictions
GET requests should be used only to retrieve data
Some notes on POST requests:
POST requests are never cached
POST requests do not remain in the browser history
POST requests cannot be bookmarked
POST requests have no restrictions on data length
(Source:W3 Schools)
I wouldn't call POST more or less secure than GET. Admittedly parameters are displayed as part of the URL when using GET, so any sensitive data will be immediately visible to the user. However, it is trivial to view and even change any part of the HTTP request, so just because POST doesn't pass data through the URL it can still easily be read. Unless you're using HTTPS both GET and POST will transfer data in an easily accessible form.
The GET method is meant for data retrieval only and should not have any side-effects. But POST is meant for that specific purpose: altering data on the server side.
GET requests can easily be foreged (see Cross-Site Request Forgery) by just placing an image on a page while forging POST requests is not that easy (this is also a reason why you should only allow authorized POST requests).