I have a WordPress webpage with posts retrieving from a public Facebook page. The FB page is not mine. However the problem is that I have millions of visitors on my Web page and every time a user visits the web page it make an API call to FB Page. Since facebook allows only a limited number of API calls in a time frame, My limit reaches instantly with such a huge number of visitors. Is there any solution to this problem. an idea in my mind is:
1. retrieve posts from Facebook and store them locally and display them every time a user visits. Is it possible? If Yes where to start ?
Or can we get more API calls by paying facebook or things like that. I am ready to pay as far as my API calls can be made sufficient for my needs.
I am open to any solution and would be very thankful for any help to resolve the problem.
There are several possible solutions to this problem
Storing responses in database
You can add a middlepoint to your requests to Facebook API using your application. This would mean that you would have a database table which stores Facebook-related information, possibly along with a lifecycle time, like:
facebook_data(user_id, lifecycle_time, ...)
Whenever you would theoretically need to send a request to Facebook API, you can check the database table to see whether the user already has a record in that table and whether it is still valid. If so, give this data to the user. If not, send an API request to Facebook and store the response in this table.
Storing responses in localStorage/memory/file
You can also store Facebook-related data in the localStorage of the web browser of the memory of an app or a file, or even a local database specific for each user. This would prevent a lot of communication and server load your app is issuing on your server(s).
Queueing user requests to be sent
If the Facebook-related data is not very urgent to your users, you can queue requests to be sent and send a single request instead of a request for each user's each visit. You can do this via a cron job.
Periodically sending requests to Facebook
You can group your users into batches and periodically update their values via Facebook and storing in your database.
Combination
Naturally, you can combine the approaches, for instance, you can store in local memory, file, or localStorage values and in the database in the same time, so first locally stored information is searched for, not needing even a request if it exists and is still valid. If not, then checking the database record and using that if it exists and is still valid. And if the data is not found in local resources, nor your database, then you can send an API request.
Related
I have a SPA website set up with a REST API. One of the issues I have come up against is how to access data unrelated to the resource being requested. Specifically, accessing the users account details so I can do things like hide admin actions when the user is not an admin.
I have an api endpoint that can get all this data so the simplest solution would be to on every page load request the profile api as well as the resource actually being accessed. This seems a little wasteful to be constantly requesting a resource that rarely changes.
Another option would be to store this data in local storage so it only has to be requested once but then I have the issue where the user updates their username or settings on one device and then the other one is left with outdated data in local storage.
I thought vuex might be able to persist this data when clicking on links and navigating to different pages but it seems like the data gets lost when the page changes.
From what I understand graphql would solve this problem by allowing the initial request to get the user data along with the other data. I'm not really sure how much more efficient this is than 2 requests and rewriting my whole api probably isn't the best solution to this.
Are there any well known solutions to this problem or is one of the options I have come up with the best way to handle this?
I am developing a feature, where a user requests some data by filling a form and posting it. The form contains user details (name, email, mobile etc.) and these details need to be saved in the database as a lead. Once the lead data is saved successfully then only the requested data will have to be retrieved from the database and shown to the user.
I started looking into this problem by dividing it into below two APIs and calling it sequentially
A POST API to create a lead resource.
A GET API to fetch the requested resource on the success of POST.
But then I realized that the solution will affect the speed of data retrieval as I will be doing two separate network request sequentially and I cannot compromise on speed.
How can I solve this problem without compromising on speed? Should I use POST request and return the requested data in its response?
P.S. I cannot call both the APIs in parallel
Yes, that's exactly what you should do - the POST request should respond with the newly created resource, that the client can subsequently use.
You have been thinking about the problem from the web developer's perspective. You have this byte stream and to send it to clients doesn't need any form submission, a GET request ought to suffice. But from the business' perspective, retrieving the file does require a form submission, namely, the user's personal data. Even from the user's perspective it is a single operation. The user sends in the requisite data in order to get out the desired response (e.g. a file download). The saving of the input data (user's private details) is a side effect, not a user-intended outcome.
As such, a single POST request which submits the user data, saves it, then responds with the requested resource, is sufficient to cover the situation.
I am using the graph api to get data about pages and the posts in the pages.
When a post is published, it gets liked, commented upon and shared over time. When I read the data next time how can I get the posts that have those changes alone?
the best way is really to set up a server to receive real time updates. Any other way would mean polling facebook endpoints. At a certain point, a single user access token would be rate limited, and would block you from making a call for a certain amount of time. Also, there would be more work to compare each post to the one you stored to see if anything has changed.
Really the most efficient way is to use real time updates in which you set up an endpoint on your server to receive messages from facebook whenever something on a page (or user) has changed. If cost of keeping a server running is your roadblock, I would recommend to setup a free Parse.com account in which you can set up a server to handle Facebook's incoming requests and act on that.
I hope that makes sense! More information on realtime updates here: https://developers.facebook.com/docs/graph-api/real-time-updates/v2.2
I have tried raising this concern on Facebook/Support/Bugs but they said I should post implementation issues here. I have read it everywhere and it seems to be quiet open issue till now. I am not sure, If this will be solved or not.
So, what we are doing is, we have clients - Android and iOS.
Apps on Android/iOS allows users to login into the app and generate the token on the basis of permissions set we have, and we are passing this token to server for fetching further data as and when required for client. As our userbase is increasing we are getting Application request limit reached quiet often.
We are fetching photos of users and their friends using FQL. So, when parallely fetching photos for around 8-10 different users, we are reaching the Application request limit sometimes, which is quiet random and we are not aware of the actual scenario when it breaks up and how. According to facebook the limit, which is 1M calls per day, but we are hitting around 80K - 1 Lac API calls in a day, but as users are increasing it is stretching a bit further, Less than or equal to 200 approax calls/user. We tried doing batch calls as well and we hit the application request limit as well.
If anyone of you could help us understand the complete concept of API limit and how this can be handled, then we will really appreciate the help. We want to understand how API limit is decided and it's rate is calculated over which interval so that we will be able to configure on our side accordingly.
Earlier in the day, we ran into a unique API call issue. Our server started to break for API calls for user tokens that are with us, we (on our systems, other than server) tried fetching the data for those tokens (Simple calls - /me or /me/home), and it was working alright for us but not for server, then we tried setting up another server and redirected the requests to our new server then this server works well for the same set of users. Not sure, what went wrong in this case and how it breaks up. Please help.
Many Thanks,
Reno Jones
Did you look at the Insights -> Developer section of developer.facebook.com for your app?
This will show you a breakdown per api call, including warnings and ones that are currently being throttled and why.
Also, are you sure you're using User token authorization and not just your App token?
Beyond that, we take the information from Insights to find api calls to cache on our side rather than hitting Facebook every time. You will likely have to do something similar if you're not already. They have limits for calling too often, as well as for requesting too much data. For those, we had to reduce the limits of historical data we requested.
Unfortunately the facebook realtime api only informs about something has changed in the friends connection of the app's users.
What do I have to do to identify that UserA has just became friend with UserX?
Currently, anytime I receive a UserA's friends have changed notification from the facebook realtime api, I receive the whole /UserA/friends.json, paging throu the whole result to just identify what has been added since the last time.
While this works, it just feels like a lot of waste in compute-cycles and I like to know if there is a more elegant approach to this...
That's the way it is designed and there is no "solution" for it.
Note that this does not include the actual data values (either from
before or after the update). To obtain those, your app can request
them as normal, subject to the usual privacy restrictions. For data
that you have access to at any time, you may wish to query for that
data immediately following this callback so that it is ready for when
the user returns to your app.
Source: https://developers.facebook.com/docs/reference/api/realtime/
It makes a lot of sense, because Facebook is able to send you ALL UPDATES while not knowing if you have the appropriate permissions to get the new data, so a very easy way to do it in terms of privacy.
If you have a valid user token (in your db) you can retrieve the updated fields via Graph API / FQL and compare it with the data in your database. Without realtime API you need to pull data every x hours/days, which is even more waste of resources.
If you don't have a valid user token you can retrieve the updated fields via Graph API / FQL when the user comes back to your app and compare it with the data in your database. Without realtime API you always need to update/check the data when the user comes back.