I have noticed that querying the status table for historical data lead to different results recently, without any notification or warning by Facebook.
Before, we could get any status updates past june 2009.
Now we can only query status from the last 100 status updates.
Can anyone confirm this ?
Thanks !
Facebook apparently is now limiting the amount of information stored for each object. They recently announced 50 for one of the lists (only via a bug report) and I presume they're heading to try to decrease their costs of warehousing so much data for the public free API.
Related
I want to analyse my music consumption on Deezer.
Researching resulted in their rest api but the amount of entries in the user history seems to be limited (see https://stackoverflow.com/a/19497151/9909548 to navigate through all entries).
In my case, it resulted in only about 140 tracks which should be thousands.
From now on I could regularly store the results of that request but all data till now would be lost.
I'm aware that, by EU law, I could request all stored user data and parse that (in case I get it digitally) but I would not consider that a solution.
Is there a different api call to get the complete history or another api or some other way to get this?
There is no other way to access the entire history:
Deezer Support on 05.12.2020
Hello, Thanks for your email, apologies for the delay in getting back to you. XXX from Support here
https://www.deezer.com/app/stateeztics/
that is the best tool, but else sadly we do not have data analysis tools or data, besides recently played. Other option though: 2020 highlights (in Music)
Kind regards, XXX
The recommended app is stuck when opened and Deezer is slowly stopping their support for app extensions anyway.
An entire table of Cloudkit records just disappeared from my public production database. The other table of records was intact. I didn't do anything to delete these records. I looked at the logs and I couldn't see anything that looked out of the ordinary. I have no idea how this could have possibly happened. Is CloudKit so unstable that this can just happen randomly? Can anyone give me some insight on this - how can I go about tracking down the source of this issue? Should I assume this is a CloudKit issue? Is there a way to tell what actually happened?
Per one of my dev friends on Twitter, it's a known issue regarding indexes (not data), and Apple's working on it. No ETA I'm aware of, as of 11PM CST.
My app has 8 tables and 3 of them are 'missing' record data.
I'm using the Core Reporting API (Reporting API V4).
Is there any way for me to determine the last time the data my query is returning was updated?
I'd like to be able to indicate whether the data being displayed was last updated several hours ago versus several minutes ago.
The API does respond with isDataGolden which tells you if the data will change again, if your website is small the data processing latency could be almost nothing.
From your question it sounds like you are more interested in not just if the data is stale but how stale. You could request the the ga:hour and ga:minute to find out when the last processed hit was recorded.
Note there is also the Realtime API which gives you a read of what is happening instantaneously.
I'm trying to download 1 minute historical stock prices from Yahoo Finance, both for the current day and the previous ones.
Yahoo (just like Google) supports up to 15 days worth of data, using the following API query:
http://chartapi.finance.yahoo.com/instrument/1.0/AAPL/chartdata;type=quote;range=1d/csv
The thing is that data keeps on changing even when the markets are closed! Try refreshing every minute or so and some minute bars change, even from the beginning of the session.
Another interesting thing is that all of these queries return slightly different data for the same bars:
http://chartapi.finance.yahoo.com/instrument/2.0/AAPL/chartdata;type=quote;range=1d/csv
Replace the bold number with 100000 and it will still work but return slightly different data.
Does anyone understand this?
Is there a modern YQL query that can fetch historical minute data instead of this API?
Thanks!
Historical minute data is not as easily accessible as we all would like. I have found that the most affordable way to gather Intraday Stock Price data is to develop automated scripts that log price information for whenever the markets are open.
Similar to the Yahoo data URL that you shared, Bloomberg maintains 1-Day Intraday Price information in JSON format like this : https://www.bloomberg.com/markets/api/bulk-time-series/price/AAPL%3AUS?timeFrame=1_DAY
The URL convention appears easy to input on your own once you have a list of Ticker Symbols and an understanding of the consistent syntax.
To arrive at that URL initially though, without having any idea for guessing / reverse-engineering it, I simply went here https://www.bloomberg.com/quote/AAPL:US and used Developer Tools on my browser and tracked a background GET request which led me to that URL. I wouldn't be surprised if you could employ similar methods on other Price Data-related websites.
You can also write scripts to track price data as fast as your internet goes. One python package that I find pretty handy and is ystockquote
You can have it request price data every couple of seconds and log that into a daily time series database.
Yes there is other APIs.
I don't know if it can still help but if you need intraday data, there is a API on rapidapi called (Quotient) which allows to pull intraday (at 1-min level), EOD market (FX, Crypto, Stocks (US, CANADIAN, UK, AUSTRALIA, EUROPE), ETFs and Futures. It also provides earnings, dividends, splits and a lot others informations.
I need to fetch the Facebook comments even though some comments are for old posts (may be a few weeks back) , but downloading the full list via Graph API and syncing it on every page view is extremely inefficient.
Does anyone knows the efficient way about how to extract the latest comments? says all comments from yesterday. Since FQL is going to be obsolete in API 2.1 so how can I do it using Graph API.
If you need to fetch the latest comments relatively often (in my case every hour) you can consider saving the last "next page cursor" for each post and try restarting from there.
From the Graph API docs -
Cursor-based pagination is the most efficient method of paging and should always be used where possible - a cursor refers to a random string of characters which mark a specific item in a list of data. Unless this item is deleted, the cursor will always point to the same part of the list, but it will be invalidated if an item is removed. Therefore, your app shouldn't store any older cursors or assume that they will still be valid.
Cursors can't be stored for long periods of time and expected to still work. They are meant to paginate through the result set in a short period of time.
Facebook expects you not to save cursors for future use, but I guess there's no harm in trying anyway and then degrade on fetching all comments (or just the first N pages) if the cursor fails.
EDIT - from my tests cursors are still valid after 12 hours. Maybe it's not a time limit expiration, it could be that a cursor is invalidated if the last comment of that page gets deleted or something.