Iam using the Sound cloud API to fetch tracks by users. But for some reason the tracks that i get back from the API is missing a lot of tracks from the users actual account. The user has about 1000 tracks but my json is only getting a fraction of those.
I'm not sure what the limit of entries in the JSON response is, but I think it's 200. That means, if a user has more than 200 tracks uploaded, and you want to get the tracks, the JSON response only includes 200 tracks, BUT there is a "next_href" cursor. Use that cursor to fetch the next 200 tracks and so on.
For example: if I fetch my latest activities, I only fetch about 30 at a time (setting limit=30). If I want to get more than 30, I just use the next_href cursor.
"next_href": "https://api.soundcloud.com/me/activities.json?limit=30&cursor=asdf1234
(I replaced my cursor with "asdf1234".)
So, just use that cursor for fetching. Hope I could help!
Related
I am working on a pipeline which executes oAuth2 flow in order to access REST API json data. Once I have the bearer token I am executing a request which returns the following structure:
As you can see, since the response is quite large, there's paging enabled and as part of the response I get a link to the next page. In order to get to that resource I need to also present MS-ContinuationToken in the headers. So, this is how I basically do it in the config of the Copy activity that I use to get the data from the REST endpoint:
and the issue here is that I only get the first 2000 rows and the next page(s) don't seem to be visited at all. Pipeline executes successfully and only the first 2000 items are fetched.
NOTE: continuationToken and links.next.headers.value have the exact same values from the initial response.
Even if you fix the other issue you’ll have an issue with the “next” URL not including “v1”. This is a known issue in the partner center api team. I’ve escalated it pretty high. But they don’t want to break backwards compatibility by changing the “next” URI to include the v1 or to be relative. They are considering other options but I wouldn’t hold your breath.
I would ditch the idea of using data factory and instead write a .NET console app using the partner center SDK
(You might think to paginate manually with loops etc but the Copy activity doesn’t return eg the http headers, so you will need a complex set up to somehow store the data in a data store and be able to look up the last page in order to get the continuation token. I couldn’t figure it out)
YouTube API eventType=live not working, does anyone have an idea why?
https://www.googleapis.com/youtube/v3/search?part=snippet&channelId=UCmyKnNRH0wH-r8I-ceP-dsg&eventType=live&type=video&key=
Without eventType was working fine (but not now):
https://www.youtube.com/embed/live_stream?channel=UCmyKnNRH0wH-r8I-ceP-dsg&autoplay=1
Looks like YouTube changed something in API, search.list really not returning live broadcast since end of last week.
If you have user's access token you can use https://www.googleapis.com/youtube/v3/liveBroadcasts?part=snippet&broadcastStatus=active&broadcastType=all
endpoint to retrieve if any broadcast is live.
If you do not have user's access token, you can try this answer
but i didn't check it if it is working
Anyway this question looks like to be a duplicate of this question
Right now the YouTube API is not working, specifically for retrieving live streams of a specific channelId. In other words if you are setting channelId in the API call, you will get 0 results.
If you're using an API key rather than OAuth (not sure if OAuth works) the only work around at the moment is to use the API to search for a specific title. Here is my query URL below.
https://www.googleapis.com/youtube/v3/search?part=id%2Csnippet&type=video&maxResults=20&order=date&q=My+Uniquely+Titled+Livestream&key=[apiKey]
The results returned by the API will be from all of YouTube. (Note: the rest is done serverside) Put the results into an array and discard any that don't match your channelId. Then check the titles in an array and only get the video ID of the one that matches your desired title. That is the basic logic and it is sort of a rigid work around that won't work for most. But at least it will get you what you need until Google fixes the API.
I've uploaded about 40k photos using the Google Photos uploader tool, and now I'm trying to get a list of those photos using the Picasa Web Albums Data API (as there is no separate Google Photos API that I'm aware of).
So far, it appears impossible to get a complete list of all of the photos because you can list only 1000 photos at a time and then use the start-index parameter to do paging, but the server returns an error once you use a start-index above 11000. With a start-index of 11000 this occurs:
gdata.photos.service.GooglePhotosException: (500, 'Internal Server Error', 'Deprecated offset is too large for a stream ID query. Please switch to using resume tokens.')
(I'm using Python, but have confirmed that the error is independent of the language library)
I'd be happy to switch to using "resume tokens" like the error indicates... except that they are not mentioned in any documentation that I could find.
This is an authenticated request and the URL I'm using looks like this:
https://picasaweb.google.com/data/feed/api/user/[myUserID]/albumid/[myAlbumID]?kind=photo&max-results=1000&start-index=11000
Can anyone show me an example request using resume tokens or point me to documentation on them? Or, if anyone knows of some other way to get a complete list of all of the photos in a large album, that'd be great too. Thanks!
EDIT: the problem happens in any language, but in Python you can reproduce it consistently with:
startIndex = 1
while 1:
print '(fetching more photos)'
photos = client.GetFeed(ALBUM_URL, limit=1000, start_index=startIndex)
for photo in photos.entry:
print 'startIndex:', startIndex
startIndex += 1
where ALBUM_URL is like the URL I listed above and client is an authenticated instance of gdata.photos.service.PhotosService.
Some of the API requests to get track details are missing the array of available_countries in which the track can be played.
For instance this Pink Floyd track:
http://api.deezer.com/track/89569653
But when testing in the API console the available_countries are present for this track ID.
It only seems to happen for some tracks though, for instance the Daft Punk API example does have the available_countries data:
http://api.deezer.com/track/3135556
Is it an issue with the API?
The node "available_countries" is there but empty, so it just means that the track is not available in any country, it is not an issue. If you check directly in our website www.deezer.com/track/89569653, you will see that the track is not available.
I'm attempting to retrieve all of my favourited songs from my profile on SoundCloud using the SoundCloud API method /users/{id}/favorites.
After registering my application with SoundCloud and retrieving my ClientID, I can successfully retrieve an array of favourited tracks in JSON using a call to http://api.soundcloud.com/users/goodforenergy/favorites?client_id={my-client-id}.
Looking at my SoundCloud profile, however, you can see I have (at this moment in time) 182 likes - but the method is only returning 48.
Any ideas as to why? I thought there may be a next_href returned in the JSON that I was expected to follow to retrieve more, but I don't see any. In any case, 48 seems a very arbitrary number to return!
According to soundcloud api docs most of the results in the API are returned in collections which are limited by default to a size of 50. I realize that it's peculiar that you are seeing 48 rather than 50 but it's probably worth modifying your call to support pagination.
Most endpoints support a linked_partitioning parameter that will allow you to page through collections. When this parameter is passed, the response will contain a next_href property if there are additional results.
So you need to update your API call to include the linked_partitioning and limit parameters in order to recieve a next_href.