Unable to paginate releases in GitHub V3 API - github

I'm trying to list all of the releases of a public repository on GitHub using their V3 API. Here's the request I'm making:
curl -is -H 'Accept: application/vnd.github.v3+json' \
https://api.github.com/repos/ffmpeg/ffmpeg/releases
The response headers I receive back are here:
HTTP/1.1 200 OK
Server: GitHub.com
Date: Fri, 29 Jan 2016 20:23:15 GMT
Content-Type: application/json; charset=utf-8
Content-Length: 29612
Status: 200 OK
X-RateLimit-Limit: 60
X-RateLimit-Remaining: 19
X-RateLimit-Reset: 1454099558
Cache-Control: public, max-age=60, s-maxage=60
ETag: "947039722a1073c5279a9fd39d00c0bf"
Vary: Accept
X-GitHub-Media-Type: github.v3; format=json
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: ETag, Link, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-
Access-Control-Allow-Origin: *
Content-Security-Policy: default-src 'none'
Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
X-Content-Type-Options: nosniff
X-Frame-Options: deny
X-XSS-Protection: 1; mode=block
Vary: Accept-Encoding
X-Served-By: b0ef53392caa42315c6206737946d931
X-GitHub-Request-Id: XXXXXXXXXXXXXXXXXXXXXXXXXXXX
Notice the lack of the Link response header? In the response body, I only get back around 7 releases and I can't seem to paginate forward or backward by manually specifying the ?page=N query parameter.
For some background, FFmpeg has about 226 releases present in its GitHub repository, and I'm only getting around 7 of those, unable to paginate through them.
Is there something I'm doing wrong here that would limit my responses back from the GitHub v3 API?

GitHub conflates its proprietary Releases feature with regular Git tags. Many of the "releases" that you see for ffmpeg are actually just tags.
Here is an example of a real release. Note how it contains much more information than a tag does. Despite their web UI showing tags mixed in with releases, GitHub's release API endpoint doesn't include regular tags:
This returns a list of releases, which does not include regular Git tags that have not been associated with a release. To get a list of Git tags, use the Repository Tags API.
Using the tags endpoint as GitHub suggests gives more results, and includes a Link header as you expect:
curl -is -H 'Accept: application/vnd.github.v3+json' \
https://api.github.com/repos/ffmpeg/ffmpeg/tags

Related

Creating a Milestone for a Github Project via REST

I would like to use the REST API of Github to manage the milestones and labels of some of my repositories, but I constantly fail to create new milestones or labels via REST. What do I miss?
I enable 2FA for my account and also create a personal access token, as it is described in Other Authentication Methods
For example I try to create a new milestone with this cURL command:
curl -X POST -v \
-H "Authorization: token MyPersonalAccessToken" \
-H "Content-Type: application/json" \
-H "Accept: application/vnd.github.v3+json" \
-d '{"title": "1", "state": "open", "description": "Milestone", "due_on": "2025-10-09T23:39:01Z"}' \
https://api.github.com/repos/obfischer/boxsack/milestones
I tried various other variants but all seems to be wrong. At least I always get only a 404 Not Found. According to the Github documentation this could also mean that my authentication failed, as Github returns a 404 Not Found response code to protect user data instead of returning a 401 Unauthorized.
But I am pretty sure that the authentication via the Person Access Token is working as I can see a user specific rate-limit headers in the response:
X-RateLimit-Limit: 5000
X-RateLimit-Remaining: 4999
Does someone have a clue why I can't create a milestone via REST?
Maybe your personal access token doesn't have the repo scope. I've just tested to send the request with a token without that scope & it also returns 404 :
< HTTP/1.1 404 Not Found
< Date: Sat, 26 Oct 2019 11:07:31 GMT
< Content-Type: application/json; charset=utf-8
< Content-Length: 125
< Server: GitHub.com
< Status: 404 Not Found
< X-RateLimit-Limit: 5000
< X-RateLimit-Remaining: 4989
< X-RateLimit-Reset: 1572091544
< X-OAuth-Scopes: [Here there should be repo] <================
< X-Accepted-OAuth-Scopes:
< X-GitHub-Media-Type: github.v3; format=json
< Access-Control-Expose-Headers: ETag, Link, Location, Retry-After, X-GitHub-OTP, X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset, X-OAuth-Scopes, X-Accepted-OAuth-Scopes, X-Poll-Interval, X-GitHub-Media-Type
< Access-Control-Allow-Origin: *
< Strict-Transport-Security: max-age=31536000; includeSubdomains; preload
< X-Frame-Options: deny
< X-Content-Type-Options: nosniff
< X-XSS-Protection: 1; mode=block
< Referrer-Policy: origin-when-cross-origin, strict-origin-when-cross-origin
< Content-Security-Policy: default-src 'none'
< X-GitHub-Request-Id: 87B2:23EED:36F2B7F:42647AE:5DB428F3
<
{
"message": "Not Found",
"documentation_url": "https://developer.github.com/v3/issues/milestones/#create-a-milestone"
}

ServiceAccout has empty calendarList after sharing a calendar has been shaired with it

On creating a new service account via Google Console, when then sharing a calendar with the service account, the calendar doesn't appear in the calendarList response for the authenticated service account.
This was working okay for some time but appears to have started failing more recently.
Oddly, if I delete the shared account entry on the calendar and then add it back in, it usually works. It doesn't appear to be a time delay as have waited hours initially and always zero results in the calendar list, until removing the shared account on the calendar and resharing.
The following are steps I've used to reproduce:
In Google Console web UI, create a new service account with 'Furnish a new private key' selected to download the JSON key.
In the Google Calendar web UI, go to calendar settings and 'Share this calendar', then share the calendar with the service account email, then save the changes.
In REST calls, authenticate with the JSON key with POST call to oauth2/v3/token
Send GET request for calendar/v3/users/me/calendarList
then optionally to show it working...
Delete the service account share from the calendar and save.
Add the service account email to the calendar share and save.
Repeat steps 2 and 3. This time it will probably work.
This is partially a manual process for our end users to create a service account and share calendars via Google web UI. Note I've been using additional calendars on my own Google account to share with the service account (this reflects our end user use case), rather than just the default calendar.
Client code is REST based. To provide an example I have shown the REST requests and responses below. There are simply two requests, one to authenticate and one to fetch the calendarList. These occur after the manual steps in the UI to create a service account and then share a calendar with that account.
---
Request:
POST https://www.googleapis.com/oauth2/v3/token HTTP/1.1
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp/105.2.3.0
Content-Type: application/x-www-form-urlencoded
Host: www.googleapis.com
Content-Length: 758
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
grant_type=urn%3Aietf%3Aparams%3Aoauth%3Agrant-type%3Ajwt-bearer&assertion=[ASSERTION_JWT_HERE]
Response:
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, max-age=0, must-revalidate
Pragma: no-cache
Expires: Mon, 01 Jan 1990 00:00:00 GMT
Date: Tue, 14 Mar 2017 20:17:18 GMT
Vary: Origin
Vary: X-Origin
Content-Type: application/json; charset=UTF-8
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Server: GSE
Alt-Svc: quic=":443"; ma=2592000; v="36,35,34"
Content-Length: 197
{
"access_token": "ya29.ElkOBAzSOzE_J2VOGFeWnTAGXdtoadW2FbnGga99SrMeamL7j6KetKomvT4aoy4jsRCcXpK-N6sxRBLFUaj_kPWFin4m6xvg_CtaTtkG5tVc_IxS7IezJDf32g",
"token_type": "Bearer",
"expires_in": 3600
}
---
---
Request
GET https://www.googleapis.com/calendar/v3/users/me/calendarList?minAccessRole=reader HTTP/1.1
Authorization: Bearer ya29.ElkOBAzSOzE_J2VOGFeWnTAGXdtoadW2FbnGga99SrMeamL7j6KetKomvT4aoy4jsRCcXpK-N6sxRBLFUaj_kPWFin4m6xvg_CtaTtkG5tVc_IxS7IezJDf32g
Accept: application/json, application/xml, text/json, text/x-json, text/javascript, text/xml
User-Agent: RestSharp/105.2.3.0
Host: www.googleapis.com
Accept-Encoding: gzip, deflate
Response:
HTTP/1.1 200 OK
Expires: Tue, 14 Mar 2017 20:17:19 GMT
Date: Tue, 14 Mar 2017 20:17:19 GMT
Cache-Control: private, max-age=0, must-revalidate, no-transform
Vary: Origin
Vary: X-Origin
Content-Type: application/json; charset=UTF-8
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
Content-Length: 202
Server: GSE
Alt-Svc: quic=":443"; ma=2592000; v="36,35,34"
{
"kind": "calendar#calendarList",
"etag": "\"p328bl14pt3bd40g\"",
"nextSyncToken": "CJC6hJno1tICEj10ZXN0MS05MjhAY2FsZW5kYXItY29ubmVjdG9yLS0tb25lbGFuLmlhbS5nc2VydmljZWFjY291bnQuY29t",
"items": []
}
---
Note, I haven't provided the actual JWT for the ouath request, but the authentication is working fine. Also note the calendar is shared as either read or modify with the service account. I can also query the calendar itself and fetch events using the service account, but it is querying the list of calendars associated with the account that fails.
Had also raised this as an issue here but posting this here also in case I'm missing something, although as said all this was working fine until more recently, and our existing unchanged client software has started failing with newly created service accounts.
When you share a calendar with someone VIA the google calendar website the code on the website automatically adds the calendar to the users calendar.list. All the calendarlist is the list on the bottom left of the google calendar website. A user can have access to a calendar without it being in their calendarlist.
When sharing a calendar wish a service account this does not always happen. None of the service accounts I have shared my calendars with have ever had anything in their calendarlist. If you need it to be in the calendarlist then you should have the service account insert it. Using CalendarList: insert just grab the calendar id in question off of the website.

How to search using Github API with enterprise

I'm trying to search through repositories, but I can't seem to figure it out with github enterprise edition. I have tried the following with no results. Any suggestions?
curl -i http://my.domain.com/api/v3/repositories "If-Modified-Since: Mon, 16 Jun 2014 01:01:01 CST"
curl -i http://my.domain.com/api/v3/search/repos?q=pushed:2014-06-17
HTTP/1.1 404 Not Found
Server: GitHub.com
Date: Wed, 18 Jun 2014 16:45:58 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Status: 404 Not Found
X-GitHub-Media-Type: github.beta
X-Content-Type-Options: nosniff
Content-Length: 29
Access-Control-Allow-Credentials: true
Access-Control-Expose-Headers: ETag, Link, X-RateLimit-Limit, X-RateLimit-Remaining, X- RateLimit-Res
et, X-OAuth-Scopes, X-Accepted-OAuth-Scopes
Access-Control-Allow-Origin: *
X-GitHub-Request-Id: b4eec0e7-1b1a-48b7-81d8-d63c28b55b37
{
"message": "Not Found"
}
One of the nice things of Github's API both public and Enterprise, is if you go to the API root, it will tell you what endpoints are available. On an enterprise instance it is: http://my.domain.com/api/v3/. Looking at my company's enterprise instance (sorry not sure of the version), I only see the legacy search API endpoints.
As a result: http://my.domain.com/api/v3/legacy/repos/search/pushed:2014-06-17 is likely the search URL you are wanting.

Unable to upload a track using the soundcloud API larger than 7MB

I am unable to upload any tracks larger than about 7MB (413 Request Entity Too Large is returned). This functionality was previously working and the soundcloud api states that tracks can be upto 500MB.
Here is an example using curl with a successful upload(4.9MB) and an unsuccessful one(7.4MB)
I have provided dropbox links to the tracks(my own production, so no copyright issues!!!) if anyone wants to try to replicate. You will need to add your oauth_token.
successful upload = 4900kb_307sec_128kbps_44100hz.mp3
curl -i -X POST "https://api.soundcloud.com/tracks.json" \
-F 'oauth_token=********' \
-F 'track[asset_data]=#4900kb_307sec_128kbps_44100hz.mp3' \
-F 'track[title]=A 4.9MB track' \
-F 'track[sharing]=public'
HTTP/1.1 100 Continue
HTTP/1.1 201 Created
Access-Control-Allow-Headers: Accept, Authorization, Content-Type, Origin
Access-Control-Allow-Methods: GET, PUT, POST, DELETE
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: Date
Age: 0
Cache-Control: no-cache
Content-Type: application/json; charset=utf-8
Date: Wed, 06 Nov 2013 18:22:57 GMT
Location: https://api.soundcloud.com/tracks/118866401
Server: nginx
Via: 1.1 varnish
X-Cache: MISS
X-Cacheable: NO:Cache-Control=no-cache
X-Runtime: 436
X-Varnish: 3652774389
Content-Length: 1623
unsuccessful upload = 7400kb_307sec_192kbps_44100hz.mp3
curl -i -X POST "https://api.soundcloud.com/tracks.json" \
-F 'oauth_token=********' \
-F 'track[asset_data]=#7400kb_307sec_192kbps_44100hz.mp3' \
-F 'track[title]=A 7.4MB track' \
-F 'track[sharing]=public'
HTTP/1.1 100 Continue
HTTP/1.1 413 Request Entity Too Large
Date: Wed, 06 Nov 2013 18:23:21 GMT
Server: ECS (lhr/4799)
Content-Length: 0
Connection: close
thanks
We have the same issue with soundcloud. Seems to be an issue in their nginx.conf (webserver
configuration).
Please contact Soundcloud support for developers.
Was an issue in SoundCloud API routing. Has been fixed now.
For details see comments at:
Soundcloud: increased 413 failures (Request Entity Too Large) on upload

Issue with Google Analytics API

I am trying to use the REST web services API from Google (Analytics) and I am getting a meaningless response instead of the expected data from Google Analytics.
Here is my request and the corresponding response:
GET /auth/analytics.readonly?ids=ga:12660456&start-date=2012-01-01&end-date=2012-02-02&metrics=ga:visits HTTP/1.1
Host: www.googleapis.com
Authorization: OAuth ya29.mytokenhere
HTTP/1.1 200 OK
status: 200
content-length: 18
x-xss-protection: 1; mode=block
content-location: https://www.googleapis.com/auth/analytics.readonly?ids=ga:12660456&start-date=2012-01-01&end-date=2012-02-02&metrics=ga:visits
x-content-type-options: nosniff
x-google-cache-control: remote-fetch
expires: Mon, 21 May 2012 19:52:57 GMT
server: GSE
via: HTTP/1.1 GWA
cache-control: private, max-age=0
date: Mon, 21 May 2012 19:52:57 GMT
x-frame-options: SAMEORIGIN
content-type: text/plain
-content-encoding: gzip
analytics.readonly
Can anyone please help?
Regards,
OAuth isn't performed by requesting the auth scope URL, like you seem to be doing. The only reason the Google's auth scopes are URLs at all are (AFAIK) so that they can be guaranteed to be globally unique.
More details about how to do OAuth 2.0 with Google here: https://developers.google.com/accounts/docs/OAuth2