The GitHub v.3 API for repository data returns an integer identifier for a given repository. This identifier is the value of the field id in the data returned. In this example, it is the number 1296269:
[
{
"id": 1296269,
"owner": {
"login": "octocat",
... stuff omitted for brevity ...
},
"name": "Hello-World",
"full_name": "octocat/Hello-World",
... stuff omitted for brevity ...
}
]
Is a given identifier value ever reused once it is assigned to a repository, even if a repository or its owner account is subsequently deleted? In other words, are the identifiers unique and permanent, and never reused for any other repositories, ever?
In this context, I don't mean simply renaming a repository; that would not count as "reusing" it in the same sense, nor would replacing the content of a repository completely with other content. I am trying to understand specifically whether the GitHub id values are ever "recycled", if you will.
(I honestly searched in the GitHub documentation and the web, but could not find a statement either way. If I missed it, I apologize and would be happy to be pointed to the appropriate documentation.)
The Ruby toolkit for the GitHub API relies on the uniqueness of a GitHub id since 2014: see issue 483 and PR 485.
At the time (2014), renamed repo were not supported, but since then (April 2015), they are
If you have information about the repo before it was renamed, you should have the id which is returned by the API. If you do then to have a resilient access to the repository, you just need to do
GET /repositories/1234
And you'll always get the repository, regardless of whether the name changes (assuming you still have access to it).
Related
I am scraping some data from GitHub. The RESTful URL to this particular PR shows that it has a merge_commit_sha value: https://api.github.com/repos/ansible/ansible/pulls/15088
However, when I try to get the same PR using GitHub GraphQL API, it shows it does not have any mergedCommit value.
resource(
url: "https://github.com/ansible/ansible/pull/15088"
) {
...on PullRequest {
id
number
title
merged
mergeCommit {
message
}
}
}
For context, the PR of interest is actually merged and should have a merged-commit value. I am looking for an explanation of the difference between these two APIs.
This link posted in the other answer contains the explanation:
As in, Git doesn’t have the originalCommit (which makes sense).
Presumably the original commit SHA is there, but the graphQL API actually checks to see if git has it, whereas the REST API doesn’t?
If you search for the commit SHA the API returns, you can't find it in the repo.
https://github.com/ansible/ansible/commit/d7b54c103050d9fc4965e57b7611a70cb964ab25
Since this is a very old pull request on an active repo, there's a good chance some old commits were cleaned up or other maintenance on the repo. It's hard to tell as that kind of maintenance obviously isn't version controlled.
Another option is the pull request was merged with fast-forward, which does not involve a merge commit. But that wouldn't explain the SHA on the REST API response.
So probably at some point they removed old merge commits to save some space, or something similar. Some objects still point to removed SHAs, but GraphQL API filters on existing objects.
Feel like it is a bug to me because if you query another PR such as 45454 , it can return the mergeCommit:
{
"data": {
"resource": {
"id": "MDExOlB1bGxSZXF1ZXN0MjE0NDYyOTY2",
"number": 45454,
"title": "win_say - fix up syntax and test issues (#45450)",
"merged": true,
"mergeCommit": {
"message": "win_say - fix up syntax and test issues (#45450)\n\n\n(cherry picked from commit c9c141fb6a51d6b77274958a2340fa54754db692)",
"oid": "f2d5954d11a1707cdb70b01dfb27c722b6416295"
}
}
}
}
Also find out other encountered the same problem at this and another similar issue at this. I suggest you can try to raise this issue to them at this.
I'm building a GitHub application to pull commit information from our internal repos. I'm using the following code to iterate over all commits:
gh = login(token=gc.ACCESS_TOKEN)
for repo in gh.iter_repos():
for commit in repo.iter_commits():
print(commit.__dict__)
print(commit.additions)
print(commit.author)
print(commit.commit)
print(commit.committer)
print(commit.deletions)
print(commit.files)
print(commit.total)
The additions/deletions/total values are all coming back as 0, and the files attribute is always []. When I click on the url, I can see that this is not the case. I've verified through curl calls that the API indeed has record of these attributes.
Reading more in the documentation, it seems that iter_commits is deprecated in favor of iter_user_commits. Might this be the case why it is not returning all information about the commits? However, this method does not return any repositories for me when I use it like this:
gh = login(token=gc.ACCESS_TOKEN)
user = gh.user()
for repo in gh.iter_user_repos(user):
In short, I'm wondering what the recommended method is to get all commits for all the repositories a user has access to.
There's nothing wrong with iter_repos with a logged in GitHub instance.
In short here's what's happening (this is described in github3.py's documentation): When listing a resource from GitHub's API, not all of the attributes are actually returned. If you want all of the information, you have to request the information for each commit. In short your code should look like this:
gh = login(token=gc.ACCESS_TOKEN)
for repo in gh.iter_repos():
for commit in repo.iter_commits():
commit.refresh()
print(commit.additions)
print(commit.deletions)
# etc.
I was ecstatic when I got a simple webhook event listener working with GitHub push events on my Azure site, but I realize now I'm not seeing the branch name or id in the json payload (example here https://developer.github.com/v3/activity/events/types/#pushevent)
I thought maybe "tree_id" would be it, but it doesn't seem to be. I couldn't find any info about this in GitHubs's doc. Maybe I need to take one of the id's from the event and make another api call to get the branch? The reason for this is I want to be able to link GitHub push events with my app portfolio, which has branches defined. So, the push events are a way to see code change activity on my different apps -- and knowing the branch is therefore important.
I wrote to GitHub support, and they told me that the branch name is part of the "ref" element in the root of the json payload. When parsing from a JToken object called jsonBody, the C# looks like this
var branchName = jsonBody["ref"].ToString().Split('/').Last();
For example in "refs/heads/master", the branch name is "master"
You need to pay closer look on WEBHOOK response mainly. Here is the trick for JSONPATH ( at-least what I did with my jenkins job):
first read your webhook whole response with character "$". You can catch it is some variable like:
$webhookres='$'
echo $webhookres
Once you have response printed, copy it and paste here: https://jsonpath.com/
Now create your pattern. For example if you want branch name (if event is push):
$.ref
Once you have the branch name( it will have extra string with /), simply trim the unwanted part using awk or cut (linux commands).
You are not limited to this only. All you need to work on pattern and you can make use of this approach for getting other values as well like, author, git repo url etc. and then these can be used in your automation further.
even if you are using any other platform like Azure, JSONPATH concept will be same. because as suggested in accepted answer, "jsonBody["ref"]", it is equivalent to $.ref, as altogether you have to identify the PATTERN ( as here PATTERN is 'ref')
In 2.6.3 Copying Resources and re-identification of DSTU1 there's a description of how clients may have to re-assign ids on resources pulled from a server. My question is what should be allowed when going in the opposite direction. I see no issue with accepting foreign absolute references when there's no re-interpretation needed (ie you accept the URI on POST/PUT and return the same URI on GET), but I'm wondering if they should be accepted if a re-identification is needed on the server side (ie you accept the URI on POST/PUT but assign a new id no the object such that subsequent GET's return a local relative URI).
Are there any guidelines in DSTU1 (or even DSTU2) related to this?
Example
The client POSTs the following:
{
"resourceType": "Patient",
"name": [{"text": "Irene"}],
"careProvider": [{"reference": "https://fhir.example.com/api/Organization/12345"}]
}
The client then does a GET and receives the following:
{
"resourceType": "Patient",
"id": "abc",
"name": [{"text": "Irene"}],
"careProvider": [{"reference": "Organization/987"}]
}
You can see that the server re-identified the Organization into a local reference.
There's no more material than what you already referenced. The material isn't meant to dictate the answers, only to suggest the kind of solutions that might be needed. Generally, then, it's not really possible to talk about what's allowed - except or one thing you mentioned: the server isn't allowed to accept a PUT and then not honor the PUT by moving the resource; it should reject the PUT and insist on a POST. But generally, the mix of clients, servers and middleware in an eco-system, I don't know that we can usefully make rules about what should and shouldn't happen
Regardless of project, I'd like to know if there's an easy way of getting all commits to all public repositories for a single username.
Since I belong to multiple organizations, I'm trying to compile a list of the projects on which I'm a contributor, as well as projects that I have accepted pull requests.
So far my google-fu and looking through the github api docs has proved insufficient.
https://connectionrequired.com/gitspective is your friend. :-) Filter out all but "Push", and you have your view, albeit without the coding work to implement it yourself first.
Inspecting what goes on with the Chrome DevTools "Network" tab might help you mimic the API queries, if you want to redo the work yourself.
The correct way to do this is via the Events API.
First you need to fetch the user's events:
GET /users/:username/events
Then you will want to filter the response array for items where type is set to PushEvent. Each one of these items corresponds to a git push by the user. The commits from that push are available in reverse chronological order in the payload.commits array.
The next step is to filter out commits made by other users by checking the author.email property of each commit object. You also have access to properties like sha, message and url on the same object, and you can eliminate duplicate commits across multiple pushes by using the distinct property.
EDIT: As pointed out by Adam Taylor in the comments, this approach is wrong. I failed to RTFM, sorry. The API lets you fetch at most 300 events and events are also limited to the last 90 days. I'll leave the answer here for completeness but for the stated question of fetching all commits, it won't work.
UPDATE 2018-11-12
The URLs mentioned below have now moved to a single URL that looks like https://github.com/AurelienLourot?from=2018-10-09 but the idea remains the same. See github-contribs.
I'd like to know if there's an easy way of getting all commits to all public repositories for a single username.
The first challenge is to list all repos a user has ever contributed to. As pointed out by others, the official API won't allow you to get this information since the beginning of time.
Still you can get that information by querying unofficial pages and parsing them in a loop:
https://github.com/users/AurelienLourot/created_commits?from=2018-05-17&to=2018-05-17
https://github.com/users/AurelienLourot/created_repositories?from=2018-05-17&to=2018-05-17
https://github.com/users/AurelienLourot/created_pull_requests?from=2018-05-17&to=2018-05-17
https://github.com/users/AurelienLourot/created_pull_request_reviews?from=2018-05-17&to=2018-05-17
(Disclaimer: I'm the maintainer.)
This is exactly what github-contribs does for you:
$ sudo npm install -g #ghuser/github-contribs
$ github-contribs AurelienLourot
✔ Fetched first day at GitHub: 2015-04-04.
⚠ Be patient. The whole process might take up to an hour... Consider using --since and/or --until
✔ Fetched all commits and PRs.
35 repo(s) found:
AurelienLourot/lsankidb
reframejs/reframe
dracula/gitk
...
The GitGub GraphQL API v4 ContributionsCollection object provides contributions grouped by repository between two dates, up to a maximum of 100 repositories. from and to can be a maximum of one year apart, so to retrieve all contributions you will need to make multiple requests.
query ContributionsView($username: String!, $from: DateTime!, $to: DateTime!) {
user(login: $username) {
contributionsCollection(from: $from, to: $to) {
commitContributionsByRepository(maxRepositories: 100) {
repository {
nameWithOwner
}
contributions {
totalCount
}
}
pullRequestContributionsByRepository(maxRepositories: 100) {
repository {
nameWithOwner
}
contributions {
totalCount
}
}
}
}
}
I know this question is quite old, but I've ended up coding my own solution to this.
In the end the solution is to find all potential repositories where the user contributed using the organization_repositories and list_repositories services (I'm using octokit).
Then we find all active branches (service branches) on these repositories and for each of them find only the commits from our user (service commits).
The sample code is a little bit extensive, but can be found here
OBS: As pointed out, this solution does not consider organizations and repositories where you contributed but are not part of.
You can get info about the user using the API method: get-a-single-user
After that you can find all user repositories and then commits with function like that:
def get_github_email(user_login, user_name, key):
'''
:param str user_login: user login for GitHub
:param str key: your client_id + client_secret from GitHub,
string like '&client_id=your_id&client_secret=yoursecret'
:param str user_name: user GitHub name (could be not equeal to user_login)
:return: email (str or None) or False
'''
url = "https://api.github.com/users/{}/repos?{}".format(user_login, key)
#get repositories
reps_req = requests.get(url)
for i in reps_req.json():
if "fork" in i:
# take only repositories created by user not forks
if i["fork"] == False:
commits_url = "https://api.github.com/repos/{}/{}/commits?{}".format(user_login, i["name"], key)
#get commits
commits_req = requests.get(commits_url)
for j in commits_req.json():
#check if author is user (there may be commits from someone else)
if j.get("commit", {}).get("author", {}).get("name") == user_name:
return j["commit"]["author"]["email"]
return False