My team recently switched to all three technologies in the past several months and have worked hard to get it up and running. Next step is automating our changelogs. We have JIRA set up look for the tags (ex. TAG-123) in github commit messages. Jenkins monitors the GitHub commits on a 5 minute timer, pulls, builds, etc.
What I would like to see is a changelog generated automatically when a build is marked as "Promoted to Production." I would like to see it do something akin to the following:
Query Jenkins for the previous build marked as a production release and get the corresponding git commit SHA1.
Run a diff in between the current Git commit and the previous commit
Find all JIRA tickets that are referenced
Compile a list of JIRA titles
Have list export to a text file and placed in build drop (bonus if it can be accessed directly through Jenkins as well)
Whether this flow is followed as written or not is irrelevant--I'm after the end result and am not looking to re-invent the wheel.. Surely somebody done something like this before?
As far as reinvention goes, I was able to find https://wiki.jenkins-ci.org/display/JENKINS/Promoted+Builds+Plugin which allows somebody to piggy-back on the Promote to Production action and run a separate script. It would then be a matter of gathering functionality to accomplish the above. (I also noticed Jenkins can tag the current GitHub commit, which my team would likely do in addition.)
Anything closer to accomplishing this would be greatly appreciated.
Thank you!
Since nobody on our team had excess time to devote to this, we ended up throwing together a quick solution.
The Process
Install and setup the All Changes plugin for Jenkins.
When we release, we use "build promotion" system which puts stars next to the previous build, so we can easily see the build# looking at the history.
Copy and paste the relevant output from All Changes into something like notepad++ (human-diff'ing ftw!)
Run a regex find/replace. Search on the regex string, replace with an empty string. (below -- there's the big-bang option or its broken up for understandability.)
Manually organize and release in whatever form is the current agreed up standard.
Everything at once
(\s*\(commit:\s[a-z0-9]{40}.\s..detail)|([\r][\n]#.*\B[\r][\n][\r][\n])|(^[ \t]*)
Remove commit hash\s*\(commit:\s[a-z0-9]{40}.\s..detail
Remove time and surrounding line breaks[\r][\n]#.*\B[\r][\n][\r][\n]
Removing leading whitespace: ^[ \t]*
The Analysis
Pros:
Effective overall
Relatively quick to implement
Cons:
Not fully automated.
Need to revert to commit id if you release from multiple Jenkins jobs.
All Changes history only appears to go back as far as the Jenkins job (I could be mistaken about the specifics of this--I just remember minor grievances with something like this at one point.)
As a whole, the cons are somewhat "the nature of the beast." I'd love to read some other solutions. (For when we have that elusive thing called Time, of course!)
Alternative to dependancy on issue trackers could be to purely use the pull requests themselves. For us they had enough context to generate release notes and we used labels for categorisation. I created PullRequestReleaseNotes and you can try it. It supports GitHub, GitLab, BitBucket and TFS and it can generate release notes in markdown from merged pull requests and its labels and optionally post it to an Atlassian Confluence page and post it to a Slack channel as a post. It can be run as part of continuous integration. Here is a sample:
1.2.1 (MASTER) - XX XXX 2016
Enhancements
Category A
Awesome new feature #1854
Fixes
Category Z
Fixed problem with widget #1792
Category Y
Fixed problem with widget #1792
Fixed exception with view layout #1848
Related
I made sweeping changes to 2 projects (call it Project A and B) in my team's repository in this PR *. After getting through testing, review, and getting comments resolved, I found out that a different team is working on conflicting changes to project A on a different PR and those changes need to go to staging first. Now I can't merge my PR because it has changes to A, but my changes to Project B need to be merged ASAP. Now the obvious answer is probably to abandon the PR, do a soft reset and start again with two branches, but my team wants to keep the comment threads on both A and B because they showcase a month of work and important design decisions made by multiple teams during the reviewing process. Is there a way to split my changes to Projects A and B into two separate PR's without losing all the comments?
My current solution is branch a new branch off the current one, manually undo my changes to Project A from the branch, push, and leave the comments to A dangling (I think there will be some comment view that the file reference is deleted), merge. Then create a new PR of the newest branch, copy and paste each of the 50+ dangling comments over to their respective files, resolve all, and merge. This is obviously not great because I lose some context and it will take hours. Other solutions I've seen on here will delete all existing comments altogether and I can't have that.
*Yes, I definitely shouldn't have done that.
There is no out of the box solution for this.
I would suggest to use Azure DevOps APIs, and even that isn’t a walk in the park.
Modus operandi
Choose your language, PowerShell, js, c#, etc and start exploring. Below a suggestion where to start:
Start with getting the original PR using the getbyid api
Since you mention the comments are the most import part, the next thing is to retrieving list with thread comments API.
Optional: Per comment iterate through the thread with the next get api, because the docs state: Represents a comment which is one of potentially many in a comment thread.
From this point, retrieve more info you need or start building the copy of the PR with, create PR and adding comments and other part you need to reuse.
Again, this is not easy and maybe retrieving and saving this in another format already helps you.
I did tried to find a solution for you online, but did find it (yet). Found other handy stuff like this quick and dirty js script to search through comments.
I'm working on a project that has a large number of json files that are never reviewed in pull requests but occasionally need to be changed. Recently we had to make minor changes to them, and github isn't allowing me to create a pull request with those changes. Instead it gives me:
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
I checked the diff locally and the actual code changes are pretty minor (maybe 200 lines changed), but there are millions of changed lines in these json files. Is there any way to tell Github to ignore them? Right now I am unable to make a PR so the changes can't go through our normal company review process.
I've tried using the .gitattributes file with *.json linguist-generated=true unfortunately that had no effect.
Edit: As suggested in the accepted answer, I contacted github support about this case. Their suggestion was to create a new branch with a small commit, create the PR, and then merge the actual branch that I want to deploy into it. This will update the PR, and while the diff still won't display, it will let me create a PR.
I have just been able to solve this problem via the gh command line tool!
gh pr create
Maybe this option did not exist when the original question was posted, but I'll definitely be doing this in future.
When you compare two branches on GitHub, GitHub has to go compute the diff for those changes using Git and then render it for you. In your case, you have millions of lines of changes, and Git doesn't perform very well in this case because the algorithm that's used to compute diffs is O((N + M)D). Thus, if you have a number of differences proportional to the number of lines, the algorithm is essentially O(N²). Having a large N makes that even worse.
GitHub has a limit on long a request can take, so your large number of changes are just not going to render in the interface. It may be possible to choose the branches you want even though the diff won't render and still open the pull request. If not, you may need to resort to using the API, which won't generate the diff for rendering and therefore is likely to work a little better.
I would encourage you to let GitHub Support know about this, if you haven't been able to find a way to do it through the UI, since they can notify someone to make sure the interface is usable to create a PR even if the diff can't render. You probably aren't the first person to encounter this.
You may also want to store these files outside of Git on some sort of artifact server and pull them down to your repository based on hash, in which case you wouldn't have this pathological case.
Is it possible to create new branch directly from Issue which is posted on Github? We have project hosted on Github with many issues, it would be easy for us just to click one button in Issue web interface and create new branch for it to start developing.
This is not possible directly from the issue interface. I can try to provide you some workarounds and maybe start a debate why one would want this feature:
Creating branches directly on the web interface
After you've noticed that you need to start working on an issue you can just go to the repository main page (keyboard shortcut gc) and create a branch with the name you want:
Chrome extensions
If this is a must for you, you can create a chrome extension that augments the UI and creates a branch directly from the Issues WebUI using the GitHub API. There's a lot of open source extensions that augment the default WebUI.
Is it the most efficient way to start a new feature?
I think this changes from developer to developer, but having worked with GitHub for 7 years using Issues I've never felt the need for this feature because unless it's a one line change that doesn't require local testing or compilation, I still want to get down to my command line and IDE. If I have to do that, I still have to at least git fetch to get the branch I just created directly through the issue interface. If that's the case I've always preferred to just look at the issue and run git checkout -b branch-name, optionally with git push if my team needs to see the branch.
Then the issue name wouldn't normally translate to a branch name, at least I wouldn't want that. So that option to create a branch from an issue would probably need to spin out a prompt to allow me to name the branch what I wanted.
This is just my personal opinion and nothing else, hope it helps :)
GitHub finally added this feature request to their roadmap.
Summary
The branch is the first thing a developer creates when the start
working on a new issue. Creating that association makes it really easy
for someone to then follow the work happening and keep everything
connected as they take their idea to code.
Intended Outcome
We want to help developers get started on work faster and signal to
their team where to find the code changes related to an issue. It
should also be really easy to then follow development to the pull
request without the user needing to do additional work to link
everything together.
How will it work?
From the issue page, a user can quickly create a branch with an
auto-generated or custom name that becomes associated with the issue.
They can then fetch the branch and switch to it in their development
environment and further development changes (such as commits and pull
requests) are automatically associated with the issue.
https://github.com/isaacs/github/issues/1125
Nope. You can subscribe this issue.
You could accomplish this by creating a small script.
I'd start by leveraging the Issues Event webhook. This will fire a JSON payload every time an issue is opened. When the webhook fires, your script can then create a new branch using the Create a Reference API endpoint. Note: URL must be formatted as heads/branch, not just branch.
There is an app that automatically creates branches for issues. You can install it here for free: https://github.com/marketplace/actions/create-issue-branch
If you want to create the branches manually instead of automatically:
It does not enable you (directly) to click on create branch from the issue. Instead you would have to write a comment in the issue consisting of: /cib. If you do that, it automatically creates the branch from the issue.
Please note I have no affiliation with the app.
Recently I saw that Github finally added this feature to its Issues Panel.
All you have to do is navigate to an Issue on Github and scroll down to the following section:
After reading and asking about how to deal with git, Github, branching, versioning and tagging, it looks like the best way to denote a new version of software, is by marking the last commit of a specific version with a specific tag.
A structure like this can be assumed:
Commit 1
Commit 2: Tag 1.0.0
Commit 3
Commit 4
Commit 5: Tag 1.1.0
Commit 6: Tag 1.1.0
Where once version 2.0.0 is released, most likely a 1.1.x branch is created to maintain the 1.1 release for hotfixes, etc.
Now I am wondering how it is possible to integrate this with a CI, like Travis for example.
What I do know is that:
A Continuous Integration system, like Travis CI, gets triggered on every commit to any branch, and it is possible to enable/disable this per branch.
I am struggling to figure out if it possible at all to start a build once you have tagged a certain commit, can anyone explain me how this can work?
The idea behind Travis and Continuous Integration systems in general is that first word: Continuous. You want it running on every commit because your tests should always be working (except when you know they're not). Running it on each tag isn't very CI-like, as it prevents the continuous part from happening, and defeats the entire point of it.
CI allows developers to catch bugs before they might've been noticed, even if your contribution docs say that you're supposed to have the build passing before pushing doesn't mean people listen. Travis, in particular, allows for lots of variation and multiple builds per commit with different setups, so you can test for a case that you might not think of when developing locally.
If you really just want to run it on each tag, you could write your build script to exit early if there wasn't a new tag defined. Travis defines a bunch of environment variables that could help you detect it.
Eric suggests that you read your team's diffs every morning. Can I get TFS to automate this in some way? Ideally I'd like an email with all of the differences in, but I'd settle for a link to each of the commits.
As someone who works for Eric and who has the behaviour of checking the diffs each morning let me explain what I do. I'd like to think that I was one of the people he was thinking about when he wrote the post, but I know for a fact that he didn't know I did the diff checking each morning :-)
In Eclipse I use the Team, Synchronize... functionality to compare my local workspace with the latest on the server. As I do a get latest frequently, this tells me what has happened since I last did this (i.e. what changed while I wasn't looking).
In Visual Studio, I can do a similar thing by right clicking on the root folder of the area that interests me and selecting Compare... and then doing a compare of the Workspace version with the latest version.
Alternatively, you can just do a "History..." on the folder that is of interest and a brief scan down the history view will show you what has been happening and you can go look at what is interesting. It also encourages you to leave good check-in comments, and to encourage your developers to do the same :-)
I used to have email alerts configured for each check-in (Team, Project Alerts...), but I just ended up ignoring them most of the time. I even have a robotic rabbit configured to talk to me when someone does a check-in or runs a build - but this is only useful during the day, not checking what has happened the previous day while I was asleep (I live in a different time-zone to the rest of my colleagues so they do a lot of work while I sleep and vice-versa, making the practise of diff-checking even more useful)
In theory it would be possible to write a program that did generate you a diff each day between the latest version and your workspace version, however I've never bothered myself. This is partly because as I find the most value of the practise comes in exploring the changes that were made each day rather than just reading about them. I also admit that I wasn't aware that anyone else in the world was doing this daily diff routine - I figured I was alone in my code voyerisum, but obviously not!
UPDATE Feb 12, 2009: The following blog post just came to my attention.
http://blogs.msdn.com/abhinaba/archive/2008/07/07/auto-generating-code-review-email-for-tfs.aspx
It talks about (and provides source for) a tool called CRMail that will generate an email from a shelveset that will contain links back to Team System Web Access to show the diffs for each change in the changeset. It would be possible to modify this source to get it to show you diffs between changesets if you wanted to. Then you would just need to hook it up to run either as a nightly scheduled task or on every check-in by subscribing to the check-in event from TFS.
Have you explored setting up a report on the project portal that would show diffs based on date? I haven't done this (and I'm at home now so I can't investigate it), but I know that there is a lot of information you can get out of the portal. Whether you can get code diffs, I don't know.
The other alternative would be automating something with tfsadmin or the power tools. Again, not at work so I can't look at it, though the power tools seem like they may make it possible to do what you want from the docs.
A quick solution would be to configure project alerts to send you one email per changeset.
Filter these into a separate folder in your email client, and review them at your leisure.