Tech writers are going to edit text in JSON files via github in our project. Since they never used a IDE (and don't need to) we think about using the web GUI of github. We have a CI, but it would be very cool, if we could run a check before the commit gets done.
Example:
Step 1: tech writer opens in github the JSON file
Step 2: tech writer updates a string
Step 3: tech writer presses button to save the changes
Step 4: Some simple script executes and checks the content of this particular file.
Step 5a: Everything fine? Then commit - END
Step 5b: There is a syntax error. Show the error message to the tech writer.
Is this possible?
GitHub doesn't provide this functionality, and it's not likely anyone else does, either.
In order to do this, GitHub would have to have a non-bare repository on their servers and let you run an arbitrary script. Bare repositories are packed and can be much smaller than a full working tree, and even if GitHub had a working tree, it wouldn't necessarily be on your branch, which would delay the process. Running an arbitrary script is a security risk, and it's not guaranteed that your script would run fast enough for the process to complete in a reasonable amount of time. Your script would also need to run via the API, which has hard limits on response times.
There is GitHub Actions, which lets you do this, but that doesn't provide real-time checking like you want. It is appropriately sandboxed and single use so you can arbitrary scripts.
I should note that Git also doesn't provide real-time checking here: the best it can do is pre-commit hooks to prevent you from committing bad changes (if you've chosen to enable them).
So I am using github pull requests for my code review needs and my only issue is that I cannot tell whether a person is using tabs or spaces for indentation. We have a standard here on this and you can fail code review for using the wrong one. Is there a way to tell which they are using with github or will I have to manually open up the file in my editor to tell the difference?
Is there a way to tell which they are using with github or will I have to manually open up the file in my editor to tell the difference?
Ideally – neither!
Whenever things can be checked in an automated way, let the computer do the work for you. Checking proper usage of whitespace among many other static rules can be checked with a variety of tools, often called linters. This highly depends on what language your project uses. Of course you can also write your own scrips if you so choose.
What you can do on Github is connect your repository to a CI tool such as Travis. This lets you automatically build all pull requests and check things such as whitespace rules. It also lets you run test suites, code formatting, … – anything you can automate, you can (and should!) run from there to minimize manual work.
I'm running a team of developers on a project and I constantly see this kind of commits:
Whenever I get about 5000 lines of code like this, i get quite frustrated
Is there any way to make github not show added whitespaces/newlines?
Set the URL parameter w=1 in the github url which enables the ignore whitepsace feature
Take a look at the difference between
https://github.com/renelink/..../9491a0ed3ae57f7b33386605d3f62f85eca8ae05
and
https://github.com/renelink/..../9491a0ed3ae57f7b33386605d3f62f85eca8ae05?w=1
But I guess the main problem is that the developers work on different OS and do not configure core.autocrlf.
My team recently switched to all three technologies in the past several months and have worked hard to get it up and running. Next step is automating our changelogs. We have JIRA set up look for the tags (ex. TAG-123) in github commit messages. Jenkins monitors the GitHub commits on a 5 minute timer, pulls, builds, etc.
What I would like to see is a changelog generated automatically when a build is marked as "Promoted to Production." I would like to see it do something akin to the following:
Query Jenkins for the previous build marked as a production release and get the corresponding git commit SHA1.
Run a diff in between the current Git commit and the previous commit
Find all JIRA tickets that are referenced
Compile a list of JIRA titles
Have list export to a text file and placed in build drop (bonus if it can be accessed directly through Jenkins as well)
Whether this flow is followed as written or not is irrelevant--I'm after the end result and am not looking to re-invent the wheel.. Surely somebody done something like this before?
As far as reinvention goes, I was able to find https://wiki.jenkins-ci.org/display/JENKINS/Promoted+Builds+Plugin which allows somebody to piggy-back on the Promote to Production action and run a separate script. It would then be a matter of gathering functionality to accomplish the above. (I also noticed Jenkins can tag the current GitHub commit, which my team would likely do in addition.)
Anything closer to accomplishing this would be greatly appreciated.
Thank you!
Since nobody on our team had excess time to devote to this, we ended up throwing together a quick solution.
The Process
Install and setup the All Changes plugin for Jenkins.
When we release, we use "build promotion" system which puts stars next to the previous build, so we can easily see the build# looking at the history.
Copy and paste the relevant output from All Changes into something like notepad++ (human-diff'ing ftw!)
Run a regex find/replace. Search on the regex string, replace with an empty string. (below -- there's the big-bang option or its broken up for understandability.)
Manually organize and release in whatever form is the current agreed up standard.
Everything at once
(\s*\(commit:\s[a-z0-9]{40}.\s..detail)|([\r][\n]#.*\B[\r][\n][\r][\n])|(^[ \t]*)
Remove commit hash\s*\(commit:\s[a-z0-9]{40}.\s..detail
Remove time and surrounding line breaks[\r][\n]#.*\B[\r][\n][\r][\n]
Removing leading whitespace: ^[ \t]*
The Analysis
Pros:
Effective overall
Relatively quick to implement
Cons:
Not fully automated.
Need to revert to commit id if you release from multiple Jenkins jobs.
All Changes history only appears to go back as far as the Jenkins job (I could be mistaken about the specifics of this--I just remember minor grievances with something like this at one point.)
As a whole, the cons are somewhat "the nature of the beast." I'd love to read some other solutions. (For when we have that elusive thing called Time, of course!)
Alternative to dependancy on issue trackers could be to purely use the pull requests themselves. For us they had enough context to generate release notes and we used labels for categorisation. I created PullRequestReleaseNotes and you can try it. It supports GitHub, GitLab, BitBucket and TFS and it can generate release notes in markdown from merged pull requests and its labels and optionally post it to an Atlassian Confluence page and post it to a Slack channel as a post. It can be run as part of continuous integration. Here is a sample:
1.2.1 (MASTER) - XX XXX 2016
Enhancements
Category A
Awesome new feature #1854
Fixes
Category Z
Fixed problem with widget #1792
Category Y
Fixed problem with widget #1792
Fixed exception with view layout #1848
At my work we currently use Aegis version control/SCM. The way we have it configured, we have a bunch of tests, and it forces the following things to be true before a change can be integrated:
The full set of tests must have been run.
All tests must have passed.
With test-driven development (TDD) these seem like sensible requirements. But I haven't heard of any way you can do this with any other version control systems. (We're not currently planning to switch, but I would like to know how to do it in the future without using Aegis.)
I would be interested in any VCS (distributed or not) that can do this, I'm also interested in any plugins/extensions to existing VCS that allow this. Preferably open source software.
ETA: OK, it seems like the usual thing to do is have VCS + continuous integration software, and running the tests is automated as part of the build, instead of as a separate step. If I understand correctly, that still lets you commit code that doesn't pass the tests, just you get notified about it -- is that right? Is there anything that would stop you from being able to integrate/commit it at all?
IMO you're much better off using a continuous integration system like CruiseControl or Hudson if you want to enforce that your tests pass, and make the build rather than the check-in dependent on the tests results. The tools are straightforward to set up, and you get the advantages of built-in notification of the results (via email, RSS or browser plugins) and test results reporting via a Web page.
Regarding the update to the question, you're right - VCS + CI lets you commit code that doesn't pass the tests; with most CI setups, you just won't get a final build of your product unless all the tests pass. If you really want to stop anyone from even committing unless all the tests pass you will have to use hooks in the VCS as others have suggested. However, this looks to me to be hard to deal with - either developers would have to run all of the tests every time they made a checkin, including tests that aren't relevant to the checkin they are making, or you'd have to make some very granular VCS hooks that only run the tests that are relevant to a given checkin. In my experience, it's much more efficient to rely on developers to run the relevant tests locally, and have the CI system pick up on the occasional mistakes.
With subversion and git you can add pre-commit hooks to do this.
It sounds like you need to look at Continuous Intergration (or a variant of).
Think Git has a hook on apply patch too.
Subversion and git both support this via pre-commit hooks.
Visual Studio Team System supports this natively via checkin policies.
I believe that Rational ClearCase also supports it, though I've never seen that demonstrated so I can't say for certain.
We use git and buildbot to do something similar, though not quite the same. We give each developer their own Git repository, and have the buildbot set to build any time pushes to one of those repositories. Then there is someone who acts as the integrator, who can check the buildbot status, review changes, and merge their changes or tell them to fix something as appropriate.
There are plenty of variations of this workflow that you could do with Git. If you didn't want to have someone be the integrator manually, you could probably set the buildbot up to run a script on success, which would automatically merge that person's change into the master repository (though it would have to deal with cases in which automatic merge didn't work, and it would have to test the merge results as well since even code that merges cleanly can sometimes introduce other problems).
I believe continuous integration software such as team city allow you to do pre-commit build and test. I don't know of any vcs that provides it directly...there may be some like the one you use but I'm not familiar with them.
You can also use pre-commit hooks in Perforce. And, if you're a .NET shop, Visual Studio can be configured to require "gated" checkins.
VSTS with custom Work Items, right? I don't see anything wrong with using this. Built in reporting. The choice to automate. Why not?
What I do here is following a branch per task pattern which lets you test the code already submmitted to version control but still keeping the mainline pristine. More on this pattern here.
You can find more information about integration strategies here and also comments about Mark Shuttleworth on version control here.
Most CI implementations have a mechanism to reject check-ins that don't meet all the criteria (most notably pass all the tests). They're called by different names.
VCS should do what they do best.. version source code.
TeamCity - Pre-tested commit
TFS - Gated check-ins