I have a weird issue. Travis OSX builds keep failing because a Jasmine unit test doesn't pass. But, this ONLY happens for commits against the main GitHub repo's master branch. Forked repos, PR's on the main repo, etc do not have this issue. Here are a couple of scenarios:
I sync local master from upstream master and push. Travis build fails.
I make a change, literally any change such as adding a random character to readme.md, commit and push, and the build passes.
I open a PR, the PR build passes
The PR is merged to upstream master, the build fails
There are no issues when running the unit tests locally, so I can only pin it as a GitHub and/or Travis issue. Tried clearing the caches in Travis and did not help. I scrolled through the raw logs side-by-side from a failed and passing build and they were fairly identical, at least nothing I wouldn't expect to be different.
So kind of at a loss here on what to do. Any suggestions?
https://github.com/Glavin001/atom-beautify/blob/master/.travis.yml
This was not an issue specifically with the master branch.
The issue was caused by the Travis environment variable, TRAVIS_COMMIT_MESSAGE. When you merge a PR on GitHub, the default commit message has a "message" and a "description", separate by a line break. Having that line break in the TRAVIS_COMMIT_MESSAGE caused path issues with Ruby and/or Rubygems.
This was resolved by blanking out the environment variable by adding the below to our .travis.yml in the before_install step at the very beginning:
- export TRAVIS_COMMIT_MESSAGE=""
Related
I haven't used Travis-CI but I can see that some GitHub commits have a green mark which shows that Travis build has succeeded and some have a red mark which shows that Travis builds failed. But I am wondering why do a few commits don't have any status?
I am looking at the commits of apache commons-math.
Presumably there were multiple commits locally, which was only pushed after commit 583d9ec. Given that the status of an old commit is irrelevant, Travis CI ignores those and only builds the most recent (source). If any issues did arise, a build could have still been manually triggered, revealing which commit broke something.
We are hosting a c/c++ based repo with multiple branches in Github , while doing CI , there are usually on the average 10 pull requests in each integration build , and if something fails , we have to track it in all of the merged PRs to figure out which one did the mess , i want to work it programaticaly so that i can know which pull requests fails the build. What can be the best approach to work on this problem.
One thing that i forgot to mention is that each PR is individually build and tested already before the merge into feature branch.
Thanks.
The best approach is to change your process a bit and only allow merging of Pull Requests that:
Build without errors (including running integration tests)
Are rebased to fit on the tip of your master branch (fast forward commits only)
Since the branch build without errors, and it was rebased on the latest master, merging it will never break master. Unrebased branches that are seemingly without conflicts can still break things if changes in not directly related files conflict with each other as the conflict algorithm is pretty simplistic.
Lacking that, git also has an option that can be used during an interactive rebase to run a command (the exec option) after each rebase step. In other words, you could rewind your branch back to a known working state, and then let git apply each commit in turn and run a command that checks if everything passes your tests with the exec option.
In this way you can automatically discover which commit broke your tests.
How is it possible that Travis the build failed for the latest push but passed the Pull Request?
On this Gist, you could find the failed and passed output log of the npm run build command that Travis gives. You could also find the configuration of Travis and here below:
install:
- npm install
- npm install -g angular-cli
language: node_js
script:
- gulp html
- gulp scss
- gulp ts
- gulp node
- npm run build
node_js:
- "6.9"
cache:
directories:
- node_modules
- bower_components
Background
This repository is configured in Travis CI to run tests on two environments - named pr and push.
A Pull Request (pr) build will be named continuous-integration/travis-ci/pr and from the docs:
Rather than test the commits that have been pushed to the branch the pull request is from, we test the merge between the origin and the upstream branch. To only build on push events, you can disable Build on Pull Requests from your repository settings.
A push build will be named continuous-integration/travis-ci/push and from the docs
Travis only runs a build on the commits you push AFTER adding the repository to Travis.
Solution
Since the merge of your branch into the base branch passed tests for continuous-integration/travis-ci/push, updating your branch to include the latest commits from the base branch will get your branch passing tests. From the image above, the GitHub UI should allow you to Update branch from the Pull Request page.
Caveat
With branch protections in place, it should be unlikely that your branch fails tests while merging into the base branch succeeds.
Be sure that you confirm that whatever was broken was actually fixed. That is, did someone "fix the build" by disabling that failing test in the base branch? As a cautious person, I would cherry-pick fixes into your branch to verify the problem is resolved.
By a comment of #osowskit, I've found the solution of the problem. He/she said:
PR will merge your changes into the base branch and run CI tests. Push will run CI tests on the current branch. Merging the base branch into your branch will likely resolve your build test on the branch.
To put it in "BDD" terms:
Background:
Given I'm contributing to a GH repo
When I create a pull request
Then Travis should build the latest commit
When I push to an existing pull request
Then Travis should build the latest commit
When I merge a pull request to master
Then Travis should build master
I was confused by Travis-CI's "build pushes" and "build PRs" settings, as:
Enabling both causes each Pull Request to be build twice by Travis
once for the commit on that branch
and once again for the merge commit of that branch into its destination
Enabling just "build PRs" causes PRs to be built, but doesn't result in post-merge builds (i.e. on master).
Enabling "pushes" brute-force satisfies the above criteria by building all pushes to the repo. You can try to finagle things by white- & black-listing branches, but that will probably bite you unless you're rigorously disciplined with branch names.
This is explained more in Travis-CI docs and GH issue #3241.
Anyone know a configuration that satisfies the above criteria?
I eventually found another GH issue (#2111) which gave me the idea to try enabling both PRs & pushes, but with a whitelist to restrict pushes to a specific branch. This seems to satisfy the criteria for my workflow. Here's what I did:
Enable both PRs & branch pushes in the Travis settings for the repo:
Change .travis.yml to white-list master branch (i.e. only build pushes to master):
branches:
only:
- master
Test it by creating a PR with the .travis.yml change, and another PR with some empty commits to verify it works for forks too.
Verify successful merge commit build from master.
Just found in travis docs
Add to .travis.yml
if: type = push
alternatively:
if: type = pull_request
Assuming you want to build all PRs, something like the following will do the trick. Enable both branch and PR builds on the settings page, and put this line as the first line in your travis.yml:
if: (type = push AND branch IN (master, dev)) OR (type = pull_request AND NOT branch =~ /no-ci/)
This will attempt a push build on all pushes and a PR build on all pushes to an open PR, but will filter out any that don't meet the condition. You might need to modify this a bit - the clause about not building branches with no-ci somewhere in their name is obviously optional, and you may not have two branches that you always want to run builds on.
You can read more on conditions and conditional builds on Travis's site.
The whitelist approach described in the accepted answer has some significant limitations. In particular, it doesn't support non-redundantly building arbitrary branches without opening a PR.
I opened an issue asking for a better solution.
You can use next workflow if you want to test not only master branch but some others branches too:
Keep both "Build pushes" and "Build pull requests" ON
Add branches:except directive to your .travis.yml:
branches:
except:
- /^pr\..*/
In this configuration:
any commit to branch feature-A will trigger the build
any commit to branch pr.feature-A will not trigger the build
if branch pr.feature-A is used in opened pull request then build will be triggered
Workflow example
temporary WIP branch shared between several developers: wip.feature-A, any commit to this branch will trigger the build
when branch is ready to be merged to master you can rename it from wip.feature-A to pr.feature-A and open pull request
if while reviewing pull request you want to apply new fixes, just push to pr.feature-A
On all the steps above only one build will be triggered.
For one of the repositories, I was working with, here is what I wanted:
There is an origin repo which is the main repo which does all the releases.
I wanted that all the pull requests coming to master branch of origin should be built with Travis only once irrespective of the fact that it comes from a forked repo or any other branch of the origin itself.
For this case, this works like a charm
if: (type == push) OR (type == pull_request AND fork == true)
Imagine situation like this on github:
repo 1
repo 2 (has dependency to repo 1 build artifact)
where both of these have travis configured.
However I'd like to run travis build of the repo 2 also in the case of successful travis build on the repo 1.
Any chance/idea how to achieve that?
The only idea I had was to:
create new user
and in case of successful travis build of repo 1 do git commit + push repo 2.
However that would lead to dirty commit history on repo 2. Well I could also remove all the new user's old commits, but that increases the complexity might be error prone.
Sorry, It's not possible to do that right now, as you can only start a travis-ci build on a commit at the moment.
Obviously not ideal, but you can, however, in repo 2, clone git repo 1, run the repo 1 unit tests in repo 2, and if they fail stop the repo 2 unit tests from running and end the build.
Unfortunately Travis CI doesn't provide this feature. There is feature request for it, but wasn't approved yet, so it's not even on roadmap so far.
This is reason why I switched to similar service: drone.io.
It's relatively new so some languages are in beta so far. Don't know if it is option for you.
It provides more features than Travis CI,
One of them is HTTPS hook for remote triggering the builds.
So you can configure repo1 Drone job to hit build hook of repo2.
This can be done via wget command (You need to wrap repo2 hook URL into double quotes).