Committing via travis ci failing - github

I am trying to use grunt-gh-pages extension to commit to my gh-branch. It works fine locally but when I use TRAVIS-CI it fails. It gives the following error message -
Warning: fatal: remote error:
You can't push to git://github.com/tusharmath/tusharm.com.git
Use https://github.com/tusharmath/tusharm.com.git
Use --force to continue.
And when I update the repo option I get the following error -
Warning: remote: Anonymous access to tusharmath/tusharm.com.git denied.
fatal: Authentication failed for 'https://github.com/tusharmath/tusharm.com.git/'
Use --force to continue.
Aborted due to warnings.
So basically I just want Travis-ci to commit the files in the gh-pages branch of my repo. Is there a way to do that?
Update The final .travis.yml that solved the problem
language: node_js
node_js:
- '0.11'
before_script:
- git config --global user.email "tusharmath#gmail.com"
- git config --global user.name "Travis-CI"
after_script:
- git config credential.helper "store --file=.git/credentials"
- echo "https://${GH_TOKEN}:#github.com" > .git/credentials
- node ./node_modules/grunt-cli/bin/grunt release
env:
global:
secure: {"lots-of-seemingly-random-characters"}

You certainly can! The first issue, like you discovered, is due to using the git:// URL to push to, but the git protocol can only be used to clone repositories.
As for the "anonymous access denied" error, that's because you need to let Travis log in to your GitHub account in order to push to the repository. Now, you probably don't want to give Travis your GitHub password, and you certainly don't have to. Instead we're going to use OAuth tokens. If you have no idea what that means, don't worry, I'll explain. An OAuth token in most cases works like a password, but it's easier to revoke access to single things.
To generate an OAuth token, go to the GitHub Applications settings page and click "Create new token" under "Personal API Access Token". You probably want to add a note for what this is, that way it's easier to keep track of and easier to revoke if you need to in the future. Note that this token is essentially a password in that it gives access to the same things a password does.
Then, you need to add the token to your .travis.yml file. First, we'll encrypt the token so only Travis can see it. For this, you need the travis Rubygem installed: gem install travis.
travis encrypt GH_TOKEN="the-token-from-github" --add
Your .travis.yml should now look something like this:
…
env:
global:
- secure: "lots-of-seemingly-random-characters"
…
Now, in order for Travis to actually use this token, you need to add some more things to your .travis.yml too.
after_script:
- git config credential.helper "store --file=.git/credentials"
- echo "https://${GH_TOKEN}:#github.com" > .git/credentials
- node ./node_modules/grunt-cli/bin/grunt release
This first tells git to look for credentials in the .git/credentials file. This can be any file you want, really, but make sure it's not one you're going to push to GitHub. Then, we add the token to the .git/credentials file. Git now knows that for pushes to https://github.com, it can use your token to authenticate.
You should be all set!
PS: If you only want to push to GitHub if the build passes, you can change after_script to after_success.

The answer by henrikhodne is great, but the solution doesn't work with grunt-gh-pages because it creates another Git repository somewhere in .grunt/grunt-gh-pages/ sub-directory. Therefore git config made in after_script or after_success section is not used by grunt-gh-pages.
It's possible to add GH_TOKEN to repository URL used by grunt-gh-pages in Gruntfile.js like this:
'gh-pages': {
// your common gh-pages config
travis: {
options: {
repo: 'https://' + process.env.GH_TOKEN + '#github.com/dim2man/csbrowser.git',
silent: true
},
src: ['**']
}
}
Note the silent: true option, it prevents publishing your token value in Travis logs.
Then your after_script or after_success section can be modified like:
after_success: grunt gh-pages:travis

Related

Terraform Git Clone does not seems to work with GITHUB_TOKEN but works when used a PAT

I am adding a Github actions workflow to execute terraform commands as part of the pipeline.
The terraform code refers refer to terraform modules from another repo as follows.
module <moduleName> {
source = "git::git#github.com:<orgName>/<moduleRepo>.git//<modulePath>?ref=<moduleTag>"
...
}
This will lead to fetching the code from given tag during terraform init command execution.
To ensure that https url is used instead of SSH git url. I am overriding the git config url as follows.
git config --global url."https://oauth2:$GITHUB_TOKEN#github.com/<orgName>/<moduleRepo>.git".insteadOf "ssh://git#github.com/<orgName>/<moduleRepo>.git"
But GITHUB_TOKEN does not allow git clone and this fails with the following error:
remote: Invalid username or password.
fatal: Authentication failed for
'https://github.com/<repoName>/<moduleRepo>.git/'
I also tried adding permission to the workflow for repositories as follows:
permissions:
repository-projects: read
The repo setting for action is set to : Allow all actions and reusable workflows
If I change the GITHUB_TOKEN with my PAT with repo permissions, then the workflow works without any issues.
Please let me know how to configure GITHUT_TOKEN with required permissions. I want to make it work with GITHUB_TOKEN rather than PAT.
Eventually I was able to figure out the issue. The GITHUB_TOKEN is made available to the Github Action workflow as a secret and not as an environment variable.
The issue was I was treating it as an environment variable and using it as such, which lead to the error.
I changed the workflow as follows to use it as a secret.
jobs:
<jobName>:
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
I updated the git config as follows to use the token properly:
git config --global url."https://oauth2:$GH_TOKEN#github.com/<orgName>/<moduleRepo>.git".insteadOf "ssh://git#github.com/<orgName>/<moduleRepo>.git"
The workflow now seems to work properly.
The usage is documented here: https://docs.github.com/en/rest/guides/getting-started-with-the-rest-api#authentication-example-for-github-actions
Adding my answer here to help others facing similar issue.

GitHub Action appleboy/ssh-action: How do I avoid that the SSH key ends up on the server?

To log in from GitHub to my external server I use/test appleboy/ssh-action. As soon as I am on the server I start a git pull to get the latest changes to the server. However, this also includes the .github/workflows folder. And in a GH action yml file is my SSH password. I would like to avoid this. But I don't know how. Somebody know how to do this?
You should not store the password in the YAML file itself. Instead, use the GitHub Actions secrets functionality in the repository settings to store the password as a secret, and then pass it in through the environment. For example, you can pipe a secret like so:
- run: echo $PASSWORD | my-program-here
env:
PASSWORD: ${{secrets.SSH_PASSWORD}}
You can see an example of how this kind of approach is use in the Git LFS release workflow.

Install private repository in build stage on GitHub Actions

I am using GitHub Actions to deploy to Azure. In this project I am using our own private repository's which we host on GitHub. These repository's will be installed during build and their links are stored in requirements.txt, for example:
git+ssh://git#github.com/org-name/package-name.git
Locally, there is no problem installing the requirements, since I have access to these private repository's with SSH. But how would I access these during build in GitHub actions.
I get the error:
Collecting git+ssh://****#github.com/org-name/package-name.git (from -r requirements.txt (line 1))
Cloning ssh://****#github.com/org-nam/package-name.git to /tmp/pip-req-build-9nud9608
ERROR: Command errored out with exit status 128: git clone -q 'ssh://****#github.com/org-name/package-name.git' /tmp/pip-req-build-9nud9608 Check the logs for full command output.
Error: Process completed with exit code 1.
Which makes sense, since it is a private repository.
You might try and include in your GitHub Action workflow the webfactory/ssh-agent action:
When running a GitHub Action workflow to stage your project, run tests or build images, you might need to fetch additional libraries or vendors from private repositories.
GitHub Actions only have access to the repository they run for.
So, in order to access additional private repositories:
create an SSH key with sufficient access privileges.
Then, use this action to make the key available with ssh-agent on the Action worker node.
Once this has been set up, git clone commands using ssh URLs will just work. Also, running ssh commands to connect to other servers will be able to use the key.
That would give a workflow like:
# .github/workflows/my-workflow.yml
jobs:
my_job:
...
steps:
- actions/checkout#v1
# Make sure the #v0.4.1 matches the current version of the
# action
- uses: webfactory/ssh-agent#v0.4.1
with:
ssh-private-key: ${{ secrets.SSH_PRIVATE_KEY }}
- ... other steps
For those wondering, another solution which I found and is easier to apply, is to use an acces token:
- name: Install requirements
run: |
git config --global url."https://${{ secrets.ACCESS_TOKEN }}#github".insteadOf https://github
pip install -r requirements.txt
Don't forget to create a personal access token and set it as ACCESS_TOKEN in your repository secrets.

travis-ci setup releases with --github-token

I am having problems using setup releases with a github token. I like travis-ci but I am not willing to hand out my github password - I need to use the token and I read the documentation as this should be possible this way. Unfortunately it still asks for pasword:
$ travis login --github-token XXXXXXXXX
Successfully logged in as ligi!
$ travis whoami
You are ligi (ligi)
$ travis setup releases
Detected repository as ligi/gobandroid, is this correct? |yes|
Username:
Here's a route which doesn't involve typing your GitHub password into the terminal. I assume you have the travis CI installed. This assumes you're using travis-ci.org, but replacing --org with --com should work otherwise.
If github.com/your/repo was your repo:
Generate a Github personal access token with the following scope: read:org, public_repo, repo:status, repo_deployment, user:email, write:repo_hook
(Optional?) Login using travis login <github token> --org
Run echo <github token> | travis encrypt --org -r your/repo
Use that secret in your .travis.yml file as described in the documentation
You may need to provide full repo scope, but for the free tier of Travis, public_repo is enough. I'm also not sure which of the other scopes are mandatory.
echo is useful on Windows because Ctrl-D doesn't work properly in Powershell.
The Travis CI CLI will not send the GitHub password to Travis CI, instead it will send it to GitHub and use it to generate a GitHub token (the same is true for travis login).
However, if you still feel uncomfortable, you can configure the deployment manually.
Add the following to your .travis.yml:
deploy:
provider: releases
api_key: "GITHUB OAUTH TOKEN"
file: "FILE TO UPLOAD"
skip_cleanup: true
on:
tags: true
all_branches: true
You can encrypt the GitHub OAuth token via travis encrypt .... It is not necessary to be logged in via the CLI for this, and the encryption happens locally.
See http://docs.travis-ci.com/user/deployment/releases/ for the full documentation
I think you can use -t/--token option, e.g.
travis login --org --github-token G1tHu8T0K3N
travis setup releases --org -t G1tHu8T0K3N

How to create a Gitlab webhook to update a mirror repo on Github?

I would like to create a webhook within Gitlab to automatically update a mirror repository on Github, whenever a push event happens. I've checked this page, but I didn't understand how it is done.
My Gitlab version is 6.5. Here is the configuration page:
What should I put in URL? Where do I need to place the script to update the repository?
You don't need a webhook for that. A regular post-receive hook will work very well.
To create and use such a hook you just have to login on the server where your gitlab is installed and create an ssh key for git user.
sudo -u git ssh-keygen -f /home/git/.ssh/reponame_key
(do not type any passphrase when prompted)
Go to your github account and add the public key (it's been created as /home/git/ssh/reponame_key.pub) to your project as a deploy key.
have a look at https://help.github.com/articles/managing-deploy-keys if you need help with that.
Once that is done, you just have to configure the connection between your git server and github's:
add an alias to git user's ssh configuration (add following lines to /home/git/.ssh/config - create it if it's not present)
Host reponame
IdentityFile /home/git/.ssh/reponame_key
HostName github.com
User git
Now add the new remote (using the alias you just created) to your repository:
cd /home/git/repositories/namespace/reponame.git
git remote add --mirror github reponame:youruser/reponame.git
Now that everything is in place you'll have to create the actual hook:
cd /home/git/repositories/namespace/reponame.git/hooks
echo "exec git push --quiet github &" >> post-receive
chmod 755 post-receive
The lastcommand is very important because git will check if a hook is executable before running it.
That's it!
(Replace reponame, namespace and youruser according to your real accounts and enjoy).
Last note: if you want your name andavatar near commits on github, make sure that the email address you are using on gitlab is one of the addresses inked to your github account as well. You'll see your gitlab username otherwise.
If you aren't hosting your own GitLab, GitLab.com has introduced this feature directly, without any workarounds.
From within a project use the gear icon to select Mirror Repository
Scroll down to Push to a remote repository
Checkmark Remote mirror repository: Automatically update the remote mirror's branches, tags, and commits from this repository every hour.
Enter the repository you want to update; for GitHub you can include your username and password in the URL, like so: https://yourgithubusername:yourgithubpassword#github.com/agaric/guts_discuss_resource.git —as noted in the comments, it is much better securitywise to use your GitHub access token here instead of login credentials; will update the answer when i've tested.
For WebHooks processing I'm using sinatra web server.
require 'sinatra'
post '/pew' do
puts JSON.parse request.body.read
# here can be placed signal code to run commit processing script
end
register webhook for push events(or other) to http://localhost:4567/pew within GitLab
and since this moment on each commit gitlab will be sending commit info to url.