Within Azure DevOps I am trying to create a Command Line Script which is pushing the actual DevOps Repo to GitLab.
git clone https://xxxx#dev.azure.com/xxxx/DBS/_git/xxx
git remote add --mirror=fetch secondary https://oauth2:%pat%#gitlab.com/username/gitlabrepo.git
git fetch origin
git push secondary --all
In the env parameter %pat% I am referencing the Personal Access Token from GitLab.
When running the pipeline with the Comman Line Script I am getting the following error:
start to push repo to gitlab
Cloning into 'gitlabrepo'...
remote: HTTP Basic: Access denied
fatal: Authentication failed for 'https://gitlab.com/***/gitlabrepo.git/'
##[debug]Exit code: 128
##[debug]Leaving Invoke-VstsTool.
##[error]Cmd.exe exited with code '128'.
How could be this achieved?
Make sure the commands work locally in the git bash.
Run git config --global--unset credential.helper command git add command
If the issue persists, try run git remote set-url origin https://usernameHere:personalAccessTokenHere#gitlab.com/usernameHere/projectNameHere
If you use self-hosted agent, go to the agene machine, and remove the credential in Credential Manager.
The Avure DevOps agent is running this on a ubuntu-18.04 Machine. It seems that the connection to gitlab with the PAT fails because this is a corporate gitlab account and have some firewall restriction.Because from my local machine connected via VPN to the corporate network it's working. Git bash locally works fine. Error is because this machine started by the Pipeline Agent is not into the VPN of the company.
Related
I have connected my managed notebook to Github using the instructions here
I cloned my repository using the HTTPS url which works fine only issues is it asks me for UN and PW every time I push or pull so I tried to clone the repository using SSH url which failed to work
I then tried these steps here to Switching remote URLs from HTTPS to SSH using
$ git remote set-url origin git#github.com:USERNAME/REPOSITORY.git
this failed to work also with VertexAI basically freezing up disconnecting from the HTTPS and not recognising the SSH
I have added the relevant SSH keys to my account using the instructions here
does anybody have any advice or any solutions
In short I want to be able to connect VertexAI managed notebook to GitHub via SSH to avoid having to re-enter by UN and PW every time I push or pull
I've tried to clone the repository using the SSH url no avail
I've also tried to change the remote repository using
$ git remote set-url origin git#github.com:USERNAME/REPOSITORY.git
I have created a pipeline to tag and push the commit, I am getting access denied when I push the code to remote origin from the gitlab job.
Note: The script is working fine in the local machine.
- Pushing release to origin
remote: You are not allowed to upload code.
fatal: unable to access 'https://gitlab-ci-token:[MASKED]#xxxxx/xxxx/test_repo.git/': The requested URL returned error: 403
origin
- Switching back to HEAD branch. (restoring local changes)
.gitlab-ci.yml
build:
stage: build
before_script:
- git config --global user.email "${CI_EMAIL}" && git config --global user.name "${CI_USERNAME}"
script:
- bash ./tag-release.sh -b main -t $release
You cannot use the builtin CI token to push to the repo. Instead, you'll need an API token with appropriate scope to push to the repo. A good way to do this is to use project access tokens. Once you have to token, you can put it in the project CICD variables settings. Then use that API token variable instead of CI_JOB_TOKEN.
I have removed the existing origin settings using
git remote rm origin
then added the remote origin with the project access tokens.
git remote add origin https://<access-token-name>:<access-token>#gitlab.com/username/reponame.git
added the personal access token(GIT_TOKEN_VALUE) and personal access token name(GIT_TOKEN_NAME) in cicd variables and masked/protected it for security purpose.
git remote add command with cicd variable is
git remote add origin https://${GIT_TOKEN_NAME}:${GIT_TOKEN_VALUE}#gitlab.com/username/reponame.git
I am new to devops, and I have been googling around, but I cannot find an answer to this simple question:
How do I make my Azure windows VM automatically pull all changes to local repo from master branch?
I could just schedule pull commands on the machine, but that does not seem very Devops to me. All windows guides I can find are more centered around pushing code to their other services.
So do I just manually add 'copy file' segments in the devops pipeline, for all the scripts I wish to deliver to the VM? It's the only way I see from the pipeline.
Sorry if this is too basic.
You can use SSH task and call command like cd /home/dev/<repo> && git pull
# SSH
# Run shell commands or a script on a remote machine using SSH
- task: SSH#0
inputs:
sshEndpoint:
runOptions: 'inline'
inline: cd /home/dev/<repo> && git pull
For endpoint:
The name of an SSH service connection containing connection details for the remote machine. The hostname or IP address of the remote machine, the port number, and the user name are required to create an SSH service connection.
The private key and the passphrase must be specified for authentication.
A password can be used to authenticate to remote Linux machines, but this is not supported for macOS or Windows systems.
Have you seen and do you have a solution for or investigation ideas for this problem:
jenkins pipeline jobs using Pipeline from GitHub SCM intermittently hang on clone/checkout
nothing reported in jenkins logs about this situation
only seems to happen when we have a lot of activity (usually right before sprint demos)
clears after some period of time - haven't been able to detect much a pattern yet
Checking out git git#github.com:MyTeam/myproject into
/var/lib/jenkins/jobs/myproject-dev-deploy/workspace#script to read deploy/dev-Jenkinsfile
git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
git config remote.origin.url git#github.com:MyTeam/myproject # timeout=10
Fetching upstream changes from git#github.com:MyTeam/myproject
git --version # timeout=10
using GIT_SSH to set credentials Deploy key for myproject
git fetch --tags --progress git#github.com:MyTeam/myproject +refs/heads/:refs/remotes/origin/```
A horrible intermittent problem in a java app? To the JVM settings we go. Luckily CloudBees wrote a great article on this Tuning Jenkins GC For Responsiveness and Stability with Large Instances
Turns out this wasn't the prime factor. The real cause was network configuration and impact of a backup job on saturation. Still, an excellent article.
Here's how we debugged it:
Github support gave us this https://github-debug.com/ script and we augmented it thus
`
#!/usr/bin/env bash
echo Clearing Previous Run
rm -rf /tmp/debug-repo-http
rm -rf /tmp/debug-repo-ssh
echo clone1: `date`
time git clone https://github.com/github/debug-repo /tmp/debug-repo-http
echo clone2: `date`
time git clone git#github.com:github/debug-repo /tmp/debug-repo-ssh
echo Ping: `date`
time ping -c 10 github.com
#echo traceroute: `date`
#traceroute github.com
Setup jenkins job to attempt checkout, fail and report if it took longer than 30s - once via pipeline + plugin and once via shell script
Began running various tests: dump all docker images and re-pull, s3 pull backup, s3 push backup etc.
Repeat tests in another network area and compare results
I had meet this issue by git clone command and it's cost me 2 days to resolved this problems.
If you run git command by windows shell/batch on jenkins, it maybe cause by git ssh key issue...
Jenkins windows shell or batch command will use 'system' account to process git command and its ssh key setting is different form your gerrit plugin or login account.
A. Different git ssh key path
The git ssh key of 'system' account is setting in following path:
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
You can check ssh key in these paths for system account.
B. Register key in know_host file
Jenkins will not auto import new key of known host to your know_host file.
You can do it mannually by following steps:
Login your local account in windows
SSH this git remote path directly ssh <git remote web site> -p <port>, type yes to register the key
Copy the registered key form local account know_host (C:\Users\.ssh) you will see the bottom of file likes:
[gerrit.cartography.com]:29168,[10.25.174.78]:29418 ssh-rsa AAAAB3NzaC1yc2EAASSSSAQABAAAAgQDPHbDKCHL3oPbzDn3qaVZR8S1EZx/rb500Th/fjQYg5jrJ7PBMVVUyNTpJ2moje5hLYjNsVtXtcoNEZjImGul2dl3f+eHrCB27X5198jr8Wj66I/xsQCLEngd7l3velsdfgd3432fsdDiFaE9INjDjgngfNzlFA+CYpFsnaorZ6XSPQ==
Paste to 'system' know_host file.
C:\Windows\System32\config\systemprofile\.ssh
C:\Windows\SysWOW64\config\systemprofile\.ssh
I think you will resolve your problem now !
Enjoy it!
I am trying to create run my rails application on ec2 using saltstack and capistrano.
Here's what I have successfully completed so far.
Using salt cloud and salt master I am able to create a new minion instance and setup everything required for the application to run i.e. ruby, rails, unicorn, mysql etc.
I have done proper configuration for capistrano. when I try to deploy I see the following error.
DEBUG [ed84c6ab] Command: ( GIT_ASKPASS=/bin/echo GIT_SSH=/pathto/git-ssh.sh /usr/bin/env git ls-remote -h git#github.com:somehost/somerepo.git )
DEBUG [ed84c6ab] Warning: Permanently added 'github.com,ip' (RSA) to the list of known hosts.
DEBUG [ed84c6ab] Permission denied (publickey).
DEBUG [ed84c6ab] fatal: Could not read from remote repository.
DEBUG [ed84c6ab]
DEBUG [ed84c6ab] Please make sure you have the correct access rights
DEBUG [ed84c6ab] and the repository exists.
DEBUG [ed84c6ab] Finished in 12.600 seconds with exit status 128 (failed).
So this means that from my local capistrano is able to connect to the minion but when it tries to checkout git repo it fails.
I know this is happening because the ssh public key of the minion is not added to the github.
so the goal is.
run salt cloud to create instance
run salt highstate to install everything required for app
run capistrano deploy to start the application
I would like to automate github authorization process too. I mean once the minion is created the minion should be able to clone git repo without any manual intervention.
I am confused as to this can be done through capistrano or saltstack.
I used github ssh forwarding to achieve this.
Here's the changes I made.
Steps to enable ssh forwarding for github
Then in capistrano deploy.rb file configure ssh forwarding by adding forward_agent: true
set :ssh_options, {
user: 'user',
auth_methods: %w(publickey),
port: <some port>,
forward_agent: true
}