There are 2 companies
Company A uses Azure Repos
Company B uses Gitlab Repos
I need to set up repos mirroring
Push Gitlab to Azure and Push Azure to Gitlab. In the first case, it was fairly simple to push Gitlab to Azure, using built-in GitLab function Mirroring.
However, I am struggling to find a way how to push Azure Repo to Gitlab repo.
I have tried creating service connection in Azure Devops project and then creating Azure Pipeline as powershell script (Note: In example below, PATs are random numbers for better visualisation):
Write-Host Starting the synchronization process
mkdir copyrepo
$sourceURL = "https://$(Gl684feafeaag56416agvr)#$(https://gitlab.com/blabla/myproject.git)"
$destURL = "https://$(Az14d4w846acf)#$(https://blabla#dev.azure.com/BlaBla/AzProject/_git/AzProject)"
Set-Location "$(Build.SourcesDirectory)/copyrepo"
git clone --mirror $sourceURL
Set-Location "$(Build.SourcesDirectory)/copyrepo/$(myproject)/"
Write-Host "*Git removing remote origin"
git remote rm origin
Write-Output "*Git remote add"
git remote add --mirror=fetch origin $destURL
Write-Output "*Git fetch origin"
git fetch $sourceURL
Write-Output "*Git push to Azure Repos"
git push origin --all -f
It ends up with: git checkout failed with exit code 1
I have even tried to enable checkout for any nested submodule, but stil getting the same error.
Is this even a good way of approaching the goal to push azure repo to gitlab, or is there a better way ?
Related
I'm part of an organization, and through the UI I can create a private repository inside that organization.
Going on repository -> setting -> Collaboration and team, I can see
I'm the admin
I've created my PAT and in fact, I can commit and push other
public repositories.
But if I clone this new repository I get "fatal: unable to access"
Is there anything specific to do when creating repos inside an organization?
If I try to create a new PAT and try to create it for specific repos, I can't see this new repo in the list of my repos! So I have to create it for "All repositories".
Other trials:
git clone https://<username>:<token>#github.com/orgName/repoName failed
git clone https://<username>#github.com/orgName/repoName failed
git clone https://github.com/orgName/repoName of course failed as well
BUT, one strange thing:
git clone https://<token>#github.com/orgName/repoName asked me for a password
I didn't go on, maybe it's recognized just as a new username so it was asking for a password
Turns out for whatever reason you have to use ssh and cannot use PAT and https. After registering a key on GitHub everything worked as expected.
Here is the guide: https://docs.github.com/en/authentication/connecting-to-github-with-ssh/checking-for-existing-ssh-keys
If it is a private repository that is accessed using the classic Personal Access Token(PAT) try resetting the fetch and push url for the remote repo by running:
git remote set-url origin https://<classic PAT >#github.com/organization_name/repo_name
In order to do the same while using the newer fine-grained token:
git remote set-url origin https://oauth2:<fine-grained PAT >#github.com/organization_name/repo_name
If indeed the Personal access token above is authorized to access that repo you should now be able to do all functions from before such as cloning, pushing and pulling.
In the company I'm working we use Azure DevOps and we have a core repository with a huge amount of branches.
In order to maintain this repository clearer as possible I am looking to set an Azure DevOps PR policy (by Azure CLI) to automatically set every PR with 'Delete {source-branch} after merging', but obviously if having an option to specify which branches we do not desire to be deleted after merging
I had look at MS docs about Azure CLI PR options but I do not see this one https://learn.microsoft.com/en-us/cli/azure/repos/pr?view=azure-cli-latest
You can set the ForcePush rights on all the feature branches for given repository like this:
tf git permission /allow:ForcePush /group:$groupContributors /collection:$collection /teamproject:$teamproject /branch:feature /repository:$repo
$collection = "Base URl of your Azure DEVOPS organization"
$teamproject = "Name of the Azure devops Project"
$repo = "Name of the Azure repo under the teamproject"
$groupContributors = "group of the users for which the ForcePush permission will be applied to. Typically the default out of the box SomeAzureDEVOPSProject\Contributors group"
Then the other standard (non admin level) users from your Azure devops project team will be able to delete the source branch on pull request merge.
You need to run the command under VISUAL STUDIO DEVELOPERS PROMPT ( so the tf command is available and on the path, it is possible to run it from your local machine against the Azure ( a sso authentication menu box will pop up)
see also:
https://learn.microsoft.com/en-us/azure/devops/repos/tfvc/git-permission-command?view=azure-devops
and
https://learn.microsoft.com/en-us/dotnet/api/microsoft.teamfoundation.sourcecontrol.webapi.gitrepositorypermissions?view=azure-devops-dotnet
In an Azure Pipelines Task, I am attempting to create and push a new branch. I am able to clone the repo using the $(System.AccessToken) variable, bit when I try to push the new branch I get the following error:
remote: TF401027: You need the Git 'GenericContribute' permission to perform this action. Details: identity 'Build\(GUID)', scope 'repository'.
If I check my repository security, I see that both the Build Service user and Project Collection Build Service Accounts group has Contribute, Create Branch, Contribute to pull request, and Create Tag permission set to "Allow", which from all the research I've done is all I should need to do.
How can I troubleshoot this issue? I assume that either I am missing something silly, or there's a permissions inheritance issue. However, if I'm setting security on the repository itself my assumption is that should override any inherited permissions.
Pipeline:
steps:
- powershell: |
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" clone "https://repoaddress/_git/common"
cd common
git checkout develop
git checkout -b release/$(build.buildNumber) $(build.buildNumber)
git -c http.extraheader="AUTHORIZATION: bearer $(System.AccessToken)" push -u origin HEAD
displayName: 'Create Branch From Tag'
Permissions:
It should caused by your build service account do not have the contribute permission for this repository.
Go Project setting --> Repositories --> click Repos you want to operate -->set repository permissions accordingly.
Note: Service account is Project Collection Build Service (org name)
Update1
I got the issue, add this service account {project name} Build Service ({Org name}) and configure the account permission, it will work.
According to the error message: Details: identity 'Build\(GUID)', scope 'repository'., we could get the service account GUID
Check this REST API, it could list the service account, we could search the service account name via the GUID, then configure the permission.
Update2
Since you are using AccessToken, it update the repo via service account, as another workaround, we could use Personal access token do the same things, and it do not need to configure service account permission.
Update2
A sample power shell script to clone the repo via PAT token:
$MyPat = 'yourPAT'
$B64Pat = [Convert]::ToBase64String([System.Text.Encoding]::UTF8.GetBytes(":$MyPat"))
git -c http.extraHeader="Authorization: Basic $B64Pat" clone https://dev.azure.com/yourOrgName/yourProjectName/_git/yourRepoName
And we will receive two notifications during the lifetime of a PAT - one upon creation and the other seven days before the expiration. You could refer to this doc for more details.
Seven days before your PAT expires, you receive a notification similar to the following example.
Then we could change the Expiration time.
In order for Jenkins to be able to have access to multiple repositories on the same server, I set the .ssh/config as follow:
Host a.github.com
HostName github.com
User git
IdentityFile ~/.ssh/project-a-id_rsa
# same for other repos
and set the Jenkins jobs' Source Code Management (SCM), to git and git#a.github.com:user/repo_a.git. It works fine.
Problem
I want those jobs to be triggered on push events so I set a webhook service in github, .i.e, Jenkins (GitHub plugin). The request received from the webhook are "POST for https://github.com/user/repo_a" which is a different host than the one set in the SCM, .i.e, a.github.com.
Because they are different, the job does not build automatically.
Ugly Solution
I got something running by setting the SCM to github.com and override the remote url of the project's git config once cloned with a.github.com. So the SCM would match the webhook, and jenkins when running git push would use the .ssh/config info.
Question
What else can I do ? Is there a better, easily automated way to achieve this?
I stopped using the deploy key and added my own account credentials on jenkins to be able to deal with all repositories without having to change the host with .ssh/config.
I would like to create a webhook within Gitlab to automatically update a mirror repository on Github, whenever a push event happens. I've checked this page, but I didn't understand how it is done.
My Gitlab version is 6.5. Here is the configuration page:
What should I put in URL? Where do I need to place the script to update the repository?
You don't need a webhook for that. A regular post-receive hook will work very well.
To create and use such a hook you just have to login on the server where your gitlab is installed and create an ssh key for git user.
sudo -u git ssh-keygen -f /home/git/.ssh/reponame_key
(do not type any passphrase when prompted)
Go to your github account and add the public key (it's been created as /home/git/ssh/reponame_key.pub) to your project as a deploy key.
have a look at https://help.github.com/articles/managing-deploy-keys if you need help with that.
Once that is done, you just have to configure the connection between your git server and github's:
add an alias to git user's ssh configuration (add following lines to /home/git/.ssh/config - create it if it's not present)
Host reponame
IdentityFile /home/git/.ssh/reponame_key
HostName github.com
User git
Now add the new remote (using the alias you just created) to your repository:
cd /home/git/repositories/namespace/reponame.git
git remote add --mirror github reponame:youruser/reponame.git
Now that everything is in place you'll have to create the actual hook:
cd /home/git/repositories/namespace/reponame.git/hooks
echo "exec git push --quiet github &" >> post-receive
chmod 755 post-receive
The lastcommand is very important because git will check if a hook is executable before running it.
That's it!
(Replace reponame, namespace and youruser according to your real accounts and enjoy).
Last note: if you want your name andavatar near commits on github, make sure that the email address you are using on gitlab is one of the addresses inked to your github account as well. You'll see your gitlab username otherwise.
If you aren't hosting your own GitLab, GitLab.com has introduced this feature directly, without any workarounds.
From within a project use the gear icon to select Mirror Repository
Scroll down to Push to a remote repository
Checkmark Remote mirror repository: Automatically update the remote mirror's branches, tags, and commits from this repository every hour.
Enter the repository you want to update; for GitHub you can include your username and password in the URL, like so: https://yourgithubusername:yourgithubpassword#github.com/agaric/guts_discuss_resource.git —as noted in the comments, it is much better securitywise to use your GitHub access token here instead of login credentials; will update the answer when i've tested.
For WebHooks processing I'm using sinatra web server.
require 'sinatra'
post '/pew' do
puts JSON.parse request.body.read
# here can be placed signal code to run commit processing script
end
register webhook for push events(or other) to http://localhost:4567/pew within GitLab
and since this moment on each commit gitlab will be sending commit info to url.