Easiest method to Trigger Azure DevOps pipeline (CI) from Bitbucket? - azure-devops
we have all our pipelines on Azure DevOps:
...but we are using a Repo from another team and it is on Bitbucket Server (company owned). We have our own branch there on the bitbucket repo we use, and I would like to be able to merge PRs to this bitbucket repo and have it "trigger" the Azure (build) pipeline. I am currently just running it manually but it is very tedious. Also, this is on a Bitbucket Server that my company owns (not the regular bitbucket cloud (public) so we have our own custom domain name i.e. www.bitbucket.MY-COMPANY-NAME.com). I tried using the built-in feature on Azure devops (CI) build pipeline to link to the bitbucket repo to trigger "Scheduled" builds at 12AM at night but it always fails for this message:
An exception occurred while polling the repository. Error:
Microsoft.TeamFoundation.Build2.Server.Extensions.ExternalConnectorException: No references received in
the response from https://bitbucket.MY-COMPANY-NAME.com/scm/some-repo/some-repo.git/info/refs?
service=git-upload-pack. Status: 502, Reason: Bad Gateway at
Microsoft.TeamFoundation.Build2.Server.Extensions.GitConnector.ReadRefs(IVssRequestContext
requestContext, HttpResponseMessage response) in
D:\v2.0\P1\_work\1\s\Tfs\Service\Build2\Extensions\SourceProviders\Git\GitConnector.cs:line 318 at
Microsoft.TeamFoundation.Build2.Server.Extensions.GitConnector.GetBranches(IVssRequestContext
requestContext, ExternalConnection connection, Int32 timeoutSeconds, Boolean useAnonymousAccess) in
D:\v2.0\P1\_work\1\s\Tfs\Service\Build2\Extensions\SourceProviders\Git\GitConnector.cs:line 125 at
Microsoft.TeamFoundation.Build2.Server.Extensions.GitSourceProvider.GetMatchingBranchRefs(IVssRequestCont
ext requestContext, BuildDefinition definition, IList`1 branchFilters) in
D:\v2.0\P1\_work\1\s\Tfs\Service\Build2\Extensions\SourceProviders\Git\GitSourceProvider.cs:line 463 at
Microsoft.TeamFoundation.Build2.Server.Extensions.GitSourceProvider.GetSourceVersionsToBuild(IVssRequestC
ontext requestContext, BuildDefinition definition, List`1 branchFilters, Boolean batchChanges, String
previousVersionEvaluated, Dictionary`2& ciData, String& lastVersionEvaluated) in
D:\v2.0\P1\_work\1\s\Tfs\Service\Build2\Extensions\SourceProviders\Git\GitSourceProvider.cs:line 369 at
Microsoft.TeamFoundation.Build2.Server.Extensions.BuildPollingJobExtension.Run(IVssRequestContext
requestContext, TeamFoundationJobDefinition jobDefinition, DateTime queueTime, String& resultMessage) in
D:\v2.0\P1\_work\1\s\Tfs\Service\Build2\Extensions\BuildPollingJobExtension.cs:line 98.
No logs available for this run
According to Microsoft's document Build on-premises Bitbucket repositories:
If your on-premises server is reachable from the servers that run
Azure Pipelines service, then:
you can set up classic build and configure CI triggers
If your on-premises server is not reachable from the servers that run Azure Pipelines service, then:
you can set up classic build pipelines and start manual builds
you cannot configure CI triggers
YAML pipelines do not work with on-premises Bitbucket repositories.
PR triggers are not available with on-premises Bitbucket repositories.
And here is a document that provides some troubleshooting advices about failing triggers:
Is your Bitbucket server accessible from Azure Pipelines? Azure Pipelines periodically polls Bitbucket server for changes. If the
Bitbucket server is behind a firewall, this traffic may not reach
your server. See Azure DevOps IP Addresses and verify that you have
granted exceptions to all the required IP addresses. These IP
addresses may have changed since you have originally set up the
exception rules. You can only start manual runs if you used an
external Git connection and if your server is not accessible from
Azure Pipelines.
Is your pipeline paused or disabled? Open the editor for the pipeline, and then select Settings to check. If your pipeline is
paused or disabled, then triggers do not work.
Have you excluded the branches or paths to which you pushed your changes? Test by pushing a change to an included path in an included
branch. Note that paths in triggers are case-sensitive. Make sure
that you use the same case as those of real folders when specifying
the paths in triggers.
Related
How can I securely deploy forks to PR authors own Azure Subscription?
I want to perform deployments of PRs originating from a GitHub.com fork to the fork/PR authors own Azure Subscription. I could do this by making the azureSubscription input on a deploy task dynamic in the pipeline, using the authors name from System.PullRequest.SourceBranch and configure a Service Connection with the same name for their subscription. However, a nefarious PR author could simply change the pipeline code in the PR to target whatever azureSubscription they choose, allowing them to deploy arbitrary/unreviewed code to whatever Azure Subscription I have configured in my Azure DevOps account to use resources where they shouldn't. How can I dynamically target fork PR deployments to the authors Azure Subscription I have configured without allowing them to target a different subscription as part of the PR by changing the pipeline code?
How Azuredevops git checkout or cloning is happening
I would like to understand how the auto git checkout is happening azuredevops build pipelines. How the authentication to ADO repo is happening from the ADO build agent. Which user is used by the ADO pipeline to clone or checkout this repo. My concern is ,when i trigger a pipeline, i could see that the repository is checkedout to build agent. Which user is used by the ADO ? when i try to push, its asking for AD authentication, So how the checkout and push are differ in ADO pipeline perspective?
when you set up your pipeline initially, you specified where you code resides, e.g. on GitHub. As part of that step a service connection will have been created (you have been asked for it). So whatever you specified there, will be used. You can check it out in the project settings under "Service Connections"
It depends on which source code tool you use. For example If you use Github, you will have to setup a connection with it. This can be accomplished on project settings under Github connections. You can then use this service connection and checkout your code. This Github integration has been made from a particular User and by navigating on github -> Settings -> Integrations -> Applications, you will notice the exact permissions. I imagine that Github/Azure Devops then will use this integration object that is created for the authentication. On Azure Repos repositories, you do not need a service connection in place. The repository can be checked out automatically. When you run the pipeline you can see the exact commands that have been executed for the authentication. git remote add origin https://ORG.visualstudio.com/test-project/_git/test-project git config gc.auto 0 git config --get-all http.https://ORG.visualstudio.com/test-project/_git/test-project.extraheader git config --get-all http.extraheader git config --get-regexp .*extraheader git config --get-all http.proxy git config http.version HTTP/1.1 git -c http.extraheader="AUTHORIZATION: bearer ***" fetch --force --tags --prune --prune-tags --progress --no-recurse-submodules origin
Microsoft-hosted agents run on secure Azure platform. However, you must be aware of the following security considerations. Although Microsoft-hosted agents run on Azure public network, they are not assigned public IP addresses. So, external entities cannot target Microsoft-hosted agents. Microsoft-hosted agents are run in individual VMs, which are re-imaged after each run. Each agent is dedicated to a single organization, and each VM hosts only a single agent. There are several benefits to running your pipeline on Microsoft-hosted agents, from a security perspective. If you run untrusted code in your pipeline, such as contributions from forks, it is safer to run the pipeline on Microsoft-hosted agents than on self-hosted agents that reside in your corporate network. When a pipeline needs to access your corporate resources behind a firewall, you have to allow the IP address range for the Azure geography. This may increase your exposure as the range of IP addresses is rather large and since machines in this range can belong to other customers as well. The best way to prevent this is to avoid the need to access internal resources. Hosted images do not conform to CIS hardening benchmarks. To use CIS-hardened images, you must create either self-hosted agents or scale-set agents. Taken from Microsoft-hosted agents - Security. The most important part is probably Microsoft-hosted agents are run in individual VMs, which are re-imaged after each run. Each agent is dedicated to a single organization, and each VM hosts only a single agent. Next to that, check Create and manage agent pools - Security of agent pools.
Azure data factory promotion from test to dev, authentication method is Managed identity for Azuressql,using azure devops CI/CD
I am using ADF and authentication method is Managedidentity -- then code is pushed to adf_publish branch in Git hub and from there we are building the CI/CD in azure devops. Problem - I used Managed identity for all the resources used in ADF like (keyvault,storage,azuresql) for Keyvault I am geeting this http url (https://BDAdhfsKV01.vault.azure.net/) this url we are paramterised in Azure devops release pipline (https://"$(vaultname)".vault.azure.net/) like this, but for Azure sql while using Managed identity I am getting blank in connection string after publishing to adf_publish branch to Git. How can be prameterised or promote to dev. Just for refrence attaching the devops screen shot how I have paramterisded the keyvault URL at CD. Same case blank connection string using Managed identity is with Azure synapse as well , Is there any help on this or any issue with Managed identity.
You just need to set the connection string as below: "connectionString": "Server=tcp:<servername>.database.windows.net,1433;Database=<databasename>;Connection Timeout=30" You can also refer to this tutorial.
YAML Pull Request Security
When reading the documentation, it says that when doing a pull request, the "source" azure pipelines file is read when doing the PR check. How is this in any way secure? Any developer that executes a pull request can now use the service connections the build might use and do whatever they want with it. In other systems, it always uses the target branch CI configuration for pull requests. Is there any way to configure Azure Devops for this behavior? What's the best practice here?
Azure DevOps yaml configuration with Bitbucket always return 'Forbidden'
I'm having issue with Az DevOps with Bitbucket connection using yaml configuration. It always returns: "Unable to configure a service on the selected Bitbucket repository. Bitbucket returned the error 'Forbidden'." Don't know what permissions are actually required. I can't use the owner account, we will need to config an account with just sufficient permissions to perform actions. Typical MS error message... Anyone has a solution for it?