I would like to have only my "defined" repository with resource "azuredevops_git_repository" [MyRepo] and not have the system generated one uninitalized with project name [MyProject], is it possible somehow on code level ?
Provider : Terraform-azuredevops
https://registry.terraform.io/providers/microsoft/azuredevops/latest/docs/resources/git_repository
resource "azuredevops_git_repository" "repo" {
project_id = azuredevops_project.project.id
name = "MyRepo"
default_branch = "refs/heads/develop"
initialization {
init_type = "Clean"
}
lifecycle {
ignore_changes = [
initialization,
]
}
}
resource "azuredevops_project" "project" {
name = "MyProject"
description = "MyDescription"
visibility = "private"
version_control = "Git"
work_item_template = "Agile"
}
Could you please help me to find and elaborate on this answer ?
Thank you in advance,
After I deleted the default repository from the Web UI of Azure DevOps, Terraform did not indicated any change on the code level, So I can not understand how can I prevent the "default" repository creation on code level e.g. by the resource "azuredevops_git_repository" or the project definition
as my knowledge, the Terraform-Azure DevOps provider can be used to configure Azure DevOps project in Microsoft Azure using Azure DevOps Service REST API.
For the create project rest api, there isn't a parameter to change the default repo name.
Thus here are two ways for your reference:
1.After creating a new project->do your shown 'create defined name repo' step, you could call corresponding delete repo rest api to delete the default repo (since a project needs at least one repo exists).
2.After creating a new project, you could directly rename the default repo by calling the rest api update repo.
I hope it could do some help.
Related
I have an ADF pipeline. The task is to productionize the pipeline. I am using azure devops CI/CD (classic). I am following this documentation
https://learn.microsoft.com/en-us/azure/data-factory/continuous-integration-delivery-resource-manager-custom-parameters
I have to move the pipeline to test and prod. Thereforem, there are many parameters that are parametrized but few parameters like sql user_name, secret_name are not parametrized.
I want to edit the ARM template and add custom parameter so that I do not have to edit the template.json and paramete-template.json and push them again to repo. The edit option in adf allows to create custom params and therefore generate these in ARM templates when exported.
I have the parameter in the template.
The parameter secretName doesnt appear in ARM template in CD flow
Which mode you are using to configure the parameters?
ARM parameter configuration is only enabled in "GIT mode". Currently it is disabled in "live mode" or "Data Factory" mode.
So, as per above official statement from Microsoft, you should be using Git repository.
Also, take note - Creating a custom Resource Manager parameter configuration creates a file named arm-template-parameters-definition.json in the root folder of your git branch. You must use that exact file name.
There are other multiple ways which you can try to pass secrets in ARM template. Refer this article from devkimchi.com.
After lot of tries and understanding the credential structure the ADF follows for different LinkedServices, we have found that to parametrize a custom nested argument, we have to specify the argument in a nested form. The parameter configuration needs to be edited like this:
For example, the secret name for SQL linked service (using password – connected to azurekeyvault) needs to be like this:
"password": {
"secretName": "="
}
But for the secret type (from azure keyvault) for storage linked service, it has to be like this:
"servicePrincipalCredential": {
"secretName": "="
}
And then these args can be passed directly from azure keyvault if variable groups are connected to keyvault. This solves the problem we were facing.
I'm looking for a way to automate/script Azure DevOps project creation. I know that I can use VSTeam PowerShell module to create a project, but it does not provide a way to programmatically set project settings (the ones on the screenshot below). Is there a way to do that with PowerShell or AZ CLI? In particular I'm looking for a way to control/change these two settings:
Limit job authorization scope to current project for non-release pipelines
Limit job authorization scope to referenced Azure DevOps repositories
You can try to use Rest API:
How to use PowerShell with rest API: Modify Azure Devops Test Case Parameters Through REST API
Rest API Methods: General Settings - Get; General Settings - Update
To create a project you will need to send a POST to https://dev.azure.com/<devopsOrgName>/_apis/projects?api-version=5.0-preview.3
with a body like the below. To get the "template type" value used in the body, you will need to set a GET to https://dev.azure.com/<devopsOrgName>/_apis/process/processes?api-version=5.0-preview.1
` {
"description": "Tailspin Toys",
"name": "Tailspin_TOYS",
"capabilities": {
"versioncontrol": {
"sourceControlType": "Git"
},
"processTemplate": {
"templateTypeId": "24268e03-7eed-4ac0-a178-700881565b99"
}
}
}`
To change specific settings, that you can't find in the documentation, I would suggest using the developer tools in your browser (f12). Watch the network tab and you should see the requests going when you change your setting.
In this instance, I can see the below request when I try these settings
URL: https://dev.azure.com/<devopsOrgName>/_apis/Contribution/HierarchyQuery
Type: POST
Body:
{ "contributionIds":["ms.vss-build-web.pipelines-general-settings-data-provider"],"dataProviderContext":{"properties":{"enforceJobAuthScopeForReleases":"false","sourcePage":{"url":"https://dev.azure.com/<devopsOrgName>/<projectName>/_settings/settings","routeId":"ms.vss-admin-web.project-admin-hub-route","routeValues":{"project":"<projectName>","adminPivot":"settings","controller":"ContributedPage","action":"Execute","serviceHost":"495d404e-cdeb-496a-8496-fccf9df3d9fa (<devopsOrgName>)"}}}} }
Azure DevOps REST isn't the most documented, but when used with your browser develop tools, I haven't found something I couldn't set.
Is it possible to fork a repo that exists in a Private Project of the SourceOrganization into another Project of DestOrganization using Azure DevOps API?
No this is not available.
The question you linked to will allow you to create a one-time mirror of another repository, but you won't have any features like pull requests, diff/merge etc.
You can perform an import request to create a clone in another organization, but it completely ignores the relationship between the repos and you can't perform any pull requests across the organisations.
The code is two-step:
Create a temporary endpoint using a PAT that has at least Code (Read) access on the source organisation.
POST https://dev.azure.com/{{ORG}}/{{PROJECT}}/_apis/serviceendpoint/endpoints
{
"authorization":{
"parameters":{
"password":"{{PAT WITH ENOUGH PERMISSIONS}}",
"username":"."
},
"scheme":"UsernamePassword"
},
"name":"{{RANDOM UNIQUE NAME}}",
"type":"git",
"url":"{{CLONE URL OF SOURCE REPO}}"
}
which will respond with an endpoint ID:
{
"data":{},
"id":"72d574f0-05f1-481f-af15-f78579b374d4",
"name":"wv1w1",
"type":"git"
...
}
Then create the import:
POST https://dev.azure.com/{{ORG}}/{{PROJECT}}/_apis/git/repositories/{{NEW REPO NAME}}/importRequests
{
"parameters":{
"deleteServiceEndpointAfterImportIsDone":true,
"gitSource":{
"overwrite":false,
"url":"{{CLONE URL OF SOURCE REPO}}"
},
"tfvcSource":null,
"serviceEndpointId":"{{ID FROM PREVIOUS REQUEST}}"
}
}
It also can't be used to sync sources between different organisations.
You could cobble something together with a Azure Pipeline that pushes the changes from one org to another. Again, no "fork", but really just a sync.
We use Synapse git Integration to deploy artifacts such as linked services generated by a Data Warehouse automation tool (JSON files)
It is different then deploying ARM template in ADF.
We created one Azure Key Vault (AKV) per environment so we do have an Azure Key Vault LinkedService in each environment and the linked services has the same name. But each AKV as his own URL so we need to change the URL in the deployed linked services during the CI/CD process.
I read this https://learn.microsoft.com/en-us/azure/synapse-analytics/cicd/continuous-integration-deployment#use-custom-parameters-of-the-workspace-template
I think I need to create a template to change "Microsoft.Synapse/workspaces/linkedServices"
But I didn't find any example on how to modify the KV url parameters.
Here is the linked services I want to modify,https://myKeyVaultDev.vault.azure.net as to be changed when deploying
{
"name": "myKeyVault",
"properties": {
"type": "AzureKeyVault",
"typeProperties": {
"baseUrl": "https://myKeyVaultDev.vault.azure.net"
}
}
}
Not much familiar with the ci/cd and azure devOps yet, but still I need to do it...
I have done this using Azure Devops. When you create the Release pipeline within Azure Devops, one of the options is to "override parameters". at this point you can specify the name of the keyvault and the corresponding value. The corresponding value is configured in a pipeline variable set - which itself can come from the same keyvault.
You don't need to create the template. Synapse already does that and stores it in the publish branch (“workspace_publish”). If you look in that branch you will see the template along with the available parameters that you can override.
More info is available here:
https://www.drware.com/how-to-use-ci-cd-integration-to-automate-the-deploy-of-a-synapse-workspace-to-multiple-environments/
https://techcommunity.microsoft.com/t5/data-architecture-blog/ci-cd-in-azure-synapse-analytics-part-1/ba-p/1964172
From the Azure Key Vault side of things, I believe you're right - you have change the Linked Services section within the template to point to the correct Key Vault base URL.
Azure Key Vault linked service
I don't know if you still are looking for the solution.
In order to parametrize linked service property and specially AKV reference, I think you should modify the template-parameters-definition.json, and add the following section:
"Microsoft.Synapse/workspaces/linkedServices":
{ "*":
{ "properties":
{ "typeProperties":
{ "baseUrl": "|:-connectionString:secureString" }
}
}
}
This will create a parameter for each linked service. The next step is to overrideParameters on SynapseWorkspaceDeployment task on Azure Devops.
I need to create the CI/CD pipelines and protect some specific branches in GITHUB for a lot of repositories. But if the remote branch doesn't exists I get an error.
It works only if I have already created the remote branch in GITHUB. But I need to do it all through Terraform or an automated way.
# Configure the GitHub Provider
provider "github" {
token = "${var.github_token}"
organization = "${var.github_organization}"
}
# Protect the CI/CD branch of the foo repository
resource "github_branch_protection" "foo" {
repository = "foo"
branch = "staging"
enforce_admins = true
required_pull_request_reviews {
required_approving_review_count = 2
}
}
Terraform result with GITHUB remote branch:
github_branch_protection.foo: Creating...
github_branch_protection.foo: Creation complete after 3s [id=foo:staging]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
(Actual error) Terraform result without GITHUB remote branch:
Error: PUT https://api.github.com/repos/jetprogramming/foo/branches/staging/protection: 404 Branch not found []
You cannot do this as branch protection is a property of a branch. If the branch does not exist you cannot enable it's branch protection property as you cannot set a property of non-existing object. This feature was introduced as in GitHub flow it is common practice to protect master branch (which is created when you create repository) so the only way to introduce changes to it is through pull request that needs to be approved first.
What you can do for now (as temporary solution) is to first create repository (with terraform) then create branches (with some script using github api) and then apply enable branch protection with terraform.
Furthermore I would also recommend you add describe your usecase in an issue on github page of terraform github provided and request feature to create branches which should solve your problem.
If it's a brand new repository being created by terraform, you can get around this by setting the
default_branch = staging
and
auto_init = true
That way the branch will exist.
It's not elegant, and I don't like it... but it does work around the issue.
If your repo already exists, do not set auto_init = true or your repo is destroyed and recreated.