I have created an "AWS for Terraform" Azure DevOps service connection to be used for authenticating with AWS. The region I used for the configuration is ap-southeast-2. This is where my S3 bucket containing my terraform state file resides.
After creating the connection, I try to create a release pipeline using a terraform init step.
I select the service connection that I just created from the drop down menu. When I click on the Bucket drop down menu "no results found" is returned. If I change the service connection's region to be us-east-1, the drop down displays a list of buckets (including the one I want). This makes absolutely no sense to me and wondered if anyone could explain?
The issue is that when I select my bucket from the drop down menu, the steps fails when it executes.
There is an apparent "work around" suggested on this page but it does not work for me. Any advice is appreciated.
Related
I am pushing an ADF factory to another environment via a CICD Pipeline and YAML Config file in Azure Devops. I can successfully deploy but one of my linked services becomes a "bad resource" although it works in the master branch when I published it.
Furthermore I cannot delete this in the target data factory nor can I edit it. Getting the bad resource error. I suspect I need to edit something in the ARM file but I don't really understand this error nor can I find much information on similar.
{"stack":"Error: Error: Unable to save [SERVICENAME]. Bad resource\n at Rl.<anonymous> (https://adf.azure.com/app.06b0e174dd8e6fa8.js:1:11274843)\n at Generator.next (<anonymous>)\n at https://adf.azure.com/main.d1fe4ec6f69aa72f.js:1:66326\n at new c
That when I deploy my ADF to a new environment it succeeds with connections intact or at least that I can fix/edit.
EDIT: Even when I recreate the Linked Service I get the same error.
The answer to this is to store all of your connection credentials as secrets in Azure Keyvault then reference that. I am unclear why using the parameters in a linked service do not transfer into the ARM template and this cause it to be a "bad resource" but the Keyvault method translates into ARM correctly and the problem doesn't persist.
I have created a Azure Container Registry in my Azure subscription and wanted to use Azure Devops to build and push my containers. I am trying to create a service connection however when selecting my subscription the screen states "No registries found"
I also tried to create a pipeline using the Docker (Build and push an image to ACR) and this screen also cannot find my registry.
The registry exists and was created as a "Basic" SKU. I have tried other SKUs as well with no luck.
Any ideas of what I can do to be able to select my registry in Devops?
This could be caused by an expired token. Generally, we can try below ways to get it work (force refresh the token).
Go to Azure DevOps user profile page and switch to another directory, and then switch it back again. During the operation, it will force you to sign out and sign in again. Thus, it will refresh the token from AAD. If you have multiple directories here, then please do the same actions one by one, and then switch it back to the original directory (AAD) which you Azure DevOps organization needs to be backed to. After that try it again
Go to the affected ACR from Azure portal, then disable and re-enable admin account. (Click Update -> Enable/Disable the “Admin user” -> Save -> Disable/Enable the “Admin user” again -> Save). After that refresh the DevOps page and check if it works now.
I'm trying to work with a custom activity in Data Factory to execute in a batch accounts pool a python batch stored in a blob storage.
I followed the Microsoft tutorial https://learn.microsoft.com/en-us/azure/batch/tutorial-run-python-batch-azure-data-factory
My problem is when I execute the ADF pipeline the activity failed:
When I check in the Batch Explorer tool, I got this BlobAccessDenied message:
Depending of the execution, it happens on all ADF reference files but also for my batch file.
I have linked the Storage Account to the Batch Accounts
I'm new to this and I'm not sure of what I must do to solve this.
Thank you in advance for your help.
I tried to reproduce the issue and it is working fine for me.
Please check the following points while creating the pipeline.
Check if you have pasted storage account connection string at line number 6 in main.py file
You need to create a Blob Storage and a Batch Linked Services in the Azure Data Factory(ADF). These linked services will be required in “Azure Batch” and “Settings” Tabs when configure ADF Pipeline. Please follow below snapshots to create Linked Services.
In ADF Portal, click on left ‘Manage’ symbol and then click on +New to create Blob Storage linked service.
Search for “Azure Blob Storage” and then click on Continue
Fill the required details as per your Storage account, test the connection and then click on apply.
Similarly, search for Azure Batch Linked Service (under Compute tab).
Fill the details of your batch account, use the previously created Storage Linked service under “Storage linked service name” and then test the connection. Click on save.
Later, when you will create custom ADF pipeline, under “Azure Batch” Tab, provide the Batch Linked Service Name.
Under “Settings” Tab, provide the Storage Linked Service name and other required information. In "Folder Path", provide the Blob name where you have main.py and iris.csv files.
Once this is done, you can Validate, Debug, Publish and Trigger the pipeline. Pipeline should run successfully.
Once pipeline ran successfully, you will see the iris_setosa.csv file in your output Blob.
In the last 6 months I have been releasing with a pipeline in Azure DevOps, but today I receive the following error:
2019-09-25T14:24:38.4296875Z ##[section]Starting: Azure App Service Deploy: AS-ServiciosNegocio-API-UAT
2019-09-25T14:24:38.4419797Z ==============================================================================
2019-09-25T14:24:38.4419900Z Task : Azure App Service deploy
2019-09-25T14:24:38.4419986Z Description : Deploy to Azure App Service a web, mobile, or API app using Docker, Java, .NET, .NET Core, Node.js, PHP, Python, or Ruby
2019-09-25T14:24:38.4420053Z Version : 3.4.31
2019-09-25T14:24:38.4420117Z Author : Microsoft Corporation
2019-09-25T14:24:38.4420182Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/deploy/azure-rm-web-app-deployment
2019-09-25T14:24:38.4420291Z ==============================================================================
2019-09-25T14:24:39.1630446Z Got connection details for Azure App Service:'AS-ServiciosNegocio-API-UAT'
2019-09-25T14:24:39.3091141Z ##[error]Error: Failed to get resource ID for resource type 'Microsoft.Web/Sites' and resource name 'AS-ServiciosNegocio-API-UAT'. Error: Could not fetch access token for Azure. Verify if the Service Principal used is valid and not expired.
2019-09-25T14:24:39.3140156Z ##[section]Finishing: Azure App Service Deploy: AS-ServiciosNegocio-API-UAT
If your existing service connection is the "Azure Resource Manager using service principal (automatic)" type (not manual), there's a simple but non-obvious way to renew the token.
Go to the service connection's settings page in Azure Devops as described in the other answers. (<YourDevAzureProject> Bottom Left → ⚙️ Project Settings → Pipelines subhead → Service Connections)
Click Edit and then Save without making any other changes. Assuming you have the right permissions, it will automatically get a new token.
NB: for some browsers you must enable pop-ups on dev.azure.com as it attempts to login to your azure account to get a list of resource groups.
(Figured this out from this forum comment.)
From reading others' comments/posts on this thread, the Azure UI might have changed so I'm posting the steps here for the later comers. I did what ecraig12345 suggested and it worked great!
Go to the deployment pipeline where the error occurs and click on Edit
Go to "Run on agent" task > Deploy Azure App Service
Click on the Manage hyperlink next to Azure Subscription label (see screenshot below)
Click on Edit
Click Save
Steps 1 - 3
Step 4
Step 5
If you look at the error message: "Verify if the Service Principal used is valid and not expired"
While I would have preferred more information, purely based on the above the likely scenario is the Key Used for the Service Connection has expired.
Visit you Azure DevOps org. and open the related Project and click on "Project
Settings" at the bottom left of the screen.
Click edit on the service connection in Azure DevOps and Click on the
link >> "To update using an existing service principal, use the full
version of the service connection dialog."
Copy the "Service principal client ID"
Now in the Azure Portal, Clic on Azure Active Directory and then Click on "App Registrations" to search for your application with the "client ID"
Go to "Certificate and Secrets" and check if your client certificate has expired.
If the cert is expired generate a new one and copy the key.
Go back to Azure DevOps "Service Connections", Click edit on the service connection in Azure DevOps and Click on the link >> "To update using an existing service principal, use the full version of the service connection dialog."
Update Service Principal Key with the copied value, Verify connection and click ok.
This should solve your issue
Although the route to the problem wasn't exactly the same (because devops changed so much again, probably), the answer from Venura was the root cause of my issue, and I was able to solve it thanks to this info.
steps I had to take:
In devops: go to releases
click correct project
edit
click on the stage that was failing
open the run agent task to deploy (should be an azure app service deploy)
click manage azure subscription
click manage service principal
in azure portal click on the expired registration
click on the red error that is has expired
click + new client secret
copy that new key
go back to devops
click edit on the screen of service connections (where we left at step 7) - (the subscript of the title here is Azure Resource Manager using service principal (manual))
paste that copied key in the field 'Service principal key'
click 'Verify and save'
That solved the issue, to confirm it was solved I just triggered a new release, which finally got through.
I followed JamesD's answer but when I got to step 13, there was nowhere for me to put the Service Principle Key that was generated. So I went back to square one and approached it a different way. Instead of trying to reuse the existing service connection that had exired, I created a new service connection and then changed my release pipelines to use that new service connection and things worked fine.
Here were my steps:
click on Project Settings in the lower left corner
On the left nav under the "Pipelines" section, click on "Service connections"
in the upper right corner, click on the button "New service connection"
select "Azure Resource Manager" and then "Next"
select "Service principle (automatic)" (this is the recommended option)
select the subscription from the drop down.
select the resource group from the drop down
give it a good name and hit save
then authenticate with your azure portal creds
Now you have a service connection created, lets go change the pipeline to use it
Go to your pipeline for the release and edit it
click on the Stage you want to edit (aim for the # tasks link)
click on Deploy Azure App Service
under the azure subscription drop down, select your new subscription entry you created above
then you will select the App Service name in that drop down
hit save and you are good to go
Now repeat for any other stages of the pipeline or any other failing release pipelines
We currently have one DevOps repository, with a functional CI/CD pipeline. We have another website hosted on a different instance (and different region) on Azure. We are trying to use our existing repo to deploy to the other Azure instance, but it is giving is the following message:
Failed to query service connection API: 'https://management.azure.com/subscriptions/c50b0601-a951-446c-b637-afa8d6bb1a1d?api-version=2016-06-01'. Status Code: 'Forbidden', Response from server: '{"error":{"code":"AuthorizationFailed","message":"The client '2317de35-b2c2-4e32-a922-e0d076a429f5' with object id '2317de35-b2c2-4e32-a922-e0d076a429f5' does not have authorization to perform action 'Microsoft.Resources/subscriptions/read' over scope '/subscriptions/c50b0601-a951-446c-b637-afa8d6bb1a1d'."}}'
I have tried all of the recommended trouble-shooting, making sure that the user is in a Global Administrator role and what-not, but still not luck. The secondary Azure subscription that we are hoping to push our builds to is a trial account. I'm not sure if it being a trial account matters.
I came across the same error. It turns out that, as the error message states, the service principal didn't have Read permission over the subscription. So the solution was to go to Azure Portal, select the subscription, select IAM and assign the role Reader to my service principal. Full explanation on here:
https://clydedz.medium.com/connecting-azure-devops-with-azure-46a908e3048f
I have the same problem. There are one repository and two instances of the application on the Azure portal. For the first instance, the subscription Pay-As-You-Go is used, and there were no problems for it when creating the service connection and CI/CD settings. For the second instance, a free subscription is used and when trying to create a new service connection (Azure Resource Manager) I get the same error.
I tried to do it with the permissions of Owner and Contributor
UPD: I was helped by the re-creation of the application in the azure portal
https://learn.microsoft.com/en-ca/azure/active-directory/develop/howto-create-service-principal-portal
Another option would be to save without verification if the Service Principle will not require permissions at the Subscription level. Like for example providing access to a Keyvault.
Check if the service connection for the second instance is correctly added in project settings: