I have created VMSS using custom image. I have hosted web application build in .Net MVC on VMSS. I have configured CI/CD from Azure DevOps by referring following https://learn.microsoft.com/en-us/azure/devops/pipelines/apps/cd/azure/deploy-azure-scaleset?view=azure-devops .
It is showing error D:\a\_temp\1575277721063\packer\packer.exe failed with return code: 1 . Any suggestion/recommendation is appreciated.
Below is some failed commands in Log:
1. azure-arm: resources.DeploymentsClient#CreateOrUpdate: Failure sending request: StatusCode=200 -- Original Error: Long running operation terminated with status 'Failed': Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details."
2. Some builds didn't complete successfully and had errors:
2019-12-02T09:57:31.5222618Z --> azure-arm: resources.DeploymentsClient#CreateOrUpdate: Failure sending request: StatusCode=200 -- Original Error: Long running operation terminated with status 'Failed': Code="DeploymentFailed" Message="At least one resource deployment operation failed. Please list deployment operations for details. Please see https://aka.ms/DeployOperations for usage details."
3. 2019-12-02T09:57:31.5222868Z ==> Builds finished but no artifacts were created.
Since the error message (DeploymentFailed) from the pipeline is a generic one, it would be tough to investigate the issue without looking at the underlying logs or your pipeline details.
For troubleshooting it further, please try the following:
View deployment history with Azure Resource Manager as mentioned in the error message itself.
Gather logs to diagnose problems, such as debug/verbose pipeline logs, worker/agent diagnostics logs etc..
Look at some common issues and resolutions if it helps.
Send feedback and report problems through the Developer Community for Azure DevOps.
Related
I'm using an Azure Pipeline to deploy to an App Service. I make it through the built stage, but am getting errors when deploying.
The deployment logs say "An unknown error has occurred. Check the diagnostic log for details."
Below are the errors in diagnostic log with system diagnostics turned on:
Error in changing file permissions : 124
##[error]Failed to deploy web package to App Service.
##[debug]Processed: ##vso[task.issue type=error;]Failed to deploy web package to App Service.
##[error]To debug further please check Kudu stack trace URL : https://$app-dev:***#app-dev-django.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace
##[debug]Processed: ##vso[task.issue type=error;]To debug further please check Kudu stack trace URL : https://$app-dev:***#app-dev.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace
##[debug]Deployment Failed with Error: Error: Package deployment using ZIP Deploy failed. Refer logs for more details.
##[debug]task result: Failed
##[error]Error: Package deployment using ZIP Deploy failed. Refer logs for more details.
I've tried restarting both the deploy VM and the app service. I've also tried disconnecting the in the App Service Deployment Center before running the pipeline.
text: No space left on device, stackTrace: at System.IO.FileStream.WriteNative(ReadOnlySpan1 source)
The cause of the issue could be that storage space of the App Service plan has reached the upper limit. It can be that the temp space in App service takes up too much space.
You can restart all the App Services present in that App Service Plan to clear out the temp space.
Or you can upgrade your APP Service Plan to get more space.
I have a pipeline deploying to my Azure web app, that most of the times errors out because it couldn't deploy to my web app. The task take around 25 mins :
...
Copying file: 'frontend/.gitignore'
Copying file: 'frontend/README.md'
Copying file: 'frontend/package.json'
Copying file: 'frontend/tsconfig.json'
Copying file: 'frontend/yarn.lock'
Omitting next output lines...
An error has occurred during web site deployment.
Kudu Sync failed
\n/opt/Kudu/Scripts/starter.sh "/home/site/deployments/tools/deploy.sh"
##[error]Failed to deploy web package to App Service.
##[error]To debug further please check Kudu stack trace URL : https://$someapp:***#someapp.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace
##[error]Error: Package deployment using ZIP Deploy failed. Refer logs for more details.
...
When i enable : system.debug = true , i see these logs repeated many time , before start copying the artifact files :
POLL URL RESULT: {"statusCode":202,"statusMessage":"Accepted","headers":{"transfer-encoding":"chunked","content-type":"application/json; charset=utf-8","location":"http://XXXXXXXXX.scm.azurewebsites.net:80/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z","server":"Kestrel","date":"Fri, 09 Jul 2021 09:23:37 GMT","connection":"close"},"body":{"id":"68a7a8811796416b993924437493ff87","status":0,"status_text":"Building and Deploying '68a7a8811796416b993924437493ff87'.","author_email":"N/A","author":"N/A","deployer":"VSTS_ZIP_DEPLOY","message":"Created via a push deployment","progress":"Running deployment command...","received_time":"2021-07-09T09:01:50.4159225Z","start_time":"2021-07-09T09:01:51.775357Z","end_time":null,"last_success_end_time":null,"complete":false,"active":false,"is_temp":false,"is_readonly":true,"url":null,"log_url":null,"site_name":"XXXXXXXXXXXXe"}}
Deployment status: 0 'Building and Deploying '68a7a8811796416b993924437493ff87'.'. retry after 5 seconds
setting affinity cookie ["ARRAffinity=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;Secure;Domain=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net","ARRAffinitySameSite=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;SameSite=None;Secure;Domain=XXXXXXXXXXXXXXX.scm.azurewebsites.net"]
[GET]https://XXXXXXXXXXX-test.scm.azurewebsites.net:443/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z
POLL URL RESULT: {"statusCode":202,"statusMessage":"Accepted","headers":{"transfer-encoding":"chunked","content-type":"application/json; charset=utf-8","location":"http://XXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net:80/api/deployments/latest?deployer=VSTS_ZIP_DEPLOY&time=2021-07-09_09-01-41Z","server":"Kestrel","date":"Fri, 09 Jul 2021 09:23:45 GMT","connection":"close"},"body":{"id":"68a7a8811796416b993924437493ff87","status":0,"status_text":"Building and Deploying '68a7a8811796416b993924437493ff87'.","author_email":"N/A","author":"N/A","deployer":"VSTS_ZIP_DEPLOY","message":"Created via a push deployment","progress":"Running deployment command...","received_time":"2021-07-09T09:01:50.4159225Z","start_time":"2021-07-09T09:01:51.775357Z","end_time":null,"last_success_end_time":null,"complete":false,"active":false,"is_temp":false,"is_readonly":true,"url":null,"log_url":null,"site_name":"XXXXXXXXXXXX"}}
Deployment status: 0 'Building and Deploying '68a7a8811796416b993924437493ff87'.'. retry after 5 seconds
setting affinity cookie ["ARRAffinity=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;Secure;Domain=XXXXXXXXXXXXXXXXXXXXXX.scm.azurewebsites.net","ARRAffinitySameSite=c06e9bb74f52245b3695b3079a52f6acbc70c3ee812f67e4fa3f5f65088ff4f7;Path=/;HttpOnly;SameSite=None;Secure;Domain=XXXXXXXXXXXXXXXXXX"]
This task fails only in specific slot in myweb app , authors slots and production slot works fine and the job take around 6 mins
Any ideas what could be wrong?
As per the discussion and troubleshooting performed here, I tried to setup a Linux App Service on Standard S1 pricing tier enabling 5 (max) slots with CI/CD configured via Azure Pipelines. Unfortunately, I wasn't able to reproduce the same error as yours despite multiple different trials.
I'd suggest you to try the following:
Kudu Sync failed in the deployment log resembles this open issue from about a year ago: ZipDelpoy on azure web app linux fails during kudu sync #2972. Please check the trace/deployment log files on kudu at https://<appname>.scm.azurewebsites.net/api/vfs/LogFiles/kudu/trace or /deployment or from Kudu's DebugConsole (/LogFiles/kudu/\*) and check if this is caused by deployment lock failures. In that case, check this wiki out for dealing with locked files during deployment.
Try a different deployment method like run from package (to avoid resource locking), using FTP/S, or local git deployment.
This should help you narrow down the issue further, whether it is caused in the App service/deployment method, or the ADO pipeline/task.
Scale up to the next higher tier and re-trigger your pipeline. If it succeeds, you may scale back down to the original tier. This would indirectly restart your SCM sites as well.
If the above workarounds don't help, you could check on the following:
Customize your deploy task with options like TakeAppOfflineFlag, DeploymentType or RenameFilesFlag to streamline your deployment.
Try restarting the app/slot just before the deployment in order to recycle the app pool.
Check if your app is running into any of the prescribed limits (ex: file system storage) for your tier.
Drill down into available metrics for your app to identify any CPU/Memory anomalies.
Try the Diagnose and solve problems tool for any additional insights about your app.
If your environment permits, try setting up and deploying to a new slot within your App Service, or try verifying if this happens to another app in a different region.
I'm having trouble troubleshooting this issue I'm running into when trying to run the sample packages for the framework.
For each infant package I get the following error:
Operation on target Throw Exception - Invalid Infant failed: Operation on target Raise Error failed: Failure happened on 'Source' side. ErrorCode=SqlOperationFailed,'Type=Microsoft.DataTransfer.Common.Shared.HybridDeliveryException,Message=A database operation failed with the following error: 'Worker pipeline [Wait 3] is not valid in target Orchestrator [adf-procfwk-rg-procfwk]',Source=,''Type=System.Data.SqlClient.SqlException,Message=Worker pipeline [Wait 3] is not valid in target Orchestrator [adf-procfwk-rg-procfwk],Source=.Net SqlClient Data Provider,SqlErrorNumber=50000,Class=16,ErrorCode=-2146232060,State=1,Errors=[{Class=16,Number=50000,State=1,Message=Worker pipeline [Wait 3] is not valid in target Orchestrator [adf-procfwk-rg-procfwk],},],'
Looks like the "Validate Pipeline" function is returning some peculiar data. I'm not much of a C# / App programmer so I'm struggling to determine what function is being called and how the database is returning the values in 'untitled4.png'
The pipelines metadata looks correct and the pipeline exists in the ADF so I'm a bit perplexed by the error. If anyone has any insights I'd greatly appreciate it.
Have you published the 'Wait 3' pipeline?
The screen shot shows your running in Debug in Git connected mode.
The framework can only trigger and interact with published Worker pipelines in the target Data Factory/Synapse instance.
Thanks
I have developed the Azure Function App One year back and published successfully and it's running. Now I have made a changes on functionality which is build succeed.
When I am publishing it's throwing error unable to publish.
I went through the path which showing screen shot showing below error.
14-09-2020 10.49.31 PM
System.AggregateException: One or more errors occurred. --->
System.Exception: Publish has encountered an error. We were unable to
determine the cause of the error. Check the output log for more
details. --- End of inner exception stack trace ---
---> (Inner Exception #0) System.Exception: Publish has encountered an error. We were unable to determine the cause of the error. Check the
output log for more details. <---
System.Exception: Publish has encountered an error. We were unable to
determine the cause of the error. Check the output log for more
details.
===================
I am unable to figure out the error here. Any one help on this?
I think you shouldn't stop because of this. And for the error you give us we can not get anything useful from it.
You just need to use another way to deploy. And it will give you the details error if it has any error.(Although I say it is another way, in fact they are essentially the same. VS Studio is actually performing packaging and running commands when you try to publish the azure function.)
Just do the following steps:
1, create the function app on azure.
2, build your function app on local.
3, Go to the compiled folder on local, and zip the content:
4, install the Azure CLI, then run az login.
5, cd to the compiled folder, then run below command:
az webapp deployment source config-zip --resource-group yourresourcegroupnameonazure --name yourfunctionappnameonazure --src xxx.zip
After above steps, everything will be ok. This is the offcial doc:
https://learn.microsoft.com/en-us/azure/app-service/deploy-zip
I've seen similar errors in two cases: locked files and previously deployment executed by DevOps pipelines. I don't know if it applies in your case. In both cases, you have to modify your Azure Functions settings on portal:
case 1 : add MSDEPLOY_RENAME_LOCKED_FILES = 1
case 2 : remove WEBSITE_RUN_FROM_PACKAGE = 1
I deleted my KF cluster last night to create a new one (using kubectl cluster command not Kfctl delete), and then when I tied to create a new one, it fails, it does not work with CLI not Console. I found other people have run into this issue before, for example (here and here)
"However, as I said even with CLI my deployment fails, the error from console is:
ailed to apply: (kubeflow.error): Code 500 with message: coordinator Apply failed for gcp: (kubeflow.error): Code 500 with message: gcp apply could not update deployment manager Error could not update storage-kubeflow.yaml; Insert deployment error: googleapi: Error 403: Request had insufficient authentication scopes.
More details:
Reason: insufficientPermissions, Message: Insufficient Permission"
and the error I get from Console is:
"Please enable APIs for your project and try again
Please enable cloud resource manager API: https://console.developers.google.com/apis/api/cloudresourcemanager.googleapis.com/ and iam API: https://console.developers.google.com/apis/api/iam.googleapis.com/"
Note that this error is wrong, all the apis are active already. I'm quite sure this is a bug of KF but not sure how to find a workaround, any thoughts?
With CLI, I'm using my own account which has "owner" privileges.
Thanks
It seems you have an issue with IAM and the installation of Kubeflow, a 3rd party product that itself is not supported by us; nevertheless I went ahead and dig some information about this Machine Learning product.
The main issues (and although it seems you already cover permissions) are permissions, number of projects and some fine grained points.
I was checking and found out the following things that may help
a) Troubleshooting Kubeflow 1
b) Deploying Kubeflow in GKE[2]
c) Kubleflow auto deployer for GKE[3]
There are also some discussion about a mismatch permissions setting in Kubeflow that may be worth reading [4]
Finally there is a group that, also on a best-effort basis due the nature of Kubeflow:"google-kubeflow-support#google.com" that may come in handy.
I trust this information will be useful for you to solve your issue