I have been trying to add an environment resource to our pipeline in azure devops but every time i install it with the powershell script they provide it asks me for tags. If i don't want to input tags it errors. If i choose to put in tags it errors. Either way it starts over every time and it will never successfully install. Anyone know if this is something on my end or on their end? I looked through the diagnostic file and the exceptions it is logging look like it's something that i can't fix but i have been having issues with this for over a week and thinking it was something that Microsoft would realize and patch. Now i'm wondering if there is something else i can do?
[2020-05-22 13:01:48Z ERR VisualStudioServices] POST request to https://DEVOPS_URL/24cca667-60da-4ba2-a323-4e05c46f3309/_apis/pipelines/environments/3/providers/virtualmachines failed. HTTP Status: InternalServerError, AFD Ref: Ref A: 56C7161B437D41698EBBDE7ACBF4CAA2 Ref B: ATAEDGE0918 Ref C: 2020-05-22T13:01:48Z
[2020-05-22 13:01:48Z INFO CommandSettings] Flag 'unattended': 'False'
[2020-05-22 13:01:48Z ERR Terminal] WRITE ERROR (exception):
[2020-05-22 13:01:48Z ERR Terminal] Microsoft.VisualStudio.Services.WebApi.VssServiceResponseException: TF400898: An Internal Error Occurred. Activity Id: acb8a36a-e602-4988-b3f7-8fbeecd729e0.
---> System.NullReferenceException: TF400898: An Internal Error Occurred. Activity Id: acb8a36a-e602-4988-b3f7-8fbeecd729e0.
--- End of inner exception stack trace ---
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable1 queryParameters, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.EnvironmentVMResourceConfigProvider.AddAgentAsync(AgentSettings agentSettings, TaskAgent agent, CommandSettings command)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command)
I was experiencing a similar error "Failed to add virtual machine resource. Linked environment pool is null."
Answer from Kevin Ross here: https://developercommunity.visualstudio.com/t/addition-of-resource-to-environment-fails-for-user/1048111
I resolved my error by following the below steps:
Get the deployment pool ID from the environment URL
Find the deployment pool settings in the organization settings and navigate to the correct pool based on ID from step 1
From the deployment pool settings, go to security and add the required user to administrator role or whatever role is required
Related
I am trying to register the agent in server to create Deployment Groups for Azure DevOps Release pipeline using the provided PowerShell script(Azure DevOps Portal). Script is able to download the azagent, but throwing the below error while configuring the agent:
Error:
Microsoft.TeamFoundation.DistributedTask.WebApi.TaskAgentPoolNotFoundException: No agent pool found with identifier 13.
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.HandleResponseAsync(HttpResponseMessage response, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync(HttpRequestMessage message, HttpCompletionOption completionOption, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpRequestMessage message, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.WebApi.VssHttpClientBase.SendAsync[T](HttpMethod method, IEnumerable1 additionalHeaders, Guid locationId, Object routeValues, ApiResourceVersion version, HttpContent content, IEnumerable1 queryParameters, Object userState, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.DeploymentGroupAgentConfigProvider.AddAgentAsync(AgentSettings agentSettings, TaskAgent agent, CommandSettings command)
at Microsoft.VisualStudio.Services.Agent.Listener.Configuration.ConfigurationManager.ConfigureAsync(CommandSettings command)
Please refer to this doc:
You're a member of a project and you want to use a set of machines owned by your team for running build and deployment jobs. First, make sure you're a member of a group in All Pools with the Administrator role.
Based on the error message, it seems that the agent pool couldn't be found. So you could check your permission for the target Deployment Group pool in Collection Settings -> Deployment Pools-> Security.
You could grant the Administrator permission for your account. Then you could check if it could work.
Here is a ticket with the similar issue, you could refer to it.
We have this issue that has been plaguing our engineering group for over a year and we cannot make any progress on the resolution. This question on Stack Overflow is eerily similar to what we experience.
We have several pipelines that use the "Visual Studio Test" task to run a set of NUnit tests from a single assembly. There are 1400-1500 unit tests in the project and it takes The NUnit Test Adapter is deployed via the NuGet task (install NUnit3TestAdapter). This test assembly is built with the following settings:
MSBuild Architecture=x86
Platform=AnyCPU
Configuration=Release
Strong Name Signed
Across multiple pipelines, this step randomly yet consistently ends early with zero error message or trace of why the process ended. Running a subsequent build of the pull request will often result in a success but occasionally another failure. In this case we have coined this a "flaky build" where given the same exact code inputs, the build can fail and then subsequently pass.
This is the only error message we received from the pipeline:
##[error]Exit code 1 returned from process: file name 'REDACTED\externals\node\bin\node.exe', arguments '"REDACTED_work_tasks\VSTest_ef087383-ee5e-42c7-9a53-ab56c98420f9\2.170.1\runvstest.js"'.
Finishing: Integration Tests
These tests are our "integration tests" that perform many setup steps to create a functional version of our system including creating a web server via Kestrel to host our API, connect to (localdb) running on the build server to restore a database .bak file and apply a database project (dacpac) as well as connect to Redis (also installed locally on the build server).
We are struggling to determine next steps so I am reaching out to the community to see if anyone else has suffered through this issue, has ideas for how to debug the issue or perhaps even a solution!
edit on 2020-11-10
Yesterday I set the pipeline variable system.debug to true and this is the output it captured during one of our flaky builds.
##[error\]Exit code 1 returned from process: file name 'E:\\servername\\externals\\node\\bin\\node.exe', arguments '"E:\\servername\\\_work\\\_tasks\\VSTest\_ef087383-ee5e-42c7-9a53-ab56c98420f9\\2.170.1\\runvstest.js"'.
##[debug\]Microsoft.VisualStudio.Services.Agent.Util.ProcessExitCodeException: Exit code 1 returned from process: file name 'E:\\servername\\externals\\node\\bin\\node.exe', arguments '"E:\\servername\\\_work\\\_tasks\\VSTest\_ef087383-ee5e-42c7-9a53-ab56c98420f9\\2.170.1\\runvstest.js"'.
at Microsoft.VisualStudio.Services.Agent.Util.ProcessInvoker.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary\`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue\`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.ProcessInvokerWrapper.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary\`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, InputQueue\`1 redirectStandardIn, Boolean inheritConsoleHandler, Boolean keepStandardInOpen, Boolean highPriorityProcess, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.DefaultStepHost.ExecuteAsync(String workingDirectory, String fileName, String arguments, IDictionary\`2 environment, Boolean requireExitCodeZero, Encoding outputEncoding, Boolean killProcessOnCancel, Boolean inheritConsoleHandler, CancellationToken cancellationToken)
at Microsoft.VisualStudio.Services.Agent.Worker.Handlers.NodeHandler.RunAsync()
at Microsoft.VisualStudio.Services.Agent.Worker.TaskRunner.RunAsync()
at Microsoft.VisualStudio.Services.Agent.Worker.StepsRunner.RunStepAsync(IStep step, CancellationToken jobCancellationToken)
Lately I have been facing this error for one of my release definition and I am not able to figure it out why is this happening.
I tried manually deleting the folders inside the r1 directory but it fails to delete the 'a' folder i.e. artifact directory and says the folder or file in it is open in another program. But its shocking that the 'a' folder is empty and don't even have a hidden files and folders in it.
As a work around, to make the release successfully get deployed I have to restart my server each time whenever I want a new release to be deployed. Which is really very annoying.
Below is the complete error that I get on the Release definition summary.
Errors (2)
Downloading artifacts failed:
Microsoft.VisualStudio.Services.Agent.Worker.Release.Artifacts.ArtifactDirectoryCreationFailedException:
Failed to create Release artifact directory 'C:\Agents\DA_CID22\r1\a'.
---> System.IO.IOException: The process cannot access the file '\?\C:\Agents\DA_CID22\r1\a' because it is being used by another
process. at System.IO.Win32FileSystem.RemoveDirectoryHelper(String
fullPath, Boolean recursive, Boolean throwOnTopLevelDirectoryNotFound)
at System.IO.Win32FileSystem.RemoveDirectory(String fullPath, Boolean
recursive) at
Microsoft.VisualStudio.Services.Agent.Util.IOUtil.DeleteDirectory(String
path, Boolean contentsOnly, Boolean continueOnContentDeleteError,
CancellationToken cancellationToken) at
Microsoft.VisualStudio.Services.Agent.Worker.Release.ReleaseFileSystemManager.EnsureEmptyDirectory(String
directoryPath, CancellationToken cancellationToken) --- End of inner
exception stack trace --- at
Microsoft.VisualStudio.Services.Agent.Worker.Release.ReleaseFileSystemManager.EnsureEmptyDirectory(String
directoryPath, CancellationToken cancellationToken) at
Microsoft.VisualStudio.Services.Agent.Worker.Release.RetryExecutor.Execute(Action
action) at
Microsoft.VisualStudio.Services.Agent.Worker.Release.ReleaseJobExtension.CreateArtifactsFolder(IExecutionContext
executionContext, String artifactsWorkingFolder) at
Microsoft.VisualStudio.Services.Agent.Worker.Release.ReleaseJobExtension.d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown --- at
System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw() at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task) at
Microsoft.VisualStudio.Services.Agent.Worker.Release.ReleaseJobExtension.d__31.MoveNext()
Failed to create Release artifact directory
'C:\Agents\DA_CID22\r1\a'.
Any help or advice would be really helpful.
This happens from time to time...I spent days trying to figure out with no easy solution (all the ones you have tried, I tried myself) but for restarting the private agent. My conclusion is that somehow the agent process got hung with the artifact Windows folder. You can see that with programs like https://lockhunter.com/. There's not an easy solution here...Sometimes you must kill the culprit process which not ended correctly freeing that file, or if you are lucky releasing the file handle to that file with a tool like said before...
I am trying to whitelist URLs for access required by Azure DevOps (VSTS Agent).
However, when trying to do a self-update, I receive the following error (file is /vsts-agent/_diag/Agent_20181127-213126-utc.log):
[2018-11-27 21:37:49Z ERR Terminal] WRITE ERROR: An error occurred: Response status code does not indicate success: 403 (Forbidden).
[2018-11-27 21:37:49Z ERR AgentProcess] System.Net.Http.HttpRequestException: Response status code does not indicate success: 403 (Forbidden).
at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
at System.Net.Http.HttpClient.FinishGetStreamAsync(Task`1 getTask)
at Microsoft.VisualStudio.Services.Agent.Listener.SelfUpdater.DownloadLatestAgent(CancellationToken token)
at Microsoft.VisualStudio.Services.Agent.Listener.SelfUpdater.SelfUpdate(AgentRefreshMessage updateMessage, IJobDispatcher jobDispatcher, Boolean restartInteractiveAgent, CancellationToken token)
at Microsoft.VisualStudio.Services.Agent.Listener.Agent.RunAsync(AgentSettings settings)
at Microsoft.VisualStudio.Services.Agent.Listener.Agent.RunAsync(AgentSettings settings)
at Microsoft.VisualStudio.Services.Agent.Listener.Agent.RunAsync(AgentSettings settings)
at Microsoft.VisualStudio.Services.Agent.Listener.Agent.ExecuteCommand(CommandSettings command)
at Microsoft.VisualStudio.Services.Agent.Listener.Program.MainAsync(IHostContext context, String[] args)
However, there is no URL listed to whitelist!
The closest I found was:
Configuring Proxy with VSTS Agents
Whitelisted URLs for VSTS
Note the agent is configured successfully and can successfully run builds. It is just every time an update comes around, it fails and won't run builds until it is removed/reconfigured (not an option - this is an agent, not interactive).
The pipeline agent is open source. The code can be found here.
Glancing through the trace, I'd guess it is throwing the error here:
using (Stream result = await httpClient.GetStreamAsync(_targetPackage.DownloadUrl))
Looking through the assets.json specifying the DownloadUrl, I'd guess this is the URL you will want:
https://vstsagentpackage.azureedge.net/agent/<AGENT_VERSION>/vsts-agent-win-x64-<AGENT_VERSION>.zip
I have a problem with a running build machine, where the agent suddenly does not want to start. It's been a part of a remote controller and for trouble shooting this issue, if started a local controller. The symptoms are, at the agent(s) initializes correct (says 'Ready'), but has the stopped icon and in the status area says 'BuildController has not been started in 1 minutes. The AD account running the build service works on another build machine (seperate controller + build agents). I've tried the following
Reinstall the build service
running with machine name, fully qualified domain name and IP address for endpoint address
un- and re-registered build service
rebooted
cleaned up build agent registrations with script
If I change the service account running the build service to my own AD account, it works. However, running under our dedicated build user failes on this particular machine, but not the other. Any suggestions what to do? Here's the error from the event log:
Service 'Default Agent - tfs2010build1' had an exception:
Exception Message: There was no endpoint listening at http://tfs2010build1:9191/Build/v3.0/Services/Controller/31 that could accept the message. This is often caused by an incorrect address or SOAP action. See InnerException, if present, for more details. (type EndpointNotFoundException)
Exception Stack Trace:
Server stack trace:
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
at System.ServiceModel.Channels.HttpOutput.Send(TimeSpan timeout)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelRequest.SendRequest(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.RequestChannel.Request(Message message, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannel.Call(String action, Boolean oneway, ProxyOperationRuntime operation, Object[] ins, Object[] outs, TimeSpan timeout)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at Microsoft.TeamFoundation.Build.Machine.IBuildControllerService.TestConnectionFromController(String agentUri)
at Microsoft.TeamFoundation.Build.Machine.ServiceProxies.ServiceProxy`1.<>c__DisplayClass3.<Do>b__2(T channel)
at Microsoft.TeamFoundation.Build.Machine.ServiceProxies.ServiceProxy`1.Do[TResult](Func`2 action)
at Microsoft.TeamFoundation.Build.Machine.BuildAgentService.<>c__DisplayClass12.<TestConnection>b__11(Object )
Inner Exception Details:
Exception Message: Unable to connect to the remote server (type WebException)
Exception Stack Trace: at System.Net.HttpWebRequest.GetRequestStream(TransportContext& context)
at System.Net.HttpWebRequest.GetRequestStream()
at System.ServiceModel.Channels.HttpOutput.WebRequestHttpOutput.GetOutputStream()
Inner Exception Details:
Exception Message: No connection could be made because the target machine actively refused it 127.0.0.1:38742 (type SocketException)
Exception Stack Trace: at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket& socket, IPAddress& address, ConnectSocketState state, IAsyncResult asyncResult, Int32 timeout, Exception& exception)
Found the problem. Proxy server was enabled under IE options. Not sure why the build service worked under my AD user account and not the dedicated build user, but it solved the problem.
---->>>>>Update!
So we have 2 machines (B1 & B2), each with 2 agents. B1 had the initial problem and was solved by disabling the proxy settings under IE. Yesterday B2 suddenly started showing the same symptons and error messages on the 2 agents. Proxy setting is NOT enabled. While it did fix B1, it's not the universal solution for this particular problem.
It's hard work keeping these build agents running :( - Miss TeamCity...
---->>>>Update again!
So yesterday when I looked at the proxy configuration, it wasn't set. However this morning the checkbox was checked. Disabled the proxy and the agents went online. Very strange behavior! Wonder if Windows Update changes these settings...
I often get this same issue with the proxy when I am forced to manually stop a build. I have not been able to find any decent resolutions for this.