VSTS Release Agent 'Failed' to deploy to IIS Site - azure-devops

I am in the process of enabling CI/CD in our NP Environments. I have recently come across an issue where I am deploying to our staging environments which consists of two servers.
The first server deploys no problem, but the second server fails (Or so it says). Upon looking at the IIS Site directory the files do actually get released, but the log would indicated otherwise:
2019-04-07T21:07:22.1153713Z Total changes: 231 (229 added, 0 deleted, 2 updated, 0 parameters changed, 53453549 bytes copied)
2019-04-07T21:07:22.1309716Z ##[debug]rc:0
2019-04-07T21:07:22.1309716Z ##[debug]rc:0
2019-04-07T21:07:22.1309716Z ##[debug]success:false
2019-04-07T21:07:22.1309716Z ##[debug]success:false
2019-04-07T21:07:22.1465719Z Error: C:\azagent\A1\_work\_tasks\IISWebAppDeploymentOnMachineGroup_1b467810-6725-4b6d-accd-886174c09bba\0.0.51\node_modules\webdeployment-common\MSDeploy3.6\msdeploy.exe failed with return code: 0
2019-04-07T21:07:22.1465719Z at ChildProcess.<anonymous> (C:\azagent\A1\_work\_tasks\IISWebAppDeploymentOnMachineGroup_1b467810-6725-4b6d-accd-886174c09bba\0.0.51\node_modules\vsts-task-lib\toolrunner.js:569:30)
2019-04-07T21:07:22.1465719Z at emitTwo (events.js:106:13)
2019-04-07T21:07:22.1465719Z at ChildProcess.emit (events.js:191:7)
2019-04-07T21:07:22.1465719Z at maybeClose (internal/child_process.js:886:16)
2019-04-07T21:07:22.1465719Z at Process.ChildProcess._handle.onexit (internal/child_process.js:226:5)
2019-04-07T21:07:22.1465719Z Retrying to deploy the package.
I've tried a couple of things:
Making sure the user the release service is running under has the correct permissions to manage that directory.
I have reinstalled the Release Agent.
Not sure what else to try or check. Any suggestions?

So I have discovered what the issue was. Funnily enough it appears the issue has stemmed from the environment variable 'COR_PROFILING_ENABLED' being set to enabled '0x01'. It looks like for whatever reason Dynatrace is configured incorrectly and so a Dynatrace specific error is being thrown during the release process.
Since disabling disabling the release successfully completes.

Related

Undefined method [] for nil when trying to run cap deploy:restart

I have a Rails 5.2 app, and with cap 3.4.1 we are suddenly getting this weird error:
[b35efe76] Phusion Passenger(R) 6.0.8
DEBUG [b35efe76] Finished in 0.305 seconds with exit status 0 (successful).
(Backtrace restricted to imported tasks)
cap aborted!
SSHKit::Runner::ExecuteError: Exception while executing as deploy#host.com: undefined method `[]' for nil:NilClass
Trying to narrow it down, it happens when attempting to restart, because this is the line that fails:
cap production deploy:restart
The question is, how do I go about finding what file is trying to call [] on a nil value on? Running cap with --trace is of no value because it just gives me internal errors - nothing in my code. Basically, how do I find out what is nil?
One more clue, currently, I have a bunch of servers, if I run the cap restart command on server A, it restarts fine, on server B, it throws this error, so I'm guessing there's an environment variable on server A but not B, but the error is just so opaque I don't know where to begin.
Thanks for any help,
kevin
A wild guess: I had a similar problem and I could solve it by upgrading capistrano-passenger to >= 0.2.1
Looks like the version change of passenger from 6.0.7 to 6.0.8 has introduces a problem. I see you are also on 6.0.8, so it might affect you, too!
Link to the capistrano-passenger issue

AWS CloudFormation Stack Upgrade fails with "Unable to satisfy 100% MinSuccessfulInstancesPercent requirement"

I tried to update an existing jenkins stack from https://github.com/widdix/aws-cf-templates and just modified the jenkins rpm and the AWS AMI Versions.
But on updating the stack I get this error message:
2019-12-17 19:14:16 UTC+0100 vpc-ci-jenkins UPDATE_ROLLBACK_IN_PROGRESS The following resource(s) failed to update: [MasterASG].
2019-12-17 19:14:15 UTC+0100 MasterASG UPDATE_FAILED Received 1 FAILURE signal(s) out of 1. Unable to satisfy 100% MinSuccessfulInstancesPercent requirement
Anyone an idea what might gone wrong with the update?
There were no changes in the MasterASG section of the template.

IBM BLUEMIX BLOCKCHAIN SDK-DEMO failing

I have been working with HFC SDK for Node.js and it used to work, but since last night I am having some problems.
When running helloblockchain.js only few times works, most time I get this error when it tries to enroll a new user:
E0113 11:56:05.983919636 5288 handshake.c:128] Security handshake failed: {"created":"#1484304965.983872199","description":"Handshake read failed","file":"../src/core/lib/security/transport/handshake.c","file_line":237,"referenced_errors":[{"created":"#1484304965.983866102","description":"FD shutdown","file":"../src/core/lib/iomgr/ev_epoll_linux.c","file_line":948}]}
Error: Failed to register and enroll JohnDoe: Error
Other times, the enroll works and the failure appears deploying the chaincode:
Enrolled and registered JohnDoe successfully
Deploying chaincode ...
E0113 12:14:27.341527043 5455 handshake.c:128] Security handshake failed: {"created":"#1484306067.341430168","description":"Handshake read failed","file":"../src/core/lib/security/transport/handshake.c","file_line":237,"referenced_errors":[{"created":"#1484306067.341421859","description":"FD shutdown","file":"../src/core/lib/iomgr/ev_epoll_linux.c","file_line":948}]}
Failed to deploy chaincode: request={"fcn":"init","args":["a","100","b","200"],"chaincodePath":"chaincode","certificatePath":"/certs/peer/cert.pem"}, error={"error":{"code":14,"metadata":{"_internal_repr":{}}},"msg":"Error"}
Or:
Enrolled and registered JohnDoe successfully
Deploying chaincode ...
E0113 12:15:27.448867739 5483 handshake.c:128] Security handshake failed: {"created":"#1484306127.448692244","description":"Handshake read failed","file":"../src/core/lib/security/transport/handshake.c","file_line":237,"referenced_errors":[{"created":"#1484306127.448668047","description":"FD shutdown","file":"../src/core/lib/iomgr/ev_epoll_linux.c","file_line":948}]}
events.js:160
throw er; // Unhandled 'error' event
^
Error
at ClientDuplexStream._emitStatusIfDone (/usr/lib/node_modules/hfc/node_modules/grpc/src/node/src/client.js:189:19)
at ClientDuplexStream._readsDone (/usr/lib/node_modules/hfc/node_modules/grpc/src/node/src/client.js:158:8)
at readCallback (/usr/lib/node_modules/hfc/node_modules/grpc/src/node/src/client.js:217:12)
E0113 12:15:27.563487641 5483 handshake.c:128] Security handshake failed: {"created":"#1484306127.563437122","description":"Handshake read failed","file":"../src/core/lib/security/transport/handshake.c","file_line":237,"referenced_errors":[{"created":"#1484306127.563429661","description":"FD shutdown","file":"../src/core/lib/iomgr/ev_epoll_linux.c","file_line":948}]}
This code worked yesterday, so I don't know what could be happening.
Does anybody know how can I fix it?
Thanks,
Javier.
ibm-bluemix
blockchain
These types of intermittent issues are usually related to GRPC. An initial suggestion is to ensure that you are using at least GRPC version 1.0.0.
If you are using a Mac, then the maximum number of open file descriptors should be checked (using ulimit -n). Sometimes this is initially set to a low value such as 256, so increasing the value could help.
There are a couple of GRPC issues with similar symptoms.
https://github.com/grpc/grpc/issues/8732
https://github.com/grpc/grpc/issues/8839
https://github.com/grpc/grpc/issues/8382
There is a grpc.initial_reconnect_backoff_ms property that is mentioned in some of these issues. Increasing the value past the 1000 ms level might help reduce the frequency of issues. Below are instructions for how the helloblockchain.js file can be modified to set this property to a higher value.
Open the helloblockchain.js file in the Hyperledger Fabric Client example and find the enrollAndRegisterUsers function.
Add “grpc.initial_reconnect_backoff_ms": 5000 to the setMemberServicesUrl call.
chain.setMemberServicesUrl(ca_url, {
pem: cert, "grpc.initial_reconnect_backoff_ms": 5000
});
Add “grpc.initial_reconnect_backoff_ms": 5000 to the addPeer call.
chain.addPeer("grpcs://" + peers[i].discovery_host + ":" + peers[i].discovery_port,
{pem: cert, "grpc.initial_reconnect_backoff_ms": 5000
});
Note that setting the grpc.initial_reconnect_backoff_ms property may reduce the frequency of issues, but it will not necessarily eliminate all issues.
The connection to the eventhub that is made in the helloblockchain.js file can also be a factor. There is an earlier version of the Hyperledger Fabric Client that does not utilize the eventhub. This earlier version could be tried to determine if this makes a difference. After running git clone https://github.com/IBM-Blockchain/SDK-Demo.git, run git checkout b7d5195 to use this prior level. Before running node helloblockchain.js from a Node.js command window, the git status command can be used to check the code level that is being used.

msdeploy - stop deploy in postsync if presync fails

I am using msdeploy -presync to backup the current deployment of a website in IIS before the -postsync deploys it, however I recently had a situation where the -presync failed (raised a warning due to a missing dll) and the -postsync continued and overwrote the code.
Both the presync and postsync run batch files.
Obviously this is bad as the backup failed so there is no backout route if the deployment has bugs or fails.
Is there anyway to stop the postsync if the presync raises warnings with msdeploy?
Perhaps the issue here is that the presync failure was raised as a warning not an error.
Supply successReturnCodes parameter set to 0 (success return code convention) to presync option such as:
-preSync:runCommand="your script",successReturnCodes=0
More info at: http://technet.microsoft.com/en-us/library/ee619740(v=ws.10).aspx

ClickOnce: DeploymentDownloadException: The operation has timed out

Symptom: ClickOnce installation starts and stops after around 600 kB (out of 2 MB).
Progress bar always stops at the same value (tried ten times).
Error log says that The operation has timed out (in inner exception) and fails with "DeploymentDownloadException (Unknown subtype)".
Error log details (irrelevant information trimmed):
ERROR DETAILS
Following errors were detected during this operation.
System.Deployment.Application.DeploymentDownloadException (Unknown subtype)
- Downloading http://fullpath/name.dll.deploy did not succeed.
- Source: System.Deployment
- Stack trace: at System.Deployment.Application.SystemNetDownloader.DownloadSingleFile(Downloa
dQueueItem next)
at
System.Deployment.Application.SystemNetDownloader.DownloadAllFiles()
at
System.Deployment.Application.FileDownloader.Download(SubscriptionState
subState)
--- Inner Exception ---
System.Net.WebException
- The operation has timed out.
- Source: System
- Stack trace:
at System.Net.ConnectStream.Read(Byte[] buffer,
Int32 offset, Int32 size)
at
System.Deployment.Application.SystemNetDownloader.DownloadSingleFile(Downloa
dQueueItem next)
This only happens for two customers. The install works OK for thousands of others. I have found numerous posts via google with no answer or generic "firewall is the issue" or "customer was using dialup".
Has anyone solved this? Is this a ClickOnce bug?
Disabling firewall software on the machine did not help because a hardware firewall installed on the network was the cause (FortiGate 30B).
I doubt that it's a bug. However, it seems like it gets stuck at one file in the deployment path. Maybe it is a type of file that is blocked by a firewall.
I would just remove all files but one from the build and see if that gets downloaded ok, and then add the rest of the files one by one (or maybe type by type) and see at what file ClickOnce gets stuck downloading.
If that doesn't seem to do anything, I'd build a dummy app and deploy it with ClickOnce and see if it installs at all on the customer's box.