I have some code that uses appfabric and was getting server unavailability errors. Following the instructions at http://msdn.microsoft.com/en-us/library/ff921031.aspx I found that my cache cluster was down. I opened up powershell as an administrator and ran Start-CacheCluster. After a few minutes I receive an error message:
Start-CacheCluster : Could not start cluster: ErrorCode<ERRCAdmin025>:SubStatus
<ES0001>:Time-out occurred in starting the cluster.
At line:1 char:19
+ Start-CacheCluster <<<<
+ CategoryInfo : NotSpecified: (:) [Start-CacheCluster], DataCach
eException
+ FullyQualifiedErrorId : ERRCAdmin025,Microsoft.ApplicationServer.Caching
.Commands.StartCacheClusterCommand
Checking the event logs reveals this:
Service cannot be started. System.TypeInitializationException: The type initializer for 'Microsoft.ApplicationServer.Caching.ConfigManager' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.Fabric.Common.ConsoleSink' threw an exception. ---> System.Configuration.ConfigurationErrorsException: Configuration system failed to initialize ---> System.Configuration.ConfigurationErrorsException: An error occurred loading a configuration file: Access to the path 'C:\Windows\System32\AppFabric\DistributedCacheService.exe.config' is denied. (C:\Windows\System32\AppFabric\DistributedCacheService.exe.config) ---> System.UnauthorizedAccessException: Access to the path 'C:\Windows\System32\AppFabric\DistributedCacheService.exe.config' is denied.
at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
at System.IO.FileStream.Init(String path, FileMode mode, FileAccess access, Int32 rights, Boolean useRights, FileShare share, Int32 bufferSize, FileOptions options, SECURITY_ATTRIBU...
Has anyone with more AppFabric experience seen this before?
-Thanks
Try Microsoft, Windows, Application Server-System Services and Admin log. Sometimes this log will give you some more info that is not present in the regular eventlog. For more information about cache cluster monitoring see Health Monitoring Tools.
Regarding your issue. The only think I would say is that after starting the cache cluster it loads configuration and because it fails, it will eventually crash the AppFabric Caching Service (you will see that information in the eventlog mentioned above).
Make sure you have AppFabric Caching Service configured to run under an account that has access to the configuration file DistributedCacheService.exe.config as well as to the configuration repository (Velocity configuration DB or configuration XML).
You need to run the "Caching administration Windows PowerShell" as Administrator.
Have you checked that the AppFabricCachingService logon has read access to DistributedCacheService.exe.config?
In my case, I am using an XML provider and have a config share named "CacheConfig". My server is not joined to a domain so I have configured a local "AppFabric" user for the cache host. I am using PowerShell to script the config:
Register-CacheHost -Provider XML -ConnectionString "\\MY-HOST\CacheConfig" -Account "AppFabric" -CachePort 22233 -ClusterPort 22234 -ArbitrationPort 22235 -ReplicationPort 22236 -HostName MY-HOST
Add-CacheHost -Provider XML -ConnectionString "\\MY-HOST\CacheConfig" -Account "AppFabric"
#...
Start-CacheCluster
#...
When I ran the script the Start-CacheCluster statement failed with the same access denied error as in your question. I then checked the AppFabricCachingService logon and found it was set to NETWORK SERVICE. When I checked the file permissions on C:\Windows\System32\AppFabric\DistributedCacheService.exe.config, I found that there was a read permission for the AppFabric user, but no permissions for NETWORK SERVICE. This indicates to me that the Register-CacheHost or Add-CacheHost applet assumes that the cache host account and the service account will be the same.
In my case I added a read permission to the file for NETWORK SERVICE and that fixed the problem.
The following isn't your exact problem, but may help: http://social.msdn.microsoft.com/Forums/en-US/velocity/thread/4fd844f6-3530-4115-8982-d7562e699627/#6bf2825a-cd1d-4659-b6ce-375a0fb0ab38
I also ran into the problem since my AppFabric server suffered something unknown. I spent much time for finding the solution. Finally the only way I tried successfully is to uninstall / re-install the AppFabric. And configure the settings once again. Hope this helps !
I had same issue cause I installed appfabric cache service more than one times. Each times hostId was changed it must be same hostId with ClusterConfig.xml and DistributedCacheService.exe.config file
Related
I want to run a deployment groups on my virtual machine.
I have run my registration script (PowerShell), but it shows the error:
Exception calling "DownloadFile" with "2" argument(s): "The remote name could not be resolved:
'vstsagentpackage.azureedge.net'"
At line:1 char:1303
+ ... nalString, $True);}; $WebClient.DownloadFile($Uri, $agentZip);Add-Typ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [], ParentContainsErrorRecordException
+ FullyQualifiedErrorId : WebException
I found in Microsoft document that I need to pass --proxyurl, --proxyusername and --proxypassword in the script.
./config.cmd --proxyurl http://127.0.0.1:8888 --proxyusername "myuser" --proxypassword "mypass"
But I don't know what value to fill in, especially the URL.
I tried to copy this script and put it in my registration script, but the same error still occurs.
Any help would be appreciated.
I think if you are using this on your office network or wherever the internet connection requires proxy, and that may have proxy settings, then you can get the proxy url from the Internet proxy settings. Creds has to be supplied if it uses a proxy username/pwd.
The below info may not be ideal for everyone, so don't blame me for this solution but this is what worked for me and for my need.
Note: THIS HAS TO HAVE ATTENTION AND UTMOST CARE IF YOU'RE RUNNING ON A SERVER
you may copy the ps scripts to a notepad,
do the steps manually on the designated server.
Download the agent zip file from web
extract it
copy the contents to the A# folder that you want to setup, mostly A1
run the config.cmd line from the ps script which would expect the appropriate creds that you need to make sure to supply as token
If you have appropriate permissions, it will go forward and install the agent on the respective server and prompt you for further steps like tags etc.
UPDATE:
I didn't have enough permissions to create the Deployment group but rather, just a deployment item (VM)
So I created the Deployment group manually from the deployment target. I had issues when creating it but going backwards by creating a group from the target was successful for me.
I have an 'Azure File Copy' task in my Build Definition for Team Services (was Visual Studio Online). Whenever I run this I get an 'Access Denied' error. I've been trying to troubleshoot this for hours with no luck.
Copy started for machine: 'MACHINE NAME'
Copy status for machine 'MACHINE NAME' : 'Failed'
Connecting to remote server [IP ADDRESS] failed with the following error message : Access is denied. For more information, see the about_Remote_Troubleshooting Help topic.To fix WinRM connection related issues, select the 'Enable Copy Prerequisites' option in the task. If set already, and the target Virtual Machines are backed by a Load balancer, ensure Inbound NAT rules are configured for target port (5986). Applicable only for ARM VMs. For more info please refer to http://aka.ms/azurefilecopyreadme
******************************************************************************
Finishing task: AzureFileCopy
******************************************************************************
Task AzureFileCopy failed. This caused the job to fail. Look at the logs for the task for more details.
******************************************************************************
Finishing Build
******************************************************************************
I've tried the following:
'Enable Copy Prerequisites' checkbox on build task
Assigning the user account directly to Microsoft.Powershell with 'Full Control' permissions using PSSession -ShowSecurityDescriptorUI
Added Computer to the TrustedHosts List
Made sure the necessary ports are unblocked (inbound and outbound)
Considering that this is a Microsoft SaaS product, connecting to a standard Azure VM using nothing but Microsoft Azure architecture, I'm surprised this is so difficult. I must be missing something simple!
Thanks in advance for any advice.
I made the mistake of using builtin\administrator as the username but after a couple of hours debugging the same error you're getting I found out that is a UPN and not allowed. I changed it to .\administrator and that worked!
Also check out this site for troubleshooting, it really goes into details with some of the things: http://www.hurryupandwait.io/blog/understanding-and-troubleshooting-winrm-connection-and-authentication-a-thrill-seekers-guide-to-adventure
I've been trying to install and start the new preview SDK, and even after several installs/uninstalls/reboots I always get this error when running DevClusterSetup:
Start-Service : Failed to start service 'Microsoft Service Fabric Host Service (FabricHostSvc)'.
At C:\Program Files\Microsoft SDKs\Service Fabric\Tools\Scripts\ClusterSetupUtilities.psm1:433 char:5
(full log below)
What I've tried, from other posts on stackoverflow:
reparing the performance counters with lodctr /R
used the system file checker with SFC /SCANNOW
checked that the windows firewall is running (and tried disabling it for the domain networks)
made sure I have enough disk space
The windows service "Microsoft Service Fabric Host Service" is always "Starting", but never starts.
I have two hints as to what the source of the problem might be, but can't solve it:
a) in the event viewer (Microsoft-Service Fabric > Admin) there are 4 errors that occur everytime the service attempts to start:
Unable to stop data collector for performance counters. The command
"logman stop FabricCounters" failed with error code -2147287038.
System.Fabric.FabricDeployer.InvalidDeploymentException: Failed to
start performance counter collection when creating or updating
deployment
FabricDeployer::Install failed with error 0xffffffff
FabricDeployer::Install failed with error 0xffffffff, Rolling back
b) In the C:\SfDevCluster\Log\Traces folder there are files named something like FabricSetup-131034051696570691.trace . All of them have the same content, and in the middle there are warnings like these:
FabricSetup.FabricSetup.EventTraceInstaller,Method QueryDataCollectorSet failed with HRESULT: -2144337918
FabricSetup.FabricSetup.EventTraceInstaller,Method StopPlaTraceSession failed with HRESULT: -2144337918
and then further down the error:
FabricSetup.FabricSetup.FabricDeployer,Configuration Deployment failed with error 0xffffffff
If I go and check the Fabric deployer files (eg, fabricdeployer-635945286697202537.trace), I have a single error at the end, after a series of Performance counter deletes:
FabricDeployer.FabricDeployer,Executing command: logman stop FabricCounters
FabricDeployer.FabricDeployer,Unable to stop data collector for performance counters. The command "logman stop FabricCounters" failed with error code -2147287038.
but this error seems to come after some other error, as part of the rollback.
Any ideas? This is very frustrating and there is very little info on the net.
I've also tried cleaning the installation with ClearCluster.ps1 and installing the dev cluster to a different folder, always with the same result.
I am running Win10 with VS2015 Update 1, Azure SDK 2.8.2.1 . My user is a liveid which is local admin.
I'll start with a short answer to unblock you. From an elevated powershell session run:
Unregister-ScheduledTask FabricCounters
I had the exact same issue but in my case the FabricCounters task wasn't there. So I did a search for other "Fabric*" tasks via Get-ScheduledTask Fabric* and found both FabricAppInfoTraces and FabricQueryTraces to be present still after uninstall.
I removed both Tasks using Unregister-ScheduledTask <name>, reinstalled the SDK and was able to start my local cluster again!
When trying to set up the local cluster the powershell script I get the following error:
Is there any way of continuing the installation or fixing the cause of this error?
Cheers,
Mike
I have completely removed the SDK and started over but I am still having the same issues. Everything boils down to the 'Connect-ServiceFabricCluster' just doesn't work at all (I have followed all of the suggestions provided).
Surely the warnings about the naming services must point to something?
Each attempt I see the following:
WARNING: Failed to contact Naming Service. Attempting to contact Failover Manager Service...
2>WARNING: Failed to connect Failover Manager Service, Attempting to contact FMM...
2>Connect-ServiceFabricCluster : A communication error caused the operation to fail.
2>At D:\Source\Play\ServiceFabricApplication\ServiceFabricApplication\Scripts\Deploy-FabricApplication.ps1:158 char:16
2>+ ... [void](Connect-ServiceFabricCluster #ClusterConnectionParameters ...
2>+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2> + CategoryInfo : InvalidOperation: (:) [Connect-ServiceFabricCluster], FabricTransientException
2> + FullyQualifiedErrorId : CreateClusterConnectionErrorId,Microsoft.ServiceFabric.Powershell.ConnectCluster
Attempting a reset from the tray:
Tray output
In my case the Cluster was not running (ie no Fabric.exe processes in Task Manager).
I was able to get things working again my opening a Powershell as Admin and running:
& "$ENV:ProgramFiles\Microsoft SDKs\Service Fabric\ClusterSetup\DevClusterSetup.ps1"
After that close the powershell window and open a new one (as Admin). Then Connect-ServiceFabricCluster worked.
This usually indicates that the main service host isn't running. If this is on our just-released public preview SDK, you can usually resolve these situations by resetting the cluster (just right click on the service fabric tray icon and click reset). If this is an older rev, well, then first you should upgrade :) But other than that you can check inside services.msc and make sure FabricHostSvc is running.
The error is a temporary communication error. Open Task Manager, go to 'Details' tab and check if 'FabricHost.exe' and 'Fabric.exe' is running. This indicates if the cluster has been setup and running.
Open a new administrator PowerShell window and try to connect to cluster using 'Connect-ServiceFabricCluster'.
If the connection still fails, try to remove the cluster using 'CleanCluster.ps1' and setup it again using 'DevClusterSetup.ps1'. This should fix the issue.
Please visit Troubleshoot your local development cluster setup.
I recently had a similar situation where all the TCP connections were erroring out with a FabricTransientException exception.
The underlying cause turned out to be the Windows Firewall. Once I disabled the firewall for the domain network, the connections were successful and the services were again accessible.
P.S> In case someone faces the same issue: Initially the problem was that after the installation Fabric Host service was just stalling with the "Starting" status. Main cause for this problem was that Windows Firewall service was disabled on the server. After enabling and starting the windows service, the Fabric Host service started as expected.
This following is not working for me -
%WINDIR%\System32\WindowsPowerShell\v1.0\powershell.exe -NoProfile -Noninteractive -command "& Invoke-Command -ComputerName "Spider_LT_86" -ScriptBlock { msiexec.exe /i "D:\3PDInstallers\ETLBackgroundWorkerSetup.msi" /qn /l*vx "D:\3PDInstallers\logs" }"
all I need to do is run msiexec on a remote machine. I am assuming that the user running the script will have required access to the remote machine.
The script doesn't work even when the Computername points to the local computer (and running msiexec with the same logged in user works fine, so it doesn't look like a permissions issue) - I have ensured that the WinRM service is running, and I also disabled to firewall to check whether that is the cause of issue, but no luck so far. What am I missing here?
This is the full error but it pretty much lists all possibilities -
Connecting to remote server failed with the following error message : WinRM cannot process the request. The following error occured while using
Kerberos authentication: The network path was not found.
Possible causes are:
-The user name or password specified are invalid.
-Kerberos is used when no authentication method and no user name are specified.
-Kerberos accepts domain user names, but not local user names.
-The Service Principal Name (SPN) for the remote computer name and port does not exist.
-The client and remote computers are in different domains and there is no trust between the two domains.
After checking for the above issues, try the following:
-Check the Event Viewer for events related to authentication.
-Change the authentication method; add the destination computer to the WinRM TrustedHosts configuration setting or use HTTPS transport.
Note that computers in the TrustedHosts list might not be authenticated.
-For more information about WinRM configuration, run the following command: winrm help config. For more information, see the about_Remote_Troubleshooting He
lp topic.
+ CategoryInfo : OpenError: (:) [], PSRemotingTransportException
+ FullyQualifiedErrorId : PSSessionStateBroken
UPDATE:
When I use -Credentials with just the username, it pops up a login window to get my password and then on entering it, gives the following error - the user does have admin access so I don't know what's wrong.
The
[spider_lt_86] Connecting to remote server failed with the following error message : Access is denied. For more information, see the about_Remote_Troubleshooting Help topic.
+ CategoryInfo : OpenError: (:) [], PSRemotingTransportException
+ FullyQualifiedErrorId : PSSessionStateBroken
This is more a workaround than an answer but as you re able to use wmi why not try to create the remote process with it? Have a look here http://www.lazywinadmin.com/2011/06/powershell-launchstart-process-on.html?m=1
After comments exchange, the problem is that you had not provide a credential with administrative privilege for the remote host.
Have you set the TrustedHosts on your computer to allow you to connect to other computers?
Run this on the computer you are trying to remote from.
Set-Item wsman::localhost\client\TrustedHosts *
Restart-Service WinRm
One thing is clear, the error message "The network path was not found" gives no indication of what the underlying issue is. I saw a post in a Microsoft forum where a respondent stated it used to work and now did not. That is where I found myself. I verified that I could ping and that I could net view shares, so I knew that the network portions were functional. I rebooted the source server and that did not seem to help. Once I rebooted the target server, the WinRM error went away and everything started to work again. It is not clear whether a service was down or misbehaving or if there was something less obvious going on. After the reboot I got one more failure (out of about 20 attempts), but random failures are not so uncommon in our environment.
I will leave this behind just in case it helps someone else.