Service Fabric Application fails to find the managed identity endpoint - azure-service-fabric

The Service Fabric cluster exists, the applications exists and are running. The user-assigned managed identity exists in the same resource group the cluster is. NOTE: I do not know how to verify whether it is assigned to the cluster or not.
Code is trying to create a Storage queues client using the identity and I get the error below, which I think means that the fabric:/System/ManagedIdentityTokenService is not running. NOTE: I do not know how to verify whether the service is running or not.
NOTE: Very similar code worked in other clusters.
NOTE: the underlying VMSS does have the managed identity associated to it.
NOTE: I am using Storage SDK 12. The C# code does the following:
ManagedIdentityCredentials cred = new ManagedIdentityCredentials(ClientId: "XYZ...");
string queueEndpoint = string.Format("https://{0}.queue.core.windows.net/{1}", accountName, queueName);
QueueClient qc = QueueClient(new Uri(queueEndpoint), cred);
bool b = await qc.CreateIfNotExistsAsync(); // This one throws the error below.
Any guidance to fix this issue would be appreciated.
Error:
Trying to create a queue (using MSI) failed with exception Azure.Identity.CredentialUnavailableException: No managed identity endpoint found.
at Azure.Identity.ExtendedAccessToken.GetTokenOrThrow()
at Azure.Identity.ManagedIdentityCredential.d__8.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Azure.Core.Pipeline.BearerTokenAuthenticationPolicy.AccessTokenCache.d__11.MoveNext()

Related

SAM Deployment failed Error- Waiter StackCreateComplete failed: Waiter encountered a terminal failure state

When I try to deploy package on SAM, the very first status comes in cloud formation console is ROLLBACK_IN_PROGRESS after that it gets changed to ROLLBACK_COMPLETE
I have tried deleting the stack and trying again, but every time same issue occurs.
Error in terminal looks like this-
Sourcing local options from ./SAMToolkit.devenv
SAM_PARAM_PKG environment variable not set
SAMToolkit will operate in legacy mode.
Please set SAM_PARAM_PKG in your .devenv file to run modern packaging.
Run 'sam help package' for more information
Runtime: java
Attempting to assume role from AWS Identity Broker using account 634668058279
Assumed role from AWS Identity Broker successfully.
Deploying stack sam-dev* from template: /home/***/1.0/runtime/sam/template.yml
sam-additional-artifacts-url.txt was not found, which is fine if there is no additional artifacts uploaded
Replacing BATS::SAM placeholders in template...
Uploading template build/private/tmp/sam-toolkit.yml to s3://***/sam-toolkit.yml
make_bucket failed: s3://sam-dev* An error occurred (BucketAlreadyOwnedByYou) when calling the CreateBucket operation: Your previous request to create the named bucket succeeded and you already own it.
upload: build/private/tmp/sam-toolkit.yml to s3://sam-dev*/sam-toolkit.yml
An error occurred (ValidationError) when calling the DescribeStacks operation: Stack with id sam-dev* does not exist
sam-dev* will be created.
Creating ChangeSet ChangeSet-2020-01-20T12-25-56Z
Deploying stack sam-dev*. Follow in console: https://aws-identity-broker.amazon.com/federation/634668058279/CloudFormation
ChangeSet ChangeSet-2020-01-20T12-25-56Z in sam-dev* succeeded
"StackStatus": "REVIEW_IN_PROGRESS",
sam-dev* reached REVIEW_IN_PROGRESS
Deploying stack sam-dev*. Follow in console: https://console.aws.amazon.com/cloudformation/home?region=us-west-2
Waiting for stack-create-complete
Waiter StackCreateComplete failed: Waiter encountered a terminal failure state
Command failed.
Please see the logs above.
I set SQS as event source for Lambda, but didn't provided the permissions like this
- Effect: Allow
Action:
- sqs:ReceiveMessage
- sqs:DeleteMessage
- sqs:GetQueueAttributes
Resource: "*"
in lambda policies.
I found this error in "Events" tab of "CloudFormation" service.

How to configure IdentityServer4 Client for production behind a Reverse Proxy to address "Correlation failed. Unknown location" Error

IdentityServer4 client through an IIS reverse proxy server getting “Exception: Correlation failed. Unknown location”.
We have a .NET Core MVC application that authenticates with our Identity server 4 application. This is working well.
However we need to deploy this into an environment where the application server and the Identity server have to be accessed through a reverse proxy server.
In our client we change the OIDC Host of the Authority option to the reverse proxy server of our Identity server.
In Identity server we change the Host of the redirect Uri for this client to point to the reverse proxy for our client.
We have configured the client to pass the redirect Uri with the hostname of the clients reverse proxy server.
With this configuration when an unauthorized user access the client, the user is redirected to login on the Identity server.
The login is successful but the user gets the following error when returning to the client application: “Exception: Correlation failed. Unknown location”.
However the use has been successfully authenticated and can access the client application.
.AddOpenIdConnect("oidc", options =>
{
options.SignInScheme = "Cookies";
options.Authority = Configuration.GetSection("Uris").GetSection("IdentityServer").Value;
options.RequireHttpsMetadata = false;
options.ClientId = "mvc";
options.ClientSecret = "secret";
options.ResponseType = "code id_token";
options.SaveTokens = true;
options.Scope.Add("api1");
options.Scope.Add("openid");
options.Scope.Add("profile");
options.Scope.Add("offline_access");
options.ClaimActions.MapJsonKey("website", "website");
options.Events.OnRedirectToIdentityProvider = async n =>
{
n.ProtocolMessage.RedirectUri = Configuration.GetSection("Uris").GetSection("RedirectUri").Value;
await Task.FromResult(0);
};
Error return in browser:
An unhandled exception occurred while processing the request.
Exception: Correlation failed.
Unknown location
Exception: An error was encountered while handling the remote login.
Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler+d__12.MoveNext()
Error Logging Information:
Application started. Press Ctrl+C to shut down.
warn: Microsoft.AspNetCore.Authentication.OpenIdConnect.OpenIdConnectHandler[15]
'.AspNetCore.Correlation.oidc.pz2cS4-GHvVSgHgHOJQQTWa8dL_CDKjEBAGqA4Sg-RY' cookie not found.
fail: Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware[0]
An unhandled exception has occurred while executing the request
System.Exception: An error was encountered while handling the remote login. ---> System.Exception: Correlation failed.
--- End of inner exception stack trace ---
at Microsoft.AspNetCore.Authentication.RemoteAuthenticationHandler`1.d__12.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at IdentityServer4.Hosting.FederatedSignOut.AuthenticationRequestHandlerWrapper.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.d__6.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.d__7.MoveNext()
When '.AddOpenIdConnect' is called, an OpenIdConnectHandler gets registered through ASP.NET Core's dependency injection system and will get invoked with the options that you configure in the lambda.
This OpenIdConnectHandler inherits from the RemoteAuthenticationHandler which is where the method to generate and validate correlation ids lives. A correlation id is a random string that's set as the name of a cookie, and is set and verified on your browser to ensure you are using the same user-agent that initiated the login.
When the correlation cookie is set but not received, the OpenIdConnectHandler will fail the request. This failed request ultimately gets handled by the RemoteAuthenticationHandler calling the 'Events.RemoteFailed' delegate on the OpenIdConnectOptions (the RemoteAuthenticationHandler knows about the events because the OpenIdConnectOptions is actually a derived class of RemoteAuthenticationOptions).
To handle this authentication failure more gracefully, you can find a way to identify this failure within the OpenIdConnectOptions' and potentially show a session expired page, or find a way to automatically re-authenticate.
TLDR: ASP.NET Core sets a correlation cookie with a fairly short expiration time that fails the authentication if too much time is taken between the redirect to login and the code exchange. To handle it gracefully you can set the 'RemoteFailure' on the option's event property.

Azure CD Issue : Failed to fetch App Service 'myAppServiceName' publishing credentials

I'm trying to deploy my release on a azure web App. It's not working and I don't know what to do. Maybe I'm missing something in the configuration in my app service or in my release pipeline. I've got the following error
Failed to fetch App Service 'myAppServiceName' publishing credentials. Error: Could not fetch access token for Managed Service Principal.
And here is a block of my debug :
2019-04-11T08:25:35.4761242Z ##[debug]Predeployment Step Started
2019-04-11T08:25:35.4776374Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data subscriptionid = xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
2019-04-11T08:25:35.4776793Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data subscriptionname = Paiement à l’utilisation
2019-04-11T08:25:35.4777798Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e auth param serviceprincipalid = null
2019-04-11T08:25:35.4778094Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data environmentAuthorityUrl = https://login.windows.net/
2019-04-11T08:25:35.4781237Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e auth param tenantid = ***
2019-04-11T08:25:35.4782509Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e=https://management.azure.com/
2019-04-11T08:25:35.4782769Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data environment = AzureCloud
2019-04-11T08:25:35.4785012Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e auth scheme = ManagedServiceIdentity
2019-04-11T08:25:35.4785626Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data msiclientId = undefined
2019-04-11T08:25:35.4785882Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data activeDirectoryServiceEndpointResourceId = https://management.core.windows.net/
2019-04-11T08:25:35.4786107Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data AzureKeyVaultServiceEndpointResourceId = https://vault.azure.net
2019-04-11T08:25:35.4786348Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data AzureKeyVaultDnsSuffix = vault.azure.net
2019-04-11T08:25:35.4786525Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e auth param authenticationType = null
2019-04-11T08:25:35.4786735Z ##[debug]33ddf4aa-03c4-4031-95fa-e2083d49cc9e data EnableAdfsAuthentication = false
2019-04-11T08:25:35.4792324Z ##[debug]{"subscriptionID":"mysubscriptionID","subscriptionName":"Paiement à l’utilisation","servicePrincipalClientID":null,"environmentAuthorityUrl":"https://login.windows.net/","tenantID":"***","url":"https://management.azure.com/","environment":"AzureCloud","scheme":"ManagedServiceIdentity","activeDirectoryResourceID":"https://management.azure.com/","azureKeyVaultServiceEndpointResourceId":"https://vault.azure.net","azureKeyVaultDnsSuffix":"vault.azure.net","authenticationType":null,"isADFSEnabled":false,"applicationTokenCredentials":{"clientId":null,"domain":"***","baseUrl":"https://management.azure.com/","authorityUrl":"https://login.windows.net/","activeDirectoryResourceId":"https://management.azure.com/","isAzureStackEnvironment":false,"scheme":0,"isADFSEnabled":false}}
2019-04-11T08:25:35.4809400Z Got service connection details for Azure App Service:'myAppServiceName'
2019-04-11T08:25:35.4846967Z ##[debug][GET]http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=https://management.azure.com/
2019-04-11T08:25:35.5443632Z ##[debug]Deployment Failed with Error: Error: Failed to fetch App Service 'myAppServiceName' publishing credentials. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5444488Z ##[debug]task result: Failed
2019-04-11T08:25:35.5501745Z ##[error]Error: Failed to fetch App Service 'myAppServiceName' publishing credentials. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5511780Z ##[debug]Processed: ##vso[task.issue type=error;]Error: Failed to fetch App Service 'myAppServiceName' publishing credentials. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5512729Z ##[debug]Processed: ##vso[task.complete result=Failed;]Error: Failed to fetch App Service 'myAppServiceName' publishing credentials. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5512828Z Failed to add release annotation. Error: Failed to get App service 'myAppServiceName' application settings. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5645194Z (node:5004) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Failed to fetch App Service 'myAppServiceName' publishing profile. Error: Could not fetch access token for Managed Service Principal. Please configure Managed Service Identity (MSI) for virtual machine 'https://aka.ms/azure-msi-docs'. Status code: 400, status message: Bad Request
2019-04-11T08:25:35.5759915Z ##[section]Finishing: Deploy Azure App Service
And some screenshot of
azure missing configuration ?
release pipeline config 1
release pipeline config 2
release pipeline config 3
Let me know if you need more informations.. I'm new in this so maybe missing simple things... Best regards
do you have setting identity Status On ?
like below
In my case, we had just moved our app service to a new resource group, but the pipeline was still referencing the old resource group. Correcting the resource group fixed the issue
A simple typo can also be the reason for this error message.
You will get this error message even though if it's just a typo or wrong value in your "slotName".
Please do ensure that the "slotName" you've given is the actual slotname (the default is 'production'). So if you've added a slot that's called 'stage' then inside the portal it will have your '/stage' or '-stage', but it's still just called 'stage'.
I know several have had this error message shown and none of the above helped them out (I faced the same issue the first time).
My research indicated this to be an intermittent problem.
I redeployed 2 times and it worked.
The first redeploy - just seemed to wait for ages to connect to an available agent, so I cancelled that too, and redeployed - which worked without any issue.
If this is still an issue or if someone had this issue, all I did was just to rerun the release and it well went well. Hopefully someone has saved time by just re-releasing, if this wont work then probably try something else.

"The upgrade policy is not allowed" ClusterConfigurationUpgrade fails on Standalone Cluster

v5.6.220.9494
I've been digging the last few hours trying to figure this out. I'm receiving the following error when attempting a clusterconfigurationupgrade:
System.Fabric.FabricDeployer.ClusterManifestValidationException: Cluster manifest validation failed with exception System.ArgumentException: The upgrade policy is not allowed
at System.Fabric.Management.WindowsFabricValidator.FabricSettingsValidator.CompareSettings(ClusterManifestType newClusterManifest, String nodeTypeNameFilter)
at System.Fabric.FabricDeployer.FabricValidatorWrapper.CompareAndAnalyze(ClusterManifestType currentClusterManifest, ClusterManifestType targetClusterManifest, Infrastructure infrastructure, DeploymentParameters parameters)
at System.Fabric.FabricDeployer.FabricValidatorWrapper.CompareAndAnalyze(ClusterManifestType currentClusterManifest, ClusterManifestType targetClusterManifest, Infrastructure infrastructure, DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperationPrivate(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.DeploymentOperation.ExecuteOperation(DeploymentParameters parameters)
at System.Fabric.FabricDeployer.Program.Main(String[] args)
I ended up having to rollback the upgrade to recover two nodes. If anyone could help point me in the right direction on resolving this error it'd be appreciated.

Failed to Create Development Service Fabric Cluster on Windows Server 2016 Standard

I am attempting to create a local development (unsecured) Service Fabric Cluster on Windows Server 2016 Standard. I have followed the instructions found in this article. However, I'm getting a rather interesting error and cannot find anything to help me resolve this.
FabricHostSvc was not installed by FabricInstallerSvc on machine
localhost. FabricSetup may have failed. CreateCluster Error:
System.AggregateException: One or more errors occurred. --->
System.Fabric.FabricServiceNotFoundExc eption: FabricHostSvc was not
installed by FabricInstallerSvc on machine localhost. FabricSetup may
have failed. at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(Str
ing machineName, ServiceController installerSvc) at
System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.<ForWorker>b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at
System.Threading.Tasks.Task.<>c__DisplayClass176_0.<ExecuteSelfReplicating>b__0(Object
) --- End of inner exception stack trace --- at
System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions) at
System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout,
CancellationToken cancellationToken) at
System.Threading.Tasks.Parallel.ForWorker[TLocal](Int32 fromInclusive,
Int32 toExclusive, ParallelOptions parallel Options, Action1 body,
Action2 bodyWithState, Func4 bodyWithLocal, Func1 localInit,
Action1 localFinally) at
System.Threading.Tasks.Parallel.ForEachWorker[TSource,TLocal](IEnumerable1
source, ParallelOptions parallelOption s, Action1 body, Action2
bodyWithState, Action3 bodyWithStateAndIndex, Func4
bodyWithStateAndLocal, Func5 bodyWithE verything, Func1 localInit,
Action1 localFinally) at
System.Threading.Tasks.Parallel.ForEach[TSource](IEnumerable1 source,
Action1 body) at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.RunFabricServices(List1
machines, FabricPacka geType fabricPackageType) at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInternal>d__7.MoveNext()
---> (Inner Exception #0) System.Fabric.FabricServiceNotFoundException: FabricHostSvc was not
installed by FabricInstall erSvc on machine localhost. FabricSetup may
have failed. at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.StartAndValidateInstallerServiceCompletion(Str
ing machineName, ServiceController installerSvc) at
System.Threading.Tasks.Parallel.<>c__DisplayClass17_01.b__1()
at System.Threading.Tasks.Task.InnerInvokeWithArg(Task childTask)
at
System.Threading.Tasks.Task.<>c__DisplayClass176_0.b__0(Object
)<---
Cleaning up faulted installation. FabricRoot not found in registry of
target machine localhost. Create Cluster failed. For more information
please look at traces in FabricLogRoot. Create Cluster failed with
exception: System.AggregateException: One or more errors occurred.
---> System.AggregateExcep tion: One or more errors occurred. at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown --- at
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task) at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task) at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManager.d__0.MoveNext()
--- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean
includeTaskCanceledExceptions) at
System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout,
CancellationToken cancellationToken) at
Microsoft.ServiceFabric.Powershell.ClusterCmdletBase.NewCluster(String
clusterConfigurationFilePath, String fabric PackageSourcePath, Boolean
cleanupOnFailure)
---> (Inner Exception #0) System.AggregateException: One or more errors occurred. at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.d__7.MoveNext()
--- End of stack trace from previous location where exception was thrown --- at
System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task
task) at
System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
task) at
Microsoft.ServiceFabric.DeploymentManager.DeploymentManager.d__0.MoveNext()<---
Has anyone encountered this error before and fixed it? How is this error resolved?
Side Note: After receiving this error I ran the CleanFabric PowerShell script and removed all the Service Fabric files from the server and tried running the installation again with the same error message.
In addition, there are no Service Fabric SDKs installed on the machine (the ones you'd use on a local development machine). The reason for this is due to the official prerequisites stated by Microsoft shown below.
Prerequisites for each machine that you want to add to the cluster:
1. A minimum of 16 GB of RAM is recommended.
2. A minimum of 40 of GB available disk space is recommended.
3. A 4 core or greater CPU is recommended.
4. Connectivity to a secure network or networks for all machines.
5. Windows Server 2012 R2 or Windows Server 2012 (you need to
have KB2858668 installed).
6. .NET Framework 4.5.1 or higher, full install.
7. Windows PowerShell 3.0. The RemoteRegistry service should be running on all the machines.
The cluster administrator deploying and configuring the cluster must have administrator privileges on each of the machines. You cannot install Service Fabric on a domain controller.
I cannot help but feel there is something obvious missing but I've followed the docs very closely so this is rather perplexing.
Service Fabric drivers have a signing issue which is preventing them from being installed on Windows Server 2016 and Windows 10 Anniversary edition. Please wait for the next version or try with version 5.2.