So we are using Azure Service Fabric and gets a weird behavior when trying to run API tests against my local development cluster.
Every time I start the test the app gets terminated, sometimes it gets restarted again but most often it just stays terminated (and even deleted from the cluster).
I guess its somehow connected against that when I run the API test it will run and build stuff that the service fabric is using, but since the outcome is different depending on something (maybe the sun?) it feels like I am either missing something or experience a bug with service fabric.
Do anyone have any idea? Consider me as a noob and assume that I have done something wrong myself (I am doing that atleast).
UPDATE
There was a question on how we do run our tests:
Starts 2 instances of Visual Studio
Open the same .sln in both of them
Start the Service Fabric project.
Wait until cluster reports OK.
Run api test through a unit test (both service bus tests and REST tests) with Resharper test runner
Now we get the messages that is attached in diagnostics.
Diagnostics:
Event #1
{
"Timestamp": "2018-10-16T08:14:03.0590414+02:00",
"ProviderName": "Microsoft-ServiceFabric",
"Id": 23083,
"Message": ApplicationHostTerminated: ApplicationId=fabric:/<MyService>, ServiceName=fabric:/<MyService>, ServicePackageName=<MyPackage>, ServicePackageActivationId=8f36ac97-9271-4a49-94ce-dd296aebffa5, IsExclusive=True, CodePackageName=Code, EntryPointType=Exe, ExeName=MyExe, ProcessId=24568, HostId=d2a820b5-5b4d-42af-ae87-350028a3fa72, ExitCode=3221225786, UnexpectedTermination=False, StartTime=10/16/2018 08:12:14. ",
"ProcessId": 22660,
"Level": "Informational",
"Keywords": "0x4000000000000001",
"EventName": "Hosting",
"ActivityID": null,
"RelatedActivityID": null,
"Payload": {
"eventInstanceId": "\"07f15452-2f75-49e3-ad5d-d16ea49bdc8f\"",
"applicationName": "MyAppName",
"ServiceName": "fabric:/MyServiceName",
"ServicePackageName": "MyPackageName",
"ServicePackageActivationId": "8f36ac97-9271-4a49-94ce-dd296aebffa5",
"IsExclusive": true,
"CodePackageName": "Code",
"EntryPointType": 1,
"ExeName": "MyExe",
"ProcessId": 24568,
"HostId": "d2a820b5-5b4d-42af-ae87-350028a3fa72",
"ExitCode": 3221225786,
"UnexpectedTermination": false,
"StartTime": "\"\/Date(1539670334917)\/\""
}
}
Event #2
{
"Timestamp": "2018-10-16T08:14:02.3557708+02:00",
"ProviderName": "Microsoft-ServiceFabric",
"Id": 29625,
"Message": "Application deleted: Application = fabric:/MyApp, Application Type = MyServiceType ",
"ProcessId": 22660,
"Level": "Informational",
"Keywords": "0x4000000000000001",
"EventName": "CM",
"ActivityID": null,
"RelatedActivityID": null,
"Payload": {
"eventInstanceId": "\"ca608cec-8d55-4606-a331-8ebfcfff8fa6\"",
"applicationName": "fabric:/MyAppName",
"applicationTypeName": "MyAppTypeName",
"applicationTypeVersion": "1.0.0"
}
}
I think you can experience a side effect of Application Debug Mode set for your .sfproj.
By default Application Debug Mode is set to Refresh Application (which if you are using 5-node cluster is automatically changed to Remove Application) or Remove Application debug mode. This instructs Visual Studio to recreate the application for each debugging session and remove it when session ends.
Changing it to Keep Application should prevent the Visual Studio from recreating the application during debugging session.
Related
I am trying to deploy data factory using ARM template. It is easy to use the exported template to create a deployment pipeline.
However, as the data factory needs to access an on-premise database server, I need to have an integrated runtime. The problem is how can I include the run time in the arm template?
The template looks like this and we can see that it is trying to include the runtime:
{
"name": "[concat(parameters('factoryName'), '/OnPremisesSqlServer')]",
"type": "Microsoft.DataFactory/factories/linkedServices",
"apiVersion": "2018-06-01",
"properties":
{
"annotations": [],
"type": "SqlServer",
"typeProperties": {
"connectionString": "[parameters('OnPremisesSqlServer_connectionString')]"
},
"connectVia": {
"referenceName": "OnPremisesSqlServer",
"type": "IntegrationRuntimeReference"
}
},
"dependsOn": [
"[concat(variables('factoryId'), '/integrationRuntimes/OnPremisesSqlServer')]"
]
},
{
"name": "[concat(parameters('factoryName'), '/OnPremisesSqlServer')]",
"type": "Microsoft.DataFactory/factories/integrationRuntimes",
"apiVersion": "2018-06-01",
"properties": {
"type": "SelfHosted",
"typeProperties": {}
},
"dependsOn": []
}
Running this template gives me this error:
\"connectVia\": {\r\n \"referenceName\": \"OnPremisesSqlServer\",\r\n \"type\": \"IntegrationRuntimeReference\"\r\n }\r\n }\r\n} and error is: Failed to encrypted linked service credentials on self-hosted IR 'OnPremisesSqlServer', reason is: NotFound, error message is: No online instance..
The problem is that I will need to type in some key in the integrated runtime's UI, so it can be registered in azure. But I can only get that key from my data factory instance's UI. So above arm template deployment will always fail at least once. I am wondering if there is a way to create the run time independently?
The problem is that I will need to type in some key in the integrated
runtime's UI, so it can be registered in azure. But I can only get
that key from my data factory instance's UI. So above arm template
deployment will always fail at least once. I am wondering if there is
a way to create the run time independently?
It seems that you already know how to create Self-Hosted IR in the ADF ARM.
{
"name": "[concat(parameters('dataFactoryName'), '/integrationRuntime1')]",
"type": "Microsoft.DataFactory/factories/integrationRuntimes",
"apiVersion": "2018-06-01",
"properties": {
"additionalProperties": {},
"description": "jaygongIR1",
"type": "SelfHosted"
}
}
Result:
Only you concern is that Windows IR Tool need to be configured with AUTHENTICATION KEY to access ADF Self-Hosted IR node.So,it should be Unavailable status once it is created.This flow is make sense i think,authenticate key should be created first,then you can use it to configure On-Premise Tool.You can't implement all of things in one step because these behaviors are operated on both of azure and on-premise sides.
Based on the Self-Hosted IR Tool document ,the Register steps can't be implemented with Powershell code. So,all steps can't be processed in the flow are creating IR and getting Auth key,not for Registering in the tool.
Situation
I'm experimenting with writing a VSCode Language Server Protocol (LSP) Extension. I've got it running as follows:
An lsp-server process which is launched by running haskell-lsp-example-exe from terminal
An lsp-client written in Typescript which, for now, basically just launches lsp-server (it's based on the lsp-sample repo)
The lsp-server is launch as follows:
# extension.ts
let serverOptions: ServerOptions = {
run: {
command: "haskell-lsp-example-exe"
},
}
The lsp-client is launched using code --extensionDevelopmentPath="path/to/extension"
I can see that it launches correctly, and I can find its pid through Activity Monitor (I'm on Mac).
Question
How can I see the logs of this process which is spawned by VSCode?
Context
I have tried the following:
In lsp-client/package.json I set the following which gives me the messages going back and forth. But not the logs of lsp-server.:
"languageServerExample.trace.server": {
"scope": "window",
"type": "string",
"enum": [
"off",
"messages",
"verbose"
],
"default": "verbose",
"description": "Traces the communication between VS Code and the language server."
}
I've also tried opening up dev tools in the launched instance of VSCode, but that gives the logs of lsp-client
The log labelled Log (Extension Host) in the launched instance of VSCode also doesn't look too useful
Thanks in advance for any help!
I am trying to integrate Azure App insights service into the service fabric app for logging and instrumentation. I am running fabric code on my local VM. I exactly followed the document here [scenario 2]. Other resources on learn.microsoft.com also seem to indicate the same steps. [ex: https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-diagnostics-event-aggregation-eventflow
For some reason, I don’t see any event entries in App insights. No errors in code when I do this:
ServiceEventSource.Current.ProcessedCountMetric("synced",sw.ElapsedMilliseconds, crc.DataTable.Rows.Count);
eventflowconfig.json contents
{
"inputs": [
{
"type": "EventSource",
"sources": [
{ "providerName": "Microsoft-ServiceFabric-Services" },
{ "providerName": "Microsoft-ServiceFabric-Actors" },
{ "providerName": "mystatefulservice" }
]
}
],
"filters": [
{
"type": "drop",
"include": "Level == Verbose"
}
],
"outputs": [
{
"type": "ApplicationInsights",
// (replace the following value with your AI resource's instrumentation key)
"instrumentationKey": "XXXXXXXXXXXXXXXXXXXXXX",
"filters": [
{
"type": "metadata",
"metadata": "metric",
"include": "ProviderName == mystatefulservice && EventName == ProcessedCountMetric",
"operationProperty": "operation",
"elapsedMilliSecondsProperty": "elapsedMilliSeconds",
"recordCountProperty": "recordCount"
}
]
}
],
"schemaVersion": "2016-08-11"
}
In ServiceEventSource.cs
[Event(ProcessedCountMetricEventId, Level = EventLevel.Informational)]
public void ProcessedCountMetric(string operation, long elapsedMilliSeconds, int recordCount)
{
if (IsEnabled())
WriteEvent(ProcessedCountMetricEventId, operation, elapsedMilliSeconds, recordCount);
}
EDIT
Adding diagnostics pipeline code from "Program.cs in fabric stateful service
using (var diagnosticsPipeline =
ServiceFabricDiagnosticPipelineFactory.CreatePipeline($"{ServiceFabricGlobalConstants.AppName}-mystatefulservice-DiagnosticsPipeline")
)
{
ServiceRuntime.RegisterServiceAsync("mystatefulserviceType",
context => new mystatefulservice(context)).GetAwaiter().GetResult();
ServiceEventSource.Current.ServiceTypeRegistered(Process.GetCurrentProcess().Id,
typeof(mystatefulservice).Name);
// Prevents this host process from terminating so services keep running.
Thread.Sleep(Timeout.Infinite);
}
Event Source is a tricky technology, I have been working with it for a while and always have problems. The configuration looks good, It is very hard to investigate without access to the environments, so I will make my suggestions.
There are a few catches you must be aware of:
If you are listening etw events from a different process, your process must be running with a user with permissions on 'Performance Log Users. Check which identity your service is running on and if it is part of performance log users, who has permissions to create event sessions to listen for these events.
Ensure the events are being emitted correctly and you can listen to them from Diagnostics Events Window, if it is not showing there, there is a problem in the provider.
For testing purposes, comment out the line if (IsEnabled()). it is an internal check to validate if your events should be emitted. I had situations where it is always false and skip the emit of events, probably it cache the results for a while, the docs are not clear how it should work.
Whenever possible, use the EventSource from the nuget package instead of the framework one, the framework version is full of bugs and lack fixes found in the nuget version.
Application Insights are not RealTime, sometimes it might take a few minutes to process your events, I would recommend to output the events to a console or file and check if it is listening correctly, afterwards, enable the AppInsights.
The link you provide is quite outdated and there's actually a much better way to log application error and exception info to Application insights. For example, the above won't help with tracking the call hierarchy of an incoming request between multiple services.
Have a look at the Microsoft App Insights Service Fabric nuget packages. It works great for:
Sending error and exception info
Populating the application map with all your services and their dependencies (including database)
Reporting on app performance metrics,
Tracing service call dependencies end-to-end,
Integrating with native as well as non-native SF applications
I'm running a simple Hyperledger network on Bluemix. I can deploy, and invoke, but not query. The chaincode function, Init sets up a value for var, "abc" ... stub.PutState("abc", []byte(strconv.Itoa(Aval)))
I should be able to query "abc" as validation the code is ready to use. Instead, I'm seeing this error:
"... Error:Failed to launch chaincode spec(Could not get deployment
transaction for - LedgerError - ResourceNotFound:
ledger: resource not found)"
The query json is:
{
"jsonrpc": "2.0",
"method": "query",
"params": {
"type": 1,
"chaincodeID": {
"name": "my chaincode id"
},
"ctorMsg": {
"function": "read",
"args": [
"abc"
]
},
"secureContext": "user_type1_3"
},
"id": 0
}
The following is the list of probable causes of the error
Could not get deployment transaction for - LedgerError -
ResourceNotFound: ledger: resource not found
1. Chaincode did not get deployed correctly. To check if the
chaincode was deployed correctly, you need to check the peer logs to
see if there were any errors when the deploy transaction was sent.
2. Chaincode got deployed correctly, but the consensus mechanism hasnt
yet completed. You should ideally wait for a few minutes after
deploying a chaincode before you try to query it.
3. The Chaincode got deployed, but the chaincode ID/name specified
while trying to send a query is incorrect. You need to make sure you
use the same chaincode ID that comes in the response when you deploy
a chaincode.
Update: I found executing script on the octopus server is now available in version 3.3, I haven't update my octopus yet but I will take that would work as designed. I'm still wondering if there is a better way to do this without octo.exe?
The task I'm trying to accomplish is after each successful production deployment, automatically schedule a DR deployment to happen next 24 hours.
My desired approach is have octopus do it.
I added a new Octopus step at the end of the deployment only runs upon success of previous step. I attempted to use octo deploy-release --deployAt can be found here in the newly created step.
My challenge is, a script step requires me to pick a target role, which means it will be executed on a tentacle. Also, presence of Octo.exe is required.
I tried to create my own octopus step template, a deployment target role is still required in my customized step.
{
"Id": "ActionTemplates-2",
"Name": "Octopus - Schedule Deployment",
"Description": "Schedule a future octopus deployment",
"ActionType": "Octopus.Script",
"Version": 3,
"Properties": {
"Octopus.Action.Script.Syntax": "PowerShell",
"Octopus.Action.Script.ScriptBody": "--hide--"
},
"SensitiveProperties": {},
"Parameters": [
{
"Name": "OctoPath",
"Label": "Path for Octo.exe",
"HelpText": "Location for octo.exe",
"DefaultValue": null,
"DisplaySettings": {
"Octopus.ControlType": "SingleLineText"
}
},
{
"Name": "projName",
"Label": "Project Name",
"HelpText": "The name of the project should be deployed",
"DefaultValue": null,
"DisplaySettings": {
"Octopus.ControlType": "SingleLineText"
}
},
{
"Name": "days",
"Label": "Days",
"HelpText": "The days in future this deployment would happen",
"DefaultValue": null,
"DisplaySettings": {
"Octopus.ControlType": "SingleLineText"
}
},
{
"Name": "hours",
"Label": "Hours",
"HelpText": "The hours in future this deployment would happen",
"DefaultValue": null,
"DisplaySettings": {
"Octopus.ControlType": "SingleLineText"
}
},
{
"Name": "env",
"Label": "Environment to deploy",
"HelpText": "The environment next deployment should happen",
"DefaultValue": null,
"DisplaySettings": {
"Octopus.ControlType": "SingleLineText"
}
}
],
"$Meta": {
"ExportedAt": "2016-04-20T13:58:54.263Z",
"OctopusVersion": "3.2.0",
"Type": "ActionTemplate"
}
}
Is there a way to alter the template to get rid of the role selection and have octopus server directly execute it as it does for Azure script step?
Is there any another way we can have octopus server automatically schedule the deployment without external help? I guess this go back to first problem, I may still need octopus to run something on the server side.
Note: We kick off production deployment manually, thus I don't have another tool waiting for the response of the deployment. I think it is possible to have a process regularly call out the last deployment and do some analysis then schedule new deployment accordingly but this is not as clean as have octopus do it directly. Injecting octo.exe to a random production machine is not desired at all
You could create new WebAPI project in C#, pull in the Octopus.Deploy nuget package,
write code that accepts HTTP requests, and deals with the scheduling logic.
Host that project on the same server as Octopus server itself. Should be 20-30 minute job to set the website up in IIS.
In your deployment process, add step that creates http request, and done. You could go even one step further, and have the site/service listen for every successful deployment, and do decisions based on that, such that other projects don't have to add extra steps to octopus deployment process.
As you said, polling is also viable option.
Alternatively, if you're on Octopus deploy 3.0, they already expose REST API, I am not sure if it's powerful enough to allow you create scheduled deployment, but you could explore that: https://github.com/OctopusDeploy/OctopusDeploy-Api/wiki/Releases
I agree floating octo.exe in production servers is bad idea. It might get out of sync, and your production server shouldn't deal with this.