How can we add Application Insight in Service Fabric Actor Reliable service - azure-service-fabric

We have 5 reliable actor services which are being called from a stateless aspnet core web api.
Now, the service fabric application is running in production but as a part of migration to azure we want all our custom events and trace to app insight.
How can we add the same to SF? Please note that I am looking specifically for reliable actor service. It's kind of simple adding that in reliable service but I am facing challenges with Actor services.
I referred to this https://learn.microsoft.com/en-us/azure/service-fabric/service-fabric-tutorial-monitoring-aspnet tutorial for reliable services but same doesn't work in case of reliable actor service written in .NET Framework.

Quoting the example from this open GitHub issue: Add documentation for configure actor services
Setting Service Context
Create your own actor service class and initialize the context there:
internal class MyActorService : ActorService
{
public MyActorService(
StatefulServiceContext context,
ActorTypeInformation actorTypeInfo,
Func<ActorService, ActorId, ActorBase> actorFactory = null,
Func<ActorBase, IActorStateProvider, IActorStateManager> stateManagerFactory = null,
IActorStateProvider stateProvider = null,
ActorServiceSettings settings = null)
: base(context, actorTypeInfo, actorFactory, stateManagerFactory, stateProvider, settings)
{
FabricTelemetryInitializerExtension.SetServiceCallContext(this.Context);
}
}
Register the actor service like below:
ActorRuntime.RegisterActorAsync<MyActor>(
(context, actorType) => new MyActorService(context, actorType)).GetAwaiter().GetResult();
Enabling Correlation for Service Remoting
Firstly, make sure Service Remoting is set up correctly. You can find more info here.
Remoting V2 (recommended)
Initialize the Service Remoting dependency/request tracking modules like below.
Note: You can also set the service context through the CreateFabricTelemetryInitializer method. In this case, you don't need to call SetServiceCallContext
public MyActorServiceNetCore(
StatefulServiceContext context,
ActorTypeInformation actorTypeInfo,
Func<ActorService, ActorId, ActorBase> actorFactory = null,
Func<ActorBase, IActorStateProvider, IActorStateManager> stateManagerFactory = null,
IActorStateProvider stateProvider = null,
ActorServiceSettings settings = null)
: base(context, actorTypeInfo, actorFactory, stateManagerFactory, stateProvider, settings)
{
var config = TelemetryConfiguration.Active;
config.InstrumentationKey = "<your ikey>";
config.TelemetryInitializers.Add(FabricTelemetryInitializerExtension.CreateFabricTelemetryInitializer(this.Context));
var requestTrackingModule = new ServiceRemotingRequestTrackingTelemetryModule();
var dependencyTrackingModule = new ServiceRemotingDependencyTrackingTelemetryModule();
requestTrackingModule.Initialize(config);
dependencyTrackingModule.Initialize(config);
}
Remoting V1
If you still want to use Service Remoting V1 and also track correlation, you can follow the instructions below, but it's strongly recommended that you upgrade to Remoting V2.
Create the proxy like below on sender side
//IActorService actorServiceProxy = ActorServiceProxy.Create(new Uri(serviceUri), partitionKey);
CorrelatingActorProxyFactory actorProxyFactory = new CorrelatingActorProxyFactory(serviceContext, callbackClient => new FabricTransportActorRemotingClientFactory(callbackClient));
IActorService actorServiceProxy = actorProxyFactory.CreateActorServiceProxy<IActorService>(new Uri(serviceUri), partitionKey);
Initialize the listener like below:
internal class MyActorService : ActorService
{
public MyActorService(
StatefulServiceContext context,
ActorTypeInformation actorTypeInfo,
Func<ActorService, ActorId, ActorBase> actorFactory = null,
Func<ActorBase, IActorStateProvider, IActorStateManager> stateManagerFactory = null,
IActorStateProvider stateProvider = null,
ActorServiceSettings settings = null)
: base(context, actorTypeInfo, actorFactory, stateManagerFactory, stateProvider, settings)
{
}
protected override IEnumerable<ServiceReplicaListener> CreateServiceReplicaListeners()
{
return new[]
{
new ServiceReplicaListener(
context => new FabricTransportActorServiceRemotingListener(
context,
new CorrelatingRemotingMessageHandler(this),
new FabricTransportRemotingListenerSettings()))
};
}
}
Register your own actor service:
// ActorRuntime.RegisterActorAsync<MyActor>(
// (context, actorType) => new ActorService(context, actorType)).GetAwaiter().GetResult();
ActorRuntime.RegisterActorAsync<MyActor>(
(context, actorType) => new MyActorService(context, actorType)).GetAwaiter().GetResult();
Originally posted by #yantang-msft in https://github.com/microsoft/ApplicationInsights-ServiceFabric/issue_comments/434430800
Other resources: service-fabric-application-insights-example

Related

Kafka Connect using REST API with Strimzi with kind: KafkaConnector

I'm trying to use Kafka Connect REST API for managing connectors, for simplicity consider the following pause implementation:
def pause(): Unit = {
logger.info(s"pause() Triggered")
val response = HttpClient.newHttpClient.send({
HttpRequest
.newBuilder(URI.create(config.connectUrl + s"/connectors/${config.connectorName}/pause"))
.PUT(BodyPublishers.noBody)
.timeout(Duration.ofMillis(config.timeout.toMillis.toInt))
.build()
}, BodyHandlers.ofString)
if (response.statusCode() != HTTPStatus.Accepted) {
throw new Exception(s"Could not pause connector: ${response.body}")
}
}
Since I'm using KafkaConnector as a resource, I cannot use Kafka Connect REST API because the connector operator has the KafkaConnetor resources as its single source of truth, manual changes such as pause made directly using the Kafka Connect REST API are reverted by the Cluster Operator.
So to pause the connector I need to edit the resource in some way.
I'm struggling to change the logic of the current function, It will be great to have some practical examples of how to handle KafkaConnetor resources.
I check out the Using Strimzi doc but couldn't find any practical example
Thanks!
After help from #Jakub i managed to create my new client:
class KubernetesService(config: Configuration) extends StrictLogging {
private[this] val client = new DefaultKubernetesClient(Config.autoConfigure(config.connectorContext))
def setPause(pause: Boolean): Unit = {
logger.info(s"[KubernetesService] - setPause($pause) Triggered")
val connector = getConnector()
connector.getSpec.setPause(pause)
Crds.kafkaConnectorOperation(client).inNamespace(config.connectorNamespace).withName(config.connectorName).replace(connector)
Crds.kafkaConnectorOperation(client)
.inNamespace(config.connectorNamespace)
.withName(config.connectorName)
.waitUntilCondition(connector => {
connector != null &&
connector.getSpec.getPause == pause && {
val desiredState = if (pause) "Paused" else "Running"
connector.getStatus.getConditions.stream().anyMatch(_.getType.equalsIgnoreCase(desiredState))
}
}, config.timeout.toMillis, TimeUnit.MILLISECONDS)
}
def delete(): Unit = {
logger.info(s"[KubernetesService] - delete() Triggered")
Crds.kafkaConnectorOperation(client).inNamespace(config.connectorNamespace).withName(config.connectorName).delete
Crds.kafkaConnectorOperation(client)
.inNamespace(config.connectorNamespace)
.withName(config.connectorName)
.waitUntilCondition(_ == null, config.timeout.toMillis, TimeUnit.MILLISECONDS)
}
def create(oldKafkaConnect: KafkaConnector): Unit = {
logger.info(s"[KubernetesService] - create(${oldKafkaConnect.getMetadata}) Triggered")
Crds.kafkaConnectorOperation(client).inNamespace(config.connectorNamespace).withName(config.connectorName).create(oldKafkaConnect)
Crds.kafkaConnectorOperation(client)
.inNamespace(config.connectorNamespace)
.withName(config.connectorName)
.waitUntilCondition(connector => {
connector != null &&
connector.getStatus.getConditions.stream().anyMatch(_.getType.equalsIgnoreCase("Running"))
}, config.timeout.toMillis, TimeUnit.MILLISECONDS)
}
def getConnector(): KafkaConnector = {
logger.info(s"[KubernetesService] - getConnector() Triggered")
Try {
Crds.kafkaConnectorOperation(client).inNamespace(config.connectorNamespace).withName(config.connectorName).get
} match {
case Success(connector) => connector
case Failure(_: NullPointerException) => throw new NullPointerException(s"Failure on getConnector(${config.connectorName}) on ns: ${config.connectorNamespace}, context: ${config.connectorContext}")
case Failure(exception) => throw exception
}
}
}
To pause the connector, you can edit the KafkaConnector resource and set the pause field in .spec to true (see the docs). There are several options how you can do it. You can use kubectl and either apply the new YAML from file (kubectl apply) or do it interactively using kubectl edit.
If you want to do it programatically, you will need to use a Kubernetes client to edit the resource. In Java, you can also use the api module of Strimzi which has all the structures for editing the resources. I put together a simple example for pausing the Kafka connector in Java using the Fabric8 Kubernetes client and the api module:
package cz.scholz.strimzi.api.examples;
import io.fabric8.kubernetes.client.DefaultKubernetesClient;
import io.fabric8.kubernetes.client.KubernetesClient;
import io.fabric8.kubernetes.client.dsl.MixedOperation;
import io.fabric8.kubernetes.client.dsl.Resource;
import io.strimzi.api.kafka.Crds;
import io.strimzi.api.kafka.KafkaConnectorList;
import io.strimzi.api.kafka.model.KafkaConnector;
public class PauseConnector {
public static void main(String[] args) {
String namespace = "myproject";
String crName = "my-connector";
KubernetesClient client = new DefaultKubernetesClient();
MixedOperation<KafkaConnector, KafkaConnectorList, Resource<KafkaConnector>> op = Crds.kafkaConnectorOperation(client);
KafkaConnector connector = op.inNamespace(namespace).withName(crName).get();
connector.getSpec().setPause(true);
op.inNamespace(namespace).withName(crName).replace(connector);
client.close();
}
}
(See https://github.com/scholzj/strimzi-api-examples for the full project)
I'm not a Scala users - but I assume it should be usable from Scala as well, but I leave rewriting it from Java to Scala to you.

How to set routing type ActiveMQ Artemis from client

I am trying to follow the hello world example. With regular ActiveMQ it works, but ActiveMQ Artemis is giving me headaches. I guess there is some configuration I am not doing correctly. The Address is made, but is it made using Multicast routing. I think I need Unicast (queue routing).
The below snippet does not work for the artemis version of ActiveMQ. Is it possible what I am trying to do? I would like to auto-create a durable Queue.
public class SimpleAmqpTest
{
[Fact]
public async Task TestHelloWorld()
{
Address address = new Address("amqp://guest:guest#localhost:5672");
Connection connection = await Connection.Factory.CreateAsync(address);
Session session = new Session(connection);
Message message = new Message("Hello AMQP");
var target = new Target
{
Address = "simple-queue",
Durable = 1,
};
SenderLink sender = new SenderLink(session, "sender-link", target, null);
await sender.SendAsync(message);
ReceiverLink receiver = new ReceiverLink(session, "receiver-link", "simple-queue");
message = await receiver.ReceiveAsync();
receiver.Accept(message);
await sender.CloseAsync();
await receiver.CloseAsync();
await session.CloseAsync();
await connection.CloseAsync();
}
}
Finally found out what I was doing wrong, as Amqp does not have the configuration of queues and topics, it can be defined in the Capabilities. For some reason Artemis creates topics by default (multicast). If you need AnyCast you can specify your need using Capabilities = new Symbol[] { new Symbol("queue") }. For the full test fact:
public async Task TestHelloWorld()
{
//strange, works using regular activeMQ and the amqp test broker from here: http://azure.github.io/amqpnetlite/articles/hello_amqp.html
//but this does not work in ActiveMQ Artemis
Address address = new Address("amqp://guest:guest#localhost:5672");
Connection connection = await Connection.Factory.CreateAsync(address);
Session session = new Session(connection);
Message message = new Message("Hello AMQP");
Target target = new Target
{
Address = "q1",
Capabilities = new Symbol[] { new Symbol("queue") }
};
SenderLink sender = new SenderLink(session, "sender-link", target, null);
sender.Send(message);
Source source = new Source
{
Address = "q1",
Capabilities = new Symbol[] { new Symbol("queue") }
};
ReceiverLink receiver = new ReceiverLink(session, "receiver-link", source, null);
message = await receiver.ReceiveAsync();
receiver.Accept(message);
await sender.CloseAsync();
await receiver.CloseAsync();
await session.CloseAsync();
await connection.CloseAsync();
}

Spring Boot Admin can't differ between multiple service instances in Cloudfoundry

I got Spring Boot Admin running locally with Eureka Service Discovery (No SBA Dependeny in the Clients). Now i tried to deploy it in Cloudfoundry. According the Documentation, Version 2.0.1 should "support CloudFoundry out of the box".
My Problem is that when I scale a service up to multiple instances, they are all registered under the same hostname and port. Eureka shows me all Instances with their InstanceID that I configured like this:
eureka:
instance:
instanceId: ${spring.application.name}:${vcap.application.instance_id:${spring.application.instance_id:${random.value}}}
But Spring Boot Admin only lists one instance with hostname:port as identifier. I think i have to configure something on the client so that it sends the instance ID per HTTP Header when registering. But i don't know how.
Apparently you have to set the ApplicationId and InstanceIndex that Cloudfoundry generates as Eureka ApplicationId and InstanceId at Startup/ContextRefresh of your Client.
CloudFoundryApplicationInitializer.kt
#Component
#Profile("cloud")
#EnableConfigurationProperties(CloudFoundryApplicationProperties::class)
class CloudFoundryApplicationInitializer {
private val log = LoggerFactory.getLogger(CloudFoundryApplicationInitializer::class.java)
#Autowired
private val applicationInfoManager: ApplicationInfoManager? = null
#Autowired
private val cloudFoundryApplicationProperties: CloudFoundryApplicationProperties? = null
#EventListener
fun onRefreshScopeRefreshed(event: RefreshScopeRefreshedEvent) {
injectCfMetadata()
}
#PostConstruct
fun onPostConstruct() {
injectCfMetadata()
}
fun injectCfMetadata() {
if(this.cloudFoundryApplicationProperties == null) {
log.error("Cloudfoundry Properties not set")
return
}
if(this.applicationInfoManager == null) {
log.error("ApplicationInfoManager is null")
return
}
val map = applicationInfoManager.info.metadata
map.put("applicationId", this.cloudFoundryApplicationProperties.applicationId)
map.put("instanceId", this.cloudFoundryApplicationProperties.instanceIndex)
}
}
CloudFoundryApplicationProperties.kt
#ConfigurationProperties("vcap.application")
class CloudFoundryApplicationProperties {
var applicationId: String? = null
var instanceIndex: String? = null
var uris: List<String> = ArrayList()
}

How to access the keyspace notifications with ServiceStack.redis

I am trying to access the keyspace notifications in a .Net Application using ServiceStack.Redis. I am new to Redis.
I enabled event notifications on cache by command:
CONFIG SET notify-keyspace-events KEs
I am subscribing to the channel "key*:*" in .Net. The following is my code:
const string ChannelName = "__key*__:*";
using (var redisConsumer = new RedisClient("localhost:6379"))
using (var subscription = redisConsumer.CreateSubscription())
{
subscription.OnSubscribe = channel =>
{
Console.WriteLine(String.Format("Subscribed to '{0}'", channel));
};
subscription.OnUnSubscribe = channel =>
{
Console.WriteLine(String.Format("UnSubscribed from '{0}'", channel));
};
subscription.OnMessage = (channel, msg) =>
{
Console.WriteLine(String.Format("Received '{0}' from channel '{1}'",
msg, channel));
};
Console.WriteLine(String.Format("Started Listening On '{0}'", ChannelName));
subscription.SubscribeToChannels(ChannelName); //blocking
}
From another .Net application, I am adding new data to the cache. I am expecting to receive an event (in OnMessage). The application is not capturing any event, on adding a new item in cache.
But, when I run the command "psubscribe 'key*:*'" on redis-cli.exe, it captures the events properly. ( when I add a new item to cache, it displays the event details in a console window.)
I am unable to capture the same in my application. Am I missing anything here?
use subscription.SubscribeToChannelsMatching(ChannelName);

how to see properties of a JmDNS service in reciever side?

One way of creating JmDNS services is :
ServiceInfo.create(type, name, port, weight, priority, props);
where props is a Map which describes some propeties of the service. Does anybody have an example illustrating the use of theese properties, for instance how to use them in the reciever part.
I've tried :
Hashtable<String,String> settings = new Hashtable<String,String>();
settings.put("host", "hhgh");
settings.put("web_port", "hdhr");
settings.put("secure_web_port", "dfhdyhdh");
ServiceInfo info = ServiceInfo.create("_workstation._tcp.local.", "service6", 80, 0, 0, true, settings);
but, then in a machine receiving this service, what can I do to see those properties?
I would apreciate any help...
ServiceInfo info = jmDNS.getServiceInfo(serviceEvent.getType(), serviceEvent.getName());
Enumeration<String> ps = info.getPropertyNames();
while (ps.hasMoreElements()) {
String key = ps.nextElement();
String value = info.getPropertyString(key);
System.out.println(key + " " + value);
}
It has been a while since this was asked but I had the same question. One problem with the original question is that the host and ports should not be put into the text field, and in this case there should actually be two service types one secure and one insecure (or perhaps make use of subtypes).
Here is an incomplete example that gets a list of running workstation services:
ServiceInfo[] serviceInfoList = jmdns.list("_workstation._tcp.local.");
if(serviceInfoList != null) {
for (int index = 0; index < serviceInfoList.length; index++) {
int port = serviceInfoList[index].getPort();
int priority = serviceInfoList[index].getPriority();
int weight = serviceInfoList[index].getWeight();
InetAddress address = serviceInfoList[index].getInetAddresses()[0];
String someProperty = serviceInfoList[index].getPropertyString("someproperty");
// Build a UI or use some logic to decide if this service provider is the
// one you want to use based on prority, properties, etc.
...
}
}
Due to the way that JmDNS is implemented the first call to list() on a given type is slow (several seconds) but subsequent calls will be pretty fast. Providers of services can change the properties by calling info.setText(settings) and the changes will be propagated out to the listeners automatically.