In my service fabric environment, I would like to send data to a specified singleton instance. If I use the create service proxy function I can only use the logical url like this:
var client = ServiceProxy.Create<IServiceInterface>(new Uri(“fabric:/platform/service”), ServicePartitionKey.Singleton, TargetReplicaSelector.RandomInstance);
TargetReplicaSelector gives me only the RandomInstanc option. Random Instance is really not working for me. I Can easily lookup all available singleton instances in my environment with:
var resolver = ServicePartitionResolver.GetDefault(); var partition = await resolver.ResolveAsync(new Uri(“fabric:/platform/service”), ServicePartitionKey.Singleton, new CancellationToken()); var endpoints
= partition.Endpoints;
but I cannot create a ServiceProxy from the endpoint data I receive.
You could use HTTP communication for that.
Have a look at this thread: Targeting a stateless replica on the same node when communicating over the ServiceProxy client
Related
Are there any recommendations on querying remote state stores between application instances that are deployed in Kubernetes? Our application instances are deployed with 2 or more replicas.
Based on documentation
https://kafka.apache.org/10/documentation/streams/developer-guide/interactive-queries.html#id7
streams.allMetadataForStore("word-count")
.stream()
.map(streamsMetadata -> {
// Construct the (fictituous) full endpoint URL to query the current remote application instance
String url = "http://" + streamsMetadata.host() + ":" + streamsMetadata.port() + "/word-count/alice";
// Read and return the count for 'alice', if any.
return http.getLong(url);
})
.filter(s -> s != null)
.findFirst();
will streamsMetadata.host() result in the POD IP? And if it does, will the call from this pod to another be allowed? Is this the correct approach?
streamsMetadata.host()
This method returns whatever you configured via application.server configuration parameter. I.e., each application instance (in your case each POD), must set this config to provide the information how it is reachable (e.g., its IP and port). Kafka Streams distributes this information for you to all application instances.
You also need to configure your PODs accordingly to allow sending/receiving query request via the specified port. This part is additional code you need to write yourself, i.e., some kind of "query routing layer". Kafka Streams has only built-in support to query local state and to distribute the metadata about which state is hosted where; but there is no built-in remove query support.
An example implementation (WordCountInteractiveQueries) of a query routing layer can be found on Github: https://github.com/confluentinc/kafka-streams-examples
I would also recommend to checkout the docs and blog post:
https://docs.confluent.io/current/streams/developer-guide/interactive-queries.html
https://www.confluent.io/blog/unifying-stream-processing-and-interactive-queries-in-apache-kafka/
I develop an API server with Hapi. I use #hapi/lab for testing. I have different test files for different API modules/routes.
There is a server initialization in each test file. A DB connection is created when the server is initialized, consequently, multiple DB connections are created at the same time when I try to test the server. And I got warning like that:
WARNING: Creating a duplicate database object for the same connection.
at Object.register (/home/.../node_modules/hapi-pg-promise/index.js:19:20)
at internals.Server.register (/home/.../node_modules/#hapi/hapi/lib/server.js:453:35)
at async Object.exports.compose (/home/.../node_modules/#hapi/glue/lib/index.js:46:9)
at async createServer (/home/.../server.js:10:115)
at async Immediate.<anonymous> (/home/.../node_modules/#hapi/lab/lib/runner.js:662:17)
So, is there an approach, how to test a Hapi server in multiple files without multiple server connections?
You will have to lazy load the servr. let's assume you have a function serverFactory which returns the server. you could do this
let serverFactoryObj;
function getServerFactory() {
if (!serverFactoryObj) {
serverFactoryObj = serverFactory();
}
return serverFactoryObj;
}
This way, no matter how many times you test the endpoints, you will always have a single instance of server. Ideally you should always test with single instance rather than creating/stopping server for each test.
I have three reliable collections defined in my StatefulService defined this way:
this.dataCount =
await StateManager.GetOrAddAsync<IReliableDictionary<string, int>>("count");
this.dataDictionary1 =
await StateManager
.GetOrAddAsync<IReliableDictionary<string, ResourceElement>>
("data1");
this.dataDictionary2 =
await StateManager
.GetOrAddAsync<IReliableDictionary<string, ResourceElement>>
("data2");
Now I would like to move collection data2 (its contents) to a separate StatefulService and I am not sure how to proceed.
Ideally, I am looking for a mechanism that would allow me to backup data2 (i don't mind if I have to add a method to my service) to a file and then resume the backup from that file into a different service.
Is there anything like this available?
Backups work at the service partition level. The only way to create a backup of a single collection, is to have a stateful service with 1 partition that holds only that collection.
What you can do, is restore backup(s) of your old service (service 1) to a new service (service 2), that you create just for the purpose of moving data from the data2 collection to another service (service 3). After that you delete service 2 and continue running service 3.
Hi I am here using jersey.1.19.1 API Client for Rest full web service.
I came to know that creating an instance of Client is an expensive.
As I am creating instance of it every time i call the web service,
response is delayed which it leads affecting the performance.
So is there any other way in creating instance of Client for multiple request.
and also how to over come in delay even for creating single instance of a Client
Is it a right approach to create a Client object pooling as that of connection pooling or one Client object for one user. Even is there any other best way for creating Client object
As there is no much info provided to have a clue on your structure, something along these lines will help
Create an instance variable of the jersey client
private Client client = null;
In a method that returns client, check if that field is null, if it is initialise it, otherwise return same instance
if(client == null)
client = ClientBuilder.newClient();
return client;
The objective that I have is to run multiple applications with some metadata embedded into applications/services so that I could query applications/services using the metadata. Is this possible?
I was looking at the following post and the answer hints at this possibility, but no specific details on how to achieve the result.
The primary piece of "metadata" you get is the service/application instance name. That's what I talked about in my other post. The way that works is by creating each service/application instance with a name that contains some information clients can use when resolving them. Clients can then query Service Fabric for named application/service instances and connect to a specific one. A service/application instance name is URI, so you can use a path hierarchy to categorize information.
Continuing with the audio/video example: Let's extend that example so we have an application that can perform specific tasks for specific media formats for audio or video. Each combination of task + media format is a unique named service instance, resulting in a deployment that looks something like this:
Application:
fabric:/avapp
Services:
fabric:/avapp/video/encoding/mp4
fabric:/avapp/video/encoding/h264
fabric:/avapp/video/captioning/english
fabric:/avapp/video/captioning/czech
fabric:/avapp/audio/encoding/aac
fabric:/avapp/audio/encoding/mp3
etc.
Now clients can query Service Fabric to discover what services are available:
FabricClient fabricClient = new FabricClient();
System.Fabric.Query.ServiceList services = await fabricClient.QueryManager.GetServiceListAsync(new Uri("fabric:/avapp"));
Then you can simply query the list of services with LINQ. For example, if I want to see all services that do video encoding:
services.Where(x => x.ServiceName.AbsolutePath.Contains("video/encoding"));
And then you can resolve an address for a specific service to connect to it:
ServicePartitionResolver resolver = ServicePartitionResolver.GetDefault();
ResolvedServicePartition servicePartition = await resolver.ResolveAsync(new Uri("fabric:/avapp/video/encoding/h264"), new ServicePartitionKey(1), cancellationToken);
ResolvedServiceEndpoint endpoint = servicePartition.GetEndpoint();
There's a bit more to the address resolution part (see here), but that's the general idea.
Application instances also allow you to set custom application parameters (key-value pairs) that can be set per instance at creation time. They don't show up in the application name, but you get that information back when you ask Service Fabric for a list of running application instances. That can potentially also be used as metadata by clients when they need to decide what application to connect to.
Update: More info on application instance parameters:
When you create a new application instance you can supply a set of key-value pairs in the application description. Then when you query Service Fabric for application instances you get back a list of Application result objects that have said parameters. This also shows up in Visual Studio, in your application project, where you have environment-specific application parameter files. Visual Studio extracts those key-value pairs from the XML files and uses them in the application description when it creates an instance of your application.