Is there any other way to get a number of requests waiting to be processed by worker threads when deploying an HTTP server as worker verticle? I need an alternative for https://vertx.io/docs/vertx-dropwizard-metrics/java/#_pool_metrics.
You can try to utilize Asynchronous Counters which you can increment whenever you send an event to specific address on event bus and then decrement when verticle is done processing (or have just picked up an event). If you have lot of verticles and don't want to modify each of them, you can set outbound interceptor:
vertx.eventBus().addOutboundInterceptor(deliveryContext -> {
//you can validate if the address is what you are looking for
if (deliveryContext.message().address().equalsIgnoreCase("http event")) {
//increment counter
}
deliveryContext.next();
});
if you have a lot of addresses to cover you can always add some specific header to the message and then look for it in the interceptor but that would mean you have to modify each .send() call in worst case scenario:
vertx.eventBus().addOutboundInterceptor(deliveryContext -> {
//looking for specific header
if (deliveryContext.message().headers().contains("incrementCounterHeader")) {
//increment counter
}
deliveryContext.next();
});
//later in code
vertx.eventBus().send("http event", message,
new DeliveryOptions().addHeader("incrementCounterHeader", "somevalue"));
last but not least if you decide to use async counter you might want to propagate message only .onComplete() or .onSuccess(), depends on your business logic.
Hope this will help!
Related
What is the recommended way in vert.x to write an Asynchronous request handler?
In this service, a request processing typically involves calling DB, calling external services, etc. I do not want to block the request handling thread however. What is the recommended way to achieve this using vet.x? In a typical asynchronous processing chain, I would use the request handling thread to emit a message to the message bus with the request object. Another handler will pick this message and do some processing such as checking request params. This handler can then emit a new message to the bus which can be picked up by the next handler which will do a remote call. This handler emits a new message with the result of the call which can be picked up by the next handler which will do error checking etc. Next handler would be responsible for creating the response and sending it to the client.
How one can create a similar pipeline using vert.x?
Everything, comprising request handlers for HttpServer, is asynchronous, isn't it?
var server = vertx.createHttpServer(HttpServerOptions())
server.requestHandler { req ->
req.setExpectMultipart(true) // for handling forms
var totalBuffer = Buffer.buffer()
req.handler { buff -> b.appendBuffer(buff) }
.endHandler { // the body has now been fully read
var formAttributes = request.formAttributes()
req.response().putHeader("Content-type","text/html");
req.response().end("Hello HTTP!");
}
// the above is so common that Vertx provides: bodyHandler{totalbuff->..}
}.listen(8080, "127.0.0.1", { res -> if(res.succeeded()) ... });
You just need to (end) write on req.response() on your final handler of your pipeline.
For a more stream-like implementation (i.e., not callback-based), you may use Vert.x Rx/ReactiveStreams API. E.g., you may use Vert.x Web Client for making requests, possibly using its Rx-fied API.
I want to refresh data every 15 seconds from an API using Reactive Cocoa 4. Since more than one subscriber can ask for this data at the same time, I want to have multiple subscribers to share one source of data.
My current approach is to have one Signal and share it to every instance that asks for the data. This Signal should start refreshing as soon as the first Signal is subscribed and end after the last has disposed.
SignalProducer<String, NoError> { observer, disposable in
self.disposable = self.repeatTimer.observeNext { _ in
NSLog("start network request")
observer.sendNext("result")
}
}.on(disposed: {
NSLog("disposed")
}).startWithSignal { signal, disposable1 in
self.updateSignal = signal
}
}
return (updateSignal, disposable!)
So for the first request I create and store the updateSignal and each following request will get that signal.
My first question: How can I know when the last subscriber disposed its signal? So when can I stop the requests?
My second question: I store the disposable from my repeatin network request in self.disposable which I also return to the subscriber. If the subscriber only disposes its Signal (which he got from Signal.observeNext()) the inner loop, where I log "start network request" is running endless. Do I really need to stop that Signal myself even when the outer Signal gets disposed?
Is there any nicer way or pattern for shared repeating requests?
Use the global timer function to perform work at specified intervals.
You could do something like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.startWithNext { _ in
//start network request here
}
But it's better if you chain your network request producer and observe the results, like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.flatMap(.Latest, transform { _ in
return self.networkRequestSignalProducer()
})
.start({ event in
//monitor the result of the network request
})
Note that you may not want to use the main queue like I did in this example, depending on how you've implemented your network requests.
If you want to avoid dealing with disposables, you can add a .takeUntil before .flatMap and terminate the timer with a signal
I have a ServiceWorker registered on my page and want to pass some data to it so it can be stored in an IndexedDB and used later for network requests (it's an access token).
Is the correct thing just to use network requests and catch them on the SW side using fetch, or is there something more clever?
Note for future readers wondering similar things to me:
Setting properties on the SW registration object, e.g. setting self.registration.foo to a function within the service worker and doing the following in the page:
navigator.serviceWorker.getRegistration().then(function(reg) { reg.foo; })
Results in TypeError: reg.foo is not a function. I presume this is something to do with the lifecycle of a ServiceWorker meaning you can't modify it and expect those modification to be accessible in the future, so any interface with a SW likely has to be postMessage style, so perhaps just using fetch is the best way to go...?
So it turns out that you can't actually call a method within a SW from your app (due to lifecycle issues), so you have to use a postMessage API to pass serialized JSON messages around (so no passing callbacks etc).
You can send a message to the controlling SW with the following app code:
navigator.serviceWorker.controller.postMessage({'hello': 'world'})
Combined with the following in the SW code:
self.addEventListener('message', function (evt) {
console.log('postMessage received', evt.data);
})
Which results in the following in my SW's console:
postMessage received Object {hello: "world"}
So by passing in a message (JS object) which indicates the function and arguments I want to call my event listener can receive it and call the right function in the SW. To return a result to the app code you will need to also pass a port of a MessageChannel in to the SW and then respond via postMessage, for example in the app you'd create and send over a MessageChannel with the data:
var messageChannel = new MessageChannel();
messageChannel.port1.onmessage = function(event) {
console.log(event.data);
};
// This sends the message data as well as transferring messageChannel.port2 to the service worker.
// The service worker can then use the transferred port to reply via postMessage(), which
// will in turn trigger the onmessage handler on messageChannel.port1.
// See https://html.spec.whatwg.org/multipage/workers.html#dom-worker-postmessage
navigator.serviceWorker.controller.postMessage(message, [messageChannel.port2]);
and then you can respond via it in your Service Worker within the message handler:
evt.ports[0].postMessage({'hello': 'world'});
To pass data to your service worker, the above mentioned is a good way. But in case, if someone is still having a hard time implementing that, there is an other hack around for that,
1 - append your data to get parameter while you load service-worker (for eg., from sw.js -> sw.js?a=x&b=y&c=z)
2- Now in service worker, fetch those data using self.self.location.search.
Note, this will be beneficial only if the data you pass do not change for a particular client very often, other wise it will keep changing the loading url of service worker for that particular client and every time the client reloads or revisits, new service worker is installed.
I am implementing a Service Bus and having a look at MassTransit. My pattern is not Publish/Subscribe but Sender/Receiver where the Receiver can be offline and came back online later.
Right now I am starting to write my tests to verify that MassTransit succesfully deliver the message using the following code:
bus = ServiceBusFactory.New(sbc =>
{
sbc.UseMsmq(
cfg =>
{
cfg.Configurator.UseJsonSerializer();
cfg.Configurator.ReceiveFrom("msmq://localhost/my_queue");
cfg.VerifyMsmqConfiguration();
});
});
Then I grab the bus and publish a message like this:
bus.Publish<TMessage>(message);
As I can notice from MSMQ, two queues are created and the message is sent cause Mass Transit does not raise any error but I cannot find any message in the queue container.
What am I doing wrong?
Update
Reading the Mass Transit newsgroup I found out that in a scenario of Sender/Receiver where the receiver can come online at any time later, the message can be Send using this code:
bus.GetEndpoint(new Uri("msmq://localhost/my_queue")).Send<TMessage>(message);
Again in my scenario I am not writing a Publisher/Subscriber but a Sender/Receiver.
First, to send, you can use a simple EndpointCacheFactory instead of a ServiceBusFactory...
var cache = EndpointCacheFactory.New(x => x.UseMsmq());
From the cache, you can retrieve an endpoint by address:
var endpoint = cache.GetEndpoint("msmq://localhost/queue_name");
Then, you can use the endpoint to send a message:
endpoint.Send(new MyMessage());
To receive, you would create a bus instance as you specified above:
var bus = ServiceBusFactory.New(x =>
{
x.UseMsmq();
x.ReceiveFrom("msmq://localhost/queue_name");
x.Subscribe(s => s.Handler<MyMessage>(x => {});
});
Once your receiver process is complete, call Dispose on the IServiceBus instance. Once your publisher is shutting down, call Dispose on the IEndpointCache instance.
Do not dispose of the individual endpoints (IEndpoint) instances, the cache keeps them available for later use until it is disposed.
I want to fetch two XML documents from the server and resume processing when both have arrived. Can I fetch them in parallel, or do I have to refrain from issuing the second request until the first has completed?
You can fetch them in parallel, but keep in mind that browsers have a limit on the number of parallel requests, see http://www.browserscope.org/?category=network (choose "Major Versions" in the dropdown on the top left to see more versions). Note especially, that IE < 8 has a limit of 2 connections per hostname!
If you still want to do this, then note that the responses can arrive in any order. So you'll have to implement something that will keep track of the requests/responses (a counter or something more sophisticated), so that you'll know when all responses you need have arrived.
The best solution is often to send just one request that asks for both XML documents, and the server returns them both at once in one response.
Make both requests, then check when either one completes whether the other is done, and continue if it is.
private String responseOne;
private String responseTwo;
public startRequests() {
makeAsyncRequestOne(new AsyncCallback<String>() {
onSuccess(String response) {
this.responseOne = response;
if (responseTwo != null) {
proceed();
}
}
});
makeAsyncRequestTwo(new AsyncCallback<String>() {
onSuccess(String response) {
this.responseTwo = response;
if (responseOne != null) {
proceed();
}
}
});
}
As Chris points out, this may hit a ceiling on maximum concurrent requests to the same hostname, so if you have lots of requests to send at once, you could keep a queue of requests and call the next one in proceed() until the queue is exhausted.
But if you plan on having a lot of concurrent requests, you probably need to redesign your service anyway, to batch operations together.