I have come across a problem a few times now and I can't really figure out how to word it to get the answer I'm looking for on Google/SA so here goes.
I have multiple http observables. Lets say I want to delete object 1, 2 and 3. I send three http delete requests to myapi/{id} one after the other. I can subscribe to each request as each returns an Observable. This allows me to do something after each has completed or failed with myHttpObservable.subscribe(successFn, failFn).
Heres where the problem comes in. I now want to know when all three delete requests have finished. To do this I can use forkJoin([deleteRequest1, deleteRequest2, deleteRequest3] but forkJoin takes an array of Observables and subscribes to them. I cannot subscribe to them individually now since forkJoin wants an array or Observables.
So how do I subscribe to each Observable with its own success and failure functions but also know when all three are done?
The forkJoin will return the response of each observable in the same order your provided them. So for example, you have three observables
const source1$ = of("Observable 1");
const source2$ = of("Observable 2");
const source3$ = of("Observable 3");
You'll get the subscription of each in forkJoin like this
forkJoin([source1$, source2$, source3$]).subscribe(([res1, res2, res3]) => {
console.log(res1);
console.log(res2);
console.log(res3);
});
Here is the sample.
Thanks.
Related
I'm building a PWA application, and I have used workbox registerroute for a few api endpoints, as well as an explicit service worker fetch event listener. During the debugging on some caching issues, I've noticed that these two seems to interfere with each other. Specifically sometimes the fetch handler is not triggered - which causes me trouble on debugging - I'm assuming this is due to the registerroute caching policy I have set via workbox.
My question is that, can I only pick one or the other, instead of having both fetch handler and registerroute? In my case, I needed fetch handler to deal with some advanced caching related to POST requests. So I think if I can only pick one, I'll have to stick with the fetch handler.
First, here's some background information about what happens when there's multiple fetch event handlers in the active service worker.
With that background info in mind, there are a few approaches for accomplishing what you're describing.
Option 1a: Register your own fetch event handler first
As long as you register your own fetch handler first, before any calls to Workbox's registerRoute(), it's guaranteed to have the "first shot" at responding the incoming fetch event.
The thing to keep in mind is that your own fetch handler needs to make a synchronous decision about whether or not to call event.respondWith(), and when you do call event.respondWith(), then Workbox's routes will not get used to respond to a given request.
So, you could do the following:
self.addEventListener('fetch', (event) => {
// Alternatively, check event.request.headers,
// or some other synchronous criteria.
if (event.request.url.endsWith('.json')) {
event.respondWith(customResponseLogic(event));
}
});
// Then, include any Workbox-specific routes you want.
registerRoute(
({request}) => request.destination === 'image',
new CacheFirst()
);
// The default handler will only apply if your own
// fetch handler didn't respond.
registerDefaultHandler(new StaleWhileRevalidate());
Option 1b: Ensure Workbox routes won't match
This is similar to 1a, but the main thing is to make sure that you don't have a "catch-all" route that will match all requests, and that you don't use registerDefaultHandler().
Assuming your Workbox routes just match a specific set of well-defined criteria, and don't match any of the requests that you want to respond to in your own handler, it shouldn't matter how you order them:
// Because this will only match image requests, it doesn't
// matter if it's listed first.
registerRoute(
({request}) => request.destination === 'image',
new CacheFirst()
);
self.addEventListener('fetch', (event) => {
// Alternatively, check event.request.headers,
// or some other synchronous criteria.
if (event.request.url.endsWith('.json')) {
event.respondWith(customResponseLogic(event));
}
});
(What's going on "under the hood" is that if there isn't a Route whose synchronous matchHandler returns a truthy value, Workbox's Router won't call event.respondWith().)
Option 2: Use custom handler logic
It should be viable to use Workbox to handle all your routing, and run your custom response generation code in either a handlerCallback (more straightforward) or a custom subclass of the Strategy base class (more reusable, but overkill for simple use cases).
The one thing to keep in mind is that if you're dealing with POST requests, you need to explicitly tell registerRoute() to respond to them, by passing in 'POST' as the (optional) third parameter.
Here's an example of how you could do this, assuming as before that you custom logic is defined in a customResponseLogic() function:
registerRoute(
({request}) => request.destination === 'image',
new CacheFirst()
);
registerRoute(
// Swap this out for whatever criteria you need.
({url}) => url.pathname.endsWith('.json'),
// As before, this assumes that customResponseLogic()
// takes a FetchEvent and returns a Promise for a Response.
({event}) => customResponseLogic(event),
// Make sure you include 'POST' here!
'POST'
);
Is there any other way to get a number of requests waiting to be processed by worker threads when deploying an HTTP server as worker verticle? I need an alternative for https://vertx.io/docs/vertx-dropwizard-metrics/java/#_pool_metrics.
You can try to utilize Asynchronous Counters which you can increment whenever you send an event to specific address on event bus and then decrement when verticle is done processing (or have just picked up an event). If you have lot of verticles and don't want to modify each of them, you can set outbound interceptor:
vertx.eventBus().addOutboundInterceptor(deliveryContext -> {
//you can validate if the address is what you are looking for
if (deliveryContext.message().address().equalsIgnoreCase("http event")) {
//increment counter
}
deliveryContext.next();
});
if you have a lot of addresses to cover you can always add some specific header to the message and then look for it in the interceptor but that would mean you have to modify each .send() call in worst case scenario:
vertx.eventBus().addOutboundInterceptor(deliveryContext -> {
//looking for specific header
if (deliveryContext.message().headers().contains("incrementCounterHeader")) {
//increment counter
}
deliveryContext.next();
});
//later in code
vertx.eventBus().send("http event", message,
new DeliveryOptions().addHeader("incrementCounterHeader", "somevalue"));
last but not least if you decide to use async counter you might want to propagate message only .onComplete() or .onSuccess(), depends on your business logic.
Hope this will help!
I am building a bot for for Facebook Messenger using Microsoft Bot Framework. I am planning to use CosmosDB for State Management and also as my backend data store. (I am not stuck to CosmosBD and can use any other store if needed)
I need to send daily/weekly proactive messages(push notifications) to users based on their time preference. I will capturing their time preference when they first interact with the bot.
What is the best way to deliver these notifications?
As I will be storing these preferences in CosmosDB, I am thinking using ComosDB trigger of creating an Azure Function and schedule it based on the user time preference. This Azure function will make a call to my webhook which will deliver these messages. If requried, I will change Function schedule when a user changes his/her preference.
My questions are:
Is this a good approach?
Are there any other alternatives (Notifications Hub?)
I should be able to set specific times for notifications (like at the top of the hour or something like that), does it make sense to schedule an Azure Function to run at these hours rather than creating a function based on user preference (I can actually combine these two approaches too)
Thank you in advance.
First, I don't think there's any "right" answer to be given here; it's going to depend a lot on your domain's specific needs. Scale is going to play a major factor in the design of this. Will you have 100 users? 10000 users? 1mil users? I'm going to assume you want to design for maximum scale up front, but it could be overkill.
First, based on what you've described, I don't think a CosmosDB trigger is necessarily the solution to your problem because that's only going to fire when the preference data is created/updated. I assume that, from that point forward, your function needs to continuously fire at the time slot they've opted into, correct?
So let's pretend you let people choose from the 24hrs in the day. A naïve approach would be to simply use a scheduled trigger that fires up every hour, queries the CosmosDB for all the documents where the preference is set to that particular hour and then begins sending out notifications from there. The problem is how you scale from there and deal with issues of idempotency in the face of failures.
First off, a timer trigger only ever spins up one instance. If you were to just go query the CosmosDB documents and start processing them one by one in the scope of that single trigger, you'd hit a ceiling relatively quickly on how many notifications you can scale to. Instead what you'd want to do is use that timer trigger to fan out the notifications to as many "worker" function instances as possible. The timer trigger can act as the orchestrator in the sense that it can own the query against the CosmosDB and then turn each document result it finds for that particular notification time window into a message that it places on a queue to be processed by a separate function which will scale out on its own.
There are actually a couple ways you can accomplish this with Azure Functions, it really depends on how early an adopter of technology you are comfortable with being.
The first is what I would call the "manual" way which would be done by simply using the existing Azure Storage Queue extension by taking an IAsyncCollector<YourNotificationWorkerMessage> as a parameter to the timer function that's bound to the worker queue and then pumping out the messages through that. Then you write a second companion function which uses a QueueTrigger, bind it to that same queue, and it will take care of processing each message. This second function is where you get the scaling, enabling process all of the queued messages as quickly as possible based on whatever scaling parameters you choose to configure. This is the "simplest" approach
The second approach would be to adopt the newer Durable Functions extension. With that model, you don't have to directly think about creating a worker queue. You simply kick off a new instance of your orchestrator function from the timer function and the orchestrator fans out the work by invoking N "concurrent" calls to an action for each notification. Now, it happens to distribute those calls using queues under the covers, but that's an implementation detail that you need no longer maintain yourself. Additionally, if the work of delivering the notification requires more involved work and/or retry logic, you might actually consider using a sub-orchestration instead of a simple action. Finally, another added benefit of this approach, is that you can "fan back in" to your main orchestrator function once all the notifications are delivered to do some follow up work... even if that's simply some kind of event logging that the notification cycle has completed for this hour.
Now, the challenge with either of these approach is actually dealing with failure in initially fetching the candidates for notification from CosmosDB, paging through the results and making sure you actually fan all of them out in an idempotent manner. You need to deal with possible hiccups as you page and you need to deal with the fact that your whole function could be torn down and you might have to restart. Perhaps on the initial run of the 8AM notifications you got through page 273 out of 371 pages and then you got hit with a complete network connectivity fail or the VM your function was running on suffered a power failure. You could resume, but you'd need to know that you left off on page 273 and that you actually processed the 27th record out of that page and start from there. Otherwise, you risk sending double notifications to your users. Maybe that's something you can accept, maybe it's not. Maybe you're ok with the 27 notifications on that page being duplicated as long as the first 272 pages aren't. Again, this is something you need to decide for your domain, but if you want to avoid this issue your orchestrator function will need to track its progress to ensure that it doesn't send out dupes. Again I would say Durable Functions has a leg up here as it comes with the ability to configure retries. Maintaining the state of a particular run is left up to the author in either approach though.
I use pro-active dialog extensively with botframwork and messenger without any issue. During your facebook approval process you simply need to inform them you will be sending notifications trough messenger with your bot. Usually if you use it to inform your user and stay away from promotional content you should be fine.
I also use azure function to trigger the pro-active dialog from a custom controller endpoint.
Bellow sample code for azure function:
public static void Run(TimerInfo notificationTrigger, TraceWriter log)
{
try
{
//Serialize request object
string timerInfo = JsonConvert.SerializeObject(notificationTrigger);
//Create a request for bot service with security token
HttpRequestMessage hrm = new HttpRequestMessage()
{
Method = HttpMethod.Post,
RequestUri = new Uri(NotificationEndPointUrl),
Content = new StringContent(timerInfo, Encoding.UTF8, "application/json")
};
hrm.Headers.Add("Authorization", NotificationApiKey);
log.Info(JsonConvert.SerializeObject(hrm));
//Call service
using (var client = new HttpClient())
{
Task task = client.SendAsync(hrm).ContinueWith((taskResponse) =>
{
HttpResponseMessage result = taskResponse.Result;
var jsonString = result.Content.ReadAsStringAsync();
jsonString.Wait();
if (result.StatusCode != System.Net.HttpStatusCode.OK)
{
//Throw what ever problem as an exception with details
throw new Exception($"AzureFunction - ERRROR - HTTP {result.StatusCode}");
}
});
task.Wait();
}
}
catch (Exception ex)
{
//TODO log
}
}
Bellow sample code for starting the pro-active dialog:
public static async Task Resume<T, R>(string resumptionCookie) where T : IDialog<R>, new()
{
//Deserialize reference to conversation
ConversationReference conversationReference = JsonConvert.DeserializeObject<ConversationReference>(resumptionCookie);
//Generate message from bot to user
var message = conversationReference.GetPostToBotMessage();
var builder = new ContainerBuilder();
using (var scope = DialogModule.BeginLifetimeScope(Conversation.Container, message))
{
//From a cold start the service is not yet authenticated with dev bot azure services
//We thus must trust endpoint url.
if (!MicrosoftAppCredentials.IsTrustedServiceUrl(message.ServiceUrl))
{
MicrosoftAppCredentials.TrustServiceUrl(message.ServiceUrl, DateTime.MaxValue);
}
var botData = scope.Resolve<IBotData>();
await botData.LoadAsync(CancellationToken.None);
//This is our dialog stack
var task = scope.Resolve<IDialogTask>();
T dialog = scope.Resolve<T>(); //Resolve the dialog using autofac
try
{
task.Call(dialog.Void<R, IMessageActivity>(), null);
await task.PollAsync(CancellationToken.None);
}
catch (Exception ex)
{
//TODO log
}
finally
{
//flush dialog stack
await botData.FlushAsync(CancellationToken.None);
}
}
}
Your dialog needs to be registered in autofac.
Your resumptionCookie needs to be saved in your db.
You might want to check FB policy regarding proactive messages
There’s a 24h limit but it might not be totally screwed in your case
https://developers.facebook.com/docs/messenger-platform/policy/policy-overview#standard_messaging
I want to refresh data every 15 seconds from an API using Reactive Cocoa 4. Since more than one subscriber can ask for this data at the same time, I want to have multiple subscribers to share one source of data.
My current approach is to have one Signal and share it to every instance that asks for the data. This Signal should start refreshing as soon as the first Signal is subscribed and end after the last has disposed.
SignalProducer<String, NoError> { observer, disposable in
self.disposable = self.repeatTimer.observeNext { _ in
NSLog("start network request")
observer.sendNext("result")
}
}.on(disposed: {
NSLog("disposed")
}).startWithSignal { signal, disposable1 in
self.updateSignal = signal
}
}
return (updateSignal, disposable!)
So for the first request I create and store the updateSignal and each following request will get that signal.
My first question: How can I know when the last subscriber disposed its signal? So when can I stop the requests?
My second question: I store the disposable from my repeatin network request in self.disposable which I also return to the subscriber. If the subscriber only disposes its Signal (which he got from Signal.observeNext()) the inner loop, where I log "start network request" is running endless. Do I really need to stop that Signal myself even when the outer Signal gets disposed?
Is there any nicer way or pattern for shared repeating requests?
Use the global timer function to perform work at specified intervals.
You could do something like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.startWithNext { _ in
//start network request here
}
But it's better if you chain your network request producer and observe the results, like this:
self.disposable =
timer(SomeTimeInterval onScheduler:QueueScheduler.mainQueueScheduler)
.flatMap(.Latest, transform { _ in
return self.networkRequestSignalProducer()
})
.start({ event in
//monitor the result of the network request
})
Note that you may not want to use the main queue like I did in this example, depending on how you've implemented your network requests.
If you want to avoid dealing with disposables, you can add a .takeUntil before .flatMap and terminate the timer with a signal
Each Sails.js model has the method publishAdd(). This notifies every listener, when a new record was added to a associated model.
This notification does not contain the newly created record. So I have to start another request from the client side to get the new record.
Is there a possibility, that Sails.js sends the new record with the notification, so I can reduce my request count?
Solution
I realized the accepted answer like that:
https://gist.github.com/openscript/7016c5fd8c5053b5e3a3
There's no way to get this record using the default publishAdd method. However, you can override that method and do the child record lookup in your implementation.
You can override publishAdd on a per-model basis by adding a publishAdd method to that model class, or override it for all models by adding the method to the config/models.js file.
I would start by copying the default publishAdd() method and then tweaking as necessary.
I know this is old, but I just had to solve this again, and didn't like the idea of dropping in duplicate code so if someone is looking for alternative, the trick is to update the model of the newly created record with an afterCreate: method.
Say you have a Game that you want to your Players to subscribe to. Games have notifications, a collection of text alerts that you only want players in the game to receive. To do this, subscribe to Game on the client by requesting it. Here I'm getting a particular game by calling game/gameId, then building my page based on what notifications and players are already on the model:
io.socket.get('/game/'+gameId, function(resData, jwres) {
let players = resData.players;
let notifications = resData.notifications;
$.each(players, function (k,v) {
if(v.id!=playerId){
addPartyMember(v);
}
});
$.each(notifications, function (k,v) {
addNotification(v.text);
});
});
Subscribed to game will only give the id's, as we know, but when I add a notification, I have both the Game Id and the notification record, so I can add the following to the Notification model:
afterCreate: function (newlyCreatedRecord, cb) {
Game.publishAdd(newlyCreatedRecord.game,'notifications',newlyCreatedRecord);
cb();}
Since my original socket.get subscribes to a particular game, I can publish only to those subscriber by using Game.publishAdd(). Now back on the client side, listen for the data coming back:
io.socket.on('game', function (event) {
if (event.attribute == 'notifications') {
addNotification(event.added.text);
}
});
The incoming records will look something like this:
{"id":"59fdd1439aee4e031e61f91f",
"verb":"addedTo",
"attribute" :"notifications",
"addedId":"59fef31ba264a60e2a88e5c1",
"added":{"game":"59fdd1439aee4e031e61f91f",
"text":"some messages",
"createdAt":"2017-11-05T11:16:43.488Z",
"updatedAt":"2017-11-05T11:16:43.488Z",
"id":"59fef31ba264a60e2a88e5c1"}}