Setup:
I have setup a pubsub service wherein the publishers publish geolocation data at regular intervals.
The subscribers receive the location data of the publishers.
The subscribers are not presence subscribed, in the sense, the subscribers are not in the publishers rosters.
Problem:
The subscribers need to know the presence status of publishers.
Is there a way for the subscribers to know the presence status of publishers?
No, since there is no direct relationship between subscribers and publishers, which is typical of any pubsub design. To accomplish this the subscribers would need to know who the publishers are, which is not a good generic pubsub design.
It sounds like what you actually want is PEP (Personal Eventing Protocol), which is a subset of pubsub. In this case, the subscribers are subscribing to nodes belonging to the actual user they are interested in. If they are subscribed to the users presence, they automatically have access to the users nodes.
NOTE: I have recently found out that the newer version of the spec does in fact support an attribute that identifies the publisher. Thus making it feasible to get their presence, but you would still have to subscribe or query for it.
Related
I'm making a social media app like Facebook using Flutter and Firebase.
I'm using Firebase Cloud Messaging to make notification service.
I want to make users who joined community or groups to subscribe their group's topic. Therefore, I can send notification to them by using method which is "subscribeToTopic()". However, I don't know how to make all users in community or groups to subscribe to certain topics.
If you know how to make all users to subscribe to certain topics, please let me know. Thank you!
There is no API to get the current subscribers to a topic, nor is there an API that verbatim subscribes all subscribers to one topic to another topic.
If you already track group membership yourself, you can either let each client subscriber themselves to the additional topic, or you can determine the list of tokens for the group members and then subscribe them to the topic on the server.
Below is a typical GCP pubsub model:
My question: is it possible for one subscriber(application or job) to subscribe to multiple subscriptions? Like this:
I mean we can filter at the subscription level that one subscription takes one event type (A or B). I know it will be easier if we have two topics (Topic A and B) and create two subscriptions, but again, it will boil down to the same question, is it possible for one subscriber to subscribe to multiple subscriptions?
Or the only alternative way I can imagine is that at the subscriber level, I can classify the event type, A or B, but that requires the publisher to pass the attribute to the topic level.
I have the control of publisher and I just wanna do one Subscriber instead of multiple subscribers.
An application can subscribe to multiple subscriptions, yes. You would need to instantiate multiple instances of the subscriber client, one for each subscription for which you want to receive messages.
If you want the subscriber to be able to receive messages without knowing the names of all of the subscriptions, then you could use push subscriptions and set the endpoint to different subscriptions to the same URL. Then, the subscriber behind that URL would receive messages from the different subscriptions.
I am trying to implement queues into our microservice architecture, to be specific AWS SNS/SQS.
For example I have this scenarion.
After order is created Orders MS raises OrderCreated event and this event publishes message to AWS OrderCreated SNS. SQS queue InvoiceCreate is subscribed to OrderCreated SNS and will get this message.
Evertyhing makes scence so far. If Invoicing MS is listening to InvoiceCreate queueu and retrieves all new messages - Invoicing MS should create an invoice, but my question is with what data?
a) contact Order MS (to order data relevant for creating invoice). If unable to do so, message will be left in queue until Invoicing MS is able to collect the relevant data
b) message published should contain all the relevant data needed to create an invoice.
If choosing A Invoicing MS will not be decoupled and it will be depending on Order MS, but on the other hand it can collect additional data other then the data packed with original message.
If choosing B, since OrderCreated event and OrderCreated SNS doesnt really know who will use message data ie. OrderCreated could be also used to perform different actions, I am confused how to preciselly decide what data should be stuffed in this message
Our architecture is set up more like your option B. To use your example, the Order service would publish it's OrderCreated event and attach - as a payload - most (or even all) of the Order information in the Payload section of the message. We format message and payload as JSON for compatibility, but you can do whatever.
In some cases, we don't publish all info, just specific fields for the Added/Edited entity - it depends on the service and the sensitivity of the information. So long as you only ever add fields to a message (don't remove any), you are honoring the contract and aren't really tightly coupled to it.
Again, to your example, the InvoiceService could get its information from one or more of several options:
Pull it directly from the OrderCreated message if you include everything needed
Pull what it can from the OrderCreated message, publish an InvoiceStarted event that triggers the OrderService (and/or others) to send it an OrderInvoiceComplete message with the rest of the details it needs
Keep a local copy of whatever key data it needs - populated by subscribing to other events - so that it can combine OrderCreated data with some local data to flesh out an invoice
It's best to avoid the InvoiceService responding to a message by making a call directly back to the OrderService - this is a pretty tight coupling that can be avoided by simply messaging back if you have to.
So, there are lots of options. I personally prefer the technique of putting all data that might be useful into the messages when things are created/updated and letting consuming services decide what to use/ignore. For our scenario, that works well but we have only a few well-contained clients accessing our services so there may be more secure ways to do it that aren't relevant for us.
Problem :
We are trying to make a chat application using AWS product AppSync and we want to achive the best performance but we're facing problem with real time subscriptions in AppSync and Graphql where a single user will need to handle hundereds of subscription in some cases which we think is not the best solution, what do you suggest ?
Problem Example:
Mutation{
addMessage(conversation_id=Int!, content:String!) : Message
}
Subscription{
subscribeForNewMessages(convesration_id: Int!):Message
#aws_subscribe(mutations: ["addMessage"])
}
the problem with this design is that the user need to invoke this subscription and keep listening for every single conversation, which we expect to be overwheelming the client in case if the conversations quantity is huge.
Questions :
Q1 :
What we are striving to achieve is one subscription for multiple (conversation_id)s, how this will be possible?
These folks (https://github.com/apollographql/apollo-client/issues/2633) are talking about something similar, we tested it and it doesn't work, is it a valid solution?
Q2:
Regarding Amplify; Will amplify perform well when listening for hundereds of subscription simulanuosly? does it make some sort of merging subscription and websockets or it will deal them separately?
Q3:
what are your comments about these designs? where there will be a service that will braodcast(invoke mutations with clients ids) the messages for chat participants , and the client will subscribe only for a single channel . like the following:
src2 : AWS AppSync for chatting application
src2 : Subscribe to a List of Group / Private Chats in AWS AppSync
Q1/Q2
You'll have to make multiple subscriptions and the aws ios/android/amplify sdks can handle subscription handshake protocols for real-time updates to data.
Take a look here
Q3
I recommend allowing clients to subscribe to specific channels (even if that means multiple subscriptions) so that the filtering logic can be done in the service rather than client side, reducing client side code which also means you don't have to worry about maintenance or scalability.
When building a social application it's common to follow other users or topics as an indication of interest in updates by the user or topic. For example, following other users on Twitter, Friending other people on Facebook or liking a product or brand on Facebook.
Pusher has the concept of channels that you subscribe to. Channels are a human readable string that provide a logical identifier to information (e.g. "some-channel-name") and therefore seems to naturally suggest that in a social application any updates on a user or topic should be sent on a channel specific to that item (e.g. "userX-status-updates" or "myBrand-status-updates").
However, this raises concerns about how efficient it is to subscribe to multiple channels if a user is following a high number of other users or topic.
Therefore, what are the appropriate strategies for structuring channels in an social status update style application that uses Pusher?
The first thing to clarify is that you need a mapping of who you are following so for the purposes of this answer I'm going to assume that it's stored in a DB on the server. It also assumed that status updates are triggered as follows:
Client (userX posts status update) -> Your Server (sanitize & validate)
Your Server -> Pusher
Pusher -> Clients (users interested in updates from UserX)
There are two possible solutions to the channel information architecture problem:
Channel Per User Status: A user subscribes to a userX-status-updates channel for all the users that they follow and users trigger update events on their own status update channel.
Users I'm Following Channel: When a user posts a status update you look up who is following that user and publish the update on a users-you-follow-updates channel.
Strategy 1. is the most optimal solution as it keeps interactions with your own infrastructure an Pusher to a minimum.
Here's the detail on these two strategies:
1. Channel Per User Status
The assumption here is that subscribing to channels is costly but that not entirely correct. Channels are simply a way of routing events. However, if you are using authenticated channels (private & presence) you need to authenticate the subscription via your own server. If you use the Pusher WebSocket libraries "out of the box" each subscription will result in a request to your server. So, a user is following 1,000 users that's 1,000 requests to your server.
But, for the pusher-js library there is a multi-auth plugin that can batch the authentication requests into a single call.
There is also a BatchAuthorizer for the Pusher WebSocket Java library, but it's only a sample solution to this scenario.
2. Users I'm Following Channel
Note: although this is an option it's probably only appropriate for smaller numbers of users
In this scenario a user sends their status update to the server, the server performs a lookup of which users are interested in the update and triggers and update even on a channel for each interested user.
For example, give users UserA, UserB and UserC each of those users will subscribe to their own update channel; UserA-followers-updates, UserB-followers-updates, and UserC-followers-updates respectively. If each of these users follows UserZ then when UserZ makes as status update that update is published on each of those channels.
This may also sound inefficient, however it is possible to trigger the same event on 10 channels at a time. So in the above example it would only require one call to the Pusher HTTP API to send the status update to all interested users. More information on multi-channel event publishing here.