We are trying to implement a notification module. It allows website internal users to send message to each other. A key feature is that it allows business users to send bulk messages to the users. We are talking about millions of users here.
Currently it is planned to be a publish/subscribe model. Once login, system shall retrieve the relevant messages for the user from a database table. The logic gets more and more complicated when each users are allow to delete and reply to the message he/she received.
Pubsubhub seems to be more server to server. XMPP seems to be too complicated for this scenario.
Anything I miss out? Can I make it simpler? Any existing library to build on? I'm open to any suggestions.
It sounds like a database is actually all you need here. You didn't mention any need for real-time notification. If this is a web application and the user is logging in, a simple relationship between users and messages may be all you need to provide the ability to send any message to one (or millions) of users. Your relationship table can include flags for read and deleted.
One option would be to use something like Joomla.
http://www.joomla.org/
Its open source, and they've solved all the problems you are trying to solve. Alternatively if you have to build it, what language are we talking about here?
Are you seriously saying you have millions of 'internal' employees? Sounds like you might need an email server!
Seriously though, please tell us more..
Related
I've been using MongoDB for a bit and I love it. Like most I migrated from using SQL. I have not had any problems with the database or the performance that it gives and I can say that over-all I extremely love the flexibility.
However, I've ran into an issue that I'm not sure how to resolve (correctly) using Mongo.
I've gone through and added friends and messaging to my application, which as you can imagine requires some relation. There's a few points here that I don't understand:
Pending friend requests
Mapping messages between two friends.
Putting all of this in the accounts document seems like it'd be ridiculous, I just don't have the slightest clue how to go about it.
I know that once a friend request is completed, I can add a pointer into an array for friends, IE:
friends: [pointer(document), pointer(documnet)]
Would doing this for all requests be sufficient though...?
How would User B know that User A sent him a request.
How would User A know that they already sent user B a request.
Any material on displaying these kind of relations in Mongo would be greatly appreciated.
I have the ecommerce store.
I faced automatization problems as my business growing.
My website is written in PHP on Kohana Framework.
I want to automize many processes, but do not know where to dig deeper.
Example:
When order is recieved I want to fire the event "order.recieved". This event knows that my system needs to fire subevents and other actions:
to log inventrory changes asynchronously;
send to the procurement module out-of-stock products and their quantity;
to recreate sliced inventory table asynchronously;
send real-time message to crm system that order is recieved;
send sms to the client with message "We will call you soon";
etc.
Solutions I have in my head:
1) I think the simplest way is to rewrite my application on Laravel framework with it's event dispatcher.
Challenges:
I want use fifo (First In, First Out) for the inventory change;
I need digging to the code to know how many events I have in my system, when they are used,
2) Use AMQP Middleware like RabbitMQ.
Challenges:
I need digging to the code to know where exactly message is sent;
Maybe It can be solved with some event dispatcher, but ->;
Again, I need digging to the code to know how many events I have in my system, when they are used
Are php workers/consumers good to handle messages? Or I need to use python, nodejs?
Challenges:
I need to have many php workers/consumers. I think PHP is not good to handle this.
3) Use some webhooks management system.
4) Study more and use some bpm engine. Only one thing I know right now I can benefit using business rule engine for complex discounts.
What solution I need to take and get deeper with it?
New to rest and not having even known what REST was, I began watching a few videos and picked up a book to help guide me towards the correct approach. Unfortunately, my first version is completely botched to hell and I'm likely going to have to break any customers using that implementation shortly. To ensure that I don't need to do this again, I need your assistance!
I have a few DB tables that I'm concerned with here:
'PrimaryBuyer' & 'AllBuyers'
They share a majority of fields, but AllBuyers has a few things Primary does not and vice versa. Each primary buyer is given a unique 'CaseNumber' when entered into the system. This in addition to a 'SequenceNumber' is then used to identify 'AllBuyers'. This CaseNumber is returned to the user of the web service to store for future use. The sequence numbers however are implied based on their location within the XML / JSON.
To specify these tables -> For example, if I were to buy a car I would be the primary buyer and would thusly be entered into BOTH Primary and AllBuyers tables. However, if my credit was bad I could have my spouse cosign on the loan. This would make her a secondary buyer, and she would be entered exclusively into 'AllBuyers' table.
I currently have one REST URI set up as '/buyers/' which mandates that all information for all buyers is entered at once. Similarly if I were to do an update on this URI, the Primary is updated in both tables and any Secondary buyers in the payload would replace previously existing ones.
Ultimately, there is no way to directly access tables 'PrimaryBuyer' and 'AllBuyers'
I've been trying to think of a solution around this problem, but have been unable to think of anything that's necessarily RESTful or not a pain for customers. Is it ridiculous to think that the user should (say on an add) POST to /primarybuyer/, take the returned casenumber, and then POST the same information and then some to /allbuyers/? That seems like it would be a little silly on bandwidth among other things. Should things be left in their current state?
Hopefully that's not too much information to answer such a seemingly simple question.
Is it ridiculous to think that the user should (say on an add) POST to
/primarybuyer/, take the returned casenumber, and then POST the same
information and then some to /allbuyers/?
When you talk about "user" do you mean the person or the system (browser) that uses your service ?
Using a REST service is normally made by a system, today that means a lot browser+Javascript. To do the job for the user person is done at a web facade, and in back is running all your (Javascript) code to make the appropiated REST calls.
Why not post the buyer's information as a serialized object to the rest server? You could do this via the parameters section of the request. When the request got to the server, you could deserialize the object, and implement the logic of updating the database.
Kind of like how amazon does it? http://docs.aws.amazon.com/ElasticMapReduce/latest/API/API_AddInstanceGroups.html
If the client application does POST /PrimaryBuyer there is no reason that the server cannot also copy that case information into the /AllBuyers resource and vice versa.
for my group at the university I'd like to set up a server-sided email-to-rss service.
It should work like that, that different people can send emails to a certain address (nothing proprietary like gmail but a certain imap or pop server) which will the be translated into an rss feed. One main and important feature has to be that one can see the sender of the email in the feed. Furthermore it would be nice (to take the load off the server) if the emails get translated to a feed only once a day or so.
Does anyone has some input on this subject? Are there any scripts/services which will allow that?
Thanks a bunch.
Instead of "reinventing the wheel", you could use a mailing list that supports RSS. Your people can then write the mails to the mailing list and you can then use the mailing list's RSS feed however you intend to.
This should help you find a solution: https://encrypted.google.com/search?q=mailing%20list%20rss
Pick a programming language you're familiar with, then use either an imap library to fetch the E-mails (through cron, every hour or something like that), or if you have access to procmail on your mail server, launch your script as an email arrives (this shouldn't be too much work to handle for a server, unless you're talking a vast amount of E-mail).
The script would just insert the E-mails into a database, before extracting them and outputting the RSS-feed directly from that (this shouldn't be more than a handful of lines of code).
There's a couple of providers that does this for you, although it seems that the most popular ones have disappeared. Advanced Email2RSS seems to be an option, although I have no idea how good they are or if it'll even solve your issue.
I'd like to have something in my app that is just like Twitter but not Twitter. Basically it will be a place people can submit messages and do not need an account. They only way they can submit is through the app. I want other app users to see the submitted messages nearly immediate. I believe push notification can do that sort of work but do I need push notification for this? How does Twitter do it?
-- EDIT --
After reading some of the responses, push might be what I need. People will be submitting messages to my server often. If someone is watching the feed, they might see one new message per minute depending on the query they are using. I'm thinking to go with a MySQL database, (which allows switching to cheaper non Windows servers w/o much hassle) and push notification. Are there any reasons those won't work for my scenario?
You only need push notification if you want the app to be able to receive new messages while closed.
Here's a rough description of one way to do this:
Your app sends a message via HTTP Post to your server.
Your server stores the message in a database, using the iPhones unique ID as an identifier.
Your app connects to the server frequently, asking for new messages.
If there are any new ones, the server hands the message to the app, which displays it.
This is approximately what twitter/iphone twitter apps do.
Your choices are fairly binary:
Use push notification
Use Polling
With Push Notification:
You control when you contact your users... Heavy Load means you can slow updates down to avoid taxing your infrastructure
Contrariwise, you have to push to clients that may not even be there anymore (And thus may need some sort of register model), high load may mean that clients don't get immediate update
You can leverage things like Amazon's EC2 to give you more processing power
Unless you're out of capacity, users are almost certain to be receiving updates as they happen
To pick up messages missed while offline, the SERVER needs to know what message was last successfully received, store older messages and forward many all at once
If you choose to use polling:
You must have a stable address to be polled
You need the ability to have lots of quick query connections checking for new data, then returning that data if required.
If your application becomes popular enough you may find you don't have enough resources
If your resources are taxed your application will go down, rather then just slow down
You don't need to register clients and keep track of their on/offline state
Parallelizing on the fly is a bit trickier
To pick up older messages, the CLIENT needs to know when they last received a message and then request the server send any message since that time
Both can be fast, but they come with different bandwidth and processing profiles. I prefer push for everything that's real-time.
Might want to take a look at XMPP.
Twitter doesn't really push events out to the iPhone in realtime. It's more like polling by the various clients.
If you really want instantaneous for the last mile you'll want to use push.
Twitter uses lots of servers and raid arrays to handle the load of millions of people posting 140 character messages. Twitter clients log in and request a list of updates for all of the people the user is following within a certain time frame.
Push wouldn't be a good candidate for this because it does not persist the "tweets". It is simply a notification mechanism. There is a text messaging app on the App Store (called Ping!) that relies completely on push notification for sending text messages. This seems to work fine, but if the developers are keeping track of the messages, it is all done on their servers. In their case push makes sense as you want to alert the user of a new message. In the case of a twitter clone, however, it would probably just annoy users if they got a new notification every time someone tweeted.
In the end you're better off just implementing it server side and then developing an iPhone client that logs in and retrieves the latest tweets for the people the user is following.