I need to implement a distributed XMPP MuC application on the lines of XEP-0289 minus some of the features, in essence I want to have a bare bones implementation of the plugin, my concern is to address fault-tolerance and as of now I do not want to worry about the performance considerations as specified in 289.
I have looked into SleekXmpp as a tool to develop server side plugins, but don't know how comfortable it would be to use it for such an implementation, other options I have looked at are OpenFire , Tigase. I am comfortable with Python/Java and other key features to consider would be good documentation, ease of use etc keeping that in mind I would like to know what would be the preferred path to take for this development.
Any guidance will be appreciated.
you should be able to write a MUC component that includes FMUC (or similar). The general way to do this would be to use a library that supports XEP-0114 components (e.g. SleekXMPP (Python), Swiften (C++)) and implement MUC+FMUC through that. You haven't said what your concerns with SleekXMPP are, but it's a fairly well-respected library in the XMPP community, so seems a fair choice (I'd pick Swiften, but I'm biased as one of the authors).
Your second option (patching the server directly) isn't generally the XMPPish way of adding customisations (as it's vendor-specific), but should also work if you can find someone sufficiently familiar with the server code, or if you're willing to become so.
To achieve fault tolerance (assuming you mean resilience to server failures) you'd need to run your XMPP server clustered, and also cluster your FMUC implementation. With that done, the usual XMPP fail-over using SRV records in DNS should ensure other servers retry connections to another host.
On a side note, the next version of FMUC (XEP-0289) will have some of the features of the current revision stripped out, and a number of improvements made based on deployment experience, so if your work is not time-critical, it might be of benefit to you to read that when it's released. I also note that there exists at least one implementation of FMUC already (Isode's M-Link, on which I work), and there is interest from other vendors, so using the standard protocol might benefit you in terms of not re-inventing the wheel.
Related
Let's assume that I have a couple of MicroServices with each exposing a set of REST end points. Assume that MicroService A is communicating with MicroService B and they exchange JSON data.
This JSON data needs to be Serialized and De-Serialized on both the MicroService A and B. This Serialization logic and the models are going to be the same on both the MicroService code base.
I can reduce this duplication by just moving the model classes into a small dependency and use it on both the MicroServices. Not a problem! This might go against the goal of a MicroService architecture, which is "share nothing". But I feel even more potential problem to address is code duplication. What do you guys think?
I do not see the point 'share nothing' in this scenario. As long as you will hold your De/Serializer as an Artifact in some nexus, you do not "share" anything, instead you are using an (somehow) external library.
If you use e.g. logging, both of your projects will use the e.g. slf4s, but they do not share it, as each uses it separately.
There are a number of things to bear in mind when separating out a functionality into communicating micro-services:
Tying of scala versions between server and client
If your server requires specific versions of scala (because, for example, you use a library that only exists for version 2.10), this should not impact your choice of scala version in the client. This points towards the idea of having the classes representing your communication path, as being in a separate project which can be cross-compiled separately.
Tying of libraries between server and client
The less requirements your shared library places on your client code, the better. Even forcing a particular choice of Play server enforces a level of rigidity and coupling between client and server that is best avoided.
The best option is that this library causes a dependency on zero other libraries.
Supporting protocol changes over time
One of the advantages of having separate services is that they can be upgraded and improved at separate points in time. You should always try and have the server support the previous version of your communications protocol, whenever it changes. This allows you to roll back an update easily, and also update the client at a different point in time.
Not allowing backwards compatibility means you need to update both services in lock-step. This not only reduces a lot of the advantages of using micro-services, it also makes it a huge pain to deal with rollbacks, if that becomes necessary.
The universal story here is to enforce as little as possible in the way of choice (scala version, library version, time period when protocol changes must happen) on the client, through what choices are made on the server.
If you can follow this approach, I don't see a problem with using code to enhance the accessibility of talking to a service.
What I want to do: I want to add communication capabilities to a couple of applications (soon to be jar libraries for Java) in Scala, and I want to do it in the most painless way, with no Tomcat, wars, paths for GET requests, RPC servers, etc.
What I have done: I've been checking a number of libraries, like Jetty, JAX-RS, Jackson, etc. But then I see the examples and they usually involve many different folders for configuration, WSDL files, etc. Most of the examples lack a main method and I don't have a clear picture about how many additional requirements may they have (e.g. Tomcat).
What I am planning to do: I'm considering to simply open a socket on the "server" to listen, then connect with the "client" and transfer some JSON, in both directions. This should be fairly standard so that I can use other programming languages in compatible ways (e.g. Python).
What I am asking: I would like to know whether there is some library that makes this easier. Not necessarily using raw sockets, but setting up some process communication in just a few lines, maybe not as simple as Node.js, but something similar.
Bonus: It would be cool to
be able to use other programming languages (e.g. Python) by using open standards
have authentication
But I don't really need any of those at this point.
I think you need RPC client/server system, I would suggest to take one of these two:
Finagle - super flexible and powerful RPC client/server from Finagle. You can define your service with Thrift, and it will generate stubs for client/server in scala. With Thrift it should be straightforward to add Python support.
Spray - much smaller library, focused on creating REST services. It's not so powerful as Finagle, however much easier. And REST allows you to use any other clients
Remotely - an elegant RPC system for reasonable people. Interesting and very promising project, however maybe difficult to start with because of extensive Scalaz+Shapeless+Macro usage
Honestly if you want something that is cross-language compatible, simple, straightforward, and concise then you do not want to use plain old sockets!
Check out dropwizard. It is amazing and I use it for small and large projects alike! It is usually configured by no more than a single configuration file. It supports authentication too!
Out of the box it gives you really great inter-process communication over JSON (using Jackson) and much much more. There is also pretty decent Scala support for dropwizard.
If you must roll your own then I'd recommend using Jackson for JSON parsing. It's super simple to use and also has great scala support.
If you've got a "controlled" use case where the client and server are on the same LAN and deployed in tandem, I'd (controversially) recommend Java RMI; it's dumb and JVM-specific (and uses a Java-specific protocol), but it's very simple to use.
If you need something more robust and cross-language, I'd recommend Apache Thrift. You write your interfaces in a platform-independent interface definition language, and it's very clear which changes are compatible and which are not; the thrift compiler generates skeleton interfaces for you to use, and then you just write an implementation of that interface and a couple of lines to start the server (as you can see from the example on the homepage). It's also got good support for async implementations if you need the performance. Thrift itself is reasonably standard and cross-platform, with its own binary protocol, or you can use JSON as a transport if you really want to (I'd recommend against that though).
RabbitMQ provides one easy way to do what you want without writing a server and implementing your own persistence, flow control, authentication, etc. You can brew or apt-get install it.
You start up a broker daemon process (i.e. manages message queues)
In the Scala producer, you can use Maven-provided Java API to send JSON strings without any fuss (e.g. no definition languages) to specified queues
Then in your other Scala program, connect to the broker, and listen for messages on the queue, and parse the incoming JSON
Because it is so popular, there are many tutorials online for different patterns you may want to use to distribute the messages, e.g. pub/sub, one-to-one, exactly-once delivery, etc.
I am currently doing my research on building a chat system with more than 10k users connected online. I came across technologies and ways to do it such as jabber(XMPP), websockets, long polling, push. As far as I now, long polling might not work given the number of users. I know there is a lot of ways to accomplish this. I also know that facebook and Google chat systems are developed on XMPP.
I would truly appreciate if anyone could point me to the right direction. I believe all these methods and technologies out there are good depending on the scale of the project. I definitely need performance and scalability.
I've used Socket.io together with NodeJS for such a chat application. It scaled to over 10K concurrent users on moderate servers and there was a lot of room to grow.
This does depend on your limitations, tho.
What kind of hardware are you planning on using?
Which operating system would power your servers?
Which client platforms are you targeting?
Do you have an existing infrastructure you need to fit this into?
Do you have a previously selected programming language?
The existing skill sets your team members have and your team's ability to adopt new platforms and languages if necessary.
Take all of the above into consideration when making your decision.
Personally, I've found XMPP to be quite adequate, but a bit bloated for my purposes. YMMV.
You are comparing a fruit basket and three different variety of oranges.
XMPP is the only protocol that you have mentioned that actually is designed to support a chat system (of which many exist). The others are simply asynchronous messaging protocols/techniques. XMPP already supports http based chat via BOSH. Without a doubt, it will also support WebSockets when the specification is finalized. There is actually a draft of this already written, but at this point it appears to be a draft using a draft, so there will probably be few, if any, implementations.
Using XMPP would allow you to build on a proven technology for implementing a chat system and would allow you to choose what transport you want to use "under the hood". You haven't actually said whether you need a http based transport or not, but with XMPP you can use the stock tcp socket based transport or a http based one (BOSH) with the knowledge that it will also support WebSockets in the future.
The other benefit is of course that this is a widely used standard that will allow reuse of existing clients, servers and libraries in pretty much all popular (and not so popular) languages and platforms.
Scalability is not too much of a concern with the numbers you are quoting, as most (maybe all) existing xmpp servers will handle that many users.
I'm in the process of deciding how to write an online auction application. I would like to provide real-time updates to the site users. My background is with LAMP (although, in my case, the 'P' would be more for Perl than PHP). I've considered ActiveMQ, but I'm wondering if there are better options.
My primary concerns are scalability and speed. It could have several simultaneous auctions taking place, with [hopefully] many users participating in each auction. Whatever solution that I decide on would have to accommodate such a scenario. Of course, this is all in theory so I have no idea how many concurrent users that I might have, but I'd like to have the means to support tens of thousands of users.
Another concern is ease of implementation. I've spent the past few days reading docs and tutorials and, so far, nothing has come across as anything less than a bit of a pain in the rear to deal with, which is actually what has led me here to seek some advice.
I was hoping to use a web framework, such as Codeigniter (PHP) or Catalyst (Perl), because I intend to pay a contractor or two to help with some of the bulk of the coding, and I like the idea of having a framework to somewhat enforce a design pattern. However, the more that I look into this, I'm just not seeing an obvious solution to 1) use a framework, and 2) provide real-time auction updates (other than Tornado, I guess - maybe I'm answering my own question. ;)).
So, with all that said, short of using polling (which I'm not really interested in doing), is there a way that I can accomplish these real-time updates using a language like Perl or PHP for my server-side code? I know that ActiveMQ supports STOMP, and I actually have this working on my local machine (using Jetty since it requires a servlet to publish/consume messages from client-side javascript), but is there a better option here?
I'm sorry that I don't have a more direct question, but after several days of looking at docs and tutorials, I'm more lost than ever!
Part of your problem is that your mixing a variety of concepts together. If I read things correctly you have a problem statement of:
I'm building an online auction site and would like to insure that my visitors have real-time updates of prices on the items they are viewing.
Now between the Browser and the Server you'll probably use a Comet style request pattern to handle communications, you could also look at socket.io as a backup pattern. This polling will require a server that is able to handle lots of simultaneous open connections, which Tornado is a good candidate (there are others, but given you asked in relationship to Tornado it's good).
Now that we've gone from 1000+ of Browsers to a handful of Tornado servers, you need a way to communicate between them. In the the last of publish/subscribe message patterns you have a few choices:
RabbitMQ (AMQP)
ZeroMQ
Redis Pub/Sub
All three a good choices, with their own pros/cons. Personally I've used Redis and Rabbit on different projects and just toyed with ZeroMQ. The message broker is a whole decision tree that is going to be based on what you have available.
I want to try developing an XMPP server component using XEP-0114: Jabber Component Protocol.
Which server do you recommend and why? I'm talking about ease of development, community support, documentation, examples, etc.
That's a hard question to answer, because I doubt there are many developers involved in developing across multiple XMPP projects and languages.
I can throw out a few personal perceptions but... I could be off-base!
What you're really looking for is which libraries would be recommended for component development. All the servers support the component protocol, so all you really need is a socket connection to the server and some helper routines to make the repetitive stuff like message parsing easier.
Where the server might matter is if you need tighter integration.
For example if you want your component to scale the same way as Ejabberd then you'll probably want to use exmpp.
If you need to deploy your component alongside Openfire into Java only enterprises, then you'll probably want to use smack.
If you are familiar with Python and want to prototype quickly use Wokkel.
I don't think documentation is going to be great for any of the libraries (haven't looked at them all though!) but that shouldn't be a huge burden. All you really need a good book on how the XMPP protocol works and then some sample code from the library and it's fairly easy to move on from there.
For an easy-to-use testing server I like openfire. Good instructions, easy to hook in components, and a good web interface for administration. Debugging is more of a "tail -f" on the logfiles, slightly java-ish.
I've used XCP professionally, but that's really for commercial use. It works well but if that's not your target deployment it's not worth the effort. I'm not sure if you can buy it separately any more.
I tried using ejabberd and I gave up quickly. I found the documentation for setup and administration awful. The config files are not self describing and there's no good walk through on the ejabberd site. It may be able to even fry my eggs in the morning for breakfast, but I couldn't get past install with the time I'd allotted to it.
For Openfire, there is something called Whack, which is a Java library for creating server components (XEP-0114).
Since the communication is over sockets, I presume the same code should work for any well designed XMPP server (such as ejabberd). However, I have only tested it with Openfire and it works quite well.