I want to run Cross-Platform XMPP Instant Messenger. What Server-Side language should I choose?
There's also http://prosody.im/ which uses Lua. Their mission statement:
Prosody is a modern flexible communications server for Jabber/XMPP written in Lua. It aims to be easy to set up and configure, and light on resources. For developers it aims to be easy to extend and give a flexible system on which to rapidly develop added functionality, or prototype new protocols.
Prosody is licensed under the permissive MIT/X11 license.
If you want to use a XMPP server, take a look at Ejabberd(written in Erlang) or maybe Tigase(written in Java)
If you want to create your own, use:
Twisted Matrix, a Python framework to develop asynchronous applications
Erlang, a functional language designed with concurrency in mind.
Java with the NIO asynchronous library
Depending on how close to the XMPP spec you want to be, C++ might be an option but it will be quite challenging, as there is a fair amount of logic to implement :-)
If you want to optimize for speed, identify the bottlenecks of your application, and look into writing specific parts in C(XML parsing or string handling).
Related
What I want to do: I want to add communication capabilities to a couple of applications (soon to be jar libraries for Java) in Scala, and I want to do it in the most painless way, with no Tomcat, wars, paths for GET requests, RPC servers, etc.
What I have done: I've been checking a number of libraries, like Jetty, JAX-RS, Jackson, etc. But then I see the examples and they usually involve many different folders for configuration, WSDL files, etc. Most of the examples lack a main method and I don't have a clear picture about how many additional requirements may they have (e.g. Tomcat).
What I am planning to do: I'm considering to simply open a socket on the "server" to listen, then connect with the "client" and transfer some JSON, in both directions. This should be fairly standard so that I can use other programming languages in compatible ways (e.g. Python).
What I am asking: I would like to know whether there is some library that makes this easier. Not necessarily using raw sockets, but setting up some process communication in just a few lines, maybe not as simple as Node.js, but something similar.
Bonus: It would be cool to
be able to use other programming languages (e.g. Python) by using open standards
have authentication
But I don't really need any of those at this point.
I think you need RPC client/server system, I would suggest to take one of these two:
Finagle - super flexible and powerful RPC client/server from Finagle. You can define your service with Thrift, and it will generate stubs for client/server in scala. With Thrift it should be straightforward to add Python support.
Spray - much smaller library, focused on creating REST services. It's not so powerful as Finagle, however much easier. And REST allows you to use any other clients
Remotely - an elegant RPC system for reasonable people. Interesting and very promising project, however maybe difficult to start with because of extensive Scalaz+Shapeless+Macro usage
Honestly if you want something that is cross-language compatible, simple, straightforward, and concise then you do not want to use plain old sockets!
Check out dropwizard. It is amazing and I use it for small and large projects alike! It is usually configured by no more than a single configuration file. It supports authentication too!
Out of the box it gives you really great inter-process communication over JSON (using Jackson) and much much more. There is also pretty decent Scala support for dropwizard.
If you must roll your own then I'd recommend using Jackson for JSON parsing. It's super simple to use and also has great scala support.
If you've got a "controlled" use case where the client and server are on the same LAN and deployed in tandem, I'd (controversially) recommend Java RMI; it's dumb and JVM-specific (and uses a Java-specific protocol), but it's very simple to use.
If you need something more robust and cross-language, I'd recommend Apache Thrift. You write your interfaces in a platform-independent interface definition language, and it's very clear which changes are compatible and which are not; the thrift compiler generates skeleton interfaces for you to use, and then you just write an implementation of that interface and a couple of lines to start the server (as you can see from the example on the homepage). It's also got good support for async implementations if you need the performance. Thrift itself is reasonably standard and cross-platform, with its own binary protocol, or you can use JSON as a transport if you really want to (I'd recommend against that though).
RabbitMQ provides one easy way to do what you want without writing a server and implementing your own persistence, flow control, authentication, etc. You can brew or apt-get install it.
You start up a broker daemon process (i.e. manages message queues)
In the Scala producer, you can use Maven-provided Java API to send JSON strings without any fuss (e.g. no definition languages) to specified queues
Then in your other Scala program, connect to the broker, and listen for messages on the queue, and parse the incoming JSON
Because it is so popular, there are many tutorials online for different patterns you may want to use to distribute the messages, e.g. pub/sub, one-to-one, exactly-once delivery, etc.
I am currently doing my research on building a chat system with more than 10k users connected online. I came across technologies and ways to do it such as jabber(XMPP), websockets, long polling, push. As far as I now, long polling might not work given the number of users. I know there is a lot of ways to accomplish this. I also know that facebook and Google chat systems are developed on XMPP.
I would truly appreciate if anyone could point me to the right direction. I believe all these methods and technologies out there are good depending on the scale of the project. I definitely need performance and scalability.
I've used Socket.io together with NodeJS for such a chat application. It scaled to over 10K concurrent users on moderate servers and there was a lot of room to grow.
This does depend on your limitations, tho.
What kind of hardware are you planning on using?
Which operating system would power your servers?
Which client platforms are you targeting?
Do you have an existing infrastructure you need to fit this into?
Do you have a previously selected programming language?
The existing skill sets your team members have and your team's ability to adopt new platforms and languages if necessary.
Take all of the above into consideration when making your decision.
Personally, I've found XMPP to be quite adequate, but a bit bloated for my purposes. YMMV.
You are comparing a fruit basket and three different variety of oranges.
XMPP is the only protocol that you have mentioned that actually is designed to support a chat system (of which many exist). The others are simply asynchronous messaging protocols/techniques. XMPP already supports http based chat via BOSH. Without a doubt, it will also support WebSockets when the specification is finalized. There is actually a draft of this already written, but at this point it appears to be a draft using a draft, so there will probably be few, if any, implementations.
Using XMPP would allow you to build on a proven technology for implementing a chat system and would allow you to choose what transport you want to use "under the hood". You haven't actually said whether you need a http based transport or not, but with XMPP you can use the stock tcp socket based transport or a http based one (BOSH) with the knowledge that it will also support WebSockets in the future.
The other benefit is of course that this is a widely used standard that will allow reuse of existing clients, servers and libraries in pretty much all popular (and not so popular) languages and platforms.
Scalability is not too much of a concern with the numbers you are quoting, as most (maybe all) existing xmpp servers will handle that many users.
I am trying to evaluate some technologies for implementing a communication process between some Ada modules with some C++/OpenGL modules. There is an (Windows XP) Ada application which communicates with a C++ application using COM, but I intend to switch the COM to a new technology. Some suggestions came up, such as direct Sockets, DSA, Polyorb, Corba and DSS/Opensplice.
DSA appears to be just Ada -implemented (not sure)
Polyorb has its last implementation on 2006, according to http://polyorb.ow2.org/
Corba someone argumented that it could be not simple enough to justify its complexity for implementing simple applications
DSS/Opensplice appears to be just C/C++ implemented, so an Ada binding should be done. It also looks to be not very simple to be implemented too.
Personally I like COM, but due to the migration, I'd rather take the sockets option due to its simplicity, and the interface architecture could be implemented very easily.
So, what you think? Could you please comment about these technologies or even suggest other more?
Thanks very much.
A big factor in your choice is the size and complexity of the system you're reengineering. Is it a broadly distributed system with lots of complex messages? Is it a relatively small system with a handful of mundane message exchanges?
For small systems I used to just roll-my-own socket-based comm modules. Now, though, I lean more towards ZeroMQ (brokerless) or STOMP (text-based). And there's some Ada support for these, zeromq-Ada and TOMI_4_Ada (supports both).
While these handle the distribution mechanics, you would still need to handle the serialization of the messages into transportable form.
CORBA/PolyORB and DDS solutions are rather heavyweight, but are complete solutions. If you don't fear IDL and managing brokers, they can do well for large-scale distributed systems. Yeah, there may need to be some Ada bindings built, but if you can get C headers or a C API to bind to, it's typically not too bad if you focus on just binding the functions and data structures you require. Rather than creating a comprehensive binding, liberally employ opaque and void pointers (void_ptr, opaque_structure_def_ptr) for structs and parameters whose internal contents you don't care about.
we intend to switch the COM to a new (suported) technology, since COM is not more supported by Microsoft
Whoever told you COM is no longer supported is totally clueless.
While COM has undergone many name changes (OLE, COM, OLE Automation, DCOM, COM+, ActiveX, WinRT) and extensions over the past decades, it is the single most important technology for MS platforms: past, present and future. The .NET runtime uses COM extensively. Much of the Win32 API is written in COM, and the portions that weren't, will be in Win8, since WinRT components are COM objects.
Also take a look at AMQP (RabbitMQ for server), there seems to be Ada library available for it http://www.gti-ia.upv.es/sma/tools/AdaBinding/index.php.
If you could find binding for Ada, Apache thrift might also be a lightweight option. Maybe you could even write your own binding, it should not be more difficult that rolling something of your own over the sockets.
If you do go sockets route, than I would suggest ZeroMQ as "supersockets".
One more option for your list should be to use Ada's distributed programming support, and write C/C++ wrappers to interface your C++ program into it.
I don't know that its the best option for your needs, but if your Ada compiler supports Annex E, it should be on the list.
Since this post, AdaCore published PolyORB on GitHub with regular updates :)
I need to create a simple Chat system like facebook chat and a twitter-like app.
What is the best concurrent program languages in this case ?
Erlang, Haskell, Scala or anything else ?
Thanks ^_^
Chat system like facebook chat
Facebook chat is written in Erlang, http://www.facebook.com/note.php?note_id=14218138919
and a twitter-like app
What aspect? Message delivery? The web frontend? It's all message routing ultimately. Haskell's been used recently for a couple of real time production trading systems, using heavy multicore concurrency. http://www.starling-software.com/misc/icfp-2009-cjs.pdf
More relevant: what's the scale: how many users do you expect to serve concurrently?
Erlang is my choice of drug for something like that. But I would also check out Node.js if you feel more comfortable in JavaScript.
Complete chat application source code using Scala/Lift.
8 minutes and 20 seconds real-time, unedited, webcast writing a chat app from scratch using Scala/Lift.
This is actually a relatively easy problem to solve that can be done by any language with decent threading support including (eg Java, C# and others). The model is fairly simple:
Each client connects to a Web server with an AJAX request that is configured on the server not to timeout;
If it does timeout for any reason the client is configured to issue another AJAX request;
The server endpoint for that AJSX call does a thread wait operation on some monitor waiting for updates;
When the user sends a chat it is sent to the server and any relevant monitors are signaled;
Any threads listening to that monitor are woken up, retrieve any messages waiting for them and return that as the AJAX result to the client;
The client renders those messages and then issues another AJAX request to listen to for messages.
This is the basic framework but isn't the end of the story. Any scalable chat system will support either internal or external federation (meaning the clients can connect to more than one server) but unless you're Google, Facebook or Twitter you're unlikely to have this problem.
If you do you need some kind of message queue/bus for inter-server communication.
This is something you need a heavy duty multithreaded language like Erlang for but it, Haskell and others are of course capable of doing it.
F# for future.
Client/Server chat F#
Concurrent Programming - A Primer
Also take a look at Twisted(Python).
I'd recommend taking a look at Akka (www.akkasource.org)
It's a Scala Actor framework with ALOT of plumbing for making scalable backend applications.
Out of the box it supports:
Actor Supervision
Remote actors
Cluster Membership
Comet over JAX-RS (Project Atmosphere)
HTTP Auth for your services
Distributed storage engine support (Cassandra, MongoDB)
+ a whole lot more
I would go for erlang, it's efficiency in comet-enabled apps is largely proven.
Go with a framework such as nitrogen, where you can start comet requests just as easily as Jquery does ajax.
If it's actually going to be a simple application (and not under very high load), then the answer is to use whichever language you already know that has decent threading. Erlang, Scala, Clojure, Haskell, F#, etc., all do a perfectly good job at things like this--but so do Java and C#. You'll be fine if you pick whichever of these you know and/or like.
If you're using this as an excuse to learn a new generally useful language, I'd pick Scala as a nice blend of highly performant (which Erlang is not in general, though it is fantastic with efficient concurrency), highly functional (which Java and C# are not), highly deployable (due to running on the JVM), and reasonably familiar (assuming you know C-ish languages).
If you're using this as an excuse to practice fault-tolerant concurrency, use Erlang. That's what it was designed for and it does it extremely well.
Has anybody done or seen a deployment of Apache Thrift in an iPhone app?
I am wondering if is a reasonable solution for a high-volume, low(er)-latency network service for iPhones compared to HTTP.
One noteworthy thing I found is a bug report about running Thrift on the iPhone, which seems to have been fixed. But that doesn't necessarily indicate that it's a done deal.
Thrift and HTTP aren't mutually exclusive. In fact thrift now ships with an HTTP transport implementation to use. It's also a really nice way to auto-generate server/client code that avoids a lot of marshalling/unmarshalling boilerplate while still being really fast. Its internal representation is basically binary JSON, so it's very similar to a RESTful web service (except being easier to code and much, much faster).
So... anyone able to answer the original question? If not, I'll dive in myself with thrift's included Cocoa support and see how it works on the iphone.
Just my two cents..
The accepted answer to this question, is an opinion to not use a technology, not an answer of whether it is possible.
Thrift, is an interface definition language, IDL, like Protobuf and Capt'n'Proto. They permit the definition of a client/server/server protocol which is platform agnostic. JSON and Plist don't provide the same level of type conformance.
Having previously lead an iOS team with 10Ms MAU using Google Protobuf v2.5 on iOS, Android, Windows, and server teams, I can attest that IDLs are great on mobile. Apple uses them for syncing iWork content.
My current team uses Thrift for iOS and Android clients, with a mostly Scala backend. I much prefer it to Protobuf.
We send Thrift payloads over HTTPS and WebSockets. Once you have defined (in Thrift) your our wire communication protocol (i.e. frame structure), it's very easy to evolve your APIs.
However, on iOS in particular there are some implementation issues. The current version of the library is quite poorly packaged, and if you hope to make an Objective-C framework (e.g. for iOS 8+), then you will not be able to out of the box with v0.9.2. This is because the library headers include local imports, (#import "TProtocol.h" instead of #import <Thrift/TProtocol.h>) with no umbrella headers. Worst of all, the Objective-C compiler generates very messy Objective-C classes, also including local imports from the Thrift library.
Some of these issues are pretty damning. It indicates to me that while use of an IDL is very much a good engineering decision, not many iOS teams are using Thrift, unless they're huge with the resources to write their own library.
I've always disliked frameworks that use a common interface definition that builds out both server and client code. It keeps both sides too much in lockstep where in reality server API changes must be very flexible in the versions of clients that are communicating with it.
There are helpful libraries that make JSON or PLIST communication over HTTP pretty easy, and decades of debugging and understanding the HTTP protocol and how to use it well. I would ignore that at your peril.
I have used thrift's objective c bindings for a large iPhone app with a few million users. As one of the posters mentioned we can use Http which gets the best of both worlds. However there is no asynchronous HTTP client for thrift. We had to build an event based wrapper to allow non-blocking I/O calls. The underlying layer still issues one call at a time which hit us in a big way because we have one server call that takes a long time but it does not block UI flow and another really fast one that does block UI flow. If the underlying layer is busy with the slow command our fast command just has to wait. I am trying to build asyc http in c++ which can then be used on the iPhone but that is someways off from being ready.
Thrift as an external API doesn't make sense. Use it internally rock and roll.