Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 months ago.
The community reviewed whether to reopen this question 5 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
Which of these technology suite for Multi user online game project?
Project requirements:
Able to handle 2k-5k user at any given time.
client library for iphone and android (native, no javascript).
client library for Microsoft Windows (most important), also for mac os x and linux.
Good documentation specially for mod development
Project is not open-source. So, can only use libraries with suitable license.
I can program in erlang and java as well, programming language isn't an issue.
I was looking at following server technologies like Openfire, Tigase, ejabberd and RabbitMQ.
All are good for my project but I want to know more about what suite my needs, AMQP or XMPP.
What AMQP offer specially for real time online games. Is it better option then xmpp?
A key difference between XMPP and AMQP is binary content. AMQP handles binary data just fine, and XMPP seems more designed for XML. Personally for online games I use Google Protocol Buffers for message formatting and parsing, and with their very small binary footprint, I'd be more inclined to use AMQP to deliver those messages.
But do consider what AMQP server you want to use. I've been bitten by using RabbitMQ for my AMQP server in the past. RabbitMQ does not have any flow control facilities, at all. So if your clients are sending messages faster than your server can consume them, buffers on the server can fill up and blow the server up. More recent versions of RabbitMQ implement flow control in an exceedingly coarse way: they halt all consumers in the system until memory clears up.
I've never tried zeromq; perhaps it'd be better for the things I've been using RabbitMQ for...
5K users doesn't tell me much about their behaviour, but if they all submited one request within the same 10 second window then let's say you'd be looking in the 500-1000 requests per second.
I've had Active/MQ running on my relatively low-powered lap-top easily handling 300 requests per second and so I'd happily recommend it here. You can also set up clusters of brokers and achieve horizontal scalability. You can use an http protocol (STOMP) or its native binary protocol. Lots of client API libraries also for C/C++, Java, JavaScript and others. There is some initial AMQP support.
You didn't mention any persistence requirement, but again I would have thought that most RDBMSs would suffice. That said, some of the document oriented and big-table type databases look interesting from a horizontal scaling perspective.
I've also found Apache Camel highly performant and I strongly recommend it. Camel is used to implement your logic layer.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 years ago.
Improve this question
I have the impression that sockets are just one of several ways that an application can communicate over the internet, maybe because of the wording in some tutorials I have read recently where it seems like sockets have been chosen as the most efficient tool out of many for the job.
Is this assumption right or do all networking applications use sockets on some level?
Edit: To clarify I'm looking for a practical answer. If I want to write some client/server program that will run on two of my computers running either Linux or Windows, should I always think sockets first (or higher level protocol that uses sockets), or are there other equivalents that may be better suited for the job than sockets.
It is operating system specific. Some (old, or academic) OSes provide weird APIs for TCP/IP, and some OSes don't even do any networking. IIRC, Gnu Hurd has something else than sockets (and probably Fuschia also has something else).
In practice, Berkeley sockets are nearly a de facto standard (this was not always the case, e.g. TLI), at least on Unix-like and POSIX systems (and probably on Windows also) and when programming in C (or related stuff, notably C++).
And you could use higher-level libraries, e.g. MPI, 0mq, or protocol specific libraries (for HTTP, libcurl on client side, libonion on server side, ....) etc ..... They are generally built above sockets.
do all networking applications use sockets on some level?
On some supercomputers, you can also have InfiniBand (and perhaps a variant of MPI using that hardware without sockets, but with something else). etc... Sometimes they have some InfinBand specific address family for socket ....
(It is more than ten years that I didn't access any supercomputer, so I could be wrong for the details)
BTW, some consortia like FIWIRE or AUTOSAR define their own networking APIs (for niche markets or specific industries).
(perhaps such consortia are defining an API above sockets; I don't know them well enough; you need to check)
See also the libraries mentioned near the end of this answer.
If I want to write some client/server program that will run on two of my computers running either Linux or Windows, should I always think sockets first (or higher level protocol that uses sockets),
Certainly yes in practice. And I would often consider using some existing free software library above Berkeley sockets.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I am learning about REST API and I am unable to understand how REST is used in distributed web application?
This is the only reference i have seen. But still I am unable to learn about REST for distributed computing.
Thank You in advance.
It isn’t really clear to me, what you are asking with your question, but generally - REST is just another way to do RMI (Remote Method Invocation) or RPC (Remote Procedure Call). However, while RMI works only within Java, REST uses the HTTP Protocol to for communication. Since HTTP is implemented in most of the technologies / libraries / languages we use today, it is an easy way to connect them.
Originally SOAP (Simple Object Access Protocol) was used to implement inter server and client-server communications. SOAP has many additional features on top of HTTP. The WSDL (Web Services Description Language) allows for automatic proxy generation, for instance. While those features make SOAP rich and perfect fit for Enterprise applications (CERN implemented their own version of SOAP which allowed state-full communication), it was also too bulky for many, quickly changing, smaller companies.
REST uses features from HTTP but can basically vary in many ways. The url's and the objects can be defined freely, as well as the format in which objects are serialised (mostly JSON). This feature paired with dynamic languages like Python, Ruby or JavaScript (Client, or NodeJS) makes it very easy to set up communication between different services as well as client and service (SPA - Single Page Application, for the later).
However, I think the very interesting fact is, that people discovered, you have to pay a high price for this elasticity:
JSON represented objects - although smaller than XML - are still larger than byte-code (this can be solved to a large part by gzip, but creates a second problem)
Serialising and Deserialising object to/from strings is very inefficient und much slower than byte-code representations. (And of course zipping the strings to reduce size also costs CPU)
So far there was the choice - HTTP, which is inefficient, or RMI, which is inflexible and can be used only by few languages. This is why there are two projects to solve this:
Protocol Buffers from Google https://code.google.com/p/protobuf/
Apache Thrift http://thrift.apache.org/
Both of those projects allow you to use a specific binary format, to you and your messages. And because both projects have implementations in different languages (Apache Thrift many more than Google's Protocol Buffer) you can use this format to communicate between different servers.
Also a direct end-to-end communication is not always what you want, which is, why there are different Messaging Queues which can fulfil many tasks additionally to message forwarding (for instance publish-subscribe, round robin delivery to a set of services, ...) The probably most widely used one is ZeroMQ.
Conclusion
You can use REST to communicate between different services in your distributed web applications. And this is also often used, due to the simplicity of implementing such a communication channel between many different hosts and technologies. However the overhead in serialising / deserialising can cost you a lot of CPU-Time, especially if you have a large backend infrastructure with many services. This is why you should rather choose one of the binary formats (Apache Thrift, Protocol Buffers), to ensure efficiency.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
I think I summed it up in the title. Regarding the question, I have been evaluating my options for utilities that allow me to craft, capture, and analyze TCP/IP packets. I am very new to network programming and I am still learning as I go, but at this point, I want to write a program that establishes a TCP connection.
Here are my goals (long term to short term):
Deploy and code a system that can establish an ssh connection between two computers behind NATS given their local host names (i.e. feynman.home to feynman.work) and the IP addresses of their routers. I would also like to be able to guide the connection across a set of hops if possible/necessary.
No using third parties or closed source code of any kind. #
Rewrite (and possibly modify) pwnat in a "cleaner" format (i.e. using some well developed library for a high level language.)
Establish ssh connection between two hosts behind the same NAT (using my own code of course.)
Establish TCP connection between two hosts behind the same NAT (using my own code of course.)
I have managed to find netexpect which seemed very promising until it failed to build on my Mac OS X (my computer is going through a glitchy period right now.) Netexpect may well be the best option, but I have come across quite a few others (e.g. scapy and its perl and ruby implementations.) Many of these are not well maintained (e.g. scapy and its perl and ruby implementations.)
So, I thought it would be best to bring the question to stackoverflow (unless there is a better stackexchange I am not aware of.)
*Given the current state of my computer (don't ask) portability is important.
*I would prefer something well maintained/under active developement as to best assure I will have all the tools I will need for my short and long term goals.
*Finally, I would prefer something that uses an interpreted language for a host of reasons ranging from my experience to my preference.
#Yes I am fully aware there are easier ways. I want to do it my way because this just as much a learning experience as it is practical.
I know you are looking for a library on an interpreted language, but libcrafter is very easy to use. Is a C++ library for crafting and parsing network packets. On the future, it will support more protocols.
Most of what you described involving routing the connection between multiple hops/NATs - isn't done in the SSH client at all. It would involve having access to several network routers and setting up all those NATs in advance, then those translations would already happen automatically when the connection is routed through.
Are you talking about proxying your connections through multiple servers? That could be accomplished by a macro that logs in to server A, then automatically invokes ssh again from server A to get to server B and so on.
Could you describe in some more detail what you are trying to accomplish?
Edit: I just read the description of pwnat. That is incredibly proprietary (and won't work in every situation anyway, since many configurations block ICMP completely). If you really wanted to try it out, you'll probably be stuck with C/C++. I don't think Java or .NET gives you the kind of low-level flexibility that you would need to packet craft, for security reasons. Have you looked at Ostinato: http://code.google.com/p/ostinato/wiki/Downloads?tm=2 or Nemesis: http://nemesis.sourceforge.net/
Pretty much everything on windows will use winpcap and on *nix it will use libpcap.
While I haven't tried it, this might work: http://www.codeproject.com/Articles/12458/SharpPcap-A-Packet-Capture-Framework-for-NET
It says it SharpPcap handles interfacing with winpcap... so depending on what they implemented, or whether you can extend that functionality, you may be able to put something together.
Or, in java, you might be able to use something like this as a starting point: http://jpcap.sourceforge.net/
libtins is an open source, multi-platform, C++ library I've been developing for the past year. You can forge, sniff and interpret packets from network interfaces using it. It's very simple to use, and it provides a high level interface so you don't have to worry about endianness, internal protocol implementations, etc.
You can have a look at it at the library's website.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
I've been engaged in a project to write a game dissemination and service platform, which would enable game developers to easily outsource game distribution and updates as well as offering certain parts of multiplayer/community such as achievements, rankings, friends and such as a webservice in form of either SOAP or REST API.
I'm not sure how viable it is in terms of attaining market share, but none the less the application should be written as to scale horizontally well in order to absorb any quantity of possible users. Since the project is still in it's early stages I have some degree of influence regarding the language and technologies to be used.
So far my research has narrowed the choice down to either Erlang or a JVM based language.
Erlang is widely known for it's focus on scalable and fault-tolerant distributed applications and proven track record, but on the other hand it's library ecosystem is nowhere as rich as JVM's.
JVM languages on the other hand enjoy a big community with a multitude libraries, many of which strive to provide for distributed and reliable services, such as Terracota or Akka (which seems to be inspired by Erlang, though I'm not sure how much of it's strength it approaches and would like to hear about it).
Since language productivity is of equal importance as platform strength and reliability I would lean towards Scala or Clojure over Java if I were to use JVM.
Of course if you think there is another language (JVM or non-) better suited to such a task I'm interested to hear about it.
I wouldn't like to read a bulletpoint list of language's cool features as that's what I can google by myself, I'd rather be interested in your own experiences with writing such an application (a bit in the vein of those blogposts - link), though not only concerning raw performance but also language and library issues.
I would like to hear what your (or your company's) language and technology choices for such a type of project were, what motivated such a choice, what your experiences using the chosen platform were and did it ultimately deliver.
All insight will be greatly appreciated ; )
Given your initial interest in Erlang, I'd have to say Scala.
Scala actors were directly inspired by Erlang, and Akka started (long ago now) as a port of Erlang's OTP to the Scala language.
Both Scala and Akka are robust solutions used in commercial settings, and are well supported by the recently formed "Typesafe" company. Plus, you get all your stated advantages of running on the JVM platform.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I've been looking at some open-source XMPP servers, and am familiar with the official page http://xmpp.org/. But thus far I've not found anything in between "The Extensible Messaging and Presence Protocol (XMPP) is an open technology for real-time communication" and a list of XEP specifications. For instance articles explaining the basics and terminology - stanzas, IQ, presence, etc, etc. Even the Wikipedia page misses this, unsurprisingly the open-source projects assume you know these things before you start digging into the code.
Is there a good, (semi-)official set of tutorials on this? Do I need to be looking for Jabber resources rather than XMPP?
Amongst other things, I'd hope to see diagrams for use-cases and flow, not just dry protocol text. I know books on XMPP exist, but generally anything in a book is available in some form online too.
This is probably way too basic, but at least it's technical: https://web.archive.org/web/20170916193014/http://www.adarshr.com/fun-with-xmpp-and-google-talk and the second part, https://web.archive.org/web/20171005104211/http://www.adarshr.com:80/fun-with-xmpp-and-google-talk-part-2
It explains what stanzas are, what types are available and stuff.
Here is what got me startet on XMPP Development:
A good book: XMPP The Definivie Guide
A mature Java API. I've chosen the Smack Library from Ignite Realtime and used the groovy language with a buch of small scripts to learn the basics.
Later i developed a plugin for the OpenFire XMPP Server. There are some tutorials and a forum on their site as well. I think that both the smack and the openfire api's are easy to learn.
If you are not into java: The book referes to the SkeekXMPP Python library and it uses it to create some examples (echo bot, ...).
As others have said, the specifications are a good introduction. It's true that they are technical in nature, and worded to be precise - but they are really some of the best specifications I've seen for any protocol, especially the latest RFCs (6120 and 6121) which clarify some of the grey areas in the originals.
E.g. you mention wanting to know the definition of a stanza, it's explained (with examples) in 6120 section 8.
If you have any feedback on how the specifications can be made clearer, then say so on the XMPP mailing list, where all feedback is considered for the next drafts of the specifications.
If the specifications are really too much for you (I appreciate some people like more pictures than I do), do consider the book (whether in paper or digital form) - it's designed exactly as an easy introduction to both the core specifications and the most common extensions, and written by people who help develop and implement them.
The RFCs (listed on the Wikipedia page) should be a quite good introduction to this topic.
For example: RFC3920: Extensible Messaging and Presence Protocol (XMPP): Core
This might be an old question, but I just wanted to keep the process I used in order to learn XMPP.
A few years ago, a few friends of mine and I were learning about how to leverage XMPP, and understanding how it fits into larger piece is quite a tedious task. I highly recommend starting off by reading the wikipedia page of XMPP:
http://en.wikipedia.org/wiki/XMPP
You'll be surprised how many people aren't able to answer questions about XMPP which are the most fundamental.
I also highly recommend reading this article:
http://www.infoworld.com/article/2682116/application-development/xmpp-rises-to-face-simple-standard.html
It'll give you a sense of the motivation behind XMPP, it's history, and it's protocols that used to be on par with it.
From there, it'll be best to read the sources of the wikipedia page to give a more indept understanding of any features you might be interested in with XMPP.
Use the xmpp asmack library from
http://beem-project.com/projects/beem/files
download asmack-android-7-beem-jingle.jar
and documentation of
http://www.igniterealtime.org/downloads/index.jsp
Hope it helps others like it helped me
Install openfire on server side and use qsmack on android side.