If you are trying to develop an interactive livestream application, you rely on ultra low (real-time) latency. For example for a video conference or a remote laboratory.
The two protocols, which should be suitable for this circumstances are:
RTSP, while transmitting the data over RTP
RTMP
*WebRTC: As I'm trying to give a bigger audience the possibility to interact with each other, WebRTC is not suitable. Because as far as I know it is not designed for a bigger audience.
My questions:
Which one should I choose for this use-case? RTSP/RTP or RTMP?
Which protocol delivers better results regarding end-to-end latency, session start-up time?
Which one consumes more hardware resources?
RTMP seems to use a persistent TCP connection. But which protocol is used for the transmission? It cannot be TCP, because this could not ensure real-time latency?
What are in general the pros and cons for using either of the protocols?
I did not find any comparison of these two protocols in scientific papers or books. Only that the famous mobile live-streaming app Periscope is using RTMP.
Other apps like Instagram or Facebook are for example providing text-based interaction with the streamer. If developers want to build the next "killer application" based on interactive live-streams: I think this question is essential to answer.
You make a lot of assumptions in your answer.
WebRTC: As I'm trying to give a bigger audience the possibility to interact with each other, WebRTC is not suitable. Because as far as I know it is not designed for a bigger audience.
That's simply not true. WebRTC doesn't know or care how you structure your applications server-side. There are plenty of off-the-shelf services for handling large group calls and low latency video distribution via WebRTC.
You should also know that for the media streams, WebRTC is RTP under the hood.
It cannot be TCP, because this could not ensure real-time latency?
Of course it can. There's some overhead with TCP, but nothing that prevents you from using it in a real time scenario. The overhead with TCP is minimal.
UDP is traditionally used for these sorts of scenarios, as reliability isn't required, but that doesn't mean TCP can't be used almost as performantly.
RTMP
RTMP is a dead protocol for Flash. No browsers support it. Other clients only support it for legacy reasons. You shouldn't use it for anything new going forward.
Only that the famous mobile live-streaming app Periscope is using RTMP.
Well, that's not a reason to do much of anything.
Which protocol delivers better results regarding end-to-end latency, session start-up time?
WebRTC
Which one consumes more hardware resources?
That's not the right question to ask. Your overhead in almost any other parts of the application is going to be far more than the transport overhead of the protocol used for distribution.
The real list of things you need to think about:
Client compatibility. What sort of clients must you support?
Do you really need low latency everywhere? Do you understand the tradeoffs you're making with that demand? Are you willing to destroy any sense of video quality and reliability for all your users if only a handful of them are going to be interactive?
What's your budget? Off-the-shelf solutions for distribution are much cheaper. If you can push off your stream to YouTube for non-interactive users, you can save yourself a ton of money. If you can't use existing infrastructure, be prepared to spend mountains of cash.
What are your actual latency requirements? Are you prepared to reduce the number of people that can use your application when these latency requirements cannot be met on crappier networks and mobile devices?
What are your quality requirements?
Where will you transcode video to a variety of bitrates?
Do your viewers need adaptive bitrate viewing?
Do you need to push streams to other platforms simultaneously?
Do you need to record the streaming for on-demand watching or going back in time?
You might also find my post here helpful: https://stackoverflow.com/a/37475943/362536
In short, check your assumptions. Understand the tradeoffs. Make decisions based on real information, not sweeping generalizations.
Related
We are planning a real-time network game application for mobiles.
After some researches and studying, we decided to use a TCP connection between the server and the clients (mobile devices) by implementing Socket.
As we are writing the protocols, we came across the following question: where should we implement the computation? In the user's device or in the server?
For example, imagine a short game between 5 players which does not require communication until the game ends (hitting the screen as fast as you can or something like that).
At the end of the round, the clients send their score to the server, and the application should display the loser or the winner.
Where should the computation take place? Should the server collect all of the scores, compute the maximum and the minimum and send back the loser and winner? Or should the server send back all the scores to the players, in which their devices will compute the results?
What are the pros and cons of each strategy?
A couple of things here:
1) Do not use a persistent socket for mobile applications. Connectivity is too spotty.
2) Use a higher level protocol, like HTTP. You have no need to use such a low-level protocol. By doing so you are going to end up doing far more work than you need to. Think about if you want to build this into a web-based game, or implement on a device that doesn't allow you to use raw sockets.
HTTP is nice because it is firewall friendly, most firewalls already let HTTP on standard ports through. This is why it is so popular these days.
3) You should think very skeptically about the devices. In general you should trust clients as little as possible. You can do processing on the client, but it should be related to rendering. Generally assuming that your client is malicious is a good approach to n-tier architecture applications. Especially with games.
I'm planning on building an application where audio media is going to be streamed to the mobile phone for the user to listen.
The targets are smartphones: iPhone/Blackberry/Android/(J2ME ?).
I see that streaming on iPhone has to be done with HTTP Live streaming, but I don't see it supported by other platforms.
Should I broadcast the streams via rstp ? http ? Is there any way to use a unified solution for all the different mobile platform ? If anyone already had to go through this, help would be greatly appreciated.
One answer to the question "what technology to use ?", for iPhone specifically is WiFi. I know that's not the type of question you are asking, but its a point worth making! Many apps that support streaming over 3G have been rejected by Apple due to bandwidth usage. You may need to be prepared to sense network connection type and limit streaming to when you have a WiFi connection only.
Blackberry works with http and RSTP on OS 4.3 or later. I'm not familiar with other platforms but I would think http would be the most compatible.
Here is a PDF that lists the supported types by the major models.
http://docs.blackberry.com/en/smartphone_users/deliverables/15801/711-01774-123_Supported_Media_Types_on_BlackBerry_Smartphones.pdf
You will probably want to do RTSP, but it doesn't really matter. HTTP Live Streaming is just a protocol on the client side I am pretty sure. All these acronyms just describe ways of transmitting data. If a browser can access the data for a given protocol....chances are a phone can too. It sounds like you are asking more of a server side question.....but that question is the least of your worries You are going to have to think more along the lines of "How am I going to scale this" rather than "What protocol should I use to transmit data". Also, the unified solution for all clients, would be to have a server that they all hit for data. You still need to develop separate clients for each OS.
Both Android and BlackBerry supports RTSP.
Note that some BlackBerry devices only support 15fps video, so you may need separated streams to give the best experience to your users.
iPhone, starting from iPhone OS 3.0, needs HTTP Live Streaming.
The only software solution I know to support all above is Wowza, but you still need an encoder. (I think Wowza supports RTP as input, but need double checking.)
iPhone can play non-streamed audio (progressive download). Considered all platforms you would normally you just needs streams that are suitable transcoded. See f.e. https://developer.apple.com/library/content/documentation/NetworkingInternet/Conceptual/StreamingMediaGuide/Introduction/Introduction.html The title says its about 'HTTP Live Streaming' but a lot applies for just downloading and playing streams.
I have a card game on the iphone and I really would like to take it to the next step by allowing players to interact with each other in real-time environment.
My questions:
Do I need a web server ?
Is there a third party specifically for iphone multiplayer games that I can use to host the game?
I would probably need a database - what is the best solution?
Should Apple push notifications be considered ?
First off, I have never done one myself, but I imagine that any massively multi-player game would entail some sort of central server to hold statistics (wins/losses), rankings, game table information, etc.
I imagine some sort of remote service providing server is required. A web server would certainly fit the bill and would most likely be the most logical solution, but certainly not your only choice.
Hosting of this service depends on the service you choose. If you're going the java route there are several places to look. Google's app engine, a straight up java web provider, amazon's cloud, etc. So many choices.
There are several open-source databases that would fit nicely with java. Postgres would come recommended by me. There are certainly others and other commercial options.
The server you use is bound to how fast the interactions with players need to be. The problem with HTTP would be that it's hard to have push notifications and it's also probably too heavy, depending on the type of game.
If it's in real time, but doesn't have a lot of interaction I'd probably use XMPP for the actual gameplay. If it's something where latency is more important, like a FPS or plataformer (actually any action game really) you'll probably need to roll your own protocol (and server obviously).
For the matchmaking a web server is enough.
Yes, you need a server for a traditional MMO, since an MMO is defined as a game where arbitrarily many people connect to each other. If you require people to be in the same vicinity, you can have an "ordinary" multiplayer game over the phones' wifi.
Doubtful. MMO servers require a lot of custom code. There are servers for high scores and such for more traditional games, however.
For your central database, this is entirely a question of preference. It sounds like cost is important to you, though, so I would go with MySQL since it's a free solution (and I think it's very nice to work with).
That depends on what gameplay will entail. If you're talking about a real-time game (like EverQuest or something), push notifications will probably not be helpful. If you're talking about something like Mafia Wars (is that the proper name of the Facebook one?), maybe.
I'd recommend looking at Parallel Kingdom, which is an existing MMO for the iPhone, iPod Touc and Android.
I am looking for the advantages of using RTSP to support streaming video to multiple devices. I have:
Benefits of TCP.
Commonly used protocol on web so supported by lots of infrastructure.
Commonly used on web so lots of tools and stacks.
Can use SSL.
Wide device support (not sure on this one)
Any other ideas?
RTSP has the benefit of being highly flexible, but at the cost of being more complicated to implement. It is also a widely supported standard, especially on mobile phones where it is a part of the 3GPP stack.
Answering directly to your list points:
RTSP can use UDP and TCP. Some clients will switch gracefully between transport protocols.
There are quite a few alternatives; both open source and comercial. The comercial offerings tend to be expensive but quite functional while the open source offerings are more flexible but tend to lack quite a lot of polish.
Although there are quite a few libraries, there are not as many finished projects as one would like, most projects are in a stable state but are quite old:
MPEG4IP (originally by Cisco) has not seen development in over a year
DSS lags behind QTSS occasionaly. Some talk of a fork has cropped up but nothing definite has shown up
VLC is the standard swiss army knife for media, including streaming, but the server is NOT designed for heavy load. VLC is also quite hard to coerce into being just a simple streamer client.
DSS does not support SSL for the RTSP connection as far as I know, and using SSL to stream the actual video may be too slow to be practical.
RTSP is supported by VLC, QuickTime Player, mplayer, RealPlayer and most 3G compatible mobile phones (mileage may vary though).
The advantage of using multicast and UDP as well as library support may make RTSP/RTP very worthwhile for internal applications. The breadth of support makes it also quite a good choice for applications which need to support a wide breadth of clients. The actual cost of complexity may not be so advantageous for external applications where compatability with preexisting technology is unnecesary, for example internet radios that do not need to support cellphones.
I'm kind of curious how I should approach a problem with iPhones and communication between two phones.
I have an idea for an application that needs to send data between two known phones. The problem is that the data could be very small or very large. I want to have intermediate storage on a server and a messaging layer, but I want there to be performant means to communicate between devices for shorter, unimportant messages.
You always hear about networking and client to client communication with popular platforms and protocols, but I'm curious what people's thoughts are on doing p2p style communication with an iPhone.
I'm pretty sure it should (and can) be treated as just a simple addressable machine... so I would have to build NAT traversing network code into the iPhone itself. A thought I had would be some sort of UPnP situation where I would ask to NAT a particular port for communication to the devices... but I guess I could just as easily turn it into a polling situation.
Granted, I understand that Apple will have a "push" mechanism for developers eventually but I want to get the app out the door before hand...
Just curious about thoughts on this type of thing... any thoughts on multi-player game situations would also directly apply to (and benefit from) this question.
Have you given the WiTap sample from Apple a try? It addresses the issue of discoverability (via Bonjour) and connectivity (via basic TCP). It may not be as robust as what you are looking for, but it's worth a look:
http://developer.apple.com/iphone/library/samplecode/WiTap/index.html
Look at the docs for Game Kit in the iPhone 3.0 SDK. It allows iPhones to connect to one another locally with Bluetooth, or via wi-fi/3G. It addresses both peer-to-peer and client-server architectures. It provides unreliable and reliable links.