“Hook” libMMS to FFmpeg for iPhone Streaming - streaming

These days, I was researching the software architechture for iPhone Streaming (Base on MMS protocol).
As we know, in order to playback MMS audio stream, we should call libMMS to read wma stream data from remote media server, and then call FFmpeg to decode the stream data from wma format into PCM data buffer, and finally, enqueue the PCM data buffer into iPhone’s audioqueue to generate real sound.
The introduction above just describe the working process of iPhone streaming. If we only need to implement this simple functionality, that is not difficult. Just follow the introduction above to call libMMS, FFMpeg and audioqueue step by step, we can achieve the streaming function. Actually, I have implemented the code last week.
But, what I need is not only a simple streaming function! I need a software architechture makes FFmpeg accessing libMMS just like accessing local filesystem!
Does anybody know how to hook the libMMS interfaces like mms_read/mms_seek onto FFmpeg filesystem interfaces like av_read_frame/av_seek_frame?

I think I have to answer my own question again this time……
After several weeks reseach and debuging, I got the truth finally.
Actually, we don’t need to “hook” libMMS onto FFMpeg. Why? Because the FFMpeg already has its native mms protocol process module “mms_protocol” (see in mms_protocol.c in FFMpeg).
All we need to do is just configuring the FFMpeg to enable the mms module like this (see in config.h in FFMpeg):
#define ENABLE_MMS_PROTOCOL 1
#define CONFIG_MMS_PROTOCOL 1
After this configuration, FFMpeg will add mms protocol into its protocol list. (Actually, the protocol list has already contained “local file system protocol”). As result, the FFMpeg could be able to treat the “mms://hostserver/abc” media file like local media file. Therefore, we can still open and read the mms media file using:
av_open_input_file();
av_read_frame();
like we did on local media file before!
By the way, in my ffmpeg version, there are still many bugs in libAVFormat module for proccessing mms protocol. It took one week for me to debug it, however, I think it will be much shorter for the guy as smart as you:-)

Related

Mac/iPhone:Streaming video file to iPhone

I have a http streaming link which gives me .flv streaming feed. I want to convert that and access in my iPhone program. How can i do that? I want to have a desktop software like VLC and input this streaming feed URL and convert to iPhone supported and stream again to iPhone. I tried VLC with H.264 and Mpeg-1 audio, but seems to be it doesn't give the supported format, so as iPhone program doesn't play the video.
Could someone please guide me how can i setup a desktop software which can stream iPhone supported file?
Thanks in advance.
I think even the great VLC can't convert FLV on the fly...(or even do anything with FLV). As far as streaming goes, you'll probably be limited to the local network (Wi-Fi). I'd start with the simple way—create an ad-hoc file server on the desktop, then use AVPlayer's initWithURL method to find that video.
On the desktop, you could query the IP address of the computer, and ask the user to enter that URL (along with an optional port assignment and file component, like http://192.168.0.2:2234/streamingVideo.mp4) onto the iDevice, then convert to NSURL.
What exactly is the http streaming link? This matters a lot as in order to stream to the iPhone you need to use HTTP Live Streaming which requires some different bits than a typical flash media, or more properly RTMP, server. Typically you need two different streaming architectures or some expensive boxes.

iPhone: HTTP live streaming without any server side processing

I want to be able to (live) stream the frames/video FROM the iPhone camera to the internet. I've seen in a Thread (streaming video FROM an iPhone) that it's possible using AVCaptureSession's beginConfiguration and commitConfiguration. But I don't know how to start designing this task. There are already a lot of tutorials about how to stream video TO the iPhone, and it is not actually what I am searching for.
Could you guys give me any ideas which could help me further?
That's a tricky one. You should be able to do it, but it won't be easy.
One way that wouldn't be live (not answering your need, but worth mentioning) is to capture from the camera and save it to a video file. see the AV Foundation Guide on how to do that. Once saved you can then use the HTTP Live Streaming segmenter to generate the proper segments. Apple has applications for Mac OSX, but there's an open source version as well that you could adapt for iOS. On top of that, you'd also have to run an http server to serve those segments. Lots of http servers out there you could adapt.
But to do it live, first as you have already found, you need to collect frames from the camera. Once you have those you want to convert them to h.264. For that you want ffmpeg. Basically you shove the images to ffmpeg's AVPicture, making a stream. Then you'd need to manage that stream so that the live streaming segmenter recognized it as a live streaming h.264 device. I'm not sure how to do that, and it sounds like some serious work. Once you've done that, then you need to have an http server, serving that stream.
What might actually be easier would be to use an RTP/RTSP based stream instead. That approach is covered by open source versions of RTP and ffmpeg supports that fully. It's not http live streaming, but it will work well enough.

live streaming in iPhone

Will the ALAC format support live streaming in iPhone ? the ALAC audio recording format is streamed to Server machine? so will i be able to play the audio chunk data, does ALAC format support?
Thank You.
Assuming you mean "Apple Lossless" audio...
I don't see why it wouldn't, but I don't know the details. You'll probably need to embed it in a transport stream instead of a MPEG 4 container (but then, I don't know how the HTTP live streaming works either).
I don't think streaming lossless audio is sensible, though.
Streaming lossless audio is possible, we have flac streaming using icecast and it works beautifully. However, we are not using HTTP Live Stream (HLS) to do it. We stream flac from the source generator to a number of servers and they create HLS's from there.
It is technically possible to mux alac into mpegts (ffmpeg can do this) as well as play it back (using ffmpeg), but there isn't a format identifier for other clients. Adding this feature to HLS will be as easy as calling/writing Apple and asking them to add ALAC to this list:
http://www.smpte-ra.org/mpegreg/mpegreg.html
and update their products accordingly. If you've purchased an Apple product less than 90 days ago, or you have AppleCare: give them a call. They have to work on the issue for you if you are covered. The more requests that get elevated to their engineers, the more likely they are to add support for alac in HLS.

Streaming live H.264 video via RTSP to iphone does work! w/example

Using FFMPEG, Live555, JSON
Not sure how it works but if you look at the source files at http://github.com/dropcam/dropcam_for_iphone you can see that they are using a combination of open source projects like FFMPEG, Live555, JSON etc. Using Wireshark to sniff the packets sent from one of the public cameras that's available to view with the free "Dropcam For Iphone App" at the App Store, I was able to confirm that the iphone was receiving H264 video via RTP/RTSP/RTCP and even RTMPT which looks like maybe some of the stream is tunneled?
Maybe someone could take a look at the open source files and explain how they got RTSP to work on the iphone.
Thanks for the info TinC0ils. After digging a little deeper I'v read that they have modified the Axis camera with custom firmware to limit the streaming to just a single 320x240 H264 feed, to better provide a consistent quality video over different networks and, as you point out, be less of a draw on the phone's hardware etc. My interest was driven by a desire to use my iphone to view live video and audio from a couple of IP cameras that I own without the jerkiness of MJPEG or the inherent latency that is involved with "http live streaming". I think Dropcam have done an excellent job with their hardware/software combo, I just don't need any new hardware at the moment.
Oh yeah, I almost forgot the reason of this post RTSP PROTOCOL DOES WORK ON THE IPHONE!
They are using open source projects to receive the frames and decoding in software instead of using hardware decoders. This will work, however, this runs counter to Apple's requirement that you use their HTTP Streaming. It will also require greater CPU resources such that it doesn't decode video at the desired fps/resolution on older devices and/or decrease battery life compared to HTTP streaming.

flv stream decoding, 3gp encoding

Are there any open source projects in any language and other recourses that I need to look at in order to implement flv to 3gp conversion? It's better to be streaming, I mean return first portion of 3gp before last portion of flv is downloaded.
Or are there any similar services already implemented - my goal is to have something like
http://converter.org?source=sourceUrl.flv&targetFormat=3gp that I can feed to 3gp player, in my case - on iPhone, and not wait until server downloads entire flv.
UPDATE: ffmpeg does really good job here, just
ffmpeg -i input.flv output.mp4
and that's it. But output file can be used only after conversion is done. Streaming is still an open question. There is ffserver that does some sort of streaming but I could not make it work.
I'm not sure if it is possible, but if there's one thing that should do it.
it's http://www.ffmpeg.org/ ..
it can convert anything to anything on an online platform. don't know if it supports streaming but definitely the best solution for online video conversion
Well this one might be a little late to the party but to stream video online you'll need a Media Streaming Server to deliver the video over a specific streaming protocol (i.e. HTTP,HTTPS,RTSP,RTMP). I've also been looking for such a "real-time" transcoding service but the closest thing I've found so far is the Video CDN's which are quite pricy, and also limited in formats/support. What would be really nice is for one of the media servers to add in a real-time transcoding feature. At the time of this writing no such service exists that I know of.
The top 10 most popular options for Media Streaming Servers are (IMHO):
VideoLAN - VLC Media Player (good for quick tests and proof-of-concept)
Kaltura - Open Source video platform
Real Media - Helix Universal Streaming Server (may be best bet for 3GP over RTSP)
Apple - Darwin Streaming Server / Quicktime (Live) Broadcaster (best for iPhone/iPad)
Red5 - Open Source Flash Streaming Server
Adobe - Flash Streaming Server
Wowza - Media Server
FluMotion - Open Source Multimedia Streaming
Microsoft - Windows Media Server (AVI, WMV, Silverlight & other formats)
FreeCast - An OGG Theoris (video) and OGG (audio) streaming/conversion platform
As you can see there are many options for streaming and you can start as simply as hosting the video on the same server and delivering to Desktop computer browsers via HTTP (the easiest way to get started with this is trial & error). Each offers different features in terms of protocols supported and transcoding, but none are truly real-time as you mention where you could feed in a source video and get an output video in the format of you choosing (i.e. 3GP). My personal choice would be to start with VLC for small-scale tests on a home network, since it is basically a swiss-army knife for desktop video that can also act as a server for any of the formats it can playback (though it may be more complicated to get this to stream to the public internet and even harder to go all the way to a single device on a private carrier network, some info is available from people who've tried):
http://forum.videolan.org/viewtopic.php?f=4&t=45782
Some basic transcoding instructions to go to MP4 (required for iPhone/iPad/iPods):
http://wiki.videolan.org/MPEG-4
I also agree about FFMPEG being by far the best solution for video conversion, as it also supports 3GP and you can at least start playing around with conversion on your own test server, you might want to try the following PHP Classes project:
http://www.phpclasses.org/package/5977-PHP-Manipulate-video-files-using-the-ffmpeg-program.html
In my experience that was an excellent contribution to speed web interaction with FFMPEG's mostly command-line and sometimes clunky interface. Who knows, maybe you'll build the first real-time transcoding service, I'd be the first to signup as a customer and/or as a contributor to help you on that!