Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 12 months ago.
Improve this question
I am extracting frames from camera and doing processing on the extracted frame.Once the processing is done, I want to stream these frames with h.264 encoding to other system.How can I do that?
You will generally want to put the H.264 into a video container like MP4 or AVI.
For example the wrapping from raw frame to streaming protocol for online video might be:
raw pixels bitmap
raw pixels encoded (e.g. h.264 encoded)
encoded video stream packaged into container with audio streams, subtitles etc (e.g. mp4 container)
container broken into 'chunks' or segments for streaming (on iOS using HLS streaming format).
Another common approach is for a camera to stream content to a dedicated streaming server and the server then provide streams to end devices using a streaming protocol like HLS or MPEG DASH. An example (at the time of writing and it appear to be kept updated) showing a stream from a camera using RTSP to a Server and then HLS or MPEG DASH from the server is here:
https://www.wowza.com/docs/how-to-re-stream-video-from-an-ip-camera-rtsp-rtp-re-streaming
If your use case is simple you will possibly not want to use a segmented ABR streaming protocol like HLS or MPEG-DASH so you could just stream the mp4 file from a regular HTTP server.
One way to approach this that will allow you build on other's examples is to use openCV in Python - you can see an example in this question and answers of writing video frames to an AVI or MP4 container: Writing an mp4 video using python opencv
Once you have your MP4 file created you can place in a folder and use a regular HTTP server to make it available for users to download or stream.
Note that if you want to stream the frames as a live stream, i.e. as you are creating them one by one, then this is trickier as you won't simply have a complete MP4 file to stream. If you do want to do this then leveraging an existing implementation would be a good place to start - this one is an example of a point to point web socket based live stream and I open source and Python based:
https://github.com/rena2damas/remote-opencv-streaming-live-video
if you want to stream data over UDP socket - use RTP protocol for streaming.
Please go through the rfc specification of RFC 6184
Media Pieline for processing the camera data:
Camera RAW data ( RGB/YUV/NV12) -> H.264 encoder -> NALU packets RTP packetization-> Socket communication.
You can use ffmpeg python interface to achieve this goal.
Related
This question already has answers here:
How can we get H.264 encoded video stream from iPhone Camera?
(3 answers)
Closed 9 years ago.
I need to publish live video stream from iPhone Camera to RTMP Server (Wowza server). The video stream must be in h.264 format. I know AVFoundation stores video to a file in h.264 compression but I don;t need to store the video to a file. I just want to capture and send it to the server. I am using following delegate method :
- (void)captureOutput:(AVCaptureOutput *)captureOutput didOutputSampleBuffer:(CMSampleBufferRef)sampleBuffer fromConnection:(AVCaptureConnection *)connection{
if (connection == videoConnection) {
//I want something like this
NSData* h264VideoData=[self h264Data:sampleBuffer];
}
}
I don't need to send audio to the server, I just want to send video.
Update for 2017:
You can do streaming Video and Audio now by using the VideoToolbox API.
Read the documentation here: VTCompressionSession
Original answer (from 2013):
That is currently not possible, you'll have to write to a file or compress the video stream with a software-encoder (you won't get HD and very bad battery times though). All possibilities to get a hand on the hardware-encoder have to write to the disk. I think this is because of memory shortage on the devices.
Methods to get hardware accelerated h264 compression:
AVAssetWriter
AVCaptureMovieFileOutput
As you can see both write to a file, writing to a pipe does not work as the encoder updates header information after a frame or GOP has been fully written. So you better don't touch the file while the encoder writes to it as it does randomly rewrite header information. Without this header information the video file will not be playable (it updates the size field, so the first header written says the file is 0 bytes).
You can, however, record 5 seconds and then switch the output-file, transmit the now "old" 5 seconds snippet and delete it afterwards. You'll have to demux the *.mov or *.mp4 container though to get to the h264 video data to send.
If you need audio: if you switch the file you'll loose some audio samples, so you'll have to roll your own buffer management for that (or just record audio separately).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have an IP camera which is streaming via RTSP and RTP.
Ideally I would like to convert RTSP to RTMP to stream it to LiveStream or similar streaming services.
Can anyone please let me know how can I convert RTSP to RTMP for the purpose of streaming it to streaming services?
Using FFMPEG you can convert rtsp stream to rtmp
For Example
ffmpeg -i "[your rtsp link]" -f flv -r -s -an "[Your rtmp link]"
run the above syntax on ubuntu or linux os . it will convert your rtsp stream to rtmp stream
After some extensive research, I have found that almost all RTSP->RTMP "solution" providers use Wowza 2 to convert RTSP->RTMP. Thats it.
Once you tell them that you need anything else too, like to convert MPEG4 part 2 to MPEG4 part 10(H.264), they tell you that they cant do that.
Wowza 3, which will be released in October 2011, will have a transcoding module which which should be able to transcode the content in addition to RTSP->RTMP stream conversion.
other potential options are:
VLCplayer
mPlayer
FFmpeg
I am still researching and will update this topic once I am done.
To summarize your options, you can use one of the following streaming servers:
Wowza, Unreal Media Server, crtmpserver, erlyvideo. All of them will receive RTSP stream and re-stream it with RTMP.
You can also use Gstreamer for it. Just create a rtsp/rtp client (source), pipe it to mux optionally (If you need any transcode, you can append here) and sink to rtmp. Compared to VLC, the performance will be faster and it is free compared to Wowza.
I am seeking the following three items, which I cannot find on STACKOVERFLOW or anywhere:
sample code for AVFoundation capturing to file chunks (~10seconds) that are ready for compression?
sample code for compressing the video and audio for transmisison across the Internet?
ffmpeg?
sample code for HTTP Live Streaming sending files from iPhone to Internet server?
My goal is to use the iPhone as a high quality AV camcorder that streams to a remote server.
If the intervening data rate bogs down, files should buffer at the iPhone.
thanks.
You can use AVAssetWriter to encode a MP4 file of your desired length. The AV media will be encoded into the container in H264/AAC. You could then simply upload this to a remote server. If you wanted you could segment the video for HLS streaming, but keep in mind that HLS is designed as a server->client streaming protocol. There is no notion of push as far as I know. You would have to create a custom server to accept pushing of segmented video streams (which does not really make a lot of sense given the way HLS is designed. See the RFC Draft. A better approach might be to simply upload the MP4(s) via a TCP socket and have your server segment the video for streaming to client viewers. This could be easily done with FFmpeg either on the command line, or via a custom program.
I also wanted to add that if you try and stream 720p video over a cellular connection your app will more then likely get rejected for excessive data use.
Capture Video and Audio using AVFouncation. You can specify the Audio and Video codecs to kCMVideoCodecType_H264 and kAudioFormatMPEG4AAC, Frame sizes, Frame rates in AVCaptureformatDescription. It will give you Compressed H264 video and AAC aduio.
Encapsulate this and transmit to server using any RTP servers like Live555 Media.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I have an IP camera which is streaming via RTSP and RTP.
Ideally I would like to convert RTSP to RTMP to stream it to LiveStream or similar streaming services.
Can anyone please let me know how can I convert RTSP to RTMP for the purpose of streaming it to streaming services?
Using FFMPEG you can convert rtsp stream to rtmp
For Example
ffmpeg -i "[your rtsp link]" -f flv -r -s -an "[Your rtmp link]"
run the above syntax on ubuntu or linux os . it will convert your rtsp stream to rtmp stream
After some extensive research, I have found that almost all RTSP->RTMP "solution" providers use Wowza 2 to convert RTSP->RTMP. Thats it.
Once you tell them that you need anything else too, like to convert MPEG4 part 2 to MPEG4 part 10(H.264), they tell you that they cant do that.
Wowza 3, which will be released in October 2011, will have a transcoding module which which should be able to transcode the content in addition to RTSP->RTMP stream conversion.
other potential options are:
VLCplayer
mPlayer
FFmpeg
I am still researching and will update this topic once I am done.
To summarize your options, you can use one of the following streaming servers:
Wowza, Unreal Media Server, crtmpserver, erlyvideo. All of them will receive RTSP stream and re-stream it with RTMP.
You can also use Gstreamer for it. Just create a rtsp/rtp client (source), pipe it to mux optionally (If you need any transcode, you can append here) and sink to rtmp. Compared to VLC, the performance will be faster and it is free compared to Wowza.
This is related to my another question
Here I'd like to ask if it is in theory (according to video file formats and codecs, etc) possible to have such scenario:
1) Client on iPhone has a reference to video in flv format. It sends http request to converting "proxy" like http://convproxy.com?source=url_of_original_video.flv by just clicking such link in Safari
2) Converting proxy starts downloading that flv file and converting it to mp4 (which iphone understands) on the fly, returning converted portion as http response, so iPhone can immediately start playing it, before entire flv is downloaded and converted.
I was playing with ffmpeg trying to do such thing, and it indeed converts flv and produces mp4 file, however that mp4 file can not be played until convertion is finished or ffmpeg is stopped. If I just kill ffmpeg process the mp4 file can not be played. If I let it finish or press ctrl-c to stop it, the part that was downloaded and converted can be played. Seems like ffmpeg does some job after it receives stop signal. Is that a necessary part of mp4 format or it can be done differently? I see that iPhone can stream video, by starting playing before the entire file is downloaded to it, so in general it seems like possible scenario for me.
I short words, I can convert flv file to mp4 file, and the question is if I can convert flv stream to mp4 stream.
According to wikipedia, the MP4 container format requires a separate "hint track" to enable streaming. I assume ffmpeg writes this at the end of the conversion. If the iPhone OS requires this track to stream, I don't see a way to stream live video outside of using a different format and having a custom decoder on the iPhone side similar to how the Orb client for iPhone does it.