The video quality of the first few seconds of encoded videos using azure do not look very good. It is lower quality and blurry. The quality improves afterwards dramatically.
What settings do you recommend to make sure the first frames are excellent quality. First perception is very important.
Thanks,
Osama
It may be possible that you observe this kind of behavior if you have encoded your file for adaptive streaming. In this case, the output asset is composed by different files of different quality (poor and high quality).
When you play an adaptive stream, the first parts downloaded are from the poor quality files and then the player detects your bandwidth and adapt the stream to a higher quality, automatically. If you look at YouTube, Netflix or Dailymotion, you will observe the exact same behavior. It allows to adapt the stream to the available bandwidth.
If you do not want an adaptive stream, you need to use a preset that encode the file in a given bitrate / quality.
The list of supported presets is here : https://msdn.microsoft.com/en-us/library/azure/mt269960.aspx
Multiple bitrates presets are for adaptive streaming.
Single bitrate presets are for single bitrate file.
For example, if your original video is 1080p, you can use this preset: https://msdn.microsoft.com/en-us/library/azure/mt269926.aspx
But be careful that all your users that will have a low bandwidth may not be able to play your content in a smooth way.
Agree with Julien,
You are probably seeing the Adaptive Bitrate streaming ramp up when playing back the content. That's normal behavior for adaptive streaming.
You can eliminate some of the lower bitrates from the encoding preset or restrict them at the client side using the Azure Media Player SDK.
Keep in mind that you can always customize your encoding presets. We support JSON schema for preset and you can define your own based on our existing presets that we ship as a "best practice" to get folks started.
I would recommend using the Azure Media Explorer tool to play around with different encoding settings and easily launch the player for preview. Go here to access the tool from our download page:
http://aka.ms/amse
Related
I am using LibVlcSharp to play an adaptive video stream (HLS) transcoded by Azure Media Services with the EncoderNamedPreset.AdaptiveStreaming setting in my Xamarin.Forms app.
When viewing my video I notice that the first few (5-6) seconds of my video are very blurry.
This is probably because the player starts with a "safe" low bitrate, and after a few chunks of data have been downloaded, it determines that the bandwith is sufficient for a higher quality video being displayed and it can switch up in quality.
This first few seconds of low quality bother me, and I would rather have it start at a higher bitrate.
I would be happy if the video would switch up sooner (<2 seconds), which would probably mean I need a different encoder setting that produces smaller "chunks" of video.
But maybe the easier solution is to set the starting bitrate to a higher value.
I have seen this done on other media players in the form of
ams.InitialBitrate = ams.AvailableBitrates.Max<uint>();
Does LibVlc have a similar option?
This is probably because the player starts with a "safe" low bitrate, and after a few chunks of data have been downloaded, it determines that the bandwith is sufficient for a higher quality video being displayed and it can switch up in quality.
Probably. You could verify that assumption by checking the bitrate at the start at the video and again when the quality improved, like this:
await media.Parse(MediaParseOptions.ParseNetwork);
foreach(var track in media.Tracks)
{
Debug.WriteLine($"{nameof(track.Bitrate)}: {track.Bitrate}");
}
Does LibVlc have a similar option?
Try this
--adaptive-logic={,predictive,nearoptimal,rate,fixedrate,lowest,highest}
Adaptive Logic
You can try it like new LibVLC("--adaptive-logic=highest")
See the docs for more info, or the forums. If that does not work, I'd set it server-side.
I am currently developing an app that will contain large amounts of audio, around 60-120 minutes. Most voice audio files. My question is really what is the best way to go about storing them. For example, one single large file, separate audio files, download-as-needed cache files.
Any suggestions on file format?
These are the audio formats decoded by iPhone hardware that should take the least power to play.
Other iPhone OS audio formats employ a hardware codec for playback.
These formats are:
AAC
ALAC (Apple Lossless)
MP3
Whether to have the audio distributed with the app or separately would depend on the use. If you could reasonably expect the user to go through the material sequentially, you may want to allow the user to download part by part or stream the audio to let them conserve space on their device, while if the audio is more random access, you'd probably want it all on the device.
Several apps, including Apple's own, appear to use the open source speex codec for compressed voice-quality audio, even though this seems not to be supported by the hardware or any public API.
As Joachim suggested you can choose from AAC/ALAC/MP3 audio formats. What I'd propose now is to also consider the issue from user experience point of view:
Convert all your audio to chosen format with quality options that
satisfy you and your potential users.
Next, calculate the size of all your files and ask yourself a questions:
"are X megabytes too much to bundle for my kind of app?" and
"will that big/small app bundle encourage users to download my app?".
Optionally play a bit with quality options to shrink files (iterate).
In the next step, decide (based on you app characteristics) whether to bundle all files. For example a game is expected to have all files in place and can be big (users accept that). If your app has e.g. podcasts only, then select the best one and bundle it - once user is hooked he can download the rest (let user trigger that), so files are stored on device. Also provide user the info how much data they are about to download and warn them if file is reasonably big and they're not on Wi-Fi; or introduce the option to download only on Wi-Fi.
I hope that sounds reasonable.
For music, the following approach would be much different.
Since it's just voice, you can reduce the sample rate significantly in the majority of cases. Try [8kHz…20kHz].
In case they are multichannel - Mono should be fine for voice.
Once that's been done, I'd recommend AAC for size and quality balance.
Do some listening tests on your devices. Tweak settings if needed. Then batch process/convert them all. That can reduce your sizes by ten or more if the sources are 16/44.1.
Unless they files are very small (e.g. seconds each) or you have to open and read many of them quickly, I wouldn't bother with the huge file. A few MB is a good size for many cases.
According to this What Techniques Are Best To Live Stream iPhone Video Camera Data To a Computer? is possible to get compressed data from iphone camera, but as I've been reading in the AVFoundation reference you only get uncompressed data.
So the questions are:
1) How to get compressed frames and audio from iPhone's camera?
2) Encoding uncompressed frames with ffmpeg's API is fast enough for real-time streaming?
Any help will be really appreciated.
Thanks.
You most likely already know....
1) How to get compressed frames and audio from iPhone's camera?
You can not do this. The AVFoundation API has prevented this from every angle. I even tried named pipes, and some other sneaky unix foo. No such luck. You have no choice but to write it to file. In your linked post a user suggest setting up the callback to deliver encoded frames. As far as I am aware this is not possible for H.264 streams. The capture delegate will deliver images encoded in a specific pixel format. It is the Movie Writers and AVAssetWriter that do the encoding.
2) Encoding uncompressed frames with ffmpeg's API is fast enough for real-time streaming?
Yes it is. However, you will have to use libx264 which gets you into GPL territory. That is not exactly compatible with the app store.
I would suggest using AVFoundation and AVAssetWriter for efficiency reasons.
I agree with Steve. I'd add that on trying with Apple's API, you're going to have to do some seriously nasty hacking. AVAssetWriter by default spends a second before spilling its buffer to file. I haven't found a way to change that with settings. The way around that seems to be to force small file writes and file close with the use of multiple AVAssetWriters. But then that introduces lots of overhead. It's not pretty.
Definitely file a new feature request with Apple (if you're an iOS developer). The more of us that do, the more likely they'll add some sort of writer that can write to a buffer and/or to a stream.
One addition I'd make to what Steve said on the x264 GPL issue is that I think you can get a commercial license for that which is better than GPL, but of course costs you money. But that means you could still use it and get pretty OK results, and not have to open up your own app source. Not as good as an augmented Apple API using their hardware codecs, but not bad.
I would like to stream video from an iPhone camera to an app running on a Mac. Think sorta like video chat but only one way, from the device to a receiver app (and it's not video chat).
My basic understanding so far:
You can use AVFoundation to get 'live' video camera data without saving to a file but it is uncompressed data and thus I'd have to handle compression on my own.
There's no built in AVCaptureOutput support for sending to a network location, I'd have to work this bit out on my own.
Am I right about the above or am I already off-track?
Apple Tech Q&A 1702 provides some info on saving off individual frames as images - is this the best way to go about this? Just saving off 30fps and then something like ffmpeg to compress 'em?
There's a lot of discussion of live streaming to the iPhone but far less info on people that are sending live video out. I'm hoping for some broad strokes to get me pointed in the right direction.
You can use AVCaptureVideoDataOutput and a sampleBufferDelegate to capture raw compressed frames, then you just need to stream them over the network. AVFoundation provides an API to encode frames to local video files, but doesn't provide any for streaming to the network. Your best bet is to find a library that streams raw frames over the network. I'd start with ffmpeg; I believe libavformat supports RTSP, look at the ffserver code.
Note that you should configure AVCaptureVideoDataOutput to give you compressed frames, so you avoid having to compress raw video frames without the benefit of hardware encoding.
This depends a lot on your target resolution and what type of frame rate performance you are targeting.
From an abstract point of view, I would probably have a capture thread to fill a buffer directly from AVCaptureOutput, and a communications thread to send and rezero the buffer (padded if need be) to a previously specified host every x milliseconds.
After you accomplish initial data transfer, I would work on achieving 15fps at the lowest resolution, and work my way up until the buffer overflows before the communication thread can transmit which would require balancing image resolution, buffer size (probably dependent on GSM, and soon to be CDMA frame sizes), and finally the maximum rate at which you can transmit that buffer.
Using FFMPEG, Live555, JSON
Not sure how it works but if you look at the source files at http://github.com/dropcam/dropcam_for_iphone you can see that they are using a combination of open source projects like FFMPEG, Live555, JSON etc. Using Wireshark to sniff the packets sent from one of the public cameras that's available to view with the free "Dropcam For Iphone App" at the App Store, I was able to confirm that the iphone was receiving H264 video via RTP/RTSP/RTCP and even RTMPT which looks like maybe some of the stream is tunneled?
Maybe someone could take a look at the open source files and explain how they got RTSP to work on the iphone.
Thanks for the info TinC0ils. After digging a little deeper I'v read that they have modified the Axis camera with custom firmware to limit the streaming to just a single 320x240 H264 feed, to better provide a consistent quality video over different networks and, as you point out, be less of a draw on the phone's hardware etc. My interest was driven by a desire to use my iphone to view live video and audio from a couple of IP cameras that I own without the jerkiness of MJPEG or the inherent latency that is involved with "http live streaming". I think Dropcam have done an excellent job with their hardware/software combo, I just don't need any new hardware at the moment.
Oh yeah, I almost forgot the reason of this post RTSP PROTOCOL DOES WORK ON THE IPHONE!
They are using open source projects to receive the frames and decoding in software instead of using hardware decoders. This will work, however, this runs counter to Apple's requirement that you use their HTTP Streaming. It will also require greater CPU resources such that it doesn't decode video at the desired fps/resolution on older devices and/or decrease battery life compared to HTTP streaming.