Is that mp4? Which exact encoding results in best performance? Couldn't find it in the class reference.
From the Apple Docs:
This class plays any movie or audio
file supported in iOS. This includes
both streamed content and fixed-length
files. For movie files, this typically
means files with the extensions .mov,
.mp4, .mpv, and .3gp and using one of
the following compression standards:
H.264 Baseline Profile Level 3.0
video, up to 640 x 480 at 30 fps. (The
Baseline profile does not support B
frames.) MPEG-4 Part 2 video (Simple
Profile) If you use this class to play
audio files, it displays a white
screen with a QuickTime logo while the
audio plays. For audio files, this
class supports AAC-LC audio at up to
48 kHz, and MP3 (MPEG-1 Audio Layer 3)
up to 48 kHz, stereo audio.
Related
I want to play video files other than MP4 in my code
is there any way to do it?
Even if there was any would apple allow such app in the appstore
Thanks in advance
Regards
Nitesh
from apple guide:
The video technologies in iOS support the playback of movie files with the .mov, .mp4, .m4v, and .3gp filename extensions and using the following compression standards:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in “Audio Technologies”
For a high-performance scientific purpose we need to render video and play it at 60fps on the device. I assume the usual frame rate of H.264 video is lower than that.
Is this possible, or is the framerate fixed? If so, what is the maximum frame rate we can get when playing H.264 video in fullscreen on the device?
Technical specifications will vary from iOS device to iOS device, so you'll need to check for the hardware you'll actually run this on. For the iPad 2, currently the most powerful of the iOS devices, Apple's technical specifications for video list the following:
Video formats supported: H.264 video up to 720p, 30 frames per second,
Main Profile level 3.1 with AAC-LC audio up to 160 Kbps, 48kHz, stereo
audio in .m4v, .mp4, and .mov file formats; MPEG-4 video, up to 2.5
Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with
AAC-LC audio up to 160 Kbps per channel, 48kHz, stereo audio in .m4v,
.mp4, and .mov file formats; Motion JPEG (M-JPEG) up to 35 Mbps, 1280
by 720 pixels, 30 frames per second, audio in ulaw, PCM stereo audio
in .avi file format
It would appear that fullscreen H.264 playback at 60 FPS is not supported on even the robust hardware of the iPad 2.
However, you can indeed render content to the screen at 60 FPS. I do this all the time in both Core Animation heavy applications and ones that use OpenGL ES. If you can generate your content in-application fast enough to display at this rate, you could render it to the screen at 60 FPS, then encode every other frame to video.
Given that video encoding is a reasonably expensive operation, and it sounds like you want to run some kind of simulation here as well, I'm guessing that you won't be able to render each frame at 60 FPS for display to the screen on current hardware simply due to the load you'll put on the system.
Yes, it is possible to encode video as a series of images and then display the images very quickly on screen. The upper limit of the video hardware and the time to decode the images and blit the images into the video card are the bottleneck in this process. As long as your image decoding logic is not too slow, it should be possible to push video data to the graphics card at 60FPS.
You could try to implement this yourself with a series of PNG images, but I think you will find that decoding the PNG image will not be fast enough to get 60 FPS playback. You can find some free example code that implements that approach with PNG images in my answer to this question
If you cannot get the performance you need, then take a look at my AVAnimator library for iOS, as it already fully solves this problem using memory mapped frames that can be sent directly to the video card from mapped memory.
I'm creating an iPhone app which can manipulate video files. I'm using AVFoundation classes (e.g., AVAsset). I just registered my application as a handler of all files conforming to public.movie (via CFBundleDocumentTypes). However, now my application is listed in the "Open With" menu for .avi files, even though I don't think iOS can play AVI movies (the Quick Look preview will try, but fails).
Is there a better way to register to open movies? I will also need to support File Sharing, so I need to distinguish which files in the Documents folder are valid movies as well, though I haven't figured out how to check the UTI of a file. The iOS Technology Overview says:
The video technologies in iOS support the playback of movie files with the .mov, .mp4, .m4v, and .3gp filename extensions and using the following compression standards:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second, Low-Complexity version of the H.264 Baseline Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second, Baseline Profile up to Level 1.3 with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps, 48 kHz, stereo audio in .m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in “Audio Technologies”
Thanks in advance for any tips you might have.
For video only, specifying the following UTIs in the Info.plist should work:
com.apple.m4v-video (.m4v)
com.apple.quicktime-movie (.qt, .mov)
public.mpeg-4 (.mp4)
You can find the UTIs supported by AVFoundation in AVMediaFormat.h:
https://developer.apple.com/library/mac/#documentation/AVFoundation/Reference/AVFoundation_Constants/Reference/reference.html%23//apple_ref/doc/uid/TP40009539-CH4g-DontLinkElementID_48
I've decided the best way to handle this is to determine the UTI of a file using the UT* functions to get a UTI from a file extension. Unfortunately this means that AVI files are still imported, because the OS knows they're movies even though they can't be played, but it doesn't seem to be causing too many problems.
What video formats are compatible with the iPhone's assets library?
In other words, for what video formats will ALAssetsLibrary's videoAtPathIsCompatibleWithSavedPhotosAlbum return YES?
I can't seem to locate any information on this in the iPhone Reference Library.
No one has mentioned this yet, but it depends on the iPhone / iOS device. In making an app that tries to copy Flickr videos to the photo album, I was getting frustrated when I kept getting invalid data results on writeVideoAtPathToSavedPhotosAlbum: calls for a non-Retina iPhone.
I ran some videoAtPathIsCompatibleWithSavedPhotosAlbum tests on Flickr videos of various sizes, as requested in this question.
iPhone iPhone iPad
(non-Retina) (Retina)
6119419764_orig.mov
H.264, 1,920 x 1,080 NO NO NO
Linear PCM, 16 bit
little-endian signed
integer, 48000 Hz,
Stereo (L R)
35.33 Mbit/s
6119419764_hd.mp4
AVC Coding, 1,280 x 720 NO YES YES
AAC, 44100 Hz, Stereo (L R)
2.15 Mbit/s
6119419764_site.mp4
AVC Coding, 640 x 360 NO YES YES
AAC, 44100 Hz, Stereo (L R)
833.71 kbit/s
6119419764_mobile.mp4
AVC Coding, 568 x 320 YES YES YES
AAC, 32000 Hz, Mono
775.14 kbit/s
6121206003_orig.mov
(Taken with iPhone 3Gs)
H.264, 480 x 360 YES YES YES
AAC, 44100 Hz, Mono
865.94 kbit/s
30 fps
6110638568_reformat.mov
H.264, 640 x 360 YES YES YES
AAC, 44100 Hz, Mono
3.57 Mbit/s
Based on this limited testing, for a given format and device, it looks like size matters most. (For the current Flickr encoding methods and url scheme, mobile videos work on all iOS device photo albums, whereas hd and site videos only work on Retina iPhones and iPads.)
An interesting side note is that HD videos will play on non-retina iPhones with the MPMoviePlayerController -- you just can't save them to the photo album.
If you want a list of supported audio/video technologies, read the iOS Technology Overview, in particular the Media Layer (scroll down to where it says "Video Technologies").
The video technologies in iOS support
the playback of movie files with the
.mov, .mp4, .m4v, and .3gp filename
extensions and using the following
compression standards:
H.264 video, up to 1.5 Mbps, 640 by 480 pixels, 30 frames per second,
Low-Complexity version of the H.264
Baseline Profile with AAC-LC audio up
to 160 Kbps, 48 kHz, stereo audio in
.m4v, .mp4, and .mov file formats
H.264 video, up to 768 Kbps, 320 by 240 pixels, 30 frames per second,
Baseline Profile up to Level 1.3 with
AAC-LC audio up to 160 Kbps, 48 kHz,
stereo audio in .m4v, .mp4, and .mov
file formats
MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second,
Simple Profile with AAC-LC audio up to
160 Kbps, 48 kHz, stereo audio in
.m4v, .mp4, and .mov file formats
Numerous audio formats, including the ones listed in “Audio
Technologies”
That's the reference information for the media frameworks in iOS.
I recently added a video export feature for the living photo burst of stills in my super-fast camera app SnappyCam Pro.
To cater for old and new devices alike, I ended up creating a few MPEG-4 "probe" videos, each with a single black frame, at a variety of 4:3 resolutions:
320x240px
640x480px
960x720px
1440x1080px
The four video files added just 12KB to the App Bundle.
By then iterating through each, with -[ALAssetsLibrary videoAtPathIsCompatibleWithSavedPhotosAlbum:], I was able to then work out which options are valid for the final Camera Roll video export.
If I had to guess, I might use the iPhone's own specifications as a guideline for testing:
Video formats supported: H.264 video up to 720p, 30 frames per second, Main Profile level 3.1 with AAC-LC audio up to 160 Kbps, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; MPEG-4 video, up to 2.5 Mbps, 640 by 480 pixels, 30 frames per second, Simple Profile with AAC-LC audio up to 160 Kbps per channel, 48kHz, stereo audio in .m4v, .mp4, and .mov file formats; Motion JPEG (M-JPEG) up to 35 Mbps, 1280 by 720 pixels, 30 frames per second, audio in ulaw, PCM stereo audio in .avi file format
There doesn't seem to be a documented way to query what formats will make videoAtPathIsCompatibleWithSavedPhotosAlbum happy and the fact that a device can decode a certain format doesn't mean the AssetLibrary will accept it. However, you would expect that a camera equipped device will always be able to save the camera's highest resolution output to the camera roll.
If you can live with this assumption, then all you have to do is enumerate the AVCaptureDevices and query the various presets with supportsAVCaptureSessionPreset.
Victor's tests show that pixel area seems to matter most, so once you've queried
AVCaptureSessionPreset1920x1080, AVCaptureSessionPreset1280x720, AVCaptureSessionPreset640x480 you should be able to choose a size that will make videoAtPathIsCompatibleWithSavedPhotosAlbum say YES.
I have no idea how you'd figure it for devices without cameras.
I wan't to play video in fullscreen mode on the iPhone, but when I try to figure out how to transcode my videos I get confused about the video format specification from Apple on
http://www.apple.com/iphone/specs.html
Video formats supported: H.264 video,
up to 1.5 Mbps, 640 by 480 pixels, 30
frames per second, Low-Complexity
version of the H.264 Baseline Profile
with AAC-LC audio up to 160 Kbps,
48kHz, stereo audio in .m4v, .mp4, and
.mov file formats; H.264 video, up to
2.5 Mbps, 640 by 480 pixels, 30 frames per second, Baseline Profile up to
Level 3.0 with AAC-LC audio up to 160
Kbps, 48kHz, stereo audio in .m4v,
.mp4, and .mov file formats; MPEG-4
video, up to 2.5 Mbps, 640 by 480
pixels, 30 frames per second, Simple
Profile with AAC-LC audio up to 160
Kbps, 48kHz, stereo audio in .m4v,
.mp4, and .mov file formats
Why is the recommended dimention 640 by 480 pixels when the fullscreen is 480 by 320?
(it doesn't even add up in ratio)
And if the ratio is correct why is it so? (Different pixel ratios?)
For pixel-exact video display on the iPhone, you'll want to use 480 x 320 H.264 with the settings they describe. I have an example video here that plays at fullscreen on the iPhone.
Videos with supported sizes that they describe will play, but will be scaled to the screen. You'll see black bars on the sides of the screen, unless you double-tap to zoom in on the video.