I'm running into a strange issue, I hope someone can help.
In my iOS app I create a video with a custom soundtrack using MutableComposition by combining a video from the user's photo library and an audio file from the app bundle. I then use an AVPlayer and AVPlayerItem to play the video back to the user using a custom video player I made.
Each time a new composition is created, the assets, the player and the composition are cleared, released and it basically starts from a clean, init state.
All works fine, until after exactly 4 successful videos created this way every other attempt to create the player fails with error Cannot Decode. It does not matter if its the same video I'm recreating, has no relation to the size/length of the video or the audio file it simply always fails exactly on the fifth attempt, like clockwork. Once it fails, it will then always fail!
This is weird, because it just decoded the same video four times with no problem, so all of a sudden it fails? So, if anyone has a clue, please let me know.
Ok everyone, I have the answer to this straight from Apple. I used one of my developer TSI lifelines to ask the question, and I'll summarize the response.
There is a limit on the number of concurrent video players that AVFoundation will allow. It is due to the limitations of iOS hardware. The limit for current devices is 4 players. If you create a 5th player, you will get the "cannot decode" error. It is not a limit on the number of instances of AVPlayer, or AVPlayerItem. Rather,it is the association of AVPlayerItem with an AVPlayer which creates a "render pipeline", and you are limited to 4 of these. For example, this causes a new render pipeline:
AVPlayer *player = [AVPlayer playerWithPlayerItem:somePlayerItem];
// assuming the AVPlayerItem is ready to go with an AVAsset that has been loaded
I was also warned that you cannot assume that you will have 4 pipelines available to you. Another App may be using one or more. Indeed, I have seen this happen on an iPad, but it was not clear which app was using a pipeline.
So, there you go, it was totally undocumented, but that is the story.
I ran into the same error message after creating 4 AVPlayer instances, the fix in my case wasn't exactly the same though. Perhaps this will help anyone else who comes across this problem.
What I eventually found is that the AVPlayers were not being released when I had thought they were. In my case I was pushing my AVPlayer View Controller onto a Navigation Controller. Even though I was only creating one AVPlayer instance at a time, when the View Controllers are popped off a nav controller they were not being released immediately. It was then very easy for me to reach 4 AVPlayer instances before the old View Controllers were cleaned up.
It wasn't until I made sure that the previous players were released that this problem went away. To be complete I released the AVPlayerItem, AVPlayer and set the player on the AVPlayerLayer to nil before releasing.
I have to wonder if there is some limit on AVPlayer instances, unintentional or not. A related bit of info from the docs:
https://developer.apple.com/library/ios/#documentation/AudioVideo/Conceptual/AVFoundationPG/Articles/02_Playback.html
"Multiple player layers: You can create arbitrarily many AVPlayerLayer objects from a single AVPlayer instance, but only the most-recently-created such layer will display any video content on-screen."
This one was absolutely killing me until I figured it out, picking up clues from this thread and a few others. The biggest single problem in my code was that I was instantiating my video player controller every time I wanted to play a video. Now, it gets instantiated once in the primary controller (in this case, my DetailViewContoller):
#interface DetailViewController () {
VideoPlayerViewController *videoPlayerViewController;
}
- (void) viewDidLoad
{
[super viewDidLoad];
videoPlayerViewController = [[VideoPlayerViewController alloc] initWithNibName: nil bundle: nil];
}
When I want to show a video, I call my DetailViewController's startVideoPlayback method:
- (void) startVideoPlayback: (NSString *)videoUID
{
videoPlayerViewController.videoUID = videoUID;
[self presentModalViewController: videoPlayerViewController animated: YES];
}
(NOTE: I'm passing it 'videoUID' -- a unique identified that was used to create the video in another part of the app.)
In the VideoPlayerViewController (which is largely cribbed from Apple's AVPlayerDemo sample), the one-time screen setup (initializing the AVPlayer, setting up the toolbar, etc.) is done in viewDidLoad -- which now only get's called once, and all per-video setup gets done within viewWillAppear, which then calls prepareToPlay:
- (void) prepareToPlay
{
[self initScrubberTimer];
[self syncPlayPauseButtons];
[self syncScrubber];
NSString *documentsDirectory = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
//*** Retrieve and play video at associated with this videoUID
NSString *destinationPath = [documentsDirectory stringByAppendingFormat: #"/%#.mov", videoUID];
if ([self fileExists: destinationPath]) {
//*** Show the activity indicator spinny thing
[pleaseWait startAnimating];
[self setURL: [NSURL fileURLWithPath: destinationPath]];
//*** Get things going with the first video in this session
if (isFirst) {
isFirst = NO;
//*** Subseqeunt videos replace the first one
} else {
[self.mPlayer replaceCurrentItemWithPlayerItem: [AVPlayerItem playerItemWithURL: [NSURL fileURLWithPath: destinationPath]]];
}
}
}
OK, I figured out a solution, I hope this is helpful to anyone who may stumble on something similar to this problem.
The solution in my case was to initialize the asset for the AVPlayer and the AVPlayerItem on the main thread and make sure I don't create the actual AVPlayerLayer before the playerItem and the player objects return with status "ReadyToPlay".
This proved to be tricky to isolate and I still don't know why it worked the first 4 times and then failed consistently on the 5th time.
Till, I couldn't really include the code, it wasn't a matter of one line or even a few functions. It was a complex problem that I couldn't isolate to begin with. Thanks for the comments though.
It seems like that issue can be caused by any decoding tasks, not only actual players.
I randomly had this problem when I implemented a background task to extract frames from currently playing videos with generateCGImagesAsynchronously
I need to display 4 videos on screen and a race condition would sometime cause the frame extraction to start before the video started playing and I would wait for isReadyForDisplay forever.
Not sure what a good recover strategy is if you can't avoid the condition in the first place, I would probably try to replaceCurrentItem
Related
My app takes a video (using a custom overlay, in case that's relevant) where I have the cameraDevice set to front facing camera. 3 videos need to be taken, though the cameraDevice alternates between front and rear each time the UIImpagePickerController is called to take the video. On each occasion the
picker.cameraDevice = UIImagePickerControllerCameraDeviceFront;
is used within the code, though as mentioned the app ignores this command every other time.
I see this problem has been asked before, though I am using ARC so the previously suggested solutions of releasing the UIImagePickerController is not an option for me.
Thanks in advance, Jim.
You cannot release the picker in ARC but you can still create a brand new one like this:
picker = nil;
picker = [[UIImagePickerController alloc] init];
// configure picker
Notice that in Apple's sample code "Using UIImagePickerController to Select Pictures and Take Photos" they call a method each time which does exactly this.
I am creating an application which displays a certain video based on an external event, which may require the playing video to change quickly - once per second or more. However, there must not be a gap or lag between the videos.
What is the best way to do this? There are only four videos, each about two megabytes.
I was considering creating four MPMoviePlayerControllers, and have their views added to the main view but hidden, and switching by pausing and hiding the current video, then unhiding and playing the next video. Is there a more elegant solution?
Edit
Here's some more information for my exact sitaution:
The different video frames share mostly common pixels- so it's OK for a frame to stick during switch, but NOT okay for black frames to appear.
Each video is only about ten seconds long, and there are only four videos. The general state transitions are 1<->2<->3<->4->1.
The video playback should compatible with simultaneous AVAudioRecorder recording. As far as I can tell, MPMoviePlayerController is not.
You'll need to set up and prebuffer all the video streams to avoid hiccups, so I don't think your multiple MPMoviePlayerController solution is too far off the mark.
However, that specific solution is potentially problematic because each movie player has its own UI. The UIs do not synchronize with each other, so one might be showing the control bar, another not; one might be in full screen mode, etc. Switching among them will cause visual discontinuities.
Since it sounds like your video switching is not necessarily user-initiated, I'm guessing you don't care too much about the standard video player UI.
I would suggest dropping down to the underlying layer, AV Foundation. In theory, you can just create an AVPlayerItem for each video. This is a stream-management object with no UI overhead, so it's perfect for what you're doing. You could then -- again, in theory -- create one AVPlayer and one AVPlayerLayer to handle the display. When you wanted to switch from one AVPlayerItem stream to another, you could call the AVPlayer's replaceCurrentItemWithPlayerItem: message to swap out the data stream.
I made a little test project (GitHub) to verify this, and unfortunately the straightforward solution isn't quite perfect. There is no video flow glitching, but in the transition from AVPlayer to AVPlayer, the presentation layer seems to briefly flash the previous movie's last-seen frame at the size appropriate to the next movie. It seems to help to allocate separate AVPlayer objects for each movie and switch among them to a constant player layer. There still seems to be an instantaneous flash of background, but at least it's a subtle defect. Here's the gist of the code:
#interface ViewController : UIViewController
{
NSMutableArray *players;
AVPlayerLayer *playerLayer;
}
#property (nonatomic) IBOutlet UIView *videoView;
- (IBAction) selectVideo:(id)sender;
#end
#implementation ViewController
#synthesize videoView;
- (void)viewDidLoad
{
[super viewDidLoad];
NSArray *videoTitles = [NSArray arrayWithObjects:#"Ultimate Dog Tease",
#"Backin Up", #"Herman Cain", nil];
players = [NSMutableArray array];
for (NSString *title in videoTitles) {
AVPlayerItem *player = [AVPlayer playerWithURL:[[NSBundle mainBundle]
URLForResource:title withExtension:#"mp4"]];
[player addObserver:self forKeyPath:#"status" options:0 context:nil];
[players addObject:player];
}
playerLayer = [AVPlayerLayer playerLayerWithPlayer:[players objectAtIndex:0]];
playerLayer.frame = self.videoView.layer.bounds;
playerLayer.videoGravity = AVLayerVideoGravityResizeAspect;
[self.videoView.layer addSublayer:playerLayer];
}
- (void) observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object
change:(NSDictionary *)change context:(void *)context
{
[object removeObserver:self forKeyPath:#"status"];
for (AVPlayer *player in players) {
if (player.status != AVPlayerStatusReadyToPlay) {
return;
}
}
// All videos are ready to go
[self playItemAtIndex:0];
}
- (void) playItemAtIndex:(NSUInteger)idx
{
AVPlayer *newPlayer = [players objectAtIndex:idx];
if (newPlayer != playerLayer.player) {
[playerLayer.player pause];
playerLayer.player = newPlayer;
}
[newPlayer play];
}
- (IBAction) selectVideo:(id)sender
{
[self playItemAtIndex:((UILabel *)sender).tag];
}
#end
Half the code is there just to observe the state of the players and make sure that playback doesn't start until all videos have been buffered.
Allocating three separate AVPlayerLayers (in addition to three AVPlayers) prevents any sort of flash. Unfortunately, an AVPlayer connected to an undisplayed AVPlayerLayer seems to assume that it doesn't need to maintain a video buffer. Every switch among layers then produces a transient video stutter. So that's no good.
A couple of things to note when using AV Foundation are:
1) The AVPlayer object doesn't have built-in support for looped playback. You'll have to observe for the end of the current video and manually seek to time zero.
2) There's no UI at all other than the video frame, but again, I'm guessing that this might actually be an advantage for you.
The MPMoviePlayerController is a singleton. Four instances will share the same pointer, the same view, etc. With the native player, I think you have only two alternatives: one is to change the contentURL property when you want a transition. If the latency this way unacceptable, the other alternative is to produce a longer video with the shorter clips concatenated. You can create very quick jumps within the single, longer clip by setting currentPlaybackTime.
I have a view controller that uses AVPlayer. this view controller can load a modal view controler where the user can record audio using AVAudioRecorder.
this is what happens:
if the user plays the composition in the fist controller with [AVPLayer play] the AVAudioRecorder will not record in the modal view controller. there are no errors but the current time returned by AVAudioRecorder is 0.0;
if the user dismisses the modal dialog and reloads it. AVAudioRecorder works fine.
this can be repeated over and over
AVAudioRecorder does not work the first time it is invoked after a [AVPlayer play] call
I have been fighting with this for days and just have reorganized my code related to both AVPlayer and AVAudioRecorder and it's still acting weird.
Any help or pointer is much appreciated
Thanks in advance
Jean-Pierre
I've been having the same problem too, and this is the solution I found.
When recording, write this line code after AVAudioRecord is initialized:
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryRecord error:NULL];
Then invoke record method.
How are you setting the AVAudioSession's category, i.e., play or record? Try something like
[[AVAudioSession sharedInstance] setCategory:AVAudioSessionCategoryPlayAndRecord error:nil];
I've been having the same problem. I use AVPlayer to play compositions (previous recordings I've used AVAudioRecord for). However, I found that once I've used AVPlayer I could no longer use AVAudioRecorder. After some searching, I discovered that so long as AVPlayer is instantiated in memory and has been played at least once (which is usually what you do immediately after instantiating it) AVAudioRecorder will not record. However, once AVPlayer is dealloc'd, AVAudioRecorder is then free to record again. It appears that AVPlayer holds on to some kind of connection that AVAudioRecorder needs, and it's greedy...it won't let it go until you pry it from it's cold dead hands.
This is the solution I've found. Some people claim that instantiating AVPlayer takes too much time to keep breaking down and setting back up. However, this is not true. Instantiating AVPlayer is actually quite trivial. So also is instantiating AVPlayerItem. What isn't trivial is loading up AVAsset (or any of it's subclasses). You really only want to do that once. They key is to use this sequence:
Load up AVAsset (for example, if you're loading from a file, use AVURLAsset directly or add it to a AVMutableComposition and use that) and keep a reference to it. Don't let it go until you're done with it. Loading it is what takes all the time.
Once you're ready to play: instantiate AVPlayerItem with your asset, then AVPlayer with the AVPlayerItem and play it. Don't keep a reference to AVPlayerItem, AVPlayer will keep a reference to it and you can't reuse it with another player anyway.
Once it's done playing, immediately destroy AVPlayer...release it, set its var to nil, whatever you need to do. **
Now you can record. AVPlayer doesn't exist, so AVAudioRecorder is free to do its thing.
When you're ready to play again, re-instantiate AVPlayerItem with the asset you've already loaded & AVPlayer. Again, this is trivial. The asset has already been loaded so there shouldn't be a delay.
** Note that destroying AVPlayer may take more than just releasing it and setting its var to nil. Most likely, you've also added a periodic time observer to keep track of the play progress. When you do this, you receive back an opaque object you're supposed to hold on to. If you don't remove this item from the player AND release it/set it to nil, AVPlayer will not dealloc. It appears that Apple creates an intentional retain cycle you must break manually. So before you destroy AVPlayer you need to (example):
[_player removeTimeObserver:_playerObserver];
[_playerObserver release]; //Only if you're not using ARC
_playerObserver = nil;
As a side note, you may also have set up NSNotifications (I use one to determine when the player has completed playing)...don't forget to remove those as well.
If you need use AVPlayer and AVAudioRecorder at the same time do following:
set audio category to "play and record" (as described above or with C-based Audio Session Function)
"record" method must be invoked after invocation of "play" method with some delay. (I set 0.5 sec)
If you don't provide this, playback will start, but recording will not start.
shouldn't it be [AVAudioRecorder record]; instead of play?
I've had the same issue: AVAudioRecorder did not start recording.
My example is a bit different: I have a tabbar based app with several AVAudioPlayers in the different views. For the sake of the example, lets say that the first loaded view has 3 AVAudioPlayers, while the second view has one AVAudioPlayer and one AVAudioRecorder. They are all initiated in the viewDidLoad method.
The recorder was not working in second view. When I hit my record button, the view slightly shaked and the recording did not begin.
Strangely, the simulator worked fine and did not show the same symptoms... (anyone has an idea why?)
After some research and reading this thread I have realized that I have not set the players in the first view to nil, so I guess they were still in the memory preventing the recorder to start. Once I have set them to nil in the viewDidDisappear method, it was working alright again.
here is what helped me:
-(void)viewDidDisappear:(BOOL)animated {
firstPlayer = nil;
secondPlayer = nil;
thirdPlayer = nil;
}
ViewDidUnload did not help because the View, and therefore the variablbes in the class itself were not unloaded when switching views.
Hope this may be useful for others as well.
#Aaron Hayman explanation is accurate but the solution for me was very simple. Just implement the didFinishPlaying method of the player delegate and release the player there.
- (void)audioPlayerDidFinishPlaying:(AVAudioPlayer *)player successfully:(BOOL)flag{
self.playerWord = nil;
}
I want to use AVAudioPlayer to play short sounds... because I have more control than System Sound.
I have several buttons which are hooked up to play several different short sounds.
Each sound file is about 1.5 - 2.0 seconds.
Everything works fine ...except ...I have to wait for the sound to stop before I can press the button and the sound will play again.
Currently the individual AVAudioPlayers are created on viewDidLoad ... and the sounds are called to play when the buttons are pressed
I tried creating the players when the button is pressed... this solves the above problem... but after a while the app crashes... all sound stops working.
Any ideas?
Thanks
Jonathan
If AVAudioPlayer stops playing sounds, it is usually because there are too many player instances created, usually leaked. You should make sure you release some after you are done.
You need to set the currentTime of the player to 0 before playing. Calling [player play] when it is already playing has no effect.
player.currentTime = 0;
[player play];
Creating new ones is fine, but make sure your header declares and does this:
- (void) audioPlayerDidFinishPlaying: (AVAudioPlayer *) player successfully: (BOOL) completed {
[player release];
}
if your app is simple, that will release the memory of anything done that's no longer necessary, as per mahboudz' suggestion.
This might not totally prevent crashing, depending on your sound file sizes, and if you dealloc the scene. You might want to add
if (self.view.superview) { //release }
or something.
This is all memory handling. You may want to brush up on pointers to know how to handle your player objects, literally within any function, and still feel comfortable placing release in dealloc OR the didFinish: delegate function, and know why it doesn't belong in the other one.
The iPhone can be interrupted, stopped, and messed with at any time, so you have to know how to handle the memory and how to deal with it when it happens. I just had a crash of a similar nature, and everything above applied to it. NSXML delegates would be a nightmare without knowing pointers... same for AVAudioPlayer
If you plan to use a lof of short sounds, you might want to think about switching to OpenAL.
It's not a lot more complicated and you will save yourselve some trouble in more complicated audio settings later on.
There's a tutorial and some useful code available here (archived here).
this is MAYBE because you have created the AVAudioPlayers inside the viewDidLoad(). I remember I had the same problem. Just initialize them inside the class but not there. hope it works!
EDIT:
ok i saw the date... 2009...
I'm using the AVAudioPlayer to play sound FX on a separate thread in my app. Right now my app is fairly simple, I have a UIScrollView for a scrolling background, and a couple of CALayers that represent a player and an enemy in a simple side-scroller type game (think Mario or Kung Fu or whatnot)...I can move the player left and right, with the view scrolling appropriately with no problems. I have an "attack" button that's playing a sound effect...and many times when I play the sound effect, I get a little bit of a hitch, graphically. Is this just par for the course on the iPhone using Quartz? Is there some way to have this perform a little more smoothly without starting to investigate OpenGL?
Use NSData dataWithContentsOfMappedFile instead of dataWithContentsOfFile.
Also AVAudioPlayer handles it's own threading, you don't need to create one. [AVAudioPlayer play] returns after it started playing, not after it is done playing.
And don't forget to release your AVAudioPlayer when you no longer need it, for example if you only want to play a sound onces, implemement the audioPlayerDidFinishPlaying:successfully: delegate and release it in there.
How are you playing the sound? If I was to make a guess it would be that each time the sound is played the OS is actually loading it from disk as well as playing it.
Update:
Ok, you really don't need to be creating a new thread to play a sound. This alone could cause a stutter in a game and is also redundant - The AVAudioPlayer play method is asynchronous - it returns immediately and not when the sound ends.
I had the same problem with slowdown. To solve it, I first play and pause the sound when it first loads at volume 0 - this works better than prepareToPlay which still seems to result in slowdown:
player.volume = 0.0;
[player play];
[player pause];
player.volume = 1.0;
When it comes time to play a sound I just [player play];. Then every update cycle I loop through all my active sounds and catch ones that are about to end and rewind them instead of letting them expire - this avoids the overhead and slowdown of starting up the sound from scratch. The example here stops the sounds 1/15th of a second from the end of the sound:
for ({each player in active sounds})
{
if ((player.duration - player.currentTime) <= (1.0/15.0))
{
[player pause];
player.currentTime = 0.0;
}
}
Now it's ready to go for the next play. Seems to work fine!
Update: Turns out I was still getting some pauses the very first time the sound was played. I think playing it at zero volume wasn't actually doing anything, but I changed it to play at 0.0001 volume and then pause - it's inaudible but seems to do the job of prepping the sound.
Had the same problem at first. Per documentation you need to call "prepareToPlay" every time a sound has finished playing. Adopting the AVAudioPlayerDelegate protocol and adding the following method will do this.
-(void)audioPlayerDidFinishPlaying:(AVAudioPlayer *)player successfully:(BOOL)flag{
[player prepareToPlay];
}
(Using an answer rather than a comment for length and so I can show code):
My sound playing method is as follows:
- (void)playSoundThreaded:(id)soundPlayer
{
if(soundPlayer == nil)
return;
NSAutoreleasePool* pool = [[NSAutoreleasePool alloc] init];
[(AVAudioPlayer*)soundPlayer play];
[pool release];
}
- (void)playSound:(AVAudioPlayer*)player
{
if(player == nil)
return;
[NSThread detachNewThreadSelector:#selector(playSoundThreaded:)
toTarget:self withObject:(id)player];
}
And the sound is loaded up as follows:
fxPlayer = [[AVAudioPlayer alloc] initWithData:
[NSData dataWithContentsOfFile:
[[NSBundle mainBundle] pathForResource:#"voicehiya" ofType:#"caf"]] error:NULL];
We've had some inkling that the sound is being reloaded from disk every time...but have no idea as to why that is or how to prevent it.
I think the thread is over complicating your player. Try this for a go:
/* setup the sound player */
AVAudioPlayer *player1=[[AVAudioPlayer alloc] initWithContentsOfURL:[NSURL fileURLWithPath:audioFilePath] error:NULL];
player1.numberOfLoops = 0;
player1.volume=1.0f;
[player1 prepareToPlay];
and then playSound becomes
[player play];
And don't forget to have the AudioSession category set correctly too.
Why not just use AudioServicesPlaySystemSound? This is well suited to sound effects.
I use it in my game (vConqr) which is also based on a UIScrollView, with numerous other views (so all having their own CALayer), and it doesn't significantly impact performance.
I've been investigating this for a few days now thanks to a project I'm on at work. I added some home-rolled profiling to my code and found that the calls to play sounds were causing frame rate issues in my OpenGL game. I've got a few recommendations for you based on what I've found:
Use an AVAudioPlayer playing an mp3 for your background music.
Use OpenAL for your sound effects.
For sound effects, use the audio converter command line tool from apple to get your sounds to .caf format. This is an uncompressed format that doesn't require the CPU to decompress your sound to memory before sending it to the sound card. It can go directly, saving you time. Instructions on using the tool can be found HERE. (scroll to the very bottom of the page)
Use the lowest bit depths that still sound good. You don't really need 128 kbps - you can probably get away with 20 for a lot of things. Lower bit rate = less processing = faster sound.
I'll let you know if I find anything else that's useful! Good luck!