I have implemented player AVPlayerViewController on tvOS using AVQueuePlayer. The playback of videos works as expected but on all videos following the first the scrubber / time slider shows thumbnails for the first video.
The player is being setup using the following code:
class TFLPlayerController: AVPlayerViewController, AVPlayerViewControllerDelegate {
var queuePlayer = AVQueuePlayer()
...
func addVideoToQueue(video: Video, after: AVPlayerItem?){
let mainVideo = AVPlayerItem(URL: NSURL(string: videoURL)!)
self.queuePlayer.insertItem(mainVideo, afterItem: nil)
}
func readyToPlay() {
self.addVideoToQueue(videoRecord, after: nil)
self.player = self.queuePlayer
if let currentItem = queuePlayer.currentItem {
// Set time to get make a call to the API for the next video
let callUpNextAPITime = CMTimeMakeWithSeconds(0.5, currentItem.asset.duration.timescale)
let callUpNextAPITimeValue = NSValue(CMTime: callUpNextAPITime)
let getUpNextObserver = queuePlayer.addBoundaryTimeObserverForTimes([callUpNextAPITimeValue], queue: dispatch_get_main_queue(), usingBlock: { () -> Void in
self.getUpNext()
})
}
self.player?.play()
}
func getUpNext(){
// Get the next video to play
let playlistId = presentingPlaylist?.identifier ?? nil
if let currentItem = self.queuePlayer.currentItem {
// Get the current items id
let currentVideoId = itemVideo[currentItem]?.identifier
if let currentVideoId = currentVideoId {
// Call API to get the next video to be played from the server
dataLayer.getNextVideo(currentVideoId, playlistIdentifier: playlistId) {
video, error in
...
if let nextVideo = video {
let currentVideo: AVPlayerItem = self.queuePlayer.currentItem!
self.addVideoToQueue(nextVideo, after: currentVideo)
}
}
}
}
}
I have also instantiated the AVQueuePlayer using the code below as I thought there might be an issue with the initial queue setup but the issue remain:
AVQueuePlayer(playerItem: mainVideo)
Related
I am making an app for playing podcasts.
the problem is when creating an AVPlayer with URL or AVPlayer item. a memory leak happens but deinit works well with no problem.
weak var presenter: AudioPlayerUpdaterProtocol?
private var audioSession: AVAudioSession?
private var audioPlayer: AVPlayer?
private var audioURL: URL?
private var playbackTimeUpdater: Timer?
deinit {
self.audioSession = nil
self.audioURL = nil
self.audioPlayer = nil
self.playbackTimeUpdater?.invalidate()
removeEndPlayingObserver()
print("deinit audio")
}
private func setupAudioPlayer(){
guard let url = audioURL else {
return
}
let playerItem = AVPlayerItem(url: URL)
// when comment the avplear init no memory leaks found.
audioPlayer = AVPlayer(playerItem: playerItem)
addEndPlayingObserver()
startPlaying()
}
when debugging with instruments I got this
enter image description here
I tried to debug this malloc 16 bytes on Xcode but I failed on this because the object is not at any memory graph.
This is because setting an AVPlayerItem on an AVPlayer creates a hidden strong reference between the two, so persisting the AVPlayerItem will hang onto the AVPlayer, causing a memory leak.
make share the AVPlayerItem and AVPlayer have a 1:1 relationship by creating a new AVPlayerItem when it needs to be assigned to an AVPlayer.
guard let url = audioURL else {
return
}
let originalAsset = AVAsset(url: url)
audioPlayer = AVPlayer(playerItem: AVPlayerItem(asset: originalAsset))
If the above approach doesn't help you try this on your viewDidDissapear:
func viewDidDisappear(_ animated: Bool) {
super.viewDidDisappear(animated)
self.audioPlayer?.replaceCurrentItem(with: nil)
}
I am working on loading a (local) movie into AVPlayer and applying processing to the audio track with an audioTapProcessor. So far I've found great GitHub examples here, here, and here. I'm using the "tap cookie" approach used in the last link and in an answer to this previous question.
Audio & video playback are working fine. However, my tapPrepare and tapProcess callbacks are not being called, but Init and Finalize are. So I'm doing something both right and wrong. relevant code attached -- Any help appreciated!
import Foundation
import AVFoundation
import AudioToolbox
import MediaToolbox
import CoreAudioTypes
class PlayerViewController: UIViewController {
class TapCookie {
weak var content: PlayerViewController?
deinit {
print("TapCookie deinit") // appears after tapFinalize
}
}
// MARK: Properties
var playerAsset: AVURLAsset?
var playerItem: AVPlayerItem! = nil
var audioProcessingFormat: AudioStreamBasicDescription?
private var tracksObserver: NSKeyValueObservation?
// MARK: Button to trigger actions
#IBAction func selectVideo(_ sender: Any) {
// starts doing stuff:
// - select a video file from device, extract movieURL string ...
playerAsset = AVURLAsset(url: movieURL)
playerItem = AVPlayerItem(url: movieURL)
//... then send asset to AVPlayer (not shown)
// set up audioProcessingTap
tracksObserver = playerItem.observe(\AVPlayerItem.tracks, options: [.initial, .new]) {
[unowned self] item, change in
installTap(playerItem: playerItem)
}
}
func installTap(playerItem: AVPlayerItem) {
let cookie = TapCookie()
cookie.content = self
var callbacks = MTAudioProcessingTapCallbacks(
version: kMTAudioProcessingTapCallbacksVersion_0,
clientInfo: UnsafeMutableRawPointer(Unmanaged.passRetained(cookie).toOpaque()),
init: tapInit,
finalize: tapFinalize,
prepare: tapPrepare,
unprepare: tapUnprepare,
process: tapProcess)
var tap: Unmanaged<MTAudioProcessingTap>?
let err = MTAudioProcessingTapCreate(kCFAllocatorDefault, &callbacks, kMTAudioProcessingTapCreationFlag_PostEffects, &tap)
assert(noErr == err);
// tapInit successfully called after MTAudioProcessingTapCreate
let audioMix = AVMutableAudioMix()
let audioTrack = playerItem.asset.tracks(withMediaType: AVMediaType.audio).first! //use first audio track
let inputParams = AVMutableAudioMixInputParameters(track: audioTrack)
inputParams.audioTapProcessor = tap?.takeRetainedValue()
audioMix.inputParameters = [inputParams]
playerItem.audioMix = audioMix
}
// MARK: install tap callbacks
let tapInit: MTAudioProcessingTapInitCallback = {
(tap, clientInfo, tapStorageOut) in
tapStorageOut.pointee = clientInfo
print("tapInit tap: \(tap)\n clientInfo: \(String(describing: clientInfo))\n tapStorageOut: \(tapStorageOut)\n")
}
// tapPrepare not called !!
let tapPrepare: MTAudioProcessingTapPrepareCallback = {
(tap, maxFrames, processingFormat) in
print("tapPrepare tap: \(tap), maxFrames: \(maxFrames)\n processingFormat: \(processingFormat)")
let cookie = Unmanaged<TapCookie>.fromOpaque(MTAudioProcessingTapGetStorage(tap)).takeUnretainedValue()
cookie.content!.audioProcessingFormat = AudioStreamBasicDescription(mSampleRate: processingFormat.pointee.mSampleRate,
mFormatID: processingFormat.pointee.mFormatID,
mFormatFlags: processingFormat.pointee.mFormatFlags,
mBytesPerPacket: processingFormat.pointee.mBytesPerPacket,
mFramesPerPacket: processingFormat.pointee.mFramesPerPacket,
mBytesPerFrame: processingFormat.pointee.mBytesPerFrame,
mChannelsPerFrame: processingFormat.pointee.mChannelsPerFrame,
mBitsPerChannel: processingFormat.pointee.mBitsPerChannel,
mReserved: processingFormat.pointee.mReserved)
}
let tapUnprepare: MTAudioProcessingTapUnprepareCallback = {
(tap) in
print("tapUnprepare \(tap)")
}
// tapProcess not called !!
let tapProcess: MTAudioProcessingTapProcessCallback = {
(tap, numberFrames, flags, bufferListInOut, numberFramesOut, flagsOut) in
print("tapProcess \(tap)\n \(numberFrames)\n \(flags)\n \(bufferListInOut)\n \(numberFramesOut)\n \(flagsOut)\n")
let status = MTAudioProcessingTapGetSourceAudio(tap, numberFrames, bufferListInOut, flagsOut, nil, numberFramesOut)
if noErr != status {
print("get audio: \(status)")
}
let cookie = Unmanaged<TapCookie>.fromOpaque(MTAudioProcessingTapGetStorage(tap)).takeUnretainedValue()
guard let cookieContent = cookie.content else {
print("Tap callback: cookie content was deallocated!")
return
}
// process audio here...
}
let tapFinalize: MTAudioProcessingTapFinalizeCallback = {
(tap) in
print("tapFinalize \(tap)")
// release cookie
Unmanaged<TapCookie>.fromOpaque(MTAudioProcessingTapGetStorage(tap)).release()
}
}
You need to create an AVPlayer
player = AVPlayer(playerItem: playerItem)
and then at some point start it playing:
player.play()
Then the prepare and process callbacks will be called.
I Created a simple audio player/recorder player using AVAudioPlayer and AVAudioRecorder. The audio player works as expected but the audio recorder doesn't seem to record or playback the recording even though it builds successfully without any errors. Im wondering if it needs permission to the mic? I'm using version 11.X The code is below any help is much appreciated.
import UIKit
import AVFoundation //must import AVFoundation
class ViewController: UIViewController {
// create property called audioPlayer equal to player
var audioPlayer : AVAudioPlayer?
var audioRecorder : AVAudioRecorder?
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
//create the audio session
let session = AVAudioSession.sharedInstance()
//set category where we want it to happen (PlayAndRecord) try? = if it doesnt work then set it nil
try? session.setCategory(AVAudioSession.Category.playAndRecord)
//play any audio through device speaker
try? session.overrideOutputAudioPort(.speaker)
// set the session to active, if it doesnt work se it to nil
try? session.setActive(true)
//setup the recording and settings to where audio will be saved (Document folder on device) and set the name and file type (iloveaudio.mp3)
if let basePath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first {
let paths = [basePath, "iloveaudioios.mp3"]
if let audioURL = NSURL.fileURL(withPathComponents: paths) {
//create a settings dictionary> orignally it's set to empty using ([:])
var settings : [String : Any] = [:]
//set the file type to mp3
settings[AVFormatIDKey] = Int(kAudioFileMP3Type)
//set the sample rate
settings[AVSampleRateKey] = 44100.0
//set number of channels
settings[AVNumberOfChannelsKey] = 2
// specify the path and settings using var and try to run the recorder if it faile set it to nil
audioRecorder = try? AVAudioRecorder(url: audioURL, settings: settings)
//prepare the recorder of ot fails set it to nil
audioRecorder?.prepareToRecord()
}
}
playerSetup(audioURL: nil)
}
//which file to play if something is recorded
func playerSetup(audioURL:URL?) {
if audioURL == nil {
if let audioPath = Bundle.main.path(forResource: "testaudio", ofType: "mp3") {
let tempAudioURL = URL(fileURLWithPath: audioPath)
audioPlayer = try? AVAudioPlayer(contentsOf: tempAudioURL)
}
} else {
audioPlayer = try? AVAudioPlayer(contentsOf: audioURL!)
}
//check if our audio fille exist? and set it to the proerty audioPath
audioPlayer?.prepareToPlay()
//create audio player, make sure it not nil and if its not the prepare to play, then play
}
#IBAction func PlayPressed(_ sender: Any) {
audioPlayer?.play()
}
#IBAction func Pausedpressed(_ sender: Any) {
audioPlayer?.pause()
}
#IBAction func Stop(_ sender: Any) {
audioPlayer?.stop()
audioPlayer?.currentTime = 0
}
#IBAction func Record(_ sender: Any) {
if let recorder = audioRecorder {
if !recorder.isRecording {
recorder.record()
}
}
}
#IBAction func StopRecord(_ sender: Any) {
if let recorder = audioRecorder {
if recorder.isRecording {
recorder.stop()
playerSetup(audioURL: recorder.url)
}
}
}
}
Yes , You need to add Permission in info.plist
Go to info.plist of your project
Add Privacy - Microphone Usage Description and set it's Value YES(true)
After Setting , when you'll start recording in your app for the first time it will ask for permission , Allow that and you're all set !
I am trying to play a video inside a subclass of JSQMessagesViewController, but it doesn't start playing onClick event.
I am downloading the video from firebase, and then I write the data on the device by appending a unique reference -LK7WSGtGAQ2anpDxkCQ. Please see below print out statement for the complete path.
//response to collection view tap events
//called every time we type on a message
override func collectionView(_ collectionView: JSQMessagesCollectionView!, didTapMessageBubbleAt indexPath: IndexPath!) {
let message = messages[indexPath.item]
print("message type is \(message.type) and message media uid is \(message.mediaMessage?.mediaUID)")
if message.type == MessageType.image {
let jsqMessage = jsqMessages[indexPath.item]
let mediaItem = jsqMessage.media as! JSQPhotoMediaItem
let photos = IDMPhoto.photos(withImages: [mediaItem.image])
let browser = IDMPhotoBrowser(photos: photos)
self.present(browser!, animated: true, completion: nil)
}
if message.type == MessageType.video {
let jsqMessage = jsqMessages[indexPath.item]
if jsqMessage.isMediaMessage {
if let mediaItem = jsqMessage.media as? JSQVideoMediaItem {
print("mediaItem.fileURL is \(mediaItem.fileURL)")
let player = AVPlayer(url: mediaItem.fileURL)
let playerViewController = AVPlayerViewController()
playerViewController.player = player
self.present(playerViewController, animated: true) {
playerViewController.player!.play()
}
}
}
}
}
}//end of extension
mediaItem.fileURL is
Optional(file:///Users/bogdanbarbulescu/Library/Developer/CoreSimulator/Devices/7FB42206-997D-4AC2-B0BD-CEE2E22DAFBE/data/Containers/Data/Application/8F17DF31-16B1-4B27-ACB2-015AA55D8979/Documents/-LK7WSGtGAQ2anpDxkCQ)
Update
The video can by played now after appending .MOV to the url above, but there is no sound when it's played. How to fix this?
The problem was that when I was saving the data on the disk, I did not append .MOV extension to the end of the path, thus AVPlayer, could not play the video.
Before:
mediaItem.fileURL is file:///Users/bogdanbarbulescu/Library/Developer/CoreSimulator/Devices/7FB42206-997D-4AC2-B0BD-CEE2E22DAFBE/data/Containers/Data/Application/8F17DF31-16B1-4B27-ACB2-015AA55D8979/Documents/-LK7WSGtGAQ2anpDxkCQ
After: file:///Users/bogdanbarbulescu/Library/Developer/CoreSimulator/Devices/7FB42206-997D-4AC2-B0BD-CEE2E22DAFBE/data/Containers/Data/Application/8F17DF31-16B1-4B27-ACB2-015AA55D8979/Documents/-LK7WSGtGAQ2anpDxkCQ.MOV
I am trying to make a game with background music looping in it. I made the song file in Adobe Audition (which is similar to audacity) and when I play it in a loop in Adobe Audition it loops how I want it.
When I play it in Xcode however, it has a lag in between the loops. I am using AVFoundations for the sound playing.
I have searched everywhere but I can't find the solution for the problem.
Is there any way you can loop audio files without there being any lags in between ? (I believe its called "seamless looping" )
here is the code:
class GameScene: SKScene {
...// Other Code
var ButtonAudio = URL(fileURLWithPath: Bundle.main.path(forResource: "Gamescene(new)", ofType: "mp3")!)
var ButtonAudioPlayer = AVAudioPlayer()
... //Other Code
}
And When I Call it:
override func didMove(to view: SKView) {
...//Code
ButtonAudioPlayer = try! AVAudioPlayer(contentsOf: ButtonAudio, fileTypeHint: nil)
ButtonAudioPlayer.numberOfLoops = -1
ButtonAudioPlayer.prepareToPlay()
ButtonAudioPlayer.play()
...//More Code
}
Can someone help me with this issue ?
Thank you in advance!
You can use AVPlayerLooper and AVQueuePlayer to do this.
import UIKit
import AVFoundation
class ViewController: UIViewController {
var queuePlayer = AVQueuePlayer()
var playerLooper: AVPlayerLooper?
override func viewDidLoad() {
super.viewDidLoad()
guard let url = Bundle.main.url(forResource: "Gamescene(new)", withExtension: "mp3") else { return }
let playerItem = AVPlayerItem(asset: AVAsset(url: url))
playerLooper = AVPlayerLooper(player: queuePlayer, templateItem: playerItem)
queuePlayer.play()
}
}
The solution proposed by #dave234 only works in iOS > 10. Since I needed to make seamless playback in iOS > 9, I did something different:
Instead of AVPlayer, I created AVQueuePlayer and immediately added two identical melodies to the queue.
Next, I made a listener to the penultimate melody.
When the listener was triggered, I added another similar record to the queue after the last one.
In fact, in order to avoid delay, I always play the penultimate record.
My code:
var player: AVQueuePlayer?
override func viewDidLoad() {
super.viewDidLoad()
if let path = Bundle.main.path(forResource: "music_file", ofType: "mp3") {
player = createPlayer(url: URL(fileURLWithPath: path))
}
}
func createPlayer(url: URL) -> AVQueuePlayer {
let player = AVQueuePlayer(items: [AVPlayerItem(url: url), AVPlayerItem(url: url)])
loopPlayer(playerItem: player.items()[player.items().count - 2])
return player
}
func loopPlayer(playerItem: AVPlayerItem) {
NotificationCenter.default.addObserver(forName: .AVPlayerItemDidPlayToEndTime, object: playerItem, queue: .main) { _ in
if let player = self.player, let url = (playerItem.asset as? AVURLAsset)?.url {
player.insert(AVPlayerItem(url: url), after: player.items()[player.items().count - 1])
self.loopPlayer(playerItem: player.items()[player.items().count - 2])
}
}
}