Apple Music & AVAudioEngine in Swift - swift

How can we access songs in the Apple Music library with AVAudioPlayerNode/AVAudioEngine for playback and processing?
I have asked this question in Apple forum.

"Apple Music" may refer to:
the music streaming service
its iOS client (known as "Music")
its macOS client, which doubles as a local media management and playback app (formerly known as iTunes, known as "Music" as of macOS Catalina).
Due to DRM restrictions it is not possible to play tracks from the Apple Music catalogue downloaded onto your device from the Apple Music macOS or iOS clients. However you can play audio files you own and which you've synced onto your device using the macOS Music app or the Finder app, as follows:
Add the NSAppleMusicUsageDescription key to your Info.plist file, and its corresponding value
Setup your AVAudioSession and AVAudioEngine
Find the URL of the media item you want to play (you can use MPMediaPickerController like in the example below or you can make your own MPMediaQuery)
Create an AVAudioFile from that URL
Create an AVAudioPlayerNode set to play that AVAudioFile
Connect the player node to the engine's output node
import UIKit
import AVFoundation
import MediaPlayer
class ViewController: UIViewController {
let engine = AVAudioEngine()
override func viewDidLoad() {
super.viewDidLoad()
let mediaPicker = MPMediaPickerController(mediaTypes: .music)
mediaPicker.allowsPickingMultipleItems = false
mediaPicker.showsItemsWithProtectedAssets = false // These items usually cannot be played back
mediaPicker.showsCloudItems = false // MPMediaItems stored in the cloud don't have an assetURL
mediaPicker.delegate = self
mediaPicker.prompt = "Pick a track"
present(mediaPicker, animated: true, completion: nil)
}
func startEngine(playFileAt: URL) {
do {
try AVAudioSession.sharedInstance().setCategory(.playback)
let avAudioFile = try AVAudioFile(forReading: playFileAt)
let player = AVAudioPlayerNode()
engine.attach(player)
engine.connect(player, to: engine.mainMixerNode, format: avAudioFile.processingFormat)
try engine.start()
player.scheduleFile(avAudioFile, at: nil, completionHandler: nil)
player.play()
} catch {
assertionFailure(String(describing: error))
}
}
}
extension ViewController: MPMediaPickerControllerDelegate {
func mediaPicker(_ mediaPicker: MPMediaPickerController, didPickMediaItems mediaItemCollection: MPMediaItemCollection) {
guard let item = mediaItemCollection.items.first else {
print("no item")
return
}
print("picking \(item.title!)")
guard let url = item.assetURL else {
return print("no url")
}
dismiss(animated: true) { [weak self] in
self?.startEngine(playFileAt: url)
}
}
func mediaPickerDidCancel(_ mediaPicker: MPMediaPickerController) {
dismiss(animated: true, completion: nil)
}
}

Related

ObservedObject class function not being called when protocol fires

While implementing the Google Interactive Media Ads (IMA) SDK protocols, my mediaPlayer/audioManager, which is an AVPlayer object, is not pausing during the adsManagerDidRequestContentPause delegate method. My mediaPlayer conforms to ObservableObject, which is where I think the problem is coming from but i'm not 100% positive.
When I press the play button, I play the audio and request the ads from my adsManager class. The issue is the preroll video plays but the audio from the content player plays over the preroll. The audio from the content player is supposed to pause when the preroll is playing and resume after it finishes. As you'll see in the code, AudioManager is also a singleton class.
Here's the code for when a user presses play.
#ObservedObject var manager = AudioManager.sharedInstance
func didTapPlayButton(){
isPlaying.toggle()
if isPlaying {
audioManager.playLiveStream(with: pageInfo.tritonMount)
adManager.requestAds()
} else {
audioManager.pause()
}
}
And here's the adManager class with the Google IMA delegate methods. After setting breakpoints at each delegate method, I found that each call to audioManager is being called successfully, however, the audio from the audioManager doesn't actually pause.
#ObservedObject var manager = AudioManager.sharedInstance
func setUpContentPlayer() {
contentPlayhead = IMAAVPlayerContentPlayhead(avPlayer: player)
// Create a player layer for the player.
playerLayer = AVPlayerLayer(player: player)
playerLayer!.frame = videoView.underlyingView.layer.bounds
videoView.underlyingView.layer.addSublayer(playerLayer!)
}
func setUpAdsLoader() {
adsLoader = IMAAdsLoader(settings: nil)
adsLoader!.delegate = self
}
func requestAds() {
// Create an ad display container for ad rendering.
let adDisplayContainer = IMAAdDisplayContainer(adContainer: videoView.underlyingView, companionSlots: nil)
// Create an ad request with our ad tag, display container, and optional user context.
let request = IMAAdsRequest(
adTagUrl: kLivePrerollVastTag,
adDisplayContainer: adDisplayContainer,
contentPlayhead: contentPlayhead,
userContext: nil)
adsLoader!.requestAds(with: request)
}
func adsLoader(_ loader: IMAAdsLoader!, adsLoadedWith adsLoadedData: IMAAdsLoadedData!) {
// Grab the instance of the IMAAdsManager and set ourselves as the delegate
adsManager = adsLoadedData.adsManager
adsManager!.delegate = self
// Create ads rendering settings and tell the SDK to use the in-app browser.
let adsRenderingSettings = IMAAdsRenderingSettings()
adsRenderingSettings.webOpenerPresentingController = viewController
// Initialize the ads manager.
adsManager!.initialize(with: adsRenderingSettings)
}
//Ads Manager recieved ad request and is loading and starting ad
func adsManager(_ adsManager: IMAAdsManager!, didReceive event: IMAAdEvent!) {
if (event.type == IMAAdEventType.LOADED) {
// When the SDK notifies us that ads have been loaded, play them.
adsManager.start()
}
}
//Ads manager failed to receive ad request
func adsLoader(_ loader: IMAAdsLoader!, failedWith adErrorData: IMAAdLoadingErrorData!) {
print("Error loading ads: \(String(describing: adErrorData.adError.message))")
audioManager.resume()
}
func adsManager(_ adsManager: IMAAdsManager!, didReceive error: IMAAdError!) {
// Something went wrong with the ads manager after ads were loaded. Log the
// error and play the content.
NSLog("AdsManager error: \(String(describing: error.message))")
audioManager.resume()
}
/*Ads manager received request, initiated avplayer and is now
requesting player be paused in order or the Ads manager to play preroll*/
func adsManagerDidRequestContentPause(_ adsManager: IMAAdsManager!) {
// The SDK is going to play ads, so pause the content.
audioManager.pause()
}
/*Ads manager received request, played the preroll and is now
requesting avplayer to resume live stream*/
func adsManagerDidRequestContentResume(_ adsManager: IMAAdsManager!) {
// The SDK is done playing ads (at least for now), so resume the content.
audioManager.resume()
}
AudioManagerClass:
/// Upon setting this property any observers for 'currentItem' as well as any time observers will
/// be removed from the old value where applicable and added to the new value where applicable.
var player: AVPlayer? {
didSet {
oldValue?.removeObserver(self, forKeyPath: "currentItem", context: &AudioManager.ObserveAVPlayerCurrentItem)
oldValue?.removeObserver(self, forKeyPath: "rate", context: &AudioManager.ObserveAVPlayerRate)
guard let player = self.player else { return }
player.addObserver(self, forKeyPath: "currentItem", options: [.initial, .new], context: &AudioManager.ObserveAVPlayerCurrentItem)
player.addObserver(self, forKeyPath: "rate", options: [.initial, .new], context: &AudioManager.ObserveAVPlayerRate)
player.automaticallyWaitsToMinimizeStalling = false
if let tmpObs = timeObserver {
print("Already have timeobserver, removing it")
oldValue?.removeTimeObserver(tmpObs)
timeObserver = nil
}
timeObserver = player.addPeriodicTimeObserver(forInterval: CMTimeMakeWithSeconds(1,preferredTimescale: 1),
queue: nil,
using: timeObserverCallback) as AnyObject?
}
}

Do you need to ask for permission to use the microphone when running an app in simulator?

I Created a simple audio player/recorder player using AVAudioPlayer and AVAudioRecorder. The audio player works as expected but the audio recorder doesn't seem to record or playback the recording even though it builds successfully without any errors. Im wondering if it needs permission to the mic? I'm using version 11.X The code is below any help is much appreciated.
import UIKit
import AVFoundation //must import AVFoundation
class ViewController: UIViewController {
// create property called audioPlayer equal to player
var audioPlayer : AVAudioPlayer?
var audioRecorder : AVAudioRecorder?
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view.
//create the audio session
let session = AVAudioSession.sharedInstance()
//set category where we want it to happen (PlayAndRecord) try? = if it doesnt work then set it nil
try? session.setCategory(AVAudioSession.Category.playAndRecord)
//play any audio through device speaker
try? session.overrideOutputAudioPort(.speaker)
// set the session to active, if it doesnt work se it to nil
try? session.setActive(true)
//setup the recording and settings to where audio will be saved (Document folder on device) and set the name and file type (iloveaudio.mp3)
if let basePath = NSSearchPathForDirectoriesInDomains(.documentDirectory, .userDomainMask, true).first {
let paths = [basePath, "iloveaudioios.mp3"]
if let audioURL = NSURL.fileURL(withPathComponents: paths) {
//create a settings dictionary> orignally it's set to empty using ([:])
var settings : [String : Any] = [:]
//set the file type to mp3
settings[AVFormatIDKey] = Int(kAudioFileMP3Type)
//set the sample rate
settings[AVSampleRateKey] = 44100.0
//set number of channels
settings[AVNumberOfChannelsKey] = 2
// specify the path and settings using var and try to run the recorder if it faile set it to nil
audioRecorder = try? AVAudioRecorder(url: audioURL, settings: settings)
//prepare the recorder of ot fails set it to nil
audioRecorder?.prepareToRecord()
}
}
playerSetup(audioURL: nil)
}
//which file to play if something is recorded
func playerSetup(audioURL:URL?) {
if audioURL == nil {
if let audioPath = Bundle.main.path(forResource: "testaudio", ofType: "mp3") {
let tempAudioURL = URL(fileURLWithPath: audioPath)
audioPlayer = try? AVAudioPlayer(contentsOf: tempAudioURL)
}
} else {
audioPlayer = try? AVAudioPlayer(contentsOf: audioURL!)
}
//check if our audio fille exist? and set it to the proerty audioPath
audioPlayer?.prepareToPlay()
//create audio player, make sure it not nil and if its not the prepare to play, then play
}
#IBAction func PlayPressed(_ sender: Any) {
audioPlayer?.play()
}
#IBAction func Pausedpressed(_ sender: Any) {
audioPlayer?.pause()
}
#IBAction func Stop(_ sender: Any) {
audioPlayer?.stop()
audioPlayer?.currentTime = 0
}
#IBAction func Record(_ sender: Any) {
if let recorder = audioRecorder {
if !recorder.isRecording {
recorder.record()
}
}
}
#IBAction func StopRecord(_ sender: Any) {
if let recorder = audioRecorder {
if recorder.isRecording {
recorder.stop()
playerSetup(audioURL: recorder.url)
}
}
}
}
Yes , You need to add Permission in info.plist
Go to info.plist of your project
Add Privacy - Microphone Usage Description and set it's Value YES(true)
After Setting , when you'll start recording in your app for the first time it will ask for permission , Allow that and you're all set !

Sound not playing in simulator, playing during debug in Xcode

I'm trying to face with the book "Intro to app development" by apple. I'm stuck on exercise AnimalSounds.
There is a SimpleSound class provided by the book, and our job is to use a sound object to reproduce a sound.
The simulator does not play any sound.
The audio files are in the build bundle and are found by the class.
I checked the simulator with other apps (e.g. youtube on safari) and sound works.
SimpleSound class (provided by example, not written by me)
import Foundation
import AudioToolbox
class SimpleSound {
private var soundID: SystemSoundID = 0
public init(named name: String) {
if let soundURL = soundURL(forName: name) {
let status = AudioServicesCreateSystemSoundID(soundURL as CFURL, &soundID)
if status != noErr {
print("Unable to create sound at URL: '\(name)'")
soundID = 0
}
}
}
public func play() {
if soundID != 0 {
print("Playing sound \(soundID)")
AudioServicesPlaySystemSound(soundID)
//AudioServicesPlaySystemSound(1001)
}
}
private func soundURL(forName name: String) -> URL? {
let fileExtensions = ["m4a", "wav", "mp3", "aac", "adts", "aif", "aiff", "aifc", "caf", "mp4"]
for fileExtention in fileExtensions {
if let soundURL = Bundle.main.url(forResource: name, withExtension: fileExtention) {
return soundURL
}
}
print("Unable to find sound file with name '\(name)'")
return nil
}
deinit {
if soundID != 0 {
AudioServicesDisposeSystemSoundID(soundID)
}
}
}
Code used in my viewController (written by me) in a button tapped action:
let meowSound = SimpleSound(named: "meow")
meowSound.play()
When debugging from Xcode line by line, when the function play() is invoked, the sound is reproduced. When app is running in simulator (with no debugging) or in actual iPhone sound does not play.
If standard system sound 1001 is uncommented, sound is played for both simulator and iPhone.
Any ideas?

Audio not playing in SwiftUI

I'm having an issue getting an audio file to play when the function is called.
I am using an AVAudioPlayer to try and play the file following the instructions form here:
https://www.hackingwithswift.com/example-code/media/how-to-play-sounds-using-avaudioplayer
After the button is pressed in the view, it calls a func to play the sound, but from what I can tell, nothing is played. There are no errors thrown, and the file is found. The app also uses a speech synthesizer when a button is pushed, and that plays fine.
I looked around stack overflow and followed the instructions from here:
Where to place code for audio playback in a SwiftUI app
But still the audio is not played when the button is pushed.
Here is the func:
func playSound() {
var sound = AVAudioPlayer()
if let path = Bundle.main.path(forResource: "TryAgain", ofType: "wav") {
do {
sound = try AVAudioPlayer(contentsOf: URL(fileURLWithPath: path))
print("Playing sound")
sound.play()
} catch {
print( "Could not find file")
}
}
Here is the class:
class Player: BindableObject {
let willChange = PassthroughSubject<Player, Never>()
var isPlaying: Bool = false {
willSet {
willChange.send(self)
}
}
func playSound() {
var sound = AVAudioPlayer()
if let path = Bundle.main.path(forResource: "TryAgainWav", ofType: "wav") {
do {
sound = try AVAudioPlayer(contentsOf: URL(fileURLWithPath: path))
print("Playing sound")
sound.play()
} catch {
print( "Could not find file")
}
}
}
}
Update: Not sure if this helps, but I built this in with IB instead of SwiftUI and noticed the same message is printed in the console when I click the button to play the audio file:
2019-07-22 11:29:41.075568-0700 PlaySoundPrac[13952:46549155] [plugin] AddInstanceForFactory: No factory registered for id F8BB1C28-BAE8-11D6-9C31-00039315CD46
Any help would be greatly appreciated
I'm pretty new myself, but I'll do my best. What happens if you move the declaration of AVAudioPlayer outside the function? For example:
import Combine
class Player: ObservableObject {
var sound: AVAudioPlayer!
let willChange = PassthroughSubject<Player, Never>()
var isPlaying: Bool = false {
willSet {
willChange.send(self)
}
}
func playSound() {
if let path = Bundle.main.path(forResource: "TryAgainWav", ofType: "wav") {
do {
sound = try AVAudioPlayer(contentsOf: URL(fileURLWithPath: path))
print("Playing sound")
sound.play()
} catch {
print( "Could not find file")
}
}
}
}
Try this out and let me know if it works or not! Thanks.
I had a problem that audio was not played on iOS 13 Simulator in Xcode 11.5 while it was okay in Xcode SwiftUI preview and physical device.

Playing sounds in Xcode while playing music from apps

I am currently using the following code to play music within my Xcode project. However, when I play music, other music apps (eg: spotify) stop playing. I understand there is AVAudioSessionCategoryAmbient, but i am not to sure how to set this function within my code.
How do i play music within my app without stopping music from other apps that are currently playing?
Any help would be much appreciated :D
Thanks!
import UIKit
import AVFoundation
let appDelegate = UIApplication.shared.delegate as! AppDelegate
#UIApplicationMain
class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
/* MUSIC */
var musicIsPlaying: Bool = false
var bgMusicLoop: AVAudioPlayer!
func playBackgroundMusic(fileName: String, withExtenstion fileExtension: String) {
let appDelegate = UIApplication.shared.delegate as! AppDelegate
if appDelegate.musicIsPlaying == false {
appDelegate.musicIsPlaying = true
let filePath: String = Bundle.main.path(forResource: fileName, ofType: fileExtension)!
if (appDelegate.bgMusicLoop != nil) {
appDelegate.bgMusicLoop.stop()
}
appDelegate.bgMusicLoop = nil
do {
appDelegate.bgMusicLoop = try AVAudioPlayer(contentsOf: NSURL.fileURL(withPath: filePath))
} catch {
print(error)
}
//A negative means it loops forever
appDelegate.bgMusicLoop.numberOfLoops = -1
appDelegate.bgMusicLoop.prepareToPlay()
appDelegate.bgMusicLoop.play()
}
}
Add try? AVAudioSession.sharedInstance().setCategory(AVAudioSessionCategoryAmbient) to your code.
AVAudioSessionCategoryAmbient "plays sounds that add polish or interest but are not essential to the app’s use."