I am currently trying to get Apple's new audio engine working with my current audio setup. Specifically, I am trying to change the pitch with Audio Engine, which apparently is possible according to this post.
I have also looked into other pitch changing solutions including Dirac and ObjectAL, but unfortunately both seem to be pretty messed up in terms of working with Swift, which I am using.
My question is how do I change the pitch of an audio file using Apple's new audio engine. I am able to play sounds using AVAudioPlayer, but I am not getting how the file is referenced in audioEngine. In the code on the linked page there is a 'format' that refers to audio file, but I am not getting how to create a format, or what it does.
I am playing sounds with this simple code:
let path = NSBundle.mainBundle().pathForResource(String(randomNumber), ofType:"m4r")
let fileURL = NSURL(fileURLWithPath: path!)
player = AVAudioPlayer(contentsOfURL: fileURL, error: nil)
player.prepareToPlay()
player.play()
You use an AVAudioPlayerNode, not an AVAudioPlayer.
engine = AVAudioEngine()
playerNode = AVAudioPlayerNode()
engine.attachNode(playerNode)
Then you can attach an AVAudioUnitTimePitch.
var mixer = engine.mainMixerNode;
auTimePitch = AVAudioUnitTimePitch()
auTimePitch.pitch = 1200 // In cents. The default value is 1.0. The range of values is -2400 to 2400
auTimePitch.rate = 2 //The default value is 1.0. The range of supported values is 1/32 to 32.0.
engine.attachNode(auTimePitch)
engine.connect(playerNode, to: auTimePitch, format: mixer.outputFormatForBus(0))
engine.connect(auTimePitch, to: mixer, format: mixer.outputFormatForBus(0))
Related
I'm trying to record voice audio with either AVAudioRecorder or AUAudioUnit.
In both, after a recording has started, whenever calling AVCaptureDevice.showSystemUserInterface(.microphoneModes) and selecting voice isolation, I get the following error:
"Voice Isolation and Wide Spectrum are currently unavailable"
TLDR: What do I need to allow the user to change to voice isolation mode?
I have an application that plays audio in real time and the audio separation mode is available by writing the following.
private let audioEngine = AVAudioEngine()
//　omission (of middle part of a text)
let audioInput = audioEngine.inputNode
audioInput.isVoiceProcessingBypassed = true
do {
try audioInput.setVoiceProcessingEnabled(true)
} catch {
print("Could not enable voice processing \(error)")
return
}
let audioFormat = audioEngine.inputNode.outputFormat(forBus: 0)
audioEngine.connect(audioInput, to: audioEngine.mainMixerNode, format:audioFormat)
Since I am using AVAudioEngine, I believe your objective can be achieved by simply changing the AVAudioEngine output destination.
I have a short video on my local machine. I am trying to open it using NSWorkspace on QuickTime Player.
let stringUrl = "~/Desktop/myrec.mov"
let url = URL(string: stringUrl)!
let config = NSWorkspace.OpenConfiguration()
config.activates = true
NSWorkspace.shared.open(
[url],
withApplicationAt: URL(string: "~/Applications/QuickTime Player")!,
configuration: config
)
The error I am getting is:
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file cap/cap.swift
Which has to do with withApplicationAt being the incorrect URL and returning nil.
I'm sure withApplicationAtURL is wrong, but I have no idea how to find the url for the quicktime player app.
I am very new to Swift and am having trouble reading the docs, especially since they don't seem up to date (e.g. my compiler says openFile is deprecated and led me to open).
I'm also not sure if this is the correct/best way to go about accomplishing what I am trying to do (open a small local video on quicktime player).
Any recommendations, tips, advice, or answers would be greatly appreciated!
Try this. Works on my Mac:
let stringUrl = "/Users/whoever/Desktop/myrec.mov"
let url = URL(fileURLWithPath: stringUrl)
let config = NSWorkspace.OpenConfiguration()
config.activates = true
NSWorkspace.shared.open(
[url],
withApplicationAt: URL(fileURLWithPath: "/System/Applications/QuickTime Player.app"),
configuration: config
)
Please note that that I replaced the initializer of URL with the correct one. Also note, that I've swapped the QuickTime Player path with the one that Finder gives me (right-click while holding option key).
I need to get the frames from a local video file so i can process them before the video is played. I already tried using AVAssetReader and VideoOutput.
[EDIT] Here is the code i used from Accesing Individual Frames using AV Player
let asset = AVAsset(URL: inputUrl)
let reader = try! AVAssetReader(asset: asset)
let videoTrack = asset.tracksWithMediaType(AVMediaTypeVideo)[0]
// read video frames as BGRA
let trackReaderOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings:[String(kCVPixelBufferPixelFormatTypeKey): NSNumber(unsignedInt: kCVPixelFormatType_32BGRA)])
reader.addOutput(trackReaderOutput)
reader.startReading()
while let sampleBuffer = trackReaderOutput.copyNextSampleBuffer() {
print("sample at time \(CMSampleBufferGetPresentationTimeStamp(sampleBuffer))")
if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
// process each CVPixelBufferRef here
// see CVPixelBufferGetWidth, CVPixelBufferLockBaseAddress, CVPixelBufferGetBaseAddress, etc
}
}
I believe AVAssetReader should work. What did you try? Have you seen this sample code from Apple? https://developer.apple.com/library/content/samplecode/ReaderWriter/Introduction/Intro.html
I found out what the problem was! It was with my implementation. The code i posted is correct. Thank you all
You can have a look at VideoToolbox : https://developer.apple.com/documentation/videotoolbox
But beware: this is close to the hardware decompressor and sparsely documented terrain.
Depending on what processing you want to do, OpenCV may be a an option - in particular if you are detecting or tracking objets in your frames. If your needs are simpler, then the effort to use OpenCV with swift may be a little too much - see below.
You can open a video, read it frame by frame, do your work on the frames and then display then - bearing in mind the need to be efficient to avoid delaying the display.
The basic code structure is quite simple - this is a python example but the same principles apply across supported languages
import numpy as np
import cv2
cap = cv2.VideoCapture('vtest.avi')
while(cap.isOpened()):
ret, frame = cap.read()
//Do whatever work you want on the frame here - in this example
//from the tutorial the image is being converted from one colour
//space to another
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
//This displays the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
More info here: http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_video_display/py_video_display.html
The one caveat is that using OpenCV with swift requires some additional effort - this is a good example, but it evolves constantly so it is worth searching for if you decide to go this way: https://medium.com/#yiweini/opencv-with-swift-step-by-step-c3cc1d1ee5f1
I am developing an application with swift. I would like to be able to increase the volume of a recorded file. Is there a way to do it directly inside the application?
I found Audiokit Here and this question but it didn't help me much.
Thanks!
With AudioKit
Option A:
Do you just want to import a file, then play it louder than you imported it? You can use an AKBooster for that.
import AudioKit
do {
let file = try AKAudioFile(readFileName: "yourfile.wav")
let player = try AKAudioPlayer(file: file)
// Define your gain below. >1 means amplifying it to be louder
let booster = AKBooster(player, gain: 1.3)
AudioKit.output = booster
try AudioKit.start()
// And then to play your file:
player.play()
} catch {
// Log your error
}
Just set the gain value of booster to make it louder.
Option B: You could also try normalizing the audio file, which essentially applies a multiple constant across the recording (with respect to the highest signal level in the recording) so it reaches a new target maximum that you define. Here, I set it to -4dB.
let url = Bundle.main.url(forResource: "sound", withExtension: "wav")
if let file = try? AKAudioFile(forReading: url) {
// Set the new max level (in dB) for the gain here.
if let normalizedFile = try? file.normalized(newMaxLevel: -4) {
print(normalizedFile.maxLevel)
// Play your normalizedFile...
}
}
This method increases the amplitude of everything to a level of dB - so it won't effect the dynamics (SNR) of your file, and it only increases by the amount it needs to reach that new maximum (so you can safely apply it to ALL of your files to have them be uniform).
With AVAudioPlayer
Option A: If you want to adjust/control volume, AVAudioPlayer has a volume member but the docs say:
The playback volume for the audio player, ranging from 0.0 through 1.0 on a linear scale.
Where 1.0 is the volume of the original file and the default. So you can only make it quieter with that. Here's the code for it, in case you're interested:
let soundFileURL = Bundle.main.url(forResource: "sound", withExtension: "mp3")!
let audioPlayer = try? AVAudioPlayer(contentsOf: soundFileURL, fileTypeHint: AVFileType.mp3.rawValue)
audioPlayer?.play()
// Only play once
audioPlayer?.numberOfLoops = 0
// Set the volume of playback here.
audioPlayer?.volume = 1.0
Option B: if your sound file is too quiet, it might be coming out the receiver of the phone. In which case, you could try overriding the output port to use the speaker instead:
do {
try AVAudioSession.sharedInstance().overrideOutputAudioPort(AVAudioSession.PortOverride.speaker)
} catch let error {
print("Override failed: \(error)")
}
You can also set that permanently with this code (but I can't guarantee your app will get into the AppStore):
try? audioSession.setCategory(AVAudioSessionCategoryPlayAndRecord, with: AVAudioSessionCategoryOptions.defaultToSpeaker)
Option C: If Option B doesn't do it for you, you might be out of luck on 'how to make AVAudioPlayer play louder.' You're best off editing the source file with some external software yourself - I can recommend Audacity as a good option to do this.
Option D: One last option I've only heard of. You could also look into MPVolumeView, which has UI to control the system output and volume. I'm not too familiar with it though - may be approaching legacy at this point.
I want to mention a few things here because I was working on a similar problem.
On the contrary to what's written on Apple Docs on the AVAudioPlayer.volume property (https://developer.apple.com/documentation/avfoundation/avaudioplayer/1389330-volume) the volume can go higher than 1.0... And actually this works. I bumped up the volume to 100.0 on my application and recorded audio is way louder and easier to hear.
Another thing that helped me was setting the mode of AVAudioSession like so:
do {
let session = AVAudioSession.sharedInstance()
try session.setCategory(.playAndRecord, options: [.defaultToSpeaker, .allowBluetooh])
try session.setMode(.videoRecording)
try session.setActive(true)
} catch {
debugPrint("Problem with AVAudioSession")
}
session.setMode(.videoRecording) is the key line here. This helps you to send the audio through the louder speakers of the phone and not just the phone call speaker that's next to the face camera in the front. I was having a problem with this and posted a question that helped me here:
AVAudioPlayer NOT playing through speaker after recording with AVAudioRecorder
There are several standard AudioKit DSP components that can increase the volume.
For example, you can use a simple method like AKBooster: http://audiokit.io/docs/Classes/AKBooster.html
OR
Use the following code,
AKSettings.defaultToSpeaker = true
See more details in this post:
https://github.com/audiokit/AudioKit/issues/599
https://github.com/AudioKit/AudioKit/issues/586
I'm currently trying to play multiple AVPlayers in parallel using AVPlayer and AVPlayerLayer on tvOS. In the simulator this is working correctly, on the device only some players play and for the rest the player layer simply stays blank (not even black but simply blank). I've heard rumors that the internal implementation only supports 24 simultaneous instances so I already limited the number to 24. However on the physical device, a number of ~15 players can play in parallel. Surprisingly that number tends to differ, sometimes its just 13, sometimes even 16.
I'm creating the players using the following code (which is executed in a closure, hence the weak self as the input and the strongSelf cast):
guard let strongSelf = self else { return }
strongSelf.player = AVPlayer(URL: localURL)
strongSelf.player?.volume = 0.0
strongSelf.player?.actionAtItemEnd = .Pause
NSNotificationCenter.defaultCenter().addObserver(strongSelf, selector: "playerPlayedToEnd:", name: AVPlayerItemDidPlayToEndTimeNotification, object: strongSelf.player?.currentItem)
strongSelf.playerLayer = AVPlayerLayer(player: strongSelf.player)
strongSelf.playerLayer?.frame = strongSelf.contentView.bounds
strongSelf.playerLayer?.videoGravity = AVLayerVideoGravityResizeAspect
strongSelf.contentView.layer.addSublayer(strongSelf.playerLayer!)
strongSelf.player?.play()
strongSelf.activityIndicatorView.stopAnimating()
Would any of you have an idea what could cause this problem? I'm also open to any workarounds if any of you could suggest one :)