OfflineContext.suspend stops the progression of OfflineContext.currentTime, but what effect does it have while rendering (OfflineContext.startRendering)? What I want to do is start the rendering process, pause it, do some other task and resume it when the other task is done. While the rendering process is paused, the imminent AudioBuffer should not be getting larger, meaning that when I export the AudioBuffer into a wav file and play it, there should be no silence corresponding to the pause that was taken by the rendering process.
I have tried OfflineContext.suspend while rendering and it does seem to add some silence in the resulting wav file, but perhaps I'm doing something wrong.
How can I pause the rendering process?
What is OfflineContext.suspend for?
suspend is intended to cause the offline context to stop at controlled times, before startRendering() is called. You can call suspend() after starting rendering, but this is not very precise, especially since rendering is probably faster than real time.
Plus you have not access to the AudioBuffer during rendering. If you want to capture the audio while rendering, use a ScriptProcessorNode or an AudioWorkletNode to save the audio data.
Related
I'm working with the raw camera image, but need to restart the preview following the capture of an image. When i call startPreview() following takePicture, the android hangs because the camera is till in use. I've waited till after the raw image has been written to disk, but the camera is still in use, so the start preview still hangs the system.
camera.takePicture(null, null, null);
(test needed here){
camera.startPreview();
}
Putting the start preview in the rawcallback hangs the android.
When calling takePicture, the jpegCallback occurs after the camera is finished, so it is safe to start the preview, but this also creates a .jpg.
The question is following the takePicture, is there any way to determine when the camera is idle? (other than the jpegCallback?).
I've found ways around the problem, such as starting the preview on a timer, but still wonder if there is any way to determine the actual status of the camera and a way to test when it is save to restart the preview.
After calling the takePicture() function, you must wait until the JPEG callback has returned for the camera to be ready to use again. There is no way to monitor its status. The documentation explicitly states the following:
After calling this method, you must not call startPreview() or take
another picture until the JPEG callback has returned.
If you need some fast way of grabbing frames (i.e. images) without stopping and restarting the camera preview, you can implement the PreviewCallback interface which will simply allow you to grab preview frames without stopping the preview at all. Just remember that you will need to convert these from YUV format (which is not difficult using Android's YUVImage class.
I am having a problem finding resources on playing an attack (start of sound) / sustain (looping sound) / decay (ending of sound) sequence with no transition breaks. Are there any good libraries for handling this, or should I roll my own with AVAudioPlayer? Is AudioQueue a better place to look? I used to use SoundEngine.cpp, but that's been long gone for a while. Is CAF still the best format to use for it?
Thanks!
From your description, it sounds as if you're trying to write a software synthesizer. The only way that you could use AVAudioPlayer for something like this would be to compose the entire duration of a note as a single WAV file and then play the whole thing with AVAudioPlayer.
To create a note sound of arbitrary duration, one that begins playing in response to a user action (like tapping a button) and then continues playing until a second user action (like tapping a "stop" button or lifting the finger off the first button) begins the process of ramping the looped region's volume down to zero (the "release" part), you will need to use AudioQueue (AVAudioPlayer can be used to play audio constructed entirely in memory, but the entire playback has to be constructed before play begins, meaning that you cannot change what is being played in response to user actions [other than to stop playback]).
Here's another question/answer that shows simply how to use AudioQueue. AudioQueue calls a callback method whenever it needs to load up more data to play - you would have to implement all the code that loops and envelope-wraps the original WAV file data.
creating your own envelope generator is very simple. the tough part will be updating your program to use lower level audio services in order to alter the signal directly.
to do this, you will need:
the audio file's samples
set up an AudioQueue (that's one approach, but i am going with it because it was mentioned in the OP, and it is relatively high level API for a user provided sample buffer)
provide a signal to the queue
determine if your program is best in realtime or pre-rendered
Realtime
Allows live variations
manage your loop points
manage your render position
be able to determine the amplitude to apply based on the sample position range you are reading
or
Prerendered
May require more memory
Requires less CPU
apply the envelope to your copy of the sample buffer
manage your render position
I also assume that you need only slow/simple transitions. If you want some crazy/fast LFO, without aliasing, you will have a lot more work to do. This approach should not produce audible aliasing unless your changes are too abrupt:
Writing a simple envelope generator (EG) is easy; check out Apple's SinSynth for a very basic EG if you need a push in that direction.
I am wondering if there is a way to manipulate the audio buffer when the audio queue is paused. So the pseudo logic goes like this:
1. pause audio queue
2. manipulate the audio buffers in the queue except the one that is being handed to the callback function.
3. start the audio queue again
I notice the problem would be when I try to manipulate the audio buffer that is being decoded and fed to the device. So anyone has ever tried this before?
I think this path will lead to pain and suffering, and one that could be simulated without breaking the paradigm that AudioQueue sets forth. The whole point of the queue is to feed buffers to a callback that you implement so you can manipulate each sample as you see fit before passing it down the chain.
Maybe if you can explain the context of what you're trying to accomplish, a more suitable solution could be offered.
I've been playing around with Apple's aurioTouch demo which is sample code for their Audio Unit tutorial. This application allows simultaneous input/output from the mic. to speaker. It also renders a stereograph of the inputted sound from the mic.
At a really high-level of this low-level process, the sample code defines an AudioComponent (in this case RemoteIO which allows for simultaneous input/output) and there is a render callback for this Audio Unit. In the callback they do some audio filtering (a DC Rejection Filter) and visualization of the stereograph based on the AudioBuffer sound data from the mic.
My ultimate goal is to create my own custom sound distortion Audio Unit based on the input from the mic. I think the proper way to do this based on the Audio Unit tutorial is to make a second Audio Unit and connect them with an Audio Processing Graph. However, I've read that iOS doesn't allow you to register your own custom Audio Units. My questions are:
Can I do direct manipulation on the AudioBufferList that I have access to in the render callback from the remoteIO Audio Unit (since they already seem to be doing this and applying an audio filter on it) and create my own custom sound distortion there?
I've tried assigning the AudioBufferList data to a constant (a value I've seen it hold from a sample run and logging of the AudioBufferList), but it appears to do nothing.
The answer to your first question is yes. That is generally how it is done.
I believe that you need to manipulate the data in the pointer directly, rather than reassigning. You may want to take a look at the code in openframeworks that handles assigning buffers and passing them to a callback: https://github.com/openframeworks/openFrameworks/blob/master/addons/ofxiPhone/src/sound/ofxiPhoneSoundStream.mm
There is other code out there that you can look at, nick collins has a basic application for getting sound off the microphone and out the speaker, whist processing inbetween: http://www.cogs.susx.ac.uk/users/nc81/code.html. He also has code there that gets sample buffers out of a iPod track that may be useful to you.
It is true that you can not add your own custom AudioUnits to the iPhone.
The way it works it like this: The speaker drives the pull-chain of data through the system. You add a render callback to the ioUnit, as you have already done.
The callback runs whenever the speaker (bus #0) is hungry and it is your job to fill in as many samples as it has requested, in the buffer that the speaker has provided. The size of the provided buffer will be a power of two that is as close as possible to the Preferred IO Buffer Duration you specified when configuring the AudioSession.
The simplest way to do that is to take the AudioBufferList you were given and pass it to AudioUnitRender on the the microphone (bus #1). After you fill in the buffer with Render(), but before the callback returns, you can manipulate the data any way you like. For example, AurioTouch zeroes it out to mute it.
The important thing to remember is that the speaker is going to read from the actual data buffer that it passed you. It is not going to look at the AudioBufferList descriptor and check if you pointed to another databuffer. If you start changing the AudioBufferList that you were given, you will run into problems. At best, you will be ignored. At worst, you will run into memory management issues.
If you don't want to be constrained to working in just the ioData buffer, then you can use your own AudioBufferList, allocated any way you like, in any size, and ask the microphone to Render() into that. Then you can do all the manipulation you like so long as in the end you copy the results into the buffer provided by the callback (i.e. ioData->mBuffers[0].mData as it was at the time the Callback was invoked).
I'm working on a project which is using Audio Toolbox for recording and playback of PCM data, and I'm having trouble with playback. In the simulator, I can record and play audio just fine, using a custom class to handle storing and sourcing PCM bytes for the recording and playback buffers as needed. On device (iPhone (3.0.1) and iPod 2G (3.1.2)) recording works fine, the audio files produced are correct, but in-app playback stutters, like it's only playing part of each playback buffer. My buffers are one second long, and I've got 3 buffers, which are preloaded before playback starts; stuttering occurs during those first 3 seconds as well, which I think rules out a latency problem.
I've written Audio Toolbox code before that worked, and I'm not doing anything strange here except that I'm using my own class to source PCM data instead of AudioFileReadBytes()
I know the data that comes out of my source is good, because it plays right in the sim, and it writes to disk as a correct audio file
I've played around with sample rates a bit; I'm normally using 11025Hz sampling to cut down on file size (it's all voice, so it sounds fine). at 44100Hz, but with the same size of buffers, I get the same stuttering problem, but the audio segments come a lot faster, about 4 times faster. That's why I think it's only playing part of each buffer.
The only reason I can conceive that it would only play part of each buffer is a latency problem... like the audio toolbox code is running out of full buffers while I'm still filling an empty one. But that would cause it to play the preloaded buffers correctly, and then start stuttering, and that doesn't happen, it stutters the whole way through
I've tried humongous buffers, like 10MB buffers, and I just get silence and a single stutter of audio at the end of playback. I've also tried preloading more buffers than normal, like 10 seconds worth of audio, and it behaves the same.
The audio session is being set with AVAudioSession, not the Audio Toolbox calls, and it's being set to the Playback category for playback
I have no idea how to try and attack this problem, it makes no sense to me that it works fine on the simulator but not the device.
Code for the playing callback and the set up for the audio queue services: http://pastebin.com/mfaa546c
It turns out that the use of NSData's GetBytes:length: was causing the problem. The buffer filled with that method was playing incorrectly. However, doing a memcpy from that buffer to another buffer would prevent the problem.