I did lots of R&D but did not get any good answer. I am working on music type app in iphone and i have to categorized songs on the basis of beats per minute.So my first task to find out BPM of song.How can we do that? I got some answers like:
1.using MPMediaItemPropertyBeatsPerMinute property of MPMediaItem class ,will it work fine. I have doubt on this because some guys said it will return null.
Do you have any sample code or logic to get this.
Thanks in advance.
You can use http://www.un4seen.com/ for detecting BPM of song.
Here is code for calculating BPM using this library.
HSTREAM mainStream = BASS_StreamCreateFile(FALSE,[pathStr UTF8String],0,0,BASS_SAMPLE_FLOAT|BASS_STREAM_PRESCAN|BASS_STREAM_DECODE);
float playBackDuration=BASS_ChannelBytes2Seconds(mainStream, BASS_ChannelGetLength(mainStream, BASS_POS_BYTE));
HSTREAM bpmStream=BASS_StreamCreateFile(FALSE, [pathStr UTF8String], 0, 0, BASS_STREAM_PRESCAN|BASS_SAMPLE_FLOAT|BASS_STREAM_DECODE);
float BpmValue= BASS_FX_BPM_DecodeGet(
bpmStream,
0.00,
playBackDuration,
MAKELONG(45,256),
BASS_FX_BPM_MULT2,
NULL);
//Check if BpmValue have any value or not.
//If it haven't any value then set default value to 128.
if(BpmValue<=0)
BpmValue = 128.00;
You can do many other things like scratching using this library.
Get Bpm of audio songs within minute:
BASS_SetConfig(BASS_CONFIG_IOS_MIXAUDIO, 0); // Disable mixing. To be called before BASS_Init.
if (HIWORD(BASS_GetVersion()) != BASSVERSION) {
NSLog(#"An incorrect version of BASS was loaded");
}
// Initialize default device.
if (!BASS_Init(-1, 44100, 0, NULL, NULL)) {
NSLog(#"Can't initialize device");
}
//NSArray *array = [NSArray arrayWithObject:#""
NSString *respath = [[NSBundle mainBundle] pathForResource:#"[Songs.PK] Paathshaala - 01 - Aye Khuda" ofType:#"mp3"];
DWORD chan1;
if(!(chan1=BASS_StreamCreateFile(FALSE, [respath UTF8String], 0, 0, BASS_SAMPLE_LOOP))) {
NSLog(#"Can't load stream!");
}
mainStream=BASS_StreamCreateFile(FALSE, [respath cStringUsingEncoding:NSUTF8StringEncoding], 0, 0, BASS_SAMPLE_FLOAT|BASS_STREAM_PRESCAN|BASS_STREAM_DECODE);
float playBackDuration=BASS_ChannelBytes2Seconds(mainStream, BASS_ChannelGetLength(mainStream, BASS_POS_BYTE));
NSLog(#"Play back duration is %f",playBackDuration);
HSTREAM bpmStream=BASS_StreamCreateFile(FALSE, [respath UTF8String], 0, 0, BASS_STREAM_PRESCAN|BASS_SAMPLE_FLOAT|BASS_STREAM_DECODE);
//BASS_ChannelPlay(bpmStream,FALSE);
BpmValue= BASS_FX_BPM_DecodeGet(bpmStream,0.0,
playBackDuration,
MAKELONG(45,256),
BASS_FX_BPM_MULT2,
NULL);
NSLog(#"BPM is %f",BpmValue);
Apple provides aurioTouch sample code which display the input audio in one of the forms, a regular time domain waveform, a frequency domain waveform (computed by performing a fast fourier transform on the incoming signal), and a sonogram view (a view displaying the frequency content of a signal over time, with the color signaling relative power, the y axis being frequency and the x as time).
Related
This is a 2-part Question. I have the following code working which grabs the current display surface and creates a video out of the surfaces (everything happens in the background).
for(int i=0;i<100;i++){
IOMobileFramebufferConnection connect;
kern_return_t result;
IOSurfaceRef screenSurface = NULL;
io_service_t framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleH1CLCD"));
if(!framebufferService)
framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleM2CLCD"));
if(!framebufferService)
framebufferService = IOServiceGetMatchingService(kIOMasterPortDefault, IOServiceMatching("AppleCLCD"));
result = IOMobileFramebufferOpen(framebufferService, mach_task_self(), 0, &connect);
result = IOMobileFramebufferGetLayerDefaultSurface(connect, 0, &screenSurface);
uint32_t aseed;
IOSurfaceLock(screenSurface, kIOSurfaceLockReadOnly, &aseed);
uint32_t width = IOSurfaceGetWidth(screenSurface);
uint32_t height = IOSurfaceGetHeight(screenSurface);
m_width = width;
m_height = height;
CFMutableDictionaryRef dict;
int pitch = width*4, size = width*height*4;
int bPE=4;
char pixelFormat[4] = {'A','R','G','B'};
dict = CFDictionaryCreateMutable(kCFAllocatorDefault, 0, &kCFTypeDictionaryKeyCallBacks, &kCFTypeDictionaryValueCallBacks);
CFDictionarySetValue(dict, kIOSurfaceIsGlobal, kCFBooleanTrue);
CFDictionarySetValue(dict, kIOSurfaceBytesPerRow, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &pitch));
CFDictionarySetValue(dict, kIOSurfaceBytesPerElement, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &bPE));
CFDictionarySetValue(dict, kIOSurfaceWidth, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &width));
CFDictionarySetValue(dict, kIOSurfaceHeight, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &height));
CFDictionarySetValue(dict, kIOSurfacePixelFormat, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, pixelFormat));
CFDictionarySetValue(dict, kIOSurfaceAllocSize, CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &size));
IOSurfaceRef destSurf = IOSurfaceCreate(dict);
IOSurfaceAcceleratorRef outAcc;
IOSurfaceAcceleratorCreate(NULL, 0, &outAcc);
IOSurfaceAcceleratorTransferSurface(outAcc, screenSurface, destSurf, dict, NULL);
IOSurfaceUnlock(screenSurface, kIOSurfaceLockReadOnly, &aseed);
CFRelease(outAcc);
// MOST RELEVANT PART OF CODE
CVPixelBufferCreateWithBytes(NULL, width, height, kCVPixelFormatType_32BGRA, IOSurfaceGetBaseAddress(destSurf), IOSurfaceGetBytesPerRow(destSurf), NULL, NULL, NULL, &sampleBuffer);
CMTime frameTime = CMTimeMake(frameCount, (int32_t)5);
[adaptor appendPixelBuffer:sampleBuffer withPresentationTime:frameTime];
CFRelease(sampleBuffer);
CFRelease(destSurf);
frameCount++;
}
P.S: The last 4-5 lines of code are the most relevant(if you need to filter).
1) The video that is produced has artefacts. I have worked on videos previously and have encountered such an issue before as well. I suppose there can be 2 reasons for this:
i. The PixelBuffer that is passed to the adaptor is getting modified or released before the processing (encoding + writing) is complete. This can be due to asynchronous calls. But I am not sure if this itself is the problem and how to resolve it.
ii. The timestamps that are passed are inaccurate (e.g. 2 frames having the same timestamp or a frame having a lower timestamp than the previous frame). I logged out the timestamp values and this doesn't seem to be the problem.
2) The code above is not able to grab surfaces when a video is played or when we play games. All I get is a blank screen in the output. This might be due to hardware accelerated decoding that happens in such cases.
Any inputs on either of the 2 parts of the questions will be really helpful. Also, if you have any good links to read on IOSurfaces in general, please do post them here.
I did a bit of experimentation and concluded that the screen surface from which the content is copied is changing even before the transfer of contents is complete (call to IOSurfaceAcceleratorTransferSurface() ). I am using a lock (tried both asynchronous and read-only) but it is being overridden by the iOS. I changed the code between the lock/unlock part to the following minimal:
IOSurfaceLock(screenSurface, kIOSurfaceLockReadOnly, &aseed);
aseed1 = IOSurfaceGetSeed(screenSurface);
IOSurfaceAcceleratorTransferSurface(outAcc, screenSurface, destSurf, dict, NULL);
aseed2 = IOSurfaceGetSeed(screenSurface);
IOSurfaceUnlock(screenSurface, kIOSurfaceLockReadOnly, &aseed);
The GetSeed function tells if the contents of the surface have changed. And, I logged a count indicating the number of frames for which the seed changes. The count was non-zero. So, the following code resolved the problem:
if(aseed1 != aseed2){
//Release the created surface
continue; //Do not use this surface/frame since it has artefacts
}
This however does affect performance since many frames/surfaces are rejected due to artefacts.
Any additions/corrections to this will be helpful.
I'm building an iPhone app that generates random guitar music by playing back individual recorded guitar notes in "caf" format. These notes vary in duration from 3 to 11 seconds, depending on the amount of sustain.
I originally used the AVAudioPlayer for playback, and in the simulator at 120 bpm, playing 16th notes it sung beautifully, but on my handset, as soon as I
upped the tempo a little over 60 bpm playing just 1/4 notes, it ran like a dog and wouldn't keep in time. My elation was very short lived.
To reduce latency, I tried to implement playback via Audio Units using the Apple MixerHost project as a template for an audio engine, but kept getting a bad access error after I bolted it on and connected everything up.
After many hours of it doing my head in, I gave up on that avenue of thought and I bolted on the Novocaine audio engine instead.
I have now run into a brick wall trying to connect it up to my model.
On the most basic level, my model is a Neck object containing an NSDictionary of Note objects.
Each Note object knows what string and fret of the guitar neck it's on and contains its own AVAudioPlayer.
I build a chromatic guitar neck containing either 122 notes (6 strings by 22 frets) or 144 notes (6 strings by 24 frets) depending on the neck size selected in the user preferences.
I use these Notes as my single point of truth so all scalar Notes generated by the music engine are pointers to this chromatic note bucket.
#interface Note : NSObject <NSCopying>
{
NSString *name;
AVAudioPlayer *soundFilePlayer;
int stringNumber;
int fretNumber;
}
I always start off playback with the root Note or Chord of the selected scale and then generate the note to play next so I am always playing one note behind the generated note. This way, the next Note to play is always queued up ready to go.
Playback control of these Notes is a achieved with the following code:
- (void)runMusicGenerator:(NSNumber *)counter
{
if (self.isRunning) {
Note *NoteToPlay;
// pulseRate is the time interval between beats
// staticNoteLength = 1/4 notes, 1/8th notes, 16th notes, etc.
float delay = self.pulseRate / [self grabStaticNoteLength];
// user setting to play single, double or triplet notes.
if (self.beatCounter == CONST_BEAT_COUNTER_INIT_VAL) {
NoteToPlay = [self.GuitarNeck generateNoteToPlayNext];
} else {
NoteToPlay = [self.GuitarNeck cloneNote:self.GuitarNeck.NoteToPlayNow];
}
self.GuitarNeck.NoteToPlayNow = NoteToPlay;
[self callOutNoteToPlay];
[self performSelector:#selector(runDrill:) withObject:NoteToPlay afterDelay:delay];
}
- (Note *)generateNoteToPlayNext
{
if ((self.musicPaused) || (self.musicStopped)) {
// grab the root note on the string to resume
self.NoteToPlayNow = [self grabRootNoteForString];
//reset the flags
self.musicPaused = NO;
self.musicStopped = NO;
} else {
// Set NoteRingingOut to NoteToPlayNow
self.NoteRingingOut = self.NoteToPlayNow;
// Set NoteToPlaNowy to NoteToPlayNext
self.NoteToPlayNow = self.NoteToPlayNext;
if (!self.NoteToPlayNow) {
self.NoteToPlayNow = [self grabRootNoteForString];
// now prep the note's audio player for playback
[self.NoteToPlayNow.soundFilePlayer prepareToPlay];
}
}
// Load NoteToPlayNext
self.NoteToPlayNext = [self generateRandomNote];
}
- (void)callOutNoteToPlay
{
self.GuitarNeck.NoteToPlayNow.soundFilePlayer.delegate = (id)self;
[self.GuitarNeck.NoteToPlayNow.soundFilePlayer setVolume:1.0];
[self.GuitarNeck.NoteToPlayNow.soundFilePlayer setCurrentTime:0];
[self.GuitarNeck.NoteToPlayNow.soundFilePlayer play];
}
Each Note's AVAudioPlayer is loaded as follows:
- (AVAudioPlayer *)buildStringNotePlayer:(NSString *)nameOfNote
{
NSString *soundFileName = #"S";
soundFileName = [soundFileName stringByAppendingString:[NSString stringWithFormat:#"%d", stringNumber]];
soundFileName = [soundFileName stringByAppendingString:#"F"];
if (fretNumber < 10) {
soundFileName = [soundFileName stringByAppendingString:#"0"];
}
soundFileName = [soundFileName stringByAppendingString:[NSString stringWithFormat:#"%d", fretNumber]];
NSString *soundPath = [[NSBundle mainBundle] pathForResource:soundFileName ofType:#"caf"];
NSURL *fileURL = [NSURL fileURLWithPath:soundPath];
AVAudioPlayer *audioPlayer = [[AVAudioPlayer alloc] initWithContentsOfURL:fileURL error:nil];
return notePlayer;
}
Here is where I come a cropper.
According to the Novocaine Github page ...
Playing Audio
Novocaine *audioManager = [Novocaine audioManager];
[audioManager setOutputBlock:^(float *audioToPlay, UInt32 numSamples, UInt32 numChannels) {
// All you have to do is put your audio into "audioToPlay".
}];
But in the downloaded project, you use the following code to load the audio ...
// AUDIO FILE READING OHHH YEAHHHH
// ========================================
NSURL *inputFileURL = [[NSBundle mainBundle] URLForResource:#"TLC" withExtension:#"mp3"];
fileReader = [[AudioFileReader alloc]
initWithAudioFileURL:inputFileURL
samplingRate:audioManager.samplingRate
numChannels:audioManager.numOutputChannels];
[fileReader play];
fileReader.currentTime = 30.0;
[audioManager setOutputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels)
{
[fileReader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels];
NSLog(#"Time: %f", fileReader.currentTime);
}];
Here is where I really start to get confused because the first method uses a float and the second one uses a URL.
How do you pass a "caf" file to a float? I am not sure how to implement Novocaine - it is still fuzzy in my head.
My questions that I hope someone can help me with are as follows ...
Are Novocaine objects similar to AVAudioPlayer objects, just more versatile and tweaked to the max for minimum latency? i.e. self contained audio playing (/recording/generating) units?
Can I use Novocaine in my model as it is? i.e. 1 Novocaine object per chromatic note or should I have 1 novocain object that contains all the Chromatic Notes? Or do I just store the URL in the note instead and pass that to a Novocaine player?
How can I put my audio into "audioToPlay" when my audio is a "caf" file and "audioToPlay" take a float?
If I include and declare a Novocaine property in Note.m do I then have to rename the class to Note.mm in order to use the Novocaine object?
How do I play multiple Novocaine objects concurrently in order to reproduce chords and intervals?
Can I loop a Novocaine object's playback?
Can I set the playback length of a note? i.e. play a 10 sec note for only 1 sec?
Can I modify the above code to use Novocaine?
Is the method I am using for runMusicGenerator the correct one to use in order to maintain a tempo that is up to professional standards?
Novocaine makes your life easier by eliminating the need for you to setup the RemoteIO AudioUnit manually. This includes having to painfully fill a bunch of CoreAudio structs and providing a bunch of callbacks such as this audio process callback.
static OSStatus PerformThru(void *inRefCon, AudioUnitRenderActionFlags *ioActionFlags, const AudioTimeStamp *inTimeStamp, UInt32 inBusNumber, UInt32 inNumberFrames, AudioBufferList *ioData);
Instead Novocaine handles that in its implementation and then calls your block, which you set by doing this.
[audioManager setOutputBlock: ^(float *audioToPlay, UInt32 numSamples, UInt32 numChannels){} ];
Whatever you write to audioToPlay gets played.
Novocaine sets up the RemoteIO AudioUnit for you. This is a low-level CoreAudio API, different from the high-level AVFoundation, and very low-latency as expected. You are right in that Novocaine is self-contained. You can record, generate, and process audio in realtime.
Novocaine is a singleton, you cannot have multiple Novocaine instances. One way to do it is to store your guitar sound/sounds in a separate class or array, and then write a bunch of methods, using Novocaine to play them.
You have a bunch of options. You can use Novocaine's AudioFileReader to play your .caf file for you. You do this by allocating an AudioFileReader and then passing the URL of the .caf file you want to play, as per example code. You then stick [fileReader retrieveFreshAudio:data numFrames:numFrames numChannels:numChannels] in your block, as per example code. Each time your block is called, AudioFileReader grabs and buffers a chunk of audio from disk and puts it in audioToPlay which subsequently gets played. There are some disadvantages with this. For short sounds (such as your guitar sound I'm assuming) repeatedly calling retrieveFreshAudio is a performance hit. It is generally a better idea (for short sounds) to perform a synchronous, sequential read of the entire file into memory. Novocaine does not provide a way to do this (yet). You will have to use ExtAudioFileServices to do this. The Apple example project MixerHost details how to do this.
If you are using AudioFileReader yes. You only rename to .mm when you are #import ing from Obj-C++ headers or #include ing C++ headers.
As mentioned earlier, only 1 Novocaine instance is allowed. You can achieve polyphony by mixing multiple audio sources. This is simply just adding buffers together. If you have made multiple versions of the same guitar sound at different pitches, just read them all in to memory, and mix away. If you only want to have one guitar sound, then you have to, in realtime, change the playback rate of however many notes you are playing and then mixdown.
Novocaine is agnostic to what you are actually playing and does not care how long you are playing a sample for. In order to loop a sound, you have to maintain a count of how many samples have elapsed, check if you are at the end of your sound, and then set that count back to 0.
Yes. Assuming a 44.1k sample rate, 1 sec of audio = 44100 samples. You would then reset your count when it reaches 44100.
Yes. It looks something like this. Assuming you have 4 guitar sounds which are mono and longer than 1 second long, and you have read them into memory float *guitarC, *guitarE, *guitarG, *guitarB; (jazzy CMaj7 chord w00t), and want to mix them down for 1 second and loop that back in mono:
[audioManager setOutputBlock:^(float *data, UInt32 numFrames, UInt32 numChannels){
static int count = 0;
for(int i=0; i<numFrames; ++i){
//Mono mix each sample of each sound together. Since result can be 4x louder, divide the total amp by 4.
//You should be using `vDSP_vadd` from the accelerate framework for added performance.
data[count] = (guitarC[count] + guitarE[count] + guitarG[count] + guitarB[count]) * 0.25;
if(++count >= 44100) count = 0; //Plays the mix for 1 sec
}
}];
Not exactly. Using performSelector or any mechanism scheduled on a runloop or thread is not guaranteed to be precise. You might experience timing irregularities when the CPU load fluctuates, for example. Use the audio block if you want sample accurate timing.
You must had gone through this before coming to my this question.How to use kAudioUnitSubType_LowShelfFilter of kAudioUnitType_Effect which controls bass in core Audio? Slowly & Steadily getting the things right for bass control of music. But yet not got succeeded in my objective. Now i got to know that i have to change the kAULowShelfParam_CutoffFrequency to change the bass.
The following code i was using before 5 to 7 days. this code plays music properly but doesn't change bass properly. have a look on this code snippet:-
- (void)awakeFromNib
{
printf("AUGraphController awakeFromNib\n");
mIsPlaying = false;
// clear the mSoundBuffer struct
memset(&mUserData.soundBuffer, 0, sizeof(mUserData.soundBuffer));
// create the URLs we'll use for source A and B
NSString *sourceA = [[NSBundle mainBundle] pathForResource:#"04 - Second Hand Jawaani - [rKmania.com]" ofType:#"mp3"];
NSString *sourceB = [[NSBundle mainBundle] pathForResource:#"Track2" ofType:#"mp4"];
sourceURL[0] = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, (CFStringRef)sourceA, kCFURLPOSIXPathStyle, false);
sourceURL[1] = CFURLCreateWithFileSystemPath(kCFAllocatorDefault, (CFStringRef)sourceB, kCFURLPOSIXPathStyle, false);
[bassSlider addTarget:self action:#selector(selectEQPreset) forControlEvents:UIControlEventValueChanged];
}
// output unit
CAComponentDescription output_desc(kAudioUnitType_Output, kAudioUnitSubType_RemoteIO, kAudioUnitManufacturer_Apple);
// iPodEQ unit
CAComponentDescription eq_desc(kAudioUnitType_Effect, kAudioUnitSubType_AUiPodEQ, kAudioUnitManufacturer_Apple);
// multichannel mixer unit
CAComponentDescription mixer_desc(kAudioUnitType_Mixer, kAudioUnitSubType_MultiChannelMixer, kAudioUnitManufacturer_Apple);
printf("add nodes\n");
- (void)selectEQPreset;
{
AUPreset *aPreset = (AUPreset*)CFArrayGetValueAtIndex(mEQPresetsArray, value);
OSStatus result = AudioUnitSetProperty(mEQ, kAudioUnitProperty_PresentPreset, kAudioUnitScope_Global, 0, aPreset, sizeof(AUPreset));
if (result) { printf("AudioUnitSetProperty result %ld %08X %4.4s\n", result, (unsigned int)result, (char*)&result); return; };
printf("SET EQ PRESET %d ", value);
CFShow(aPreset->presetName);
}
Now after this code i got to know that i have to change frequency to change the bass, then i use the following code snippet but in the following code snippet music is not playing and its giving excess_bad too. i'm just writing the changes i made to the code.have a look on current code snippet:-
// iPodEQ unit
CAComponentDescription eq_desc(kAudioUnitType_Effect, kAudioUnitSubType_LowShelfFilter, kAudioUnitManufacturer_Apple);
- (void)selectEQPreset;
{
AudioUnit lowShelfAU;
assert(lowShelfAU);
float frequencyInHz = 120.0f;
frequencyInHz = bassSlider.value;
OSStatus result = AudioUnitSetParameter(lowShelfAU,kAULowShelfParam_CutoffFrequency,kAudioUnitScope_Global,0,frequencyInHz,0);
if (noErr != result)
{
assert(0 && "error!");
return ;
}
}
This code now i'm using but this is not changing the frequency. Its even stop playing the music and giving the excess_bad error on this line of code..
AudioUnitSetParameter(lowShelfAU,kAULowShelfParam_CutoffFrequency,kAudioUnitScope_Global,0,frequencyInHz,0);
Please anybody help me regarding this tell me how can i change the kAULowShelfParam_CutoffFrequency so that i can adjust the bass of music Via slider. Any help regarding this would be highly appreciable.
Thanks :)
Before you can use an AudioUnit you need to create it. If you're using an AUGraph your code will look something like:
AudioComponentDescription filterDescription = { kAudioUnitType_Effect, kAudioUnitSubType_LowShelfFilter, kAudioUnitSubType_LowShelfFilter, 0, 0 };
AUNode filterNode = -1;
OSStatus result = AUGraphAddNode(mAUGraph, &filterDescription, &filterNode);
if(noErr != result) {
// Handle error
}
AudioUnit filterUnit = nullptr;
result = AUGraphNodeInfo(mAUGraph, filterNode, nullptr, &filterUnit);
if(noErr != result) {
// Handle error
}
// Set parameters on filterUnit
The reason your code is failing is that the line
AudioUnit lowShelfAU;
initializes lowShelfAU with an undetermined value. An AudioUnit is a pointer type so without initialization it points to an unknown area of memory. I think it is a programming best practice to always initialize your variables when they are declared, to catch these kinds of bugs:
AudioUnit lowShelfAU = nullptr;
`AudioUnit lowShelfAU;` <-- that is an uninitialized garbage value
you need to actually create an AU instance (the Low Shelf) and add it to an AUGraph.
Note: The compiler/analyzer will identify this problem for you. I recommend people turn the warning levels waaaay up, then build, analyze, and remove all issues. Repeat until clean.
`as you know ,apple create 3 audio units according their purpose.
remoteIO --> for input and output;mixer --> for audio mixer ;for some audio effect is eq and Filter so,if u wanna cutoff the audio frequency,u need to create a filter unit and connect them together.
I am currently working on an audio DSP App development. The project requires direct access and modification of audio data. Right now I can successfully access and modify the raw audio data using AudioQueue but encounters error during playback. The output audio after any modification turns out be noise.
In short, the code is something like this:
(Modified from Speakhere sample code. The rest remains unchanged.)
void AQPlayer::AQBufferCallback(void * inUserData,
AudioQueueRef inAQ,
AudioQueueBufferRef inCompleteAQBuffer)
{
AQPlayer *THIS = (AQPlayer *)inUserData;
if (THIS->mIsDone) return;
UInt32 numBytes;
UInt32 nPackets = THIS->GetNumPacketsToRead();
OSStatus result = AudioFileReadPackets(THIS->GetAudioFileID(),
false,
&numBytes,
inCompleteAQBuffer->mPacketDescriptions,
THIS->GetCurrentPacket(),
&nPackets,
inCompleteAQBuffer->mAudioData);
if (result)
printf("AudioFileReadPackets failed: %d", (int)result);
if (nPackets > 0) {
inCompleteAQBuffer->mAudioDataByteSize = numBytes;
inCompleteAQBuffer->mPacketDescriptionCount = nPackets;
//My modification starts from here
//Modifying audio data
SInt16 *testBuffer = (SInt16*)inCompleteAQBuffer->mAudioData;
for (int i = 0; i < (inCompleteAQBuffer->mAudioDataByteSize)/sizeof(SInt16); i++)
{
//printf("before modification %d", (int)*testBuffer);
*testBuffer = (SInt16) *testBuffer/2; //Say some simple modification
//printf("after modification %d", (int)*testBuffer);
testBuffer++;
}
AudioQueueEnqueueBuffer(inAQ, inCompleteAQBuffer, 0, NULL);
}
During debugging, the data in buffer is displayed as expected, but the actual output is nothing but noise.
Here are some other strange behaviors of the code that makes both the whole team crazy:
If there is no change to the data (add/sub by 0, multiply by 1) or the whole buffer is assigned to a constant (say 0, then the audio will be muted), the playback behaves normally (Of course!) But if I perform anything more than it, it still turns out to be noise.
In the case I hardcode a single tone as test audio, the output noise spreads into another channel also.
So where is the bug in this code? Or if I am on the wrong track, what is the correct approach to modify the audio data and perform playback CORRECTLY? Any insight will be sincerely appreciated.
Thank you very much :-)
Cheers,
Manca
are you SURE the sample format is SInt16? And how many channels are there? You seem to treat the audio as a single channel short stream, but suppose the format is actually dual channel Float32 or so, and you do the modifications there, than the effect would be exactly as you describe, including the noise on other channels.
I have analyzed "SpeakHere" sample code of iPhone dev forum.
There is a code for starting AudioQueue as following..
AudioTimeStamp ats = {0};
AudioQueueStart(mQueue, &ats);
But I have no idea that how to start middle of file.
I changed AudioTimeStamp with various values include negative. But it does not works.
Please let me know your great opinion. Thanks.
AudioQueueStart is not the function that will help you to do that. The time there is like a delay, if you pass NULL then it means that the queue will start ASAP.
You have to pass the frame you want to play and enqueue it, to calculate that you have to know the number of frames your file has and the (relative) position you want to play.
These are instructions how to make it in SpeakHere
In the new (objc++ based) SpeakHere
In AQPlayer.h add a private instance variable:
UInt64 mPacketCount;
and a public method:
void SetQueuePosition(float position) { mCurrentPacket = mPacketCount*position; };
In AQPlayer.mm inside AQPlayer::SetupNewQueue() before mIsInitialized = true; add:
// get the total number of packets
UInt32 sizeOfPacketsCount = sizeof(mPacketCount);
XThrowIfError (AudioFileGetProperty (mAudioFile, kAudioFilePropertyAudioDataPacketCount, &sizeOfPacketsCount, &mPacketCount), "get packet count");
Now you have to use it (In SpeakHereControler.mm add this and link it to a UISlider for example):
- (IBAction) sliderValueChanged:(UISlider *) sender
{
float value = [sender value];
player->SetQueuePosition(position);
}
Why this works:
The playback callback function AudioQueueOutputCallback that feeds the queue with new packets and which in the new SpeakHere is: void AQPlayer::AQBufferCallback( , , ) calls AudioFileReadPackets to read and enqueue a certain part of a file. For that task mCurrentPacket is used and that is what we just adjusted in above methods, hence the part you wanted to play is read, enqueued and finally played :)
Just for historical reasons :)
In the old (objc based) SpeakHere
In AudioPlayer.h add an instance variable:
UInt64 totalFrames;
AudioPlayer.m inside
- (void) openPlaybackFile: (CFURLRef) soundFile
add:
UInt32 sizeOfTotalFrames = sizeof(UInt64);
AudioFileGetProperty (
[self audioFileID],
kAudioFilePropertyAudioDataPacketCount,
&sizeOfTotalFrames,
&totalFrames
);
Then add a method to AudioPlayer.h and .m
- (void) setRelativePlaybackPosition: (float) position
{
startingPacketNumber = totalFrames * position;
}
Now you have to use it (In AudioViewController add this and link it to a UISlider for example):
- (IBAction) setPlaybackPosition: (UISlider *) sender
{
float value = [sender value];
[audioPlayer setRelativePlaybackPosition: value];
}
When value is 0 you will play from the beggining, 0.5 from the middle, etc.
Hope this helps.