I have a problem with the this function AudioConverterConvertBuffer. Basically I want to convert from this format
_
streamFormat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked |0 ;
_streamFormat.mBitsPerChannel = 16;
_streamFormat.mChannelsPerFrame = 2;
_streamFormat.mBytesPerPacket = 4;
_streamFormat.mBytesPerFrame = 4;
_streamFormat.mFramesPerPacket = 1;
_streamFormat.mSampleRate = 44100;
_streamFormat.mReserved = 0;
to this format
_streamFormatOutput.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kLinearPCMFormatFlagIsPacked|0 ;//| kAudioFormatFlagIsNonInterleaved |0;
_streamFormatOutput.mBitsPerChannel = 16;
_streamFormatOutput.mChannelsPerFrame = 1;
_streamFormatOutput.mBytesPerPacket = 2;
_streamFormatOutput.mBytesPerFrame = 2;
_streamFormatOutput.mFramesPerPacket = 1;
_streamFormatOutput.mSampleRate = 44100;
_streamFormatOutput.mReserved = 0;
and what i want to do is to extract an audio channel(Left channel or right channel) from an LPCM buffer based on the input format to make it mono in the output format. Some logic code to convert is as follows
This is to set the channel map for PCM output file
SInt32 channelMap[1] = {0};
status = AudioConverterSetProperty(converter, kAudioConverterChannelMap, sizeof(channelMap), channelMap);
and this is to convert the buffer in a while loop
AudioBufferList audioBufferList;
CMBlockBufferRef blockBuffer;
CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampBuffer, NULL, &audioBufferList, sizeof(audioBufferList), NULL, NULL, 0, &blockBuffer);
for (int y=0; y<audioBufferList.mNumberBuffers; y++) {
AudioBuffer audioBuffer = audioBufferList.mBuffers[y];
//frames = audioBuffer.mData;
NSLog(#"the number of channel for buffer number %d is %d",y,audioBuffer.mNumberChannels);
NSLog(#"The buffer size is %d",audioBuffer.mDataByteSize);
numBytesIO = audioBuffer.mDataByteSize;
convertedBuf = malloc(sizeof(char)*numBytesIO);
status = AudioConverterConvertBuffer(converter, audioBuffer.mDataByteSize, audioBuffer.mData, &numBytesIO, convertedBuf);
char errchar[10];
NSLog(#"status audio converter convert %d",status);
if (status != 0) {
NSLog(#"Fail conversion");
assert(0);
}
NSLog(#"Bytes converted %d",numBytesIO);
status = AudioFileWriteBytes(mRecordFile, YES, countByteBuf, &numBytesIO, convertedBuf);
NSLog(#"status for writebyte %d, bytes written %d",status,numBytesIO);
free(convertedBuf);
if (numBytesIO != audioBuffer.mDataByteSize) {
NSLog(#"Something wrong in writing");
assert(0);
}
countByteBuf = countByteBuf + numBytesIO;
But the insz problem is there... so it cant convert. I would appreciate any input
Thanks in advance
First, you cannot use AudioConverterConvertBuffer() to convert anything where input and output byte size is different. You need to use AudioConverterFillComplexBuffer(). This includes performing any kind of sample rate conversions, or adding/removing channels.
See Apple's documentation on AudioConverterConvertBuffer(). This was also discussed on Apple's CoreAudio mailing lists, but I'm afraid I cannot find a reference right now.
Second, even if this could be done (which it can't) you are passing the same number of bytes allocated for output as you had for input, despite actually requiring half of the number of bytes (due to reducing number of channels from 2 to 1).
I'm actually working on using AudioConverterConvertBuffer() right now, and the test files are mono while I need to play stereo. I'm currently stuck with the converter performing conversion only of the first chunk of the data. If I manage to get this to work, I'll try to remember to post the code. If I don't post it, please poke me in comments.
Related
I am working on an iOS project that needs to encode and decode Speex audio using a remoteIO audio unit as input / output.
The problem I am having is although speex doesn't print any errors, the audio I get is somewhat recognizable as voice but very distorted, sort of sounds like the gain was just cranked up in a robotic way.
Here are the encode and decode functions (Input to encode is 320 bytes of signed integers from the audio unit render function, Input to decode is 62 bytes of compressed data ):
#define AUDIO_QUALITY 10
#define FRAME_SIZE 160
#define COMP_FRAME_SIZE 62
char *encodeSpeexWithBuffer(spx_int16_t *buffer, int *insize) {
SpeexBits bits;
void *enc_state;
char *outputBuffer = (char *)malloc(200);
speex_bits_init(&bits);
enc_state = speex_encoder_init(&speex_nb_mode);
int quality = AUDIO_QUALITY;
speex_encoder_ctl(enc_state, SPEEX_SET_QUALITY, &quality);
speex_bits_reset(&bits);
speex_encode_int(enc_state, buffer, &bits);
*insize = speex_bits_write(&bits, outputBuffer, 200);
speex_bits_destroy(&bits);
speex_encoder_destroy(enc_state);
return outputBuffer;
}
short *decodeSpeexWithBuffer(char *buffer) {
SpeexBits bits;
void *dec_state;
speex_bits_init(&bits);
dec_state = speex_decoder_init(&speex_nb_mode);
short *outTemp = (short *)malloc(FRAME_SIZE * 2);
speex_bits_read_from(&bits, buffer, COMP_FRAME_SIZE);
speex_decode_int(dec_state, &bits, outTemp);
speex_decoder_destroy(dec_state);
speex_bits_destroy(&bits);
return outTemp;
}
And the audio unit format:
// Describe format
audioFormat.mSampleRate = 8000.00;
audioFormat.mFormatID = kAudioFormatLinearPCM;
audioFormat.mFormatFlags = kAudioFormatFlagIsSignedInteger |
kAudioFormatFlagsNativeEndian |
kAudioFormatFlagIsPacked;
audioFormat.mFramesPerPacket = 1;
audioFormat.mChannelsPerFrame = 1;
audioFormat.mBitsPerChannel = 16;
audioFormat.mBytesPerPacket = 2;
audioFormat.mBytesPerFrame = 2;
No errors are reported anywhere and I have confirmed that the Audio Unit is processing at a sample rate of 8000
After a few days of going crazy over this I finally figured it out. The trick with Speex is that you must initialize a SpeexBit and encoder void* and use them throughout the entire session. Because I was recreating them for every piece of the encode it was causing strange sounding results.
Once I moved:
speex_bits_init(&bits);
enc_state = speex_encoder_init(&speex_nb_mode);
Out of the while loop everything worked great.
I am currently in the process of building an application that reads in audio from my iPhone's microphone, and then does some processing and visuals. Of course I am starting with the audio stuff first, but am having one minor problem.
I am defining my sampling rate to be 44100 Hz and defining my buffer to hold 4096 samples. Which is does. However, when I print this data out, copy it into MATLAB to double check accuracy, the sample rate I have to use is half of my iPhone defined rate, or 22050 Hz, for it to be correct.
I think it has something to do with the following code and how it is putting 2 bytes per packet, and when I am looping through the buffer, the buffer is spitting out the whole packet, which my code assumes is a single number. So what I am wondering is how to split up those packets and read them as individual numbers.
- (void)setupAudioFormat {
memset(&dataFormat, 0, sizeof(dataFormat));
dataFormat.mSampleRate = kSampleRate;
dataFormat.mFormatID = kAudioFormatLinearPCM;
dataFormat.mFramesPerPacket = 1;
dataFormat.mChannelsPerFrame = 1;
// dataFormat.mBytesPerFrame = 2;
// dataFormat.mBytesPerPacket = 2;
dataFormat.mBitsPerChannel = 16;
dataFormat.mReserved = 0;
dataFormat.mBytesPerPacket = dataFormat.mBytesPerFrame = (dataFormat.mBitsPerChannel / 8) * dataFormat.mChannelsPerFrame;
dataFormat.mFormatFlags =
kLinearPCMFormatFlagIsSignedInteger |
kLinearPCMFormatFlagIsPacked;
}
If what I described is unclear, please let me know. Thanks!
EDIT
Adding the code that I used to print the data
float *audioFloat = (float *)malloc(numBytes * sizeof(float));
int *temp = (int*)inBuffer->mAudioData;
int i;
float power = pow(2, 31);
for (i = 0;i<numBytes;i++) {
audioFloat[i] = temp[i]/power;
printf("%f ",audioFloat[i]);
}
I found the problem with what I was doing. It was a c pointer issue, and since I have never really programmed in C before, I of course got them wrong.
You can not directly cast inBuffer->mAudioData to an int array. So what I simply did was the following
SInt16 *buffer = malloc(sizeof(SInt16)*kBufferByteSize);
buffer = inBuffer->mAudioData;
This worked out just fine and now my data is of correct length and the data is represented properly.
I saw your answer, there also is an underlying issue which gives wrong sample data bytes which is because of an endian issue of bytes being swapped.
-(void)feedSamplesToEngine:(UInt32)audioDataBytesCapacity audioData:(void *)audioData {
int sampleCount = audioDataBytesCapacity / sizeof(SAMPLE_TYPE);
SAMPLE_TYPE *samples = (SAMPLE_TYPE*)audioData;
//SAMPLE_TYPE *sample_le = (SAMPLE_TYPE *)malloc(sizeof(SAMPLE_TYPE)*sampleCount );//for swapping endians
std::string shorts;
double power = pow(2,10);
for(int i = 0; i < sampleCount; i++)
{
SAMPLE_TYPE sample_le = (0xff00 & (samples[i] << 8)) | (0x00ff & (samples[i] >> 8)) ; //Endianess issue
char dataInterim[30];
sprintf(dataInterim,"%f ", sample_le/power); // normalize it.
shorts.append(dataInterim);
}
I want to read a sound file from application bundle, copy it, play with its maximum volume level(Gain value or peak power, I'm not sure about the technical name of it), and then write it as another file to the bundle again.
I did the copying and writing part. Resulting file is identical to input file. I use AudioFileReadBytes() and AudioFileWriteBytes() functions of AudioFile services in AudioToolbox framework to do that.
So, I have the input file's bytes and also its audio data format(via use of AudioFileGetProperty() with kAudioFilePropertyDataFormat) but I can't find a variable in these to play with the original file's maximum volume level.
To clarify my purpose, I'm trying to produce another sound file of which volume level is increased or decreased relative to the original one, so I don't care about the system's volume level which is set by the user or iOS.
Is that possible to do with the framework I mentioned? If not, are there any alternative suggestions?
Thanks
edit:
Walking through Sam's answer regarding some audio basics, I decided to expand the question with another alternative.
Can I use AudioQueue services to record existing sound file(which is in the bundle) to another file and play with the volume level(with the help of framework) during the recording phase?
update:
Here's how I'm reading the input file and writing the output. Below code lowers the sound level for "some" of the amplitude values but with lots of noise. Interestingly, if I choose 0.5 as amplitude value it increases the sound level instead of lowering it, but when I use 0.1 as amplitude value it lowers the sound. Both cases involve disturbing noise. I think that's why Art is talking about normalization, but I've no idea about normalization.
AudioFileID inFileID;
CFURLRef inURL = [self inSoundURL];
AudioFileOpenURL(inURL, kAudioFileReadPermission, kAudioFileWAVEType, &inFileID)
UInt32 fileSize = [self audioFileSize:inFileID];
Float32 *inData = malloc(fileSize * sizeof(Float32)); //I used Float32 type with jv42's suggestion
AudioFileReadBytes(inFileID, false, 0, &fileSize, inData);
Float32 *outData = malloc(fileSize * sizeof(Float32));
//Art's suggestion, if I've correctly understood him
float ampScale = 0.5f; //this will reduce the 'volume' by -6db
for (int i = 0; i < fileSize; i++) {
outData[i] = (Float32)(inData[i] * ampScale);
}
AudioStreamBasicDescription outDataFormat = {0};
[self audioDataFormat:inFileID];
AudioFileID outFileID;
CFURLRef outURL = [self outSoundURL];
AudioFileCreateWithURL(outURL, kAudioFileWAVEType, &outDataFormat, kAudioFileFlags_EraseFile, &outFileID)
AudioFileWriteBytes(outFileID, false, 0, &fileSize, outData);
AudioFileClose(outFileID);
AudioFileClose(inFileID);
You won't find amplitude scaling operations in (Ext)AudioFile, because it's about the simplest DSP you can do.
Let's assume you use ExtAudioFile to convert whatever you read into 32-bit floats. To change the amplitude, you simply multiply:
float ampScale = 0.5f; //this will reduce the 'volume' by -6db
for (int ii=0; ii<numSamples; ++ii) {
*sampOut = *sampIn * ampScale;
sampOut++; sampIn++;
}
To increase the gain, you simply use a scale > 1.f. For example, an ampScale of 2.f would give you +6dB of gain.
If you want to normalize, you have to make two passes over the audio: One to determine the sample with the greatest amplitude. Then another to actually apply your computed gain.
Using AudioQueue services just to get access to the volume property is serious, serious overkill.
UPDATE:
In your updated code, you're multiplying each byte by 0.5 instead of each sample. Here's a quick-and-dirty fix for your code, but see my notes below. I wouldn't do what you're doing.
...
// create short pointers to our byte data
int16_t *inDataShort = (int16_t *)inData;
int16_t *outDataShort = (int16_t *)inData;
int16_t ampScale = 2;
for (int i = 0; i < fileSize; i++) {
outDataShort[i] = inDataShort[i] / ampScale;
}
...
Of course, this isn't the best way to do things: It assumes your file is little-endian 16-bit signed linear PCM. (Most WAV files are, but not AIFF, m4a, mp3, etc.) I'd use the ExtAudioFile API instead of the AudioFile API as this will convert any format you're reading into whatever format you want to work with in code. Usually the simplest thing to do is read your samples in as 32-bit float. Here's an example of your code using ExtAudioAPI to handle any input file format, including stereo v. mono
void ScaleAudioFileAmplitude(NSURL *theURL, float ampScale) {
OSStatus err = noErr;
ExtAudioFileRef audiofile;
ExtAudioFileOpenURL((CFURLRef)theURL, &audiofile);
assert(audiofile);
// get some info about the file's format.
AudioStreamBasicDescription fileFormat;
UInt32 size = sizeof(fileFormat);
err = ExtAudioFileGetProperty(audiofile, kExtAudioFileProperty_FileDataFormat, &size, &fileFormat);
// we'll need to know what type of file it is later when we write
AudioFileID aFile;
size = sizeof(aFile);
err = ExtAudioFileGetProperty(audiofile, kExtAudioFileProperty_AudioFile, &size, &aFile);
AudioFileTypeID fileType;
size = sizeof(fileType);
err = AudioFileGetProperty(aFile, kAudioFilePropertyFileFormat, &size, &fileType);
// tell the ExtAudioFile API what format we want samples back in
AudioStreamBasicDescription clientFormat;
bzero(&clientFormat, sizeof(clientFormat));
clientFormat.mChannelsPerFrame = fileFormat.mChannelsPerFrame;
clientFormat.mBytesPerFrame = 4;
clientFormat.mBytesPerPacket = clientFormat.mBytesPerFrame;
clientFormat.mFramesPerPacket = 1;
clientFormat.mBitsPerChannel = 32;
clientFormat.mFormatID = kAudioFormatLinearPCM;
clientFormat.mSampleRate = fileFormat.mSampleRate;
clientFormat.mFormatFlags = kLinearPCMFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved;
err = ExtAudioFileSetProperty(audiofile, kExtAudioFileProperty_ClientDataFormat, sizeof(clientFormat), &clientFormat);
// find out how many frames we need to read
SInt64 numFrames = 0;
size = sizeof(numFrames);
err = ExtAudioFileGetProperty(audiofile, kExtAudioFileProperty_FileLengthFrames, &size, &numFrames);
// create the buffers for reading in data
AudioBufferList *bufferList = malloc(sizeof(AudioBufferList) + sizeof(AudioBuffer) * (clientFormat.mChannelsPerFrame - 1));
bufferList->mNumberBuffers = clientFormat.mChannelsPerFrame;
for (int ii=0; ii < bufferList->mNumberBuffers; ++ii) {
bufferList->mBuffers[ii].mDataByteSize = sizeof(float) * numFrames;
bufferList->mBuffers[ii].mNumberChannels = 1;
bufferList->mBuffers[ii].mData = malloc(bufferList->mBuffers[ii].mDataByteSize);
}
// read in the data
UInt32 rFrames = (UInt32)numFrames;
err = ExtAudioFileRead(audiofile, &rFrames, bufferList);
// close the file
err = ExtAudioFileDispose(audiofile);
// process the audio
for (int ii=0; ii < bufferList->mNumberBuffers; ++ii) {
float *fBuf = (float *)bufferList->mBuffers[ii].mData;
for (int jj=0; jj < rFrames; ++jj) {
*fBuf = *fBuf * ampScale;
fBuf++;
}
}
// open the file for writing
err = ExtAudioFileCreateWithURL((CFURLRef)theURL, fileType, &fileFormat, NULL, kAudioFileFlags_EraseFile, &audiofile);
// tell the ExtAudioFile API what format we'll be sending samples in
err = ExtAudioFileSetProperty(audiofile, kExtAudioFileProperty_ClientDataFormat, sizeof(clientFormat), &clientFormat);
// write the data
err = ExtAudioFileWrite(audiofile, rFrames, bufferList);
// close the file
ExtAudioFileDispose(audiofile);
// destroy the buffers
for (int ii=0; ii < bufferList->mNumberBuffers; ++ii) {
free(bufferList->mBuffers[ii].mData);
}
free(bufferList);
bufferList = NULL;
}
I think you should avoid working with 8 bits unsigned chars for audio, if you can.
Try to get the data as 16 bits or 32 bits, that would avoid some noise/bad quality issues.
For most common audio file formats there isn't a single master volume variable. Instead you will need to take (or convert to) the PCM sound samples and perform at least some minimal digital signal processing (multiply, saturate/limit/AGC, quantization noise shaping, and etc.) on each sample.
If the sound file is normalized, there's nothing you can do to make the file louder. Except in the case of poorly encoded audio, volume is almost entirely the realm of the playback engine.
http://en.wikipedia.org/wiki/Audio_bit_depth
Properly stored audio files will have peak volume at or near the maximum value available for the file's bit depth. If you attempt to 'decrease the volume' of a sound file, you'll essentially just be degrading the sound quality.
I am writing an iPhone app that records and plays audio simultaneously using the I/O audio unit as per Apple's recommendations.
I want to apply some sound effects (reverb, etc) on the recorded audio before playing it back. For these effects to work well, I need the samples to be floating point numbers, rather than integers. It seems this should be possible, by creating an AudioStreamBasicDescription with kAudioFormatFlagIsFloat set on mFormatFlags. This is what my code looks like:
AudioStreamBasicDescription streamDescription;
streamDescription.mSampleRate = 44100.0;
streamDescription.mFormatID = kAudioFormatLinearPCM;
streamDescription.mFormatFlags = kAudioFormatFlagIsFloat;
streamDescription.mBitsPerChannel = 32;
streamDescription.mBytesPerFrame = 4;
streamDescription.mBytesPerPacket = 4;
streamDescription.mChannelsPerFrame = 1;
streamDescription.mFramesPerPacket = 1;
streamDescription.mReserved = 0;
OSStatus status;
status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input, 0, &streamDescription, sizeof(streamDescription));
if (status != noErr)
fprintf(stderr, "AudioUnitSetProperty (kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input) returned status %ld\n", status);
status = AudioUnitSetProperty(audioUnit, kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output, 1, &streamDescription, sizeof(streamDescription));
if (status != noErr)
fprintf(stderr, "AudioUnitSetProperty (kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output) returned status %ld\n", status);
However, when I run this (on an iPhone 3GS running iPhoneOS 3.1.3), I get this:
AudioUnitSetProperty (kAudioUnitProperty_StreamFormat, kAudioUnitScope_Input) returned error -10868
AudioUnitSetProperty (kAudioUnitProperty_StreamFormat, kAudioUnitScope_Output) returned error -10868
(-10868 is the value of kAudioUnitErr_FormatNotSupported)
I didn't find anything of value in Apple's documentation, apart from a recommendation to stick to 16 bit little-endian integers. However, the aurioTouch example project contains at least some support code related to kAudioFormatFlagIsFloat.
So, is my stream description incorrect, or is kAudioFormatFlagIsFloat simply not supported on iPhoneOS?
It's not supported, as far as I know. You can pretty easily convert to floats, though using AudioConverter. I do this conversion (both ways) in real time to use the Accelerate framework with iOS audio. (note: this code is copied and pasted from more modular code, so there may be some minor typos)
First, you'll need the AudioStreamBasicDescription from the input. Say
AudioStreamBasicDescription aBasicDescription = {0};
aBasicDescription.mSampleRate = self.samplerate;
aBasicDescription.mFormatID = kAudioFormatLinearPCM;
aBasicDescription.mFormatFlags = kAudioFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
aBasicDescription.mFramesPerPacket = 1;
aBasicDescription.mChannelsPerFrame = 1;
aBasicDescription.mBitsPerChannel = 8 * sizeof(SInt16);
aBasicDescription.mBytesPerPacket = sizeof(SInt16) * aBasicDescription.mFramesPerPacket;
aBasicDescription.mBytesPerFrame = sizeof(SInt16) * aBasicDescription.mChannelsPerFrame
Then, generate a corresponding AudioStreamBasicDescription for float.
AudioStreamBasicDescription floatDesc = {0};
floatDesc.mFormatID = kAudioFormatLinearPCM;
floatDesc.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagIsPacked;
floatDesc.mBitsPerChannel = 8 * sizeof(float);
floatDesc.mFramesPerPacket = 1;
floatDesc.mChannelsPerFrame = 1;
floatDesc.mBytesPerPacket = sizeof(float) * floatDesc.mFramesPerPacket;
floatDesc.mBytesPerFrame = sizeof(float) * floatDesc.mChannelsPerFrame;
floatDesc.mSampleRate = [controller samplerate];
Make some buffers.
UInt32 intSize = inNumberFrames * sizeof(SInt16);
UInt32 floatSize = inNumberFrames * sizeof(float);
float *dataBuffer = (float *)calloc(numberOfAudioFramesIn, sizeof(float));
Then convert. (ioData is your AudioBufferList containing the int audio)
AudioConverterRef converter;
OSStatus err = noErr;
err = AudioConverterNew(&aBasicDescription, &floatDesct, &converter);
//check for error here in "real" code
err = AudioConverterConvertBuffer(converter, intSize, ioData->mBuffers[0].mData, &floatSize, dataBuffer);
//check for error here in "real" code
//do stuff to dataBuffer, which now contains floats
//convert the floats back by running the conversion the other way
I'm doing something unrelated to AudioUnits but I am using AudioStreamBasicDescription on iOS. I was able to use float samples by specifying:
dstFormat.mFormatFlags = kAudioFormatFlagIsFloat | kAudioFormatFlagIsNonInterleaved | kAudioFormatFlagsNativeEndian | kLinearPCMFormatFlagIsPacked;
The book Learning Core Audio: A Hands-on Guide to Audio Programming for Mac and iOS was helpful for this.
It is supported.
The problem is you must also set kAudioFormatFlagIsNonInterleaved on mFormatFlags. If you don't do this when setting kAudioFormatFlagIsFloat, you will get a format error.
So, you want to do something like this when preparing your AudioStreamBasicDescription:
streamDescription.mFormatFlags = kAudioFormatFlagIsFloat |
kAudioFormatFlagIsNonInterleaved;
As for why iOS requires this, I'm not sure - I only stumbled across it via trial and error.
From the Core Audio docs:
kAudioFormatFlagIsFloat
Set for floating point, clear for integer.
Available in iPhone OS 2.0 and later.
Declared in CoreAudioTypes.h.
I don't know enough about your stream to comment on its [in]correctness.
You can obtain an interleaved float RemoteIO with the following ASBD setup:
// STEREO_CHANNEL = 2, defaultSampleRate = 44100
AudioStreamBasicDescription const audioDescription = {
.mSampleRate = defaultSampleRate,
.mFormatID = kAudioFormatLinearPCM,
.mFormatFlags = kAudioFormatFlagIsFloat,
.mBytesPerPacket = STEREO_CHANNEL * sizeof(float),
.mFramesPerPacket = 1,
.mBytesPerFrame = STEREO_CHANNEL * sizeof(float),
.mChannelsPerFrame = STEREO_CHANNEL,
.mBitsPerChannel = 8 * sizeof(float),
.mReserved = 0
};
This worked for me.
has anyone been able to make ffmpeg work with audio queues, I get an error when I try to create the queue.
ret = avcodec_open(enc, codec);
if (ret < 0) {
NSLog(#"Error: Could not open video decoder: %d", ret);
av_close_input_file(avfContext);
return;
}
if (audio_index >= 0) {
AudioStreamBasicDescription
audioFormat;
audioFormat.mFormatID = -1;
audioFormat.mSampleRate =
avfContext->streams[audio_index]->codec->sample_rate;
audioFormat.mFormatFlags = 0;
switch (avfContext->streams[audio_index]->codec->codec_id)
{
case CODEC_ID_MP3:
audioFormat.mFormatID = kAudioFormatMPEGLayer3;
break;
case CODEC_ID_AAC:
audioFormat.mFormatID = kAudioFormatMPEG4AAC;
audioFormat.mFormatFlags = kMPEG4Object_AAC_Main;
break;
case CODEC_ID_AC3:
audioFormat.mFormatID = kAudioFormatAC3;
break;
default:
break;
}
if (audioFormat.mFormatID != -1) {
audioFormat.mBytesPerPacket = 0;
audioFormat.mFramesPerPacket =
avfContext->streams[audio_index]->codec->frame_size;
audioFormat.mBytesPerFrame = 0;
audioFormat.mChannelsPerFrame = avfContext->streams[audio_index]->codec->channels;
audioFormat.mBitsPerChannel = 0;
if (ret = AudioQueueNewOutput(&audioFormat, audioQueueOutputCallback, self, NULL, NULL, 0, &audioQueue)) {
NSLog(#"Error creating audio output queue: %d", ret);
}
The issues only with the audio,
Video is perfect if only I can figure out how to get audio queues to work.
http://web.me.com/cannonwc/Site/Photos_6.html
I though of remoteio but there is'nt much doc on that.
I will share the code for the complete class with anyone that helps me get it to work.
The idea is to have a single view controller that plays any streaming video passed to it, similar to ffplay on the iphone but without the sdl overhead.
You could be very well missing some important specifications in the AudioStreamBasicDescription structure: i don't know about ffmpeg, but specifying zero bytes per frame and zero bytes per packet won't work ;)
Here is how i would fill the structure, given the samplerate, the audio format, the number of channels and the bits per sample:
iAqc.mDataFormat.mSampleRate = iSampleRate;
iAqc.mDataFormat.mFormatID = kAudioFormatLinearPCM;
iAqc.mDataFormat.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger | kAudioFormatFlagIsPacked;
iAqc.mDataFormat.mBytesPerPacket = (iBitsPerSample >> 3) * iNumChannels;
iAqc.mDataFormat.mFramesPerPacket = 1;
iAqc.mDataFormat.mBytesPerFrame = (iBitsPerSample >> 3) * iNumChannels;
iAqc.mDataFormat.mChannelsPerFrame = iNumChannels;
iAqc.mDataFormat.mBitsPerChannel = iBitsPerSample;
I assume here you are writing PCM samples to the audio device.
As long as you know the audio format you are working with, there should be no problems adapting it: the important thing to remember is what all this stuff mean.
Here i'm working with one sample frame per packet, so the number of bytes per packet coincides with the number of bytes per sample frame.
Most of the problems come out because there is a lot of bad usage of words such as "samples", "sample frames" in the wrong contexts and so on: a sample frame can be thought as the atomic unit of audio data that embrace all the available channels, a sample refers to a single sub-unit of data composing the sample frame.
For example, you have an audio stream of 2 channels with a resolution of 16 bits per sample: a sample will be 2 bytes big (16bps/8 or 16 >> 3), the sample frame will also take the number of channels into account, so it will be 4 bytes big (2bytes x 2channels).
IMPORTANT
The theory behind this doesn't apply only to the iPhone, but to audio coding in general!
It just happens the AudioQueues ask you for well-defined specifications about your audio stream, and that's good, but you could be asked for bytes instead, so expressing audio data sizes as audio frames is always good, you can always convert your data sizes and be sure about it.