iOS issues having large uint8_t[ ] array - iphone

Actually I m receiving huge byte array as long as 35 mb in size. This byte array is bytes of a pdf document. I m receiving this from tcp socket input stream.
The code I m using is
+ (NSString *) recvToFile:(NSString *)_fileName: (long)_sz {
#try {
uint8_t t[_sz];
NSMutableData *data = nil;
NSMutableData *fileData = [[NSMutableData alloc] init];
long _pos = 0;
NSString *_fullPath = [Misc createTempFile:_fileName];
while (_sz > _pos) {
long _c = [m_sin read:t maxLength:_sz];
_pos += _c;
data = [NSData dataWithBytes:t length:_c];
if([Misc checkTempFileExists:_fileName]==nil)
[[NSFileManager defaultManager] createFileAtPath:_fullPath contents:nil attributes:nil];
[fileData appendData:data];
}
[fileData writeToFile:_fullPath atomically:YES];
NSDictionary *attr = [[NSFileManager defaultManager] attributesOfItemAtPath:_fullPath error:nil];
NSLog(#"ATTR - %#",attr);
long long length = [[attr valueForKey:#"NSFileSize"] intValue];
if (length >= _sz)
return (_fullPath);
}
#catch (NSException * e) {
}
return (nil);
}
The problem is that the app crashes giving "EXC_BAD_ACCESS(code=2 address=0x2461ae4)" error at first line that is
+(NSString *) recvToFile:(NSString *)_fileName: (long)_sz
When i explicitly declare the size of uint8_t array to 800000 as uint8_t[800000];
then it works in my iOS simulator but not on my iPad
Please help. Thanks

You are most likely running out of stack space. Even if you allocated the memory on the heap, 35MB is quite a lot for iOS devices. You should write the data out to a file rather than an in-memory buffer.
UPDATE:
So it looks like this code can already chunk the data and write it to the file in small pieces. All you have to do is change from this:
uint8_t t[_sz];
To something like this:
uint8_t t[CHUNK_SIZE];
where CHUNK_SIZE can be something reasonable like a few KB.

This line looks wrong to me:
(NSString *) recvToFile:(NSString *)_fileName: (long)_sz {
You should have something like:
+ (ReturnType *) methodname:(Type) _var1 secondArg:(type) _var2
So maybe you want:
+ (NSString *) recvToFile:(NSString *)_fileName size:(long)_sz
I think the reason you're getting a crash is that you're trying to read a variable called _sz. By changing this to a fixed value you're not reading it here, and by some miracle the rest of your code is running.
Looking at your code I think you might need a refresher on some of basics of objective C syntax. Maybe you're more familiar with another language? I'm told by my iOS colleagues that try and catch are used rarely in objective C, and the way you've used them could create various problems.

Related

Objective C read file wrong encoding

Hi all I have a problem when I download file from internet from which I need to mine some data. I open it and try to buffer it, but it gives me wrong chars because this file is in Czech...
My code:
- (void) sync {
NSString * path = #"/Users/syky/Documents/stats.csv";
NSFileHandle * fileHandle = [NSFileHandle fileHandleForReadingAtPath:path];
NSData * buffer = nil;
while ((buffer = [fileHandle readDataOfLength:1024])) {
//do something with the buffer
NSString * s = [[NSString alloc]initWithData:buffer encoding:nil];
NSLog(s);
break;
}
No matter which encoding I choose I always get broken chars such as
"Poø.";"Jméno"
I need to get:
"Příjmení";"Jméno"
This file is originaly generated by Microsoft Excel such as *.csv export file...
When I try to open this file by any MAC OS X Text editor I get broken chars as well, but when I open it on other Windows based maschine with Microsoft Excel it works just fine...
Thank you for your help
Solution:
- (void) sync {
NSString * path = #"/Users/syky/Documents/stats.csv";
NSFileHandle * fileHandle = [NSFileHandle fileHandleForReadingAtPath:path];
NSData * buffer = nil;
while ((buffer = [fileHandle readDataOfLength:1024])) {
NSStringEncoding encoding = CFStringConvertEncodingToNSStringEncoding(kCFStringEncodingWindowsLatin2);
NSString *string = [[NSString alloc] initWithData:buffer encoding:encoding];
NSLog(string);
break;
}
First, I'm not a Czech speaker. Second, I think "use UTF-8" is akin to saying "throw a barrel at it." It's heavy-handed in the same way.
From what I've researched, you could use ISO Latin 2 or Apple's Central European Roman encoding. You'll find the former represented among NSStringEncodings, but not the latter, so look to Core Foundation's support:
NSStringEncoding encoding = CFStringConvertEncodingToNSStringEncoding(kCFStringEncodingMacCentralEurRoman);
NSString *string = [[NSString alloc] initWithData:buffer encoding:encoding];
Otherwise, you could (and probably already have, from what you've said) use:
NSString *string = [[NSString alloc] initWithData:buffer encoding:NSISOLatin2StringEncoding];
I'm really curious to see if using CFStringEncoding encodings improves your situation.
EDIT:
If your source was generated by Microsoft Excel, perhaps kCFStringEncodingWindowsLatin2 will work instead of kCFStringEncodingMacCentralEurRoman. Like before, you'll need to convert it using CFStringConvertEncodingToNSStringEncoding.
There's one other approach you might want to try. Since CFStringRef is "toll-bridged" to NSString (and so is CFDataRef to NSData), perhaps working entirely in Core Foundation might work:
CFStringRef stringRef = CFStringCreateFromExternalRepresentation(kCFAllocatorDefault, (CFDataRef)buffer, kCFStringEncodingMacCentralEurRoman);
NSString *string = (NSString *)stringRef;
In this case, don't forget that stringRef has to be released.
Good luck to you in your endeavors.

reading local webarchive files -occasionally- returns null WebResourceData

Aloha,
I've come across a problem in iOS 6.1.3 reading webarchive files where -occasionally- the WebResourceData returns a null.
These are known good files (created in TextEdit) stored inside the bundle, and usually read fine. It's just that every so often, they don't.
In a simple test shown below, I read 3 different files over and over until I spot an error. For iOS 6.1.3, I hit an error between 1 and 200 iterations, every time I run the test. I've run this on various devices and the simulator with the same results.
// trying to find out why reading webarchives occasionally
// returns with empty WebResourceData from known good files.
//
- (BOOL) testMe {
NSMutableDictionary *plist = [[NSMutableDictionary alloc] initWithCapacity:100] ;
int iteration = 1;
BOOL ok = TRUE;
// keep going until we have an error
while (ok) {
NSArray *fileNames = #[#"file1",
#"file2",
#"file3" ];
// LOOP through the webArchives...
//
for (NSString *webarchiveName in fileNames) {
// get the webarchive template
//
NSURL *fileURL = [[NSBundle mainBundle] URLForResource:webarchiveName withExtension:#"webarchive"];
//
NSData *plistData = [NSData dataWithContentsOfURL:fileURL];
NSString *error;
NSPropertyListFormat format;
//
plist = (NSMutableDictionary *)[NSPropertyListSerialization propertyListFromData:plistData
mutabilityOption:NSPropertyListMutableContainersAndLeaves
format:&format
errorDescription:&error];
// check to see if it loaded
//
//
if(!plist){
NSLog(#"ERROR: did not load %#", webarchiveName);
}else{
NSData *foundWebResourceData = [[plist objectForKey:#"WebMainResource"] objectForKey:#"WebResourceData"];
NSString *foundHTML = [NSString stringWithUTF8String:[foundWebResourceData bytes]];
if (foundHTML == NULL) {
NSLog(#" %# descr = %#", webarchiveName, foundHTML);
[errorOutlet setText:[NSString stringWithFormat:#"%# returned with no content (null) in WebResourceData", webarchiveName]];
ok = FALSE;
}
} //---- end of if plist exists
} // loop through all 3 files
[countOutlet setText:[NSString stringWithFormat:#"%d", iteration]];
++ iteration;
} // keep looping until error
return ok;
} // end of testMe
The error shows up in these two lines:
NSData *foundWebResourceData = [[plist objectForKey:#"WebMainResource"] objectForKey:#"WebResourceData"];
NSString *foundHTML = [NSString stringWithUTF8String:[foundWebResourceData bytes]];
but it's not consistent. I've isolated the code by making a new xcode project where this is the only activity, other than displaying the iteration count and a retest button. The files always load, and always have a WebMainResource with a WebResourceData key.
A possible clue is that if I instead insert the code into ViewDidLoad, it runs for many more iterations but still finds a null. Calling [self testMe] from a button action hits an error much faster...not sure why.
I'm a bit at a loss, and hoping that it's not an iOS bug, but rather something basic I'm just missing. Any help would be appreciated.
You might try using the NSString initializer designed for reading NSData:
NSString *foundHTML = [[NSString alloc] initWithData:foundWebResourceData encoding:NSUTF8StringEncoding];

Uploading Large NSData to the Web

I'm currently working on an application that has to upload large files (mainly movies/videos) to the web. After reading what I can, I went the the approach of converting the movie to NSData and then including that as the NSURLConnection's HTTPBody. However, upon converting the movie (which was originally an ALAsset) into NSData, I receive a memory warning and then a subsequent crash.
I have no idea how I would go about uploading these types of large files, if that data just causes an instant crash. One solution that I was thinking of is writing to the filesystem and then uploading a file directly from there, but I have not been able to find any information on how one would accomplish this.
Here is the relevant code that I use. If there is something that I'm doing wrong right here, I'd love to know.
ALAssetRepresentation *representation = [asset defaultRepresentation];
Byte *buffer = (Byte *)malloc([representation size]);
NSUInteger buffered = [representation getBytes:buffer fromOffset:0.0 length:[representation size] error:nil];
uploadData = [NSData dataWithBytes:buffer length:buffered];
free(buffer);
Assuming that it makes sense to upload the movie in its native format, you can really make this easier using the BSD (ie Unix) section 3 interface:
given a filePath, open the file and get an int file descriptor (fd)
with fd, get the length of the file
keep track of how much you've loaded so you know where to get more data
use mmap(3) to map in JUST the data you want to upload at any time, and use the void * pointer returned by mmap as the location of the data
when the data has been sent, munmap the old data chunk and mmap a new chunk
after all data is sent, munmap the last chunk, the close(fd).
No temporary memory - no mallocs. I use mmap whenever I have to deal with huge files.
Edit: you can also use NSData dataWithContentsOfFile:options with options set to use mmap. You would then use the byte pointer to read small chunks as you need them.
In case anyone got here and couldn't solve your problems, I figured out a way to do this.
You have to firstly write your ALAssetRepresentation to disk (as described here):
NSUInteger chunkSize = 100 * 1024;
NSString *tempFile = [NSTemporaryDirectory() stringByAppendingPathComponent:#"temp.tmp"];
uint8_t *chunkBuffer = malloc(chunkSize * sizeof(uint8_t));
NSUInteger length = [rep size];
NSFileHandle *fileHandle = [[NSFileHandle fileHandleForWritingAtPath: tempFile] retain];
if(fileHandle == nil) {
[[NSFileManager defaultManager] createFileAtPath:tempFile contents:nil attributes:nil];
fileHandle = [[NSFileHandle fileHandleForWritingAtPath:tempFile] retain];
}
NSUInteger offset = 0;
do {
NSUInteger bytesCopied = [rep getBytes:chunkBuffer fromOffset:offset length:chunkSize error:nil];
offset += bytesCopied;
NSData *data = [[NSData alloc] initWithBytes:chunkBuffer length:bytesCopied];
[fileHandle writeData:data];
[data release];
} while (offset < length);
[fileHandle closeFile];
[fileHandle release];
free(chunkBuffer);
chunkBuffer = NULL;
Then you have to create an NSData object that can map the disk without using memory resources (kind of like David's answer, but inspired by this answer):
NSError *error;
NSData *fileData = [NSData dataWithContentsOfFile:tempFile options:NSDataReadingMappedIfSafe error:&error];
if (!fileData) {
NSLog(#"Error %# %#", error, [error description]);
NSLog(#"%#", tempFile);
//do what you need with the error
}
EDIT Although, if you are uploading the file somewhere, you should open a connection and send small buffers of the file, kind of like what I did above. I had to write a C++ class to handle the socket and the connection
You probably shouldn't be trying to read the whole asset in one shot:
Byte *buffer = (Byte *)malloc([representation size]);
NSUInteger buffered = [representation getBytes:buffer fromOffset:0.0 length:[representation size] error:nil];
Instead, set up a loop and read from the asset in chunks. I've outlined the basic approach. You'll need to fill in a few gaps, but it should solve the memory issue.
You might also want to consider running this in a thread so you don't lock up the UI.
NSError error;
int bufferSize = 1000;
float offset=0.0;
//TODO: Open Connection
while (1)
{
Byte *buffer = (Byte *)malloc(bufferSize);
NSUInteger buffered = [representation getBytes:buffer fromOffset:offset length:bufferSize error:&error];
//TODO: Write data
//TODO: Increment offset, check errors
free(buffer);
//if (done){
//break;
//}
}
//TODO close eonnection

size of NSCachesDirectory in iphone

how do i get size of folder NSCachesDirectory i.e /Library/Cache. i want to know size of this folder so that i can eventually clear this.
thanks.
Edit: here is my code.
NSDictionary *attributes = [[NSFileManager defaultManager] attributesOfItemAtPath:folderPath error:&error];
if (attributes != nil) {
if (fileSize = [attributes objectForKey:NSFileSize]) {
NSLog(#"size of :%# = %qi\n",folderPath, [fileSize unsignedLongLongValue]);
}
}
when i run this it gives my file size 768(dont know bytes or KB) and i check in finder it shows me folder size 168KB. i dont know whats wrong.
Something like the following should help get you started:
- (unsigned long long int) cacheFolderSize {
NSFileManager *_manager = [NSFileManager defaultManager];
NSArray *_cachePaths = NSSearchPathForDirectoriesInDomains(NSCachesDirectory, NSUserDomainMask, YES);
NSString *_cacheDirectory = [_cachePaths objectAtIndex:0];
NSArray *_cacheFileList;
NSEnumerator *_cacheEnumerator;
NSString *_cacheFilePath;
unsigned long long int _cacheFolderSize = 0;
_cacheFileList = [_manager subpathsAtPath:_cacheDirectory];
_cacheEnumerator = [_cacheFileList objectEnumerator];
while (_cacheFilePath = [_cacheEnumerator nextObject]) {
NSDictionary *_cacheFileAttributes = [_manager fileAttributesAtPath:[_cacheDirectory stringByAppendingPathComponent:_cacheFilePath] traverseLink:YES];
_cacheFolderSize += [_cacheFileAttributes fileSize];
}
return _cacheFolderSize;
}
EDIT
The value returned will be in bytes: cf. http://developer.apple.com/mac/library/documentation/Cocoa/Reference/Foundation/Classes/NSFileManager_Class/Reference/Reference.html#//apple_ref/doc/c_ref/NSFileSize
Assuming you are running this in the Simulator, Finder is probably reporting usage of file blocks for those bytes. Those blocks will necessarily be larger than the file data itself. Read up on the HFS+ system to learn about blocks: http://en.wikipedia.org/wiki/HFS_Plus
I'm not sure what file system is used on the iPhone, or what the file block size will be on the device, so while the byte total will be the same, the actual disk usage may be different between Simulator and device.
Do you really mean /Library/Cache, or do you mean ~/Library/Cache (the application's cache directory). You generally have no control over the former, so I'll assume you mean the latter.
Use NSFileManager's -enumeratorAtPath: to walk the directory and use -attributesOfItemAtPath:error: to fetch the fileSize. I recommend doing this slowly on a background thread to avoid blocking your app.

On iPhone, how can I use fileHandle to download a mp3 file on the fly?

I am trying to implement the AudioFileStreamSeek feature on my streaming app. But there is no way I can get this running. Even Matt Gallagher said on his blog:
Icidentally, the AudioFileStreamSeek function seems completely broken. If you can't get it to work (as I couldn't) just seek to a new point in the file, set discontinuous to true and let AudioFileStream deal with it.
My code kindda looks like this but I can't get it to work:
NSString *path = [[NSString alloc] initWithContentsOfURL: url];
NSLog(#"url = %#", path);
SInt64 currentOffset;
UInt32 flags = 0;
OSStatus status = AudioFileStreamSeek( audioFileStream, 150, &currentOffset, &flags );
NSLog(#"Setting next byte offset to: %qi, flags: %d", (long long)currentOffset, flags);
NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingAtPath: path];
// then read data from the new offset set by AudioFileStreamSeek
[fileHandle seekToFileOffset:currentOffset];
NSData * data = [fileHandle readDataOfLength:4096];
NSLog(#"data length %d, bytes %d", [data length], [data bytes]);
if (discontinuous)
{
err = AudioFileStreamParseBytes(audioFileStream, length, bytes, kAudioFileStreamParseFlag_Discontinuity);
if (err)
{
[self failWithErrorCode:AS_FILE_STREAM_PARSE_BYTES_FAILED];
return;
}
}
else
{
err = AudioFileStreamParseBytes(audioFileStream, length, bytes, 0);
if (err)
{
[self failWithErrorCode:AS_FILE_STREAM_PARSE_BYTES_FAILED];
return;
}
}
Please help...
Lacking any other solutions, what you can do is create an NSConnection and then as you receive the NSData you can effectively create streaming by processing each new chunk of NSData that you receive to your NSConnectionDelegate. NSConnection will make sure to send it to you in order, so you won't have to worry about getting it ordered correctly. Note though that, depending on your application, you may need to do this outside the main application thread, so that the user can still work with your app even if the download stalls and you have to rebuffer.