stxxl and overriding cxx_constructor - iphone

I'm trying to use stxxl as a backing store to a vector. If I do nothing it will automatically allocate 1 gig of disk space and everything works perfectly. However I don't need that much space allocated and, in fact, I need to gracefully scale from my optimal 512 megs down to a min of 128 megs of storage.
Unfortunately I have stxxl::vector defined in my objective-c class and on instantiation of that class the cxx_constructor function is called which starts up stxxl and allocated a gig whether i like it or not.
Is there any way I can override the cxx_constructor call and add my init in before it goes on to instantiate my class? I did try and create a simple object that would get instantiated by cxx_constructor. Unfortunately, however, for some unknown reason cxx_constructor calls my class's constructor twice.
Is the only option to add a static to that class preventing it from getting instantiated more than once? This is definitely an option just not very elegant. For one I'd love to know why it gets called twice.
Any info much appreciated!
Edit: Here is the code I wrote.
namespace stxxl
{
class Config
{
float GetFreeDiskspace()
{
float totalSpace = 0.0f;
float totalFreeSpace = 0.0f;
NSError *error = nil;
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSDictionary *dictionary = [[NSFileManager defaultManager] attributesOfFileSystemForPath:[paths lastObject] error: &error];
if (dictionary) {
NSNumber *fileSystemSizeInBytes = [dictionary objectForKey: NSFileSystemSize];
NSNumber *freeFileSystemSizeInBytes = [dictionary objectForKey:NSFileSystemFreeSize];
totalSpace = [fileSystemSizeInBytes floatValue];
totalFreeSpace = [freeFileSystemSizeInBytes floatValue];
NSLog(#"Memory Capacity of %f MiB with %f MiB Free memory available.", ((totalSpace/1024.0f)/1024.0f), ((totalFreeSpace/1024.0f)/1024.0f));
} else {
NSLog(#"Error Obtaining System Memory Info: Domain = %#, Code = %#", [error domain], [error code]);
}
return totalFreeSpace;
}
public:
Config()
{
char cacheFileName[256];
NSString* pTempDir = NSTemporaryDirectory();
strcpy( cacheFileName, [pTempDir UTF8String] );
strcat( cacheFileName, "/stxxlcache" );
const uint64_t kFreeSpace = (uint64_t)GetFreeDiskspace();
const uint64_t kMaxCacheSize = 512*1024*1024;
const uint64_t kCacheSize = (kFreeSpace > kMaxCacheSize) ? kMaxCacheSize : kFreeSpace;
const uint64_t kMinCacheSize = 2000 * ((1 << 15) >> 1) * sizeof( float );
stxxl::config* pCfg = stxxl::config::get_instance();
pCfg->init( cacheFileName, kCacheSize, false );
}
};
};
I then declare it inside my app delegate as follows:
stxxl::Config mCallOnceConfig;
Then when I run my application with a breakpoint inside the stxxl::Config constructor the breakpoint gets hit twice. I don't declare it anywhere else. cxx_constructor calls the constructor twice.
(It also worth noting that I've added my own stxxl::config::init function and blanked out the default one)
Edit 2: Placing a static bool around the constructor's code does sort out the double intialisation and everything works as I'd expect. Its a really rubbish solution though :(

Ok I've come up with a fairly elegant solution to my problem. I've dropped the whole class instance method and now I define my stxxl setup within the app delegates "+ (void) initialize" function. This ONLY gets called once.
Hopefully that will save some people some hassle if they suffer similar issues in the future :)

Related

loading file content to NSArray

I'm using this code to load content to NSArray and it seem to work fine however Instrument to detect leaks point that there is a problem that I can't put my finger on:
- (void) loadPlan: (NSString *) fName
{
short j1;
fName= [NSString stringWithFormat:#"/%#", fName];
[self NewCase];
NSArray *arrayPaths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *docDirectory = [arrayPaths objectAtIndex:0];
NSString *filePath = [docDirectory stringByAppendingString:fName];
BOOL fileExists = [[NSFileManager defaultManager] fileExistsAtPath:filePath];
if (!fileExists) return;
NSString *fileContents = [NSString stringWithContentsOfFile:filePath
encoding:NSUTF8StringEncoding error:nil];
NSArray *chunks = [fileContents componentsSeparatedByString: #"#"];
for (i = 0; i <= 100; i++)
{
InputV[i] = [[chunks objectAtIndex: i+5] doubleValue];
}
...
for (j1 = 0; j1 <= 10; j1++)
{
GroupMode[j1] = [[chunks objectAtIndex: 206+j1] retain];
}
...
}
and on a init method someplace i have:
for (j1 = 0; j1 <= 10; j1++)
{
GroupMode[j1] = [[NSString alloc] initWithFormat:#""];
}
Instrument points to the NSAraay *chunks line code but i'm not sure what's the issue. Do i need to release it at some point ?
I appreciate any help.
In the comments you mention being able to call release. Therefore you are not using ARC and since I noticed you tagged with iphone you are not using GC. This leaves manual memory management.
The problem seems to be that either the chunks array or some chunks themseves are overretained (or underreleased). You are not showing all the code, so it's hard to say.
Make sure you do not retain either of them somewhere else in the code you did not show. Maybe show us the rest of the loadPlan method implementation.
edit: Now that you added more code, I can also expand this answer.
Answer this question: where is the retain call to the chunks matched with a release?
Also what is the declaration of GroupMode? It seems to be just an array of pointers. If so you probably need to release the old value before setting the new one.
Let me try another answer based on what you posted.
I'm assuming GroupMode is an instance variable of some class and is declared like this:
NSString* GroupMode[11];
The second loop in loadPlan should be:
for (j1 = 0; j1 <= 10; j1++)
{
NSString* aChunk = [chunks objectAtIndex: 206+j1];
if ( GroupMode[j1] != aChunk ) {
[GroupMode[j1] release];
GroupMode[j1] = [aChunk retain];
}
}
You should do something similar every time you change an element of GroupMode and you should make sure you release all GroupMode held objects in the dealloc method of that class.
I however suggest you do not use plain arrays and instead switch to using NSArray and/or NSMutableArray.
Take a look at this answer:
"componentsSeparatedByString" Memory leak
The problem is probably that something using the results is over-retaining the stuff from chunks. Instruments is pointing at this line because it's where the memory was first allocated, but it may not be the source of your problem.

retainCount shows MaxInt

After trying to print retainCount of object I get 2147483647. Why do I get such a result? It should be 1, shouldn't?
NSString *myStr = [NSString new];
NSUInteger retCount = [myStr retainCount];
NSLog(#"refCount = %u", retCount);
2011-09-08 17:59:18.759 Test[51972:207] refCount = 2147483647
I use XCode Version 4.1. Tried compilers GCC 4.2 and LLVM GCC 4.2 - the same result.
Garbage Collection option was set to unsupported.
NSString is somewhat special when it comes to memory management. String literals (something like #"foo") are effectively singletons, every string with the same content is the same object because it can't be modified anyway. As [NSString new] always creates an empty string that cannot be modified, you'll always get the same instance which cannot be released (thus the high retainCount).
Try this snippet:
NSString *string1 = [NSString new];
NSString *string2 = [NSString new];
NSLog(#"Memory address of string1: %p", string1);
NSLog(#"Memory address of string2: %p", string2);
You'll see that both strings have the same memory address and are therefore the same object.
This doesn't directly answer your question, but retainCount is not really all that useful and should not be used for testing. See this SO post for details.
When to use -retainCount?
While NSString's are an odd case (there are others in the framework) you might also run across this in other clases - it's one of the ways of creating a singleton object.
A singleton only exists once in the app and it's pretty important that it never gets released! Therefore, it will overwrite some methods of NSObject including (but not limited to):
- (id)release {
// Ignore the release!
return self;
}
- (NSUInteger)retainCount {
// We are never going to be released
return UINT_MAX;
}
This object can never be released and tells the framework that it's a singleton by having a ludicrously high retain count.
Checkout this link for more information about singleton objects.
I've seen this a couple of times regarding NSStrings, the retainCount returns the maximum count instead of the actual one when you try to look at retainCounts of strings in this manner.
Try this;
NSString *myStr = [[NSString alloc] init];
NSUInteger retCount = [myStr retainCount];
NSLog(#"refCount = %u", retCount);
Edit: Restored NSUInteger

GameKit's sendDataToAllPeers isn't sending packets quick enough

I'm trying to send files over a bluetooth connection.
I've got this to work thanks to my previous post, but the method wasn't memory efficient.
The whole file was loaded into memory before it was sent, and this created problems (and crashed the app) for files > ~20 MB. So I've come up with a new method of only reading parts of the file I need at a specific time, creating packets from the data, sending them and repeating the process for each 8KB chunk of the file.
So I made a class method that generates the packet, informs the controller that a packet is available (through a protocol) and repeats this process for each packet that's available.
Here's the code for the packet generator:
+ (void)makePacketsFromFile:(NSString *)path withDelegate:(id <filePacketDelegate>)aDelegate {
if (![[NSFileManager defaultManager] fileExistsAtPath:path] || aDelegate == nil) return;
id <filePacketDelegate> delegate;
delegate = aDelegate;
const NSUInteger quanta = 8192;
uint filesize;
NSDictionary *fileAttributes = [[NSFileManager defaultManager] attributesOfItemAtPath:path error:nil];
filesize = [[fileAttributes objectForKey:NSFileSize] intValue];
int numOfPackets = (int)ceil(filesize/quanta);
if (numOfPackets == 0) numOfPackets = 1;
NSLog(#"filesize = %d, numOfPackets = %d or %.3f", filesize, numOfPackets, (float)ceil(filesize/quanta));
NSFileHandle *handle = [NSFileHandle fileHandleForReadingAtPath:path];
int offset = 0;
int counter = 0;
while (counter != (numOfPackets + 1)) {
uint len = (filesize < quanta) ? filesize : quanta;
if (counter == numOfPackets) {
len = filesize - offset;
}
[handle seekToFileOffset:offset];
NSData *fileData = [handle readDataOfLength:len];
file_packet *packet = [[file_packet alloc] initWithFileName:[path lastPathComponent] ofType:0 index:counter];
packet.packetContents = fileData;
[fileData release];
packet.checksum = #"<to be done>";
packet.numberOfPackets = [NSString stringWithFormat:#"%d", numOfPackets];
[delegate packetIsReadyForSending:packet];
[packet release];
offset += quanta;
counter++;
}
[handle closeFile];
}
And receiving and sending the file:
- (void)packetIsReadyForSending:(file_packet *)packet {
NSData *fileData = [packet dataForSending];
[self.connectionSession sendDataToAllPeers:fileData withDataMode:GKSendDataReliable error:nil];
}
- (void)sendFileViaBluetooth {
[file_packet makePacketsFromFile:selectedFilePath withDelegate:self];
}
However, the memory use is quite large. Not what I expected.
I'm a bit stuck on this, as I wouldn't like to restrict bluetooth sharing to files smaller than 20MB.
Any help appreciated.
Edit:
I've been thinking about this for a while and have come to the conclusion that it's not my code that's causing the memory allocation issue, it's GameKit's stack for sending the packets.
I think I'm generating too many packets too fast, and I don't think GameKit is sending them quick enough.
So I am now thinking about a way to see when GameKit has sent a packet, and only generating another one once GameKit confirms it was sent.
There’s one clear memory management issue in your code:
NSData *fileData = [handle readDataOfLength:len];
fileData wasn’t obtained via NARC (a method whose name contains new, alloc, retain, copy), so you don’t own it, hence you don’t release it.
[fileData release];
Oops.
As for the memory use, one thing to consider is that even though you release packet you can’t really tell what’s going on inside -[GKSession sendDataToAllPeers:withDataMode:error:]. It is possible that the method internally creates autoreleased objects that only end up being released after +makePacketsFromFile:withDelegate: has finished executing. I suggest you use a new autorelease pool in every loop iteration:
while (counter != (numOfPackets + 1)) {
NSAutoreleasePool *pool = [NSAutoreleasePool new];
uint len = (filesize < quanta) ? filesize : quanta;
…
counter++;
[pool drain];
}
By doing this, any autoreleased object inside that loop will be effectively released at the end of each loop iteration.
Checkout the NSInputStream. With it you can open a stream and only read out a buffer of bytes at a time.

Class initialization breaking application

I've recently created a new class for my iPhone application which will hold information read from a text file containing the street address and GPS points of points of interest.
The issue though is that whenever I add code to initialize the class my application loads up and the instantly quits with no errors in the console. When I remove it, everything is fine. I simply cannot see anything wrong with the code.
Here is the constructor:
#import "GPSCoordinate.h"
#implementation GPSCoordinate
-(GPSCoordinate*) initWithData:(NSString *)rawData size:(int)size
{
self = [super init];
location = [NSMutableArray arrayWithCapacity:size];
coordinates = [NSMutableArray arrayWithCapacity:(int)size];
NSArray *tokens = [rawData componentsSeparatedByString:#"#"];
for (int i = 0; i < size - 1; i++) {
//Sub tokens
NSString *line = [tokens objectAtIndex:i];
NSArray *lineTokens = [line componentsSeparatedByString:#":"];
//Store address
[location addObject:[lineTokens objectAtIndex:0]];
//Store GPS coords
NSString *coords = [lineTokens objectAtIndex:1];
coords = [[coords stringByReplacingCharactersInRange:NSMakeRange(0, 1) withString:#""]
stringByReplacingCharactersInRange:NSMakeRange([coords length]-2, 1) withString:#""];
NSArray *coordsTokens = [coords componentsSeparatedByString:#" "];
CLLocationCoordinate2D coord;
coord.latitude = [[coordsTokens objectAtIndex:0] doubleValue];
coord.longitude =[[coordsTokens objectAtIndex:1] doubleValue];
[coordinates addObject:coords];
[line release];
[lineTokens release];
[coords release];
[coordsTokens release];
}
return self;
}
#end
Here is the call I make to it in another class:
self.gps = [[GPSCoordinate alloc] initWithData:gpsRawData size:[[gpsRawData componentsSeparatedByString:#"#"] count]];
Where am I going wrong with this?
I see a number of problems.
You're not checking the return value of [super init].
You're storing autoreleased arrays in what are presumably ivars (location and coordinates).
You're passing a separate size parameter which is calculated from the rawData outside of the call, but -initWithData: makes the exact same calculation inside the method. The size: parameter seems completely superfluous here.
You're skipping the last token entirely. You should take that for loop and make the condition simply i < size. Alternately if you're targetting iOS 4.0 or above you can turn the entire loop into
[tokens enumerateObjectsUsingBlock:^(id obj, NSUInteger idx, BOOL *stop){
NSString *line = obj;
// rest of loop body
}];
Since you don't seem to need the index inside the loop, you could also just use a for-in loop (this will work on pre-4.0 iOS devices):
for (NSString *line in tokens) {
// body of loop
}
You're not checking that your data is valid. If a line contains "foo", your program will crash when it tries to access [lineTokens objectAtIndex:1]. Similarly it'll crash if you have the string "foo:" as it tries to remove the first character of the coordinates variable. In fact anything less than 2 characters after the colon will crash. It'll also crash if there's no spaces after the colon.
And finally, all those calls to -release at the end will crash. All 4 of those objects are autoreleased objects, so by calling -release on them now you're simply guaranteeing that the app will crash when the autorelease pool is drained.
You're also storing coords (e.g. the string) in your coordinates array. Presumably you meant to store coord, though you'll need to wrap it in an NSValue in order to store it in an NSArray.
I see several issues.
1) Most fundamentally, you are releasing a lot of objects that you didn't allocate. For example:
NSString *line = [tokens objectAtIndex:i];
....
[line release];
is incorrect. Review the Cocoa Memory Management Rules.
2) Why are you doing [[gpsRawData componentsSeparatedByString:#"#"] count to pass the size to
your initWithData:size: method, when you're just going to have to repeat the -componentsSeparatedByString: call inside your method. Passing a separate "size" doesn't gain you anything, involves a redundant parse of the input, and opens up more possible bugs (what if the caller passes in a "size" that doesn't match the number of "#"s in the input - you aren't handling that error condition).
3) I also see that you are assigning latitude/longitude to CLLocationCoordinate2D coord; but not doing anything with it. Is that deliberate?

Truncated Core Data NSData objects

I am saving arrays of doubles in an NSData* object that is persisted as a binary property in a Core Data (SQLite) data model. I am doing this to store sampled data for graphing in an iPhone app. Sometimes when there are more than 300 doubles in the binary object not all the doubles are getting saved to disk. When I quit and relaunch my app there may be as few as 25 data points that have persisted or as many as 300.
Using NSSQLitePragmasOption with synchronous = FULL and this may be making a difference. It is hard to tell, as bug is intermittent.
Given the warnings about performance problems as a result of using synchronous = FULL, I am seeking advice and pointers.
Thanks.
[[Edit: here is code.]]
The (as yet unrealized) intent of -addToCache: is to add each new datum to the cache but only flush (fault?) Data object periodically.
From Data.m
#dynamic dataSet; // NSData * attribute of Data entity
- (void) addDatum:(double_t)datum
{
DLog(#"-[Data addDatum:%f]", datum);
[self addToCache:datum];
}
- (void) addToCache:(double_t)datum
{
if (cache == nil)
{
cache = [NSMutableData dataWithData:[self dataSet]];
[cache retain];
}
[cache appendBytes:&datum length:sizeof(double_t)];
DLog(#"-[Data addToCache:%f] ... [cache length] = %d; cache = %p", datum, [cache length], cache);
[self flushCache];
}
- (void) wrapup
{
DLog(#"-[Data wrapup]");
[self flushCache];
[cache release];
cache = nil;
DLog(#"[self isFault] = %#", [self isFault] ? #"YES" : #"NO"); // [self isFault] is always NO.
}
- (void) flushCache
{
DLog(#"flushing cache to store");
[self setDataSet:cache];
DLog(#"-[Data flushCache:] [[self dataSet] length] = %d", [[self dataSet] length]);
}
- (double*) bytes
{
return (double*)[[self dataSet] bytes];
}
- (NSInteger) count
{
return [[self dataSet] length]/sizeof(double);
}
- (void) dump
{
ALog(#"Dump Data");
NSInteger numDataPoints = [self count];
double *data = (double*)[self bytes];
ALog(#"numDataPoints = %d", numDataPoints);
for (int i = 0; i
I was trying to get behavior as if my Core Data entity could have an NSMutableData attribute. To do this my NSManagedObject (called Data) had an NSData attribute and an NSMutableData ivar. My app takes sample data from a sensor and appends each data point to the data set - this is why I needed this design.
On each new data point was appended to the NSMutableData and then the NSData attribute was set to the NSMutableData.
I suspect that because the NSData pointer wasn't changing (though its content was), that Core Data did not appreciate the amount of change. Calling -hasChanged on the NSManagedObjectContext showed that there had been changes, and calling -updatedObjects even listed the Data object as having changed. But the actual data that was being written seems to have been truncated (sometimes).
To work around this I changed things slightly. New data points are still appended to NSMutableData but NSData attribute is only set when sampling is completed. This means that there is a chance that a crash might result in truncated data - but for the most part this work around seems to have solved the problem.
Caveat emptor: the bug was always intermittent, so it is possible that is still there - but just harder to manifest.