I am playing around with Dave DeLong's excellent CHCSVParser for Objective-C with an extremely long .CSV file and am running into some trouble using it. I would use the arrayWithContentsOfCSVFile method, but I'm running the code on an iPhone and parsing the whole file into memory would take more memory than is available.
In my code below, the parser opens the document and calls the delegate methods perfectly, but where in the delegate do I stop after each line and access the data (to create and save a Core Data object to the data store)? I assume that would be in - (void) parser:(CHCSVParser *)parser didEndLine:(NSUInteger)lineNumber, but how do I get an NSArray (or whatever) of the data from the parser when it's done with the line?
Here is my code so far:
//
// The code from a method in my view controller:
//
NSArray *paths = NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES);
NSString *documentsDirectory = [paths objectAtIndex:0];
NSFileManager *manager = [NSFileManager defaultManager];
NSError *err = nil;
NSArray *fileList = [manager contentsOfDirectoryAtPath:documentsDirectory error:&err];
NSString *fileName = [fileList objectAtIndex:1];
NSURL *inputFileURL = [NSURL fileURLWithPath: [documentsDirectory stringByAppendingPathComponent:fileName]];
NSStringEncoding encoding = 0;
CHCSVParser *p = [[CHCSVParser alloc] initWithContentsOfCSVFile:[inputFileURL path] usedEncoding:&encoding error:nil];
[p setParserDelegate:self];
[p parse];
[p release];
...
#pragma mark -
#pragma mark CHCSVParserDelegate methods
- (void) parser:(CHCSVParser *)parser didStartDocument:(NSString *)csvFile {
NSLog(#"Parser started!");
}
- (void) parser:(CHCSVParser *)parser didStartLine:(NSUInteger)lineNumber {
//NSLog(#"Parser started line: %i", lineNumber);
}
- (void) parser:(CHCSVParser *)parser didEndLine:(NSUInteger)lineNumber {
NSLog(#"Parser ended line: %i", lineNumber);
}
- (void) parser:(CHCSVParser *)parser didReadField:(NSString *)field {
//NSLog(#"Parser didReadField: %#", field);
}
- (void) parser:(CHCSVParser *)parser didEndDocument:(NSString *)csvFile {
NSLog(#"Parser ended document: %#", csvFile);
}
- (void) parser:(CHCSVParser *)parser didFailWithError:(NSError *)error {
NSLog(#"Parser failed with error: %# %#", [error localizedDescription], [error userInfo]);
}
Thanks!
I'm glad to see that my code is proving useful! :)
CHCSVParser is similar in behavior to an NSXMLParser, in that every time it finds something interesting, it's going to let you know via one of the delegate callbacks. However, if you choose to ignore the data that it gives you in the callback, then it's gone. These parsers (CHCSVParser and NSXMLParser) are pretty stupid. They just know the format of the stuff they're trying to parse, but don't really do much beyond that.
So the answer, in a nutshell, is "you have to save it yourself". If you look at the code for the NSArray category, you'll see in the .m file that it's using a simple NSObject subclass as the parser delegate, and that subclass is what's aggregating the fields into an array, and then adding that array to the overall array. You'll need to do something similar.
Example delegate:
#interface CSVParserDelegate : NSObject <CHCSVParserDelegate> {
NSMutableArray * currentRow;
}
#end
#implementation CSVParserDelegate
- (void) parser:(CHCSVParser *)parser didStartLine:(NSUInteger)lineNumber {
currentRow = [[NSMutableArray alloc] init];
}
- (void) parser:(CHCSVParser *)parser didReadField:(NSString *)field {
[currentRow addObject:field];
}
- (void) parser:(CHCSVParser *)parser didEndLine:(NSUInteger)lineNumber {
NSLog(#"finished line! %#", currentRow);
[self doSomethingWithLine:currentRow];
[currentRow release], currentRow = nil;
}
#end
However, I could be convinced to modify the behavior of the parser to aggregate the row itself, but if I go down that route, why not just have the parser aggregate the entire file? (Answer: it shouldn't)
I tried using this today, based on #DaveDeLong's excellent answer and code, but I think the software has been revised since his (2010) answer. At the time of writing, I found I had to use this:
#interface CSVParserDelegate : NSObject <CHCSVParserDelegate> {
NSMutableArray * currentRow;
}
#end
#implementation CSVParserDelegate
- (void) parser:(CHCSVParser *)parser didBeginLine:(NSUInteger)lineNumber {
currentRow = [[NSMutableArray alloc] init];
}
- (void) parser:(CHCSVParser *)parser didReadField:(NSString *)field atIndex:(NSInteger)fieldIndex {
[currentRow addObject:field];
}
- (void) parser:(CHCSVParser *)parser didEndLine:(NSUInteger)lineNumber {
NSLog(#"finished line! %#", currentRow);
[self doSomethingWithLine:currentRow];
[currentRow release], currentRow = nil;
}
#end
i.e., parser:didStartLine:lineNumber: has become parser:didBeginLine:lineNumber: and parser:didReadField: has become parser:didReadField:atIndex:.
To use CHCSVParser with Swift you can use a swift wrapper for basic needs
Related
I need to download and parse 40mb of json. Right now I'm using AFJSONRequestOperation, which on older devices causes memory warning and crash. How it should be done? I think the only way to do it correctly, is to stream json but I've got no idea how to do it or which library will be best. Please provide examples. Thanks a lot!
For anybody who has the same problem, here's how I solved it:
1. Download JSON file to local storage using AFHTTPRequestOperation's output stream.
2. Parse little chunks of NSData using YAJLParser.
Result: I was testing it on 50mb json on iPad (1), without any memory warnings (memory around 10mb).
Example:
NSError *error = nil;
NSData *data = [NSData dataWithContentsOfFile:path
options:NSDataReadingMappedAlways | NSDataReadingUncached
error:&error];
YAJLParser *parser = [[YAJLParser alloc] initWithParserOptions:YAJLParserOptionsAllowComments];
parser.delegate = self;
[parser parse:data];
parser.delegate = nil;
parser = nil;
YAJLParser delegate:
// first declare in header file NSMutableArray *stack and NSString *mapKey
- (void)parserDidStartDictionary:(YAJLParser *)parser
{
NSString *dictName = mapKey;
if (mapKey == nil)
{
dictName = (stack.count == 0) ? #"" : [stack lastObject];
}
[stack addObject:(dictName)];
}
- (void)parserDidEndDictionary:(YAJLParser *)parser
{
mapKey = nil;
[stack removeLastObject];
}
- (void)parserDidStartArray:(YAJLParser *)parser
{
NSString *arrayName = mapKey;
if (mapKey == nil)
{
arrayName = stack.count == 0 ? #"" : [stack lastObject];
}
[stack addObject:(arrayName)];
if([mapKey isEqualToString:#"something"])
{
// do something
}
}
- (void)parserDidEndArray:(YAJLParser *)parser
{
if([mapKey isEqualToString:#"some1"])
{
// do something
}
mapKey = nil;
[stack removeLastObject];
}
- (void)parser:(YAJLParser *)parser didMapKey:(NSString *)key
{
mapKey = key;
}
- (void)parser:(YAJLParser *)parser didAdd:(id)value
{
if([mapKey isEqualToString:#"id"])
{
// do something
}
}
Write your data to a file, then use NSData's dataWithContentsOfFile:options:error: and specify the NSDataReadingMappedAlways and NSDataReadingUncached flags. This will tell the system to use mmap() to reduce the memory footprint, and not to burden the file system cache with blocks of memory (that makes it slower, but much less of a burden to iOS).
You can find the answer here
iPad - Parsing an extremely huge json - File (between 50 and 100 mb)
According to this post I can use the encryption/decryption methods to store/retrieve plist file securely.
But the problem is:
Q: After I have decrypted the plist file, how can I parse and store the plist file as NSDictrionary object
Probably NSPropertyListSerialization is what you are looking for.
As seen in this Post:
Plist Array to NSDictionary
You could use core foundation approach here with the method CFPropertyListCreateFromXMLData
If the plist represents the NSDictionary content, the following check should be passed:
if ([(id)plist isKindOfClass:[NSDictionary class]])
and the plist object might be safely casted to NSDictionary. If no, something is wrong with the data or decription process.
The easiest way would be creating a category for NSDictionary like this:
NSDictionaryWithData.h:
#interface NSDictionary (NSDictionaryWithData)
+ (id)dictionaryWithData:(NSData *)data;
- (id)initWithData:(NSData *)data;
#end
NSDictionaryWithData.m:
#implementation NSDictionary (NSDictionaryWithData)
+ (id)dictionaryWithData:(NSData *)data
{
return [[[NSDictionary alloc] initWithData:data] autorelease];
}
- (id)initWithData:(NSData *)data
{
self = (NSDictionary *)[NSPropertyListSerialization propertyListFromData:data
mutabilityOption:NSPropertyListImmutable
format:NULL
errorDescription:nil];
return [self retain];
}
#end
Usage:
NSDictionary* myDict = [[NSDictionary alloc] initWithData:decryptedData];
One of the possible reasons dictionaryWithData doesn't exist is a property list is not necessarily a dictionary at the root level. It could equally be an NSArray.
Here is my take on a solution: a category that utilises NSPropertyListSerialization
Features
Silently discards data that contains arrays at the root level.
Checks which method to use ( propertyListFromData:mutabilityOption:format:errorDescription: is depreciated )
NSMutableDictionary also supported
Note - this takes an unorthodox approach of wrapping a class factory method with an init method. This is for efficiency - most of the time you will be using the factory method, which just wraps NSPropertyListSerialization, which internally invokes alloc/init/autorelease to return an appropriate object.
NSDictionary+DictionaryWithData.h
#import <Foundation/Foundation.h>
#interface NSDictionary (DictionaryWithData)
+ (id)dictionaryWithData:(NSData *)data;
- (id)initWithData:(NSData *)data;
#end
NSDictionary+DictionaryWithData.m
#import "NSDictionary+DictionaryWithData.h"
#implementation NSDictionary (DictionaryWithData)
+(NSPropertyListMutabilityOptions) mutabilityOption {
return NSPropertyListImmutable;
}
+ (id)dictionaryWithData:(NSData *)data
{
static BOOL methodChecked = NO;
static BOOL use_propertyListWithData = NO;
if (!methodChecked) {
SEL sel = #selector(propertyListWithData:options:format:error:);
use_propertyListWithData = [[NSPropertyListSerialization class]
respondsToSelector:sel];
methodChecked = YES;
}
id result;
if (use_propertyListWithData) {
result = [NSPropertyListSerialization propertyListWithData:data
options:[self mutabilityOption]
format:nil
error:nil];
} else {
result = [NSPropertyListSerialization propertyListFromData:data
mutabilityOption:[self mutabilityOption]
format:NULL
errorDescription:nil];
}
return [result isKindOfClass:[NSDictionary class]] ? result : nil;
}
- (id)initWithData:(NSData *)data
{
id result = [[self class] dictionaryWithData:data];
self = result ? [self initWithDictionary:result ] : nil;
return self;
}
#end
#implementation NSMutableDictionary (DictionaryWithData)
+(NSPropertyListMutabilityOptions) mutabilityOption {
return NSPropertyListMutableContainersAndLeaves;
}
#end
All,
Disclaimer: I'm aware that there are other questions that ask similar questions, but none provide an answer that I understand or an answer that applies to my situation.
I have two classes, classA and classB. In classA there is a void instance method that creates a NSMutableDictionary (if you want specifics, it's a XMLparser). After classA's XMLParser is run and the NSMutableDictionary has been created and filled, classB is called which needs to do some other things with that dictionary. For some reason, classB cannot access the NSMutableDictionary in classA (actually, it can access it, but for some reason it shows up as "NULL"). What should I do?
Thanks in advance.
EDIT: You asked for the source code, you got it. ClassA below, dictionary in question is called "response."
#import "XMLParser.h"
#import "CardSetupViewController.h"
#implementation XMLParser
#synthesize response;
- (XMLParser *) initXMLParser
{
self = [super init];
// init dictionary of response data
response = [[NSMutableDictionary alloc] init];
return self;
}
//Gets Start Element of SessionData
- (void)parser:(NSXMLParser *)parser
didStartElement:(NSString *)elementName
namespaceURI:(NSString *)namespaceURI
qualifiedName:(NSString *)qualifiedName
attributes:(NSDictionary *)attributeDict
{
if ([elementName isEqualToString:#"SessionData"])
{
NSLog(#"Found SessionData in the return XML! Continuing...");
//response is a NSMutableArray instance variable
//THIS SHOULD NEVER NEED TO BE USED
if (!response)//if array is empty, it makes it!
{
NSLog(#"Dictionary is empty for some reason, creating...");
response = [[NSMutableDictionary alloc] init];
}
//END: THIS SHOULD NEVER BE USED
return;
}
else
{
currentElementName = elementName;
NSLog(#"Current Element Name = %#", currentElementName);
return;
}
}
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string
{
if (!currentElementValue) {
// init the ad hoc string with the value
currentElementValue = [[NSMutableString alloc] initWithString:string];
} else {
[currentElementValue setString:string];
NSLog(#"Processing value for : %#", string);
}
}
//Gets End Element of SessionData
- (void)parser:(NSXMLParser *)parser
didEndElement:(NSString *)elementName
namespaceURI:(NSString *)namespaceURI
qualifiedName:(NSString *)qName {
if ([elementName isEqualToString:#"SessionData"])
{
// We reached the end of the XML document
// dumps dictionary into log
NSLog(#"Dump:%#", [response description]);
return;
}
else
{
//Adds key and object to dictionary
[response setObject:currentElementValue forKey:currentElementName];
NSLog(#"Set values, going around again... brb.");
}
currentElementValue = nil;
currentElementName = nil;
}
#end
You probably want to look into using singletons.
http://www.galloway.me.uk/tutorials/singleton-classes/
http://pixeleap.com/?p=19
http://www.iphonedevsdk.com/forum/iphone-sdk-development/5302-how-make-global-variables-objectivec.html
Those should get you started, if you need clarification just ask.
Are currentElementName and currentElementValue ivars ? If so, every time you catch either an element or a value, you must then release your ivar and retain the new value you catch.
Also, as the comments are saying you didn't follow any rules about writing a correct initializer.
What about classB ? Isn't there some logic that makes your dictionary be released ?
Try managing a bit more the memory.
you have several options... so many options actually
Option A:(set)
#interface ClassB{
NSDictionary * someDict;
}
-(void)setSomeDict:(NSDictionary *)aDict;
#end
#implementation ClassB
-(void)setSomeDict:(NSDictionary *)aDict
{
someDict = [aDict retain];//or copy depending on your needs;
}
#end
somewhere in ClassA...
ClassB * b = [ClassB new];
[b setSomeDict: someOtherDict];
Option B:(push)
#interface ClassB{
NSDictionary * someDict;
}
-(void)doSomethingWithDict:(NSDictionary *)aDict;
#end
#implementation ClassB
-(void)doSomethingWithDict:(NSDictionary *)aDict
{
NSLog(#"did something with aDict: %#",aDict);
}
#end
somewhere in ClassA...
ClassB * b = [ClassB new];
[b doSomethingWithDict: someOtherDict];
Option c:(init)
#interface ClassB{
NSDictionary * someDict;
}
-(id)initWithDict:(NSDictionary *)aDict;
#end
#implementation ClassB
-(id)initWithDict:(NSDictionary *)aDict
{
self = [super init];
if(self)
{
someDict = [someDict retain]; //or copy depending on your needs
}
NSLog(#"did something with aDict: %#",aDict);
}
#end
somewhere in ClassA...
ClassB * b = [[ClassB alloc] initWithDict:someOtherDict];
[b doSomethingelse];
you can also use properties etc... there are many many options, but you should understand object ownership as well so you don't end up leaking the dictionary.
-(void)dealloc
{
[someDict release];
}
should be added to the classB.
First of all check if u get the response in this delegate method of NSXMLParser
- (void)parserDidEndDocument:(NSXMLParser *)parser;
and send the response via a delegate (which u should implement in this class) and the delegate method is to implemented in the other class ...Simple !!!
I am trying to open a PDF via the QuickLook framework without using UIScrollView...
I believe I'm missing something...
Where I believe I'm going wrong is that I need to use a QLPreviewController and on the QLPreviewController is a dataSource that has to conform to QLPreviewItem. The documentation states that NSURL does conform to QLPriewItem so I'm setting the preview.dataSource to an NSURL which is throwing an error:
[NSURL numberOfPreviewItemsInPreviewController:]: unrecognized selector sent to instance
Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[NSURL numberOfPreviewItemsInPreviewController:]: unrecognized selector sent to instance 0x5b5f200'
Which makes me think that NSURL does not conform.
all the code I think is necessary...
- (BOOL)previewController:(QLPreviewController *)controller shouldOpenURL:(NSURL *)url forPreviewItem:(id <QLPreviewItem>)item {
return YES;
}
- (NSInteger) numberOfPreviewItemsInPreviewController: (QLPreviewController *) controller {
return [documents count];
}
- (id <QLPreviewItem>) previewController: (QLPreviewController *) controller previewItemAtIndex: (NSInteger) index {
return [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:[documents objectAtIndex:index] ofType:nil]];
}
- (void)pushPDF {
QLPreviewController *preview = [[QLPreviewController alloc] init];
preview.dataSource = [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:#"MCIT_Quiz" ofType:#"pdf"]];
//preview.currentPreviewItemIndex = 0;
[self presentModalViewController:preview animated:YES];
[preview release];
}
I ended up just creating another class to hold my values and use as a datasource, a bit quick and dirty but it works.
//
// documentList.h
//
#import <Foundation/Foundation.h>
#import <QuickLook/QuickLook.h>
#interface DocumentList : NSObject <QLPreviewControllerDataSource, QLPreviewControllerDelegate> {
NSArray *documents;
}
#property (nonatomic, retain) NSArray *documents;
-(void)createList;
-(NSInteger) numberOfPreviewItemsInPreviewController: (QLPreviewController *) controller;
- (id <QLPreviewItem>) previewController: (QLPreviewController *) controller previewItemAtIndex: (NSInteger) index;
#end
inserting text to break up the files
//
// documentList.m
//
#import "DocumentList.h"
#implementation DocumentList
#synthesize documents;
-(void) createList {
documents = [[NSArray arrayWithObjects:#"Quiz.pdf", nil] retain];
}
-(NSInteger) numberOfPreviewItemsInPreviewController: (QLPreviewController *) controller {
return [documents count];
}
- (id <QLPreviewItem>) previewController: (QLPreviewController *) controller previewItemAtIndex: (NSInteger) index {
return [NSURL fileURLWithPath:[[NSBundle mainBundle] pathForResource:[documents objectAtIndex:index] ofType:nil]];
}
#end
Well, I don't see where an NSURL conforms to QLPreviewControllerDataSource. I think you want
preview.dataSource = self;
And then your already written routines (numberOfPreviewItemsInPreviewController and previewController) would return the appropriate NSURL (although it's not clear how "documents" gets filled.).
I am reading a xml and finaly just need to remove the CDATA Infos in my results
For example: I get:
"<![CDATA[iPhone 4-Rückgaberecht: Deutsche Telekom kulant]]>"
just need "iPhone 4-Rückgaberecht: Deutsche Telekom kulant"
thx
chris
Edit to your answers:
I am not using NSXMLParser (thats the reason I make my own parser)
Found some suggestions with:
- (NSString *)stringByDecodingXMLEntities;
but dont know how to implement. I always get
> YourController may not respond to '-stringByDecodingXMLEntities" <
Ok, i solved it with that:
NSMutableString* resultString;
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)s {
resultString = [[NSMutableString alloc] init];
[resultString appendString:s];
}
- (NSString*)convertEntiesInString:(NSString*)s {
if(s == nil) {
NSLog(#"ERROR : Parameter string is nil");
}
NSString* xmlStr = [NSString stringWithFormat:#"<d>%#</d>", s];
NSData *data = [xmlStr dataUsingEncoding:NSUTF8StringEncoding allowLossyConversion:YES];
NSXMLParser* xmlParse = [[NSXMLParser alloc] initWithData:data];
[xmlParse setDelegate:self];
[xmlParse parse];
NSString* returnStr = [[NSString alloc] initWithFormat:#"%#",resultString];
return returnStr;
}
call: myConvertedString = [self convertEntiesInString:myOriginalString];
use
(void)parser:(NSXMLParser *)parser foundCDATA:(NSData *)CDATABlock
method instead of
(void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string
thats it
you could try a regex
replace <!\[CDATA\[(.*)\]\]> with $1
If you already have the String in String format with the you can remove it like so:
//Declare what you wish to remove
NSString * suffixTorRemove = #"<![CDATA[";
NSString * prefixToRemove = #"]]>";
//Now create a new string which uses your existing string and removes the declared occurrences above
NSString * newString = [yourString stringByReplacingOccurrencesOfString:suffixTorRemove withString:#""];
//Now the first part has changed, time to remove the second part
NSString * newString2 = [newString stringByReplacingOccurrencesOfString:prefixTorRemove withString:#""];
Quick and simple :-)