This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
NSMutableArray addObject not working
I'm making an iPhone app, and so far, I'm receiving data from my server, creating objects using that data and filling an array with those objects.
My data is in XML format, and its saved into a string, which is transformed into a NSData object, like this:
NSMutableURLRequest *myRequest = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:#"https://my.URL/data.php"]];
[myRequest setHTTPMethod:#"POST"];
NSURLResponse *response;
NSError *error = NULL;
NSData *myReturn = [NSURLConnection sendSynchronousRequest:myRequest returningResponse:&response error:&error];
NSString *returnString = [[NSString alloc] initWithData:myReturn encoding:NSASCIIStringEncoding];
NSData *tempData = [return dataUsingEncoding:NSUTF8StringEncoding];
After that, I do the standard objective-C XML event parsing, but I don't create anything until the following method:
- (void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName
{
if ([elementName isEqual:#"id"])
{
hailID = currentValue;
return;
}
if ([elementName isEqual:#"timestamp"])
{
timeStamp = currentValue;
return;
}
if ([elementName isEqual: #"lat"])
{
hailLat = currentValue;
return;
}
if ([elementName isEqual:#"lng"])
{
hailLng = currentValue;
return;
}
if ([elementName isEqual:#"address"])
{
address = currentValue;
return;
}
if ([elementName isEqual:#"serviceType"])
{
serviceType = currentValue;
return;
}
if ([elementName isEqual: #"hail"])
{
Hail *newHail = [[Hail alloc]init];
newHail.hailID = hailID;
newHail.hailLat = hailLat;
newHail.hailLng = hailLng;
newHail.address = address;
newHail.timeStamp = timeStamp;
newHail.serviceType = serviceType;
[hails addObject:newHail];
NSLog(#"%u", [hails count]);
return;
}
}
hails is declared in the header file, and it's just a NSMutableArray with a capacity of 10000 items. The Hail object is a separate class. The NSLog always returns 0, even though I know that the XML code itself is valid, and the hail object exists.
Any ideas on why the hails array is always zero?
Edit: Sorry, I forgot to mention that the hails array is initialized in the -(void)viewDidLoad method as this:
hails = [[NSMutableArray alloc]initWithCapacity:10000];
Your hail is always nil because you never allocated + initialized it. You don't even mention its initialization into your code
Related
I need to download and parse 40mb of json. Right now I'm using AFJSONRequestOperation, which on older devices causes memory warning and crash. How it should be done? I think the only way to do it correctly, is to stream json but I've got no idea how to do it or which library will be best. Please provide examples. Thanks a lot!
For anybody who has the same problem, here's how I solved it:
1. Download JSON file to local storage using AFHTTPRequestOperation's output stream.
2. Parse little chunks of NSData using YAJLParser.
Result: I was testing it on 50mb json on iPad (1), without any memory warnings (memory around 10mb).
Example:
NSError *error = nil;
NSData *data = [NSData dataWithContentsOfFile:path
options:NSDataReadingMappedAlways | NSDataReadingUncached
error:&error];
YAJLParser *parser = [[YAJLParser alloc] initWithParserOptions:YAJLParserOptionsAllowComments];
parser.delegate = self;
[parser parse:data];
parser.delegate = nil;
parser = nil;
YAJLParser delegate:
// first declare in header file NSMutableArray *stack and NSString *mapKey
- (void)parserDidStartDictionary:(YAJLParser *)parser
{
NSString *dictName = mapKey;
if (mapKey == nil)
{
dictName = (stack.count == 0) ? #"" : [stack lastObject];
}
[stack addObject:(dictName)];
}
- (void)parserDidEndDictionary:(YAJLParser *)parser
{
mapKey = nil;
[stack removeLastObject];
}
- (void)parserDidStartArray:(YAJLParser *)parser
{
NSString *arrayName = mapKey;
if (mapKey == nil)
{
arrayName = stack.count == 0 ? #"" : [stack lastObject];
}
[stack addObject:(arrayName)];
if([mapKey isEqualToString:#"something"])
{
// do something
}
}
- (void)parserDidEndArray:(YAJLParser *)parser
{
if([mapKey isEqualToString:#"some1"])
{
// do something
}
mapKey = nil;
[stack removeLastObject];
}
- (void)parser:(YAJLParser *)parser didMapKey:(NSString *)key
{
mapKey = key;
}
- (void)parser:(YAJLParser *)parser didAdd:(id)value
{
if([mapKey isEqualToString:#"id"])
{
// do something
}
}
Write your data to a file, then use NSData's dataWithContentsOfFile:options:error: and specify the NSDataReadingMappedAlways and NSDataReadingUncached flags. This will tell the system to use mmap() to reduce the memory footprint, and not to burden the file system cache with blocks of memory (that makes it slower, but much less of a burden to iOS).
You can find the answer here
iPad - Parsing an extremely huge json - File (between 50 and 100 mb)
We want to import a huge XML-file (13MB) to Core Data. At the moment, the XML-File includes around 64000 entries, but this number will increase in future.
XML-Structure:
<entry name='...' doctype='' last-modified='...' [some more attributes] />
After a lot of research which included the XMLSchema Sample Project, Ray Wenderlich XML Tutorial and some stackoverflow entries, we didn't found a solution yet.
We first download the XML-File, and afterwards start parsing and insert the data to CoreData
Here is our implementation:
- (void)importXMLFile:(NSString*)fileName {
NSInputStream* theStream = [[NSInputStream alloc] initWithFileAtPath:fileName];
_theParser = [[NSXMLParser alloc] initWithStream:theStream];
_theParser.delegate = self;
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
[_theParser parse];
});
}
- (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName attributes:(NSDictionary *)attributeDict {
if ([elementName isEqualToString:#"entry"]) {
Importer* __weak weakSelf = self;
NSManagedObjectContext* theContext = self.importContext;
[theContext performBlock:^{
CustomObject* mo;
// Create ManagedObject
// Read values from parsed XML element
dispatch_async(dispatch_get_main_queue(), ^{
// Call a handler, just for information "added object"
});
NSError *error = nil;
if ([theContext hasChanges] && ![theContext save:&error]) {
NSLog(#"Unresolved error %#, %#", error, [error userInfo]);
abort();
} else {
DLOGError(error);
}
}];
}
}
Using this methods, memory usage explodes leading to a crash. The XML file seems to be parsed completely before even one block is being processed by Core Data. So the question is:
Is it possible to process parts of the XML file (f.e. 30 entries a time), than save to CoreData and after that continue parsing?
Or more commonly asked: How can memory usage be optimized?
You want to use a stream based parser so you don't need to load the whole XML to memory at the same time. Perhaps this or something from github.
You should also batch your save operation. Don't save every individual object, save groups of perhaps 100 objects. If this is inside a tight loop you should have an autorelease pool.
guess our memory problem occurred with a line we didn't publish, while creating our ManagedObject. We had to free the xmlChar
Instead of
xmlChar *xmlString = xmlTextReaderGetAttribute(reader, (xmlChar*)"someAttribute");
NSString *someAttributeToString = [NSString stringWithUTF8String:(const char *)xmlString];
we used
xmlChar * nameString = xmlTextReaderGetAttribute(reader, (xmlChar*)"someAttribute");
if (attributeString)
{
[elementDict setValue:[NSString stringWithUTF8String:(const char*)attributeString] forKey:#"someAttribute"];
xmlFree(nameString);
}
And we pause our parser after parsing 100elements and wait, till those elements are written to CoreData. After that, we parse the next 100 bundle
Parser
// Start the data parse
- (void) parse {
_dictionaryQeue = [NSMutableArray new];
xmlTextReaderPtr reader = xmlReaderForMemory([data bytes], [data length], NULL, NULL,
(XML_PARSE_NOBLANKS | XML_PARSE_NOCDATA | XML_PARSE_NOERROR | XML_PARSE_NOWARNING));
if (!reader) {
NSLog(#"Failed to create xmlTextReader");
return;
}
while (xmlTextReaderRead(reader)) {
#autoreleasepool {
while (_isPaused) {
//[NSThread sleepForTimeInterval:0.1];
}
switch (xmlTextReaderNodeType(reader)) {
case XML_READER_TYPE_ELEMENT: {
NSMutableDictionary* elementDict = [NSMutableDictionary new];
//Create Object
xmlChar * nameString = xmlTextReaderGetAttribute(reader, (xmlChar*)"name");
if (nameString)
{
[elementDict setValue:[NSString stringWithUTF8String:(const char*)nameString] forKey:#"name"];
xmlFree(nameString);
}
//...
if (self.collectDictionaries) {
[_dictionaryQeue addObject:elementDict];
NSArray* dictArray = [NSArray arrayWithArray:_dictionaryQeue];
if ([dictArray count] == self.maxCollectedDictionaries) {
dispatch_async(dispatch_get_main_queue(), ^{
if (saxDelegate && [(NSObject*)saxDelegate respondsToSelector:#selector(SAXDictionaryElements:finished:)]) {
[saxDelegate SAXDictionaryElements:dictArray finished:FALSE];
}
});
[_dictionaryQeue removeAllObjects];
_isPaused = TRUE;
}
}
elementDict = nil;
}
break;
case XML_READER_TYPE_END_ELEMENT: {
DLOGcomment(#"XML_READER_TYPE_END_ELEMENT");
if (self.collectDictionaries) {
NSArray* dictArray = [NSArray arrayWithArray:_dictionaryQeue];
if ([dictArray count] > 0) {
dispatch_async(dispatch_get_main_queue(), ^{
if (saxDelegate && [(NSObject*)saxDelegate respondsToSelector:#selector(SAXDictionaryElements:finished:)]) {
[saxDelegate SAXDictionaryElements:dictArray finished:TRUE];
}
});
data = nil;
[_dictionaryQeue removeAllObjects];
_dictionaryQeue = nil;
}
}
}
break;
}
}
}
xmlTextReaderClose(reader);
xmlFreeTextReader(reader);
reader = NULL;
}
DOM based parsers are quite convenient (TBXML, TouchXML, KissXML, TinyXML, GDataXML, RaptureXML, etc) especially those with XPATH support. But, memory becomes an issue as a DOM is created.
I am phasing the same memory constrains, so I started looking at wrappers for the Libxml2 XmlTextReader and so far I only found one IGXMLReader
IGXMLReader parses an XML document similar to the way a cursor would
move. The Reader is given an XML document, and return a node (an
IGXMLReader object) to each calls to nextObject.
Example,
IGXMLReader* reader = [[IGXMLReader alloc] initWithXMLString:#"<x xmlns:edi='http://ecommerce.example.org/schema'>\
<edi:foo>hello</edi:foo>\
</x>"];
for (IGXMLReader* node in reader) {
NSLog(#"node name: %#", node.name);
}
This is a different approach to that of the NSXMLParser.
I am sending a query to a server and getting response using this code
NSString *urlString = #"myPHPQuery";
NSURL *parserUrl = [[[NSURL alloc] initWithString:urlString] autorelease];
NSXMLParser *parser = [[[NSXMLParser alloc] initWithContentsOfURL:parserUrl] autorelease];
[parser setDelegate:self];
[parser parse];
I can get this type of response
<users>
<username>nothan</username>
<score>1000</score>
</users>
<users>
<username>nothan</username>
<score>1000</score>
</users>
I am using this code to parse the data in Mutable Array
int arrayCount = 0;
NSString *elementname;
NSInteger * count;
- (void)parser:(NSXMLParser *)parser didStartElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName attributes:(NSDictionary *)attributeDict
{
elementname = elementName;
}
-(void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string{
if([elementname isEqualToString:#"username"])
{
if ( count == 0 )
{
NSLog(#"%#" , string);
[self.playerNames replaceObjectAtIndex:arrayCount withObject:string];
arrayCount = arrayCount + 1;
count = count + 1 ;
}
else
count = 0;
}
else if([elementname isEqualToString:#"score"])
{
if ( count == 0 )
{
NSLog(#"%#" , string);
[self.scores replaceObjectAtIndex:arrayCount withObject:string];
count = count + 1 ;
}
else
count = 0;
}
}
- (void)parserDidEndDocument:(NSXMLParser *)parser
{
for (int i = 0 ; i <10 ; i++)
{
NSLog(#"%d - %#" , i,[playerNames objectAtIndex:i]);
}
}
The problem with this code is, it retrieve only one
<users> .... </users> How can I change this code to make it retrieve more users.
Best Regards
For the sample XML you posted above, the parser ran correctly.
Extensible Markup Language (XML) 1.0 (Fifth Edition) §2.1 Well-Formed XML Documents
There is exactly one element, called the root, or document element
After the first <users>…</users> element is parsed, the document has finished and the parser stops.
To have a list if things in an XML document, you must wrap them in an outer element.
<allusers>
<users>
<username>nothan</username>
<score>1000</score>
</users>
<users>
<username>nothan</username>
<score>1000</score>
</users>
</allusers>
When the parser foundCharacters method is called the int count is checked. If it is zero then the first entry in the array is replaced, and count is incremented. On the next call to this method count is not zero, so nothing is replaced, and then count is set back to zero. For the next call to this method the count is zero.....and so on.
Also bear in mind that when the data being retrieved off the net is large enough to be split into blocks by tcp/ip then the initWithContentsOfURL may not return all of it. It is better to use NSURLConnection to build up the received data as an NSData object and then pass that to the parser. In addition there may be more than one call to foundCharacters for a given element and so the element received should be built up and the placing into the array should be done at the delegate didEndElement method.
Ok, my problem is that whenever i collect data from the parser into an array where the string contains Swedish ÅÄÖ characters. In my example the
[schemaInfoArray objectAtIndex:3]
is supposed to be #"Lördag" but is saved as #"L" and the
[schemaInfoArray objectAtIndex:4]
contains the rest of the string that gets presented as
#"ördag"
-(void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string
{
tempStrang = string;
[schemaInfoArray insertObject:tempStrang atIndex:uppraknare];
uppraknare++;
}
-(void)parser:(NSXMLParser *)parser didEndElement:(NSString *)elementName namespaceURI:(NSString *)namespaceURI qualifiedName:(NSString *)qName
{
if ( [elementName isEqualToString:#"schemaInfo"] )
{
}
if ( [elementName isEqualToString:#"modfromtid"] )
{
frommodarbtid = [schemaInfoArray objectAtIndex:0];
}
if ([elementName isEqualToString:#"modtomtid"] )
{
tommodarbtid = [schemaInfoArray objectAtIndex:1];
}
if ([elementName isEqualToString:#"modrast"] )
{
modrast = [schemaInfoArray objectAtIndex:2];
}
if ([elementName isEqualToString:#"benamning"] )
{
benamning = [schemaInfoArray objectAtIndex:3];
}
if ([elementName isEqualToString:#"fromnormarb"] )
{
fromnormarbtid = [schemaInfoArray objectAtIndex:4];
}
if ([elementName isEqualToString:#"tomnormarb"] )
{
tomnormarbtid = [schemaInfoArray objectAtIndex:5];
}
if ([elementName isEqualToString:#"rast"] )
{
normrast = [schemaInfoArray objectAtIndex:6];
}
}
Does anyone have any thoughts about how to actually get #"Lördag" to be saved into ONE index instead of getting split into several indexes? This really destroys the structure of things that is supposed to be presented.
This is a documented design choice from Apple, and has nothing to do with Swedish characters:
Because string may be only part of
the total character content for the
current element, you should append it
to the current accumulation of
characters until the element changes.
So you should do just as they say: use a NSMutableString to accumulate the results, and when the element changes, save the buffer to a permanent, (preferrably) immutable NSString.
As requested, here's an example. It was written without any kind of IDE, so chances are that it'll work, but there's no guarantee that it will either compile or work.
#interface Foo : NSObject<NSXMLParserDelegate> {
NSMutableString* accumulator;
NSMutableArray* schemaInfoArray;
int uppraknare; // whatever 'uppraknare' means
}
/* snip */
#end
#implementation Foo
-(void)parser:(NSXMLParser*)parser foundCharacters:(NSString*)string
{
// only accumulate characters, until we get notified that we went through
// the whole XML element
[accumulator appendString:string];
}
-(void)parser:(NSXMLParser*)parser didEndElement:(NSString*)elementName namespaceURI:(NSString*)nsuri qualifiedName:(NSString*)qName
{
// we went through the whole element! time to save!
NSString* immutableResult = [accumulator copy];
[schemaInfoArray insertObject:immutableResult atIndex:uppraknare];
uppraknare++;
[immutableResult release];
// clear the accumulator for the next element
[accumulator deleteCharactersInRange:NSMakeRange(0, [accumulator length])];
/* the rest of your code here */
}
#end
-(void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string is not guaranteed to contain the complete contents of the string. You need to have a class instance variable that is a NSMutableString that can append all of foundCharacters between the calls to didStartElement and didEndElement. Inside of didEndElement add the the string to the schemaInfoArray.
I am reading a xml and finaly just need to remove the CDATA Infos in my results
For example: I get:
"<![CDATA[iPhone 4-Rückgaberecht: Deutsche Telekom kulant]]>"
just need "iPhone 4-Rückgaberecht: Deutsche Telekom kulant"
thx
chris
Edit to your answers:
I am not using NSXMLParser (thats the reason I make my own parser)
Found some suggestions with:
- (NSString *)stringByDecodingXMLEntities;
but dont know how to implement. I always get
> YourController may not respond to '-stringByDecodingXMLEntities" <
Ok, i solved it with that:
NSMutableString* resultString;
- (void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)s {
resultString = [[NSMutableString alloc] init];
[resultString appendString:s];
}
- (NSString*)convertEntiesInString:(NSString*)s {
if(s == nil) {
NSLog(#"ERROR : Parameter string is nil");
}
NSString* xmlStr = [NSString stringWithFormat:#"<d>%#</d>", s];
NSData *data = [xmlStr dataUsingEncoding:NSUTF8StringEncoding allowLossyConversion:YES];
NSXMLParser* xmlParse = [[NSXMLParser alloc] initWithData:data];
[xmlParse setDelegate:self];
[xmlParse parse];
NSString* returnStr = [[NSString alloc] initWithFormat:#"%#",resultString];
return returnStr;
}
call: myConvertedString = [self convertEntiesInString:myOriginalString];
use
(void)parser:(NSXMLParser *)parser foundCDATA:(NSData *)CDATABlock
method instead of
(void)parser:(NSXMLParser *)parser foundCharacters:(NSString *)string
thats it
you could try a regex
replace <!\[CDATA\[(.*)\]\]> with $1
If you already have the String in String format with the you can remove it like so:
//Declare what you wish to remove
NSString * suffixTorRemove = #"<![CDATA[";
NSString * prefixToRemove = #"]]>";
//Now create a new string which uses your existing string and removes the declared occurrences above
NSString * newString = [yourString stringByReplacingOccurrencesOfString:suffixTorRemove withString:#""];
//Now the first part has changed, time to remove the second part
NSString * newString2 = [newString stringByReplacingOccurrencesOfString:prefixTorRemove withString:#""];
Quick and simple :-)