iOS - Amazon S3 Slow Download - iphone

I am creating an iOS app that allows users to write lets say a status update and people can comment on it, like it, interact with it in multiple ways, basically the status update has multiple properties that would be stored with it. Imagine an app with a home-screen of more than 50 of these status updates represented into a table view.
Now take your eyes away and focus on a practice/demo app, a developer trying to master the techniques before the big game (thats me!) So essentially, I have started out by setting up an S3 bucket based in Singapore. I live in Singapore, so there is a server nearby and really everything should be fast and smooth. Except, its just not. Its slow, and its starting to make me annoyed.
I know an app that uses S3 that loads high-definition panorama images with comments, likes etc. and it takes a second or a bit more for all this data to load. I am not sure how they actually carry out the process, I know they store the images on S3 but thats all I know. In my starter demo, I simply upload some pieces of text (say 20) then download them and it takes like 15 seconds under my 60 mbps wifi! These pieces of text don't even exceed a sentence each, they are phrases, so I am really kind of confused here.
I have CloudFront setup but isn't it for websites? I have the manage distributions and URL things all setup but how to setup in my code? This is probably my biggest question to nail down for release later in my other app. Even so, I live in Singapore and the bucket is in the Singapore server, so CloudFront for self-testing / practicing wouldn't be mandatory.
I find this extremely confusing, here is some annotated code I have produced, any problems, misconceptions that is leading it to be slow?
- (void)loadObjects {
S3ListObjectsRequest *listObjectRequest = [[S3ListObjectsRequest alloc] initWithName: #"testbucketquotes.rohanprostudios"];
S3ListObjectsResponse *listObjectResponse = [[AmazonClientManager s3] listObjects:listObjectRequest];
if(listObjectResponse.error != nil)
{
}
else
{
S3ListObjectsResult *listObjectsResults = listObjectResponse.listObjectsResult;
if (objects == nil) {
objects = [[NSMutableArray alloc] initWithCapacity:[listObjectsResults.objectSummaries count]];
}
else {
[objects removeAllObjects];
}
// By defrault, listObjects will only return 1000 keys
// This code will fetch all objects in bucket.
NSString *lastKey = #"";
for (S3ObjectSummary *objectSummary in listObjectsResults.objectSummaries) {
if ([[objectSummary key] rangeOfString: #"UploadedQuote"].location != NSNotFound) {
[objects addObject:[objectSummary key]];
lastKey = [objectSummary key];
}
}
while (listObjectsResults.isTruncated) {
listObjectRequest = [[S3ListObjectsRequest alloc] initWithName: #"testbucketquotes.rohanprostudios"];
listObjectRequest.marker = lastKey;
listObjectResponse = [[AmazonClientManager s3] listObjects:listObjectRequest];
if(listObjectResponse.error != nil)
{
break;
}
listObjectsResults = listObjectResponse.listObjectsResult;
for (S3ObjectSummary *objectSummary in listObjectsResults.objectSummaries) {
if ([[objectSummary key] rangeOfString: #"UploadedQuote"].location != NSNotFound) {
[objects addObject:[objectSummary key]];
}
lastKey = [objectSummary key];
}
}
if (objects.count) {
for (int i = 0; i <= objects.count - 1; i++) {
S3GetObjectRequest *req = [[S3GetObjectRequest alloc] initWithKey:[objects objectAtIndex: i] withBucket: #"testbucketquotes.rohanprostudios"];
`// asynchronously loads text (adds to operation queue)`
AsyncQuoteDownloader *quote = [[AsyncQuoteDownloader alloc] initWithRequest:req andViewController: self];
[operationQueue addOperation: quote];
// in 'AsyncQuoteDownloader' when finished calls a method in this view controller adding the response to an array and reloading a table
}
}
}
});
}
Anything wrong with my code that is making it lag so much? I would have thought this would take a matter of milliseconds if an imaging service would take 1 second to load HQ images with likes and comments etc takes 1-2 seconds.
Thank you for any help...
Update
Ok, so the iteration of keys doesn't seem to be the problem here but rather the downloading of the objects. Any thoughts? Thanks...

This is the first time I've ever even seen Objective C, so I may be completely wrong here. But... it looks like you're iterating through the keys of the entire bucket. This is going to be really, really slow with any appreciable quantity of keys.
A better design would be store a lookup table in something like DynamoDB (since you're already using AWS). You'd query there, get an array of matching id's (S3 keys), and then fetch the match objects from S3.

Another option, as I used in my own iOS app, is to use Heroku as the app layer and create a POSTGRESQL record that points to your content. This way you can create rich queries in rails and mediate upload and download with the power of Rails, rather then paying for both DynamoDB and S3.

Turns out my bucket was set to US region and not Singapore region... #doh
Its faster now, its working fine
I just needed to set the AmazonS3Client's endpoint to Singapore (SEA) region

Related

How to Efficiently Reset Attributes in a Core Data Entity

My app uses Core Data and has an attribute called 'beenSeen'. When a user refreshes the app, all 'beenSeen' values of 1 are changed to 0. On an iPod Touch 2nd gen with over 2000 objects, refreshing takes over a minute. My code looks like this:
for (Deck *deck in self.deckArray) {
if ([deck.beenSeen isEqualToNumber:[NSNumber numberWithInt:1]]) {
[deck setBeenSeen:[NSNumber numberWithInt:0]];
[self.managedObjectContext save:&error];
}
}
I'm also considering deleting the sqlite file and having an alert ask the user to restart the app themselves. Doing that sure is a whole lot quicker than what I have now. Is there a quicker way to refresh an entity? Could I have a 'backup' entity and copy it over? Thanks for any help.
Hm. The first optimization I'd suggest would be
for (Deck *deck in self.deckArray) {
if ([deck.beenSeen isEqualToNumber:[NSNumber numberWithInt:1]]) {
[deck setBeenSeen:[NSNumber numberWithInt:0]];
}
}
[self.managedObjectContext save:&error];
I suspect it might speed things up to do one big context save, instead of 2,000 little ones.
The second suggestion would be to try getting rid of the if test – if the majority of your beenSeen values are changing from 1 to 0, and the others are already 0, then you might as well just set all of them to 0 and save the time of checking each one individually. (On the other hand, if there are 10,000 objects and you're resetting 2,000 of them, then getting rid of the test might not be optimal.)
for (Deck *deck in self.deckArray) {
[deck setBeenSeen:[NSNumber numberWithInt:0]];
}
[self.managedObjectContext save:&error];
}
The third suggestion would be to think about implementing this another way – for instance, your deck object could implement a lastSeen attribute, storing the date and time when the deck was last seen, and then instead of doing a mass reset (and writing 2,000 Core Data rows) you could just test each deck's lastSeen date and time against the timestamp of the last user refresh.
Try this, First, filter the array using a predicate:
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"beenSeen == %#",
[NSNumber numberWithInt:1]];
NSArray* filtered = [self.deckArray filteredArrayUsingPredicate:predicate];
Now set the new value:
[filtered setValue:[NSNumber numberWithInt:0] forKeyPath:#"beenSeen"];
Finally save the context:
[self.managedObjectContext save:&error];
Hope this helps :)

Core data only storing last object of JSON feed

I´m using Core Data as local storage in my app. I´ve set it up properly and made subclasses of NSManagedObject for each entity. However, when I´m trying to insert values into my store, it only inserts the last object from my JSON feed.
res = [JSONHandler requestJSONResponse:jsonString];
shows = [res valueForKeyPath:#"Show.Name"];
NSUInteger showIndex = 0;
for(NSString *showName in shows){
showObject = [NSEntityDescription insertNewObjectForEntityForName:#"Show" inManagedObjectContext:managedObjectContext_];
showObject.name = showName;
showObject.iD = [[res valueForKeyPath:#"Show.Id"]objectAtIndex:showIndex];
showObject.desc = [[res valueForKeyPath:#"Show.Description"]objectAtIndex:showIndex];
showObject.activityType = [[res valueForKeyPath:#"Show.ActivityType"]objectAtIndex:showIndex];
showIndex++;
}
This only stores the last object from my JSON feed. Any idea why?
EDIT: It works fine when I do this:
res = [JSONHandler requestJSONResponse:jsonString];
shows = [res valueForKeyPath:#"Show.Name"];
NSUInteger index = 0;
for(NSString *showName in shows){
show = [NSEntityDescription insertNewObjectForEntityForName:#"Show" inManagedObjectContext:managedObjectContext_];
[show setValue:showName forKey:#"name"];
[show setValue:[[res valueForKeyPath:#"Show.Id"]objectAtIndex:index] forKey:#"iD"];
[show setValue:[[res valueForKeyPath:#"Show.Description"]objectAtIndex:index] forKey:#"desc"];
[show setValue:[[res valueForKeyPath:#"Show.ActivityType"]objectAtIndex:index] forKey:#"activityType"];
index++;
}
It´s basically the same thing, isn´t it? But I want to use subclasses of NSManagedObject instead of doing like I did above. Because in the snippet above show is NSManagedObject *show instead of what it should be: Show *show.
How many shows are there? You can find this by doing: NSLog(#"Number of shows: %d.", shows.count);, assuming that shows is an NSArray. It could be that your Core Data code is fine and the JSON parsing itself is at fault.
EDIT: Also, are you correctly saving the changes to the persistent store?
Usually when you see just one of several objects being saved like this, the problem is that a relationship that should be to-many is improperly set as to-one. No matter how many objects you try to add to the relationship, only the last one is set because the relationship can hold only one value.
I think in this circumstance the problem is most likely in the code of the custom subclass instead of the data model itself given that the data model works with generic NSManagedObjects.

How to sync a section in a MutableArray with UITextView

I have one MutableArray (NSMutableArray *NoteBook;) in which I would like to store Notepages (I set up an object called NotePage.m which simply contains one NSString *noteText and a method to setNoteText). I also have an integer to keep track of my pages (NSUInteger currentPageCounter;).
I start my little program by setting up a new instance of NotePage:
NotePage *newPage = [[NotePage alloc] init];
[newPage setNoteText:#"Sample Text for a certain page..."];
I then copy this *newPage 3 times into my NoteBook Mutable Array:
[self.NoteBook insertObject:newPage atIndex:(self.currentPageCounter)];
This will give me 3 pages, at the indices 0,1,2. So far so good. Everything works splendid. But now comes the enemy, UITextView. First of all, I would like to display the contents of a page within my NoteBook in an UITextView. So I did this to sync one section in the MutableArray (e.g. page at index 0) with the UITextView which works fine:
NotePage *thisPage = [self.NoteBook objectAtIndex:self.currentPageCounter];
TextViewNote.text = thisPage.noteText;
The problem is, however, if I want to edit my UITextView and sync the updated text with the MutableArray. This is where the crashing happens... I have coded this in a separate method which can be clicked once the user is done editing the UITextView:
-(IBAction) savePage
{
NSString *tempString;
tempString = [NSString stringWithFormat:#"%#",TextViewNote.text];
NotePage *newPage = [[NotePage alloc] init];
[newPage setNoteText:tempString];
[self.NoteBook insertObject:newPage atIndex:(self.currentPageCounter)]; // e.g. at index 0 for page 1
[self.NoteBook removeObjectAtIndex:(currentPageCounter+1)]; // i.e. this is to remove the old page 1 which used to be at index 0 but now is at index 1
[self syncTextView];
[self syncCounter];
}
I guess there is a less cumbersome way (I'm still a beginner...) to simply replace an object at a certain index at a Mutable Array. As it stands now, it will simply crash once I try to move onwards to the next index position which is apparently not found anymore.
Thanks for any ideas!
I guess there is a less cumbersome way
(I'm still a beginner...) to simply
replace an object at a certain index
at a Mutable Array.
Indeed. Look at -[NSMutableArray replaceObjectAtIndex:withObject:].
Considering the error you posted, it sounds like you've got a bad pointer somewhere. The error is complaining that somewhere, a -[UITouchData length] message is being sent to an instance of NotePage. Turn on the 'Stop on Objective-C Exceptions' option in the debugger (under the Run menu). That should help you see where the problem is occurring.
NSMutableArray has a method replaceObjectAtIndex:withObject: that should do what you are trying to do.
You've got a memory management issue, however, unless you are calling release on the newPage after you add it to the array. Unless you are planning on making the NotePage class more complex, it might make sense simply to change the item's text rather than substituting a new object:
[[self.noteBook objectAtIndex:currentPageCounter] setNoteText:tempString];
(Also, be aware that inserting the newPage object into the array four times does not copy the object; it simply inserts the same reference four times. If you want four distinct objects, you will need to allocate four objects in a loop:
for(i = 0; i < 4; i++){
NotePage *newPage = [[NotePage alloc] init];
[newPage setNoteText:#"Dummy text"];
[self.notebook addObject:newPage];
[newPage release];
}

Parsing my flat-file on iPhone is a pain in the ***, please Help me out

I am programming an iPhone App which is supposed to parse a flat-file from the web, create managed objects from the flat-file and later on should display them in an UITableView.
There are no problems with the saving and the displaying, but I just can't get the hang of a good Parser.
Thats the file I want to parse: Flat-file
AS far as I know, I can't use the NSXMLParser for this task (because obviously there are no tags).
So I at first tried to programm a NSScanner which should get me the interesting properties --> didn't work out
Now I am using this method:
- (void) parseMemberDataWithURL: (NSString *)urlString
{
self.memberTempCounter = 1;
//Get data from web
self.downloadedText = [NSString stringWithContentsOfURL: [NSURL URLWithString: urlString] encoding:NSUTF8StringEncoding error:nil ];
memberArray = [downloadedText componentsSeparatedByString:#";"];
while (self.memberTempCounter<[memberArray count])
{
[[ExhibitorController sharedController] createExhibitorWithName:[memberArray objectAtIndex:self.memberTempCounter]
street:[memberArray objectAtIndex:self.memberTempCounter+2]
zip:[memberArray objectAtIndex:self.memberTempCounter+3]
city:[memberArray objectAtIndex:self.memberTempCounter+4]
email:[memberArray objectAtIndex:self.memberTempCounter+7]
phone:[memberArray objectAtIndex:self.memberTempCounter+5]
website:[memberArray objectAtIndex:self.memberTempCounter+8]
produktbereiche:[[memberArray objectAtIndex:self.memberTempCounter+9] componentsSeparatedByString:#","]];
self.memberTempCounter= self.memberTempCounter+13;
}
}
I am using the memberTempCounter to identify the property.
The problems are:
This only works out in like 3 of 4 times.1 of 4 times the App crashes and I have no Idea why...
The method has a performance like a 1962 VW Beetle. Parsing the whole chunk of data takes up to 3 Minutes on my iPhone 3G
Any Ideas or a simpler way to do this?
I would be really gratefull. Thanks in advance: -)
You might as well do all the parsing in the background, and then display as the information gets parsed.
As for memory issues, try doing temporary autorelease pools and release every 50 or so iterations through the loop.
int count = 0;
NSAutoreleasePool * loopPool = [[NSAutoreleasePool alloc] init];
while(someInsanelyLargeCondition){
// Do your stuff here
// .............
count++;
if (count > 50) {
count = 0;
[loopPool release];
loopPool = [[NSAutoreleasePool alloc] init];
}
}
Recursive-descent (LL1) parsers are pretty simple, light on memory, and for speed they go almost as fast as you can run a pointer through characters. Building your data structure would probably be the dominant time-taker.
I was finally able to fix my performance problem.
I have a method in another class, which ads Tags for the different Exhibitors. Therefore it first checks if the Tag already is stored in the database or else creates it.
With an growing Set of Tags in my database the search-process took longer and longer and this led to the long parsing time.
Anyone else having this problem: Take a look at the Performance Core Data Programming guide of apple in the "Implementing Find-or-Create Efficiently"-section:
http://developer.apple.com/mac/library/documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html

Bulk update & occasional insert (coredata) - Too slow

Update: Currently looking into NSSET's minusSet
links: Comparing Two Arrays
Hi guys,
Could benefit from your wisdom here..
I'm using Coredata in my app, on first launch I download a data file and insert over 500 objects (each with 60 attributes) - fast, no problem.
Each subsequent launch I download an updated version of the file, from which I need to update all existing objects' attributes (except maybe 5 attributes) and create new ones for items which have been added to the downloaded file.
So, first launch I get 500 objects.. say a week later my file now contains 507 items..
I create two arrays, one for existing and one for downloaded.
NSArray *peopleArrayDownloaded = [CoreDataHelper getObjectsFromContext:#"person" :#"person_id" :YES :managedObjectContextPeopleTemp];
NSArray *peopleArrayExisting = [CoreDataHelper getObjectsFromContext:#"person" :#"person_id" :YES :managedObjectContextPeople];
If the count of each array is equal then I just do this:
NSUInteger index = 0;
if ([peopleArrayExisting count] == [peopleArrayDownloaded count]) {
NSLog(#"Number of people downloaded is same as the number of people existing");
for (person *existingPerson in peopleArrayExisting) {
person *tempPerson = [peopleArrayDownloaded objectAtIndex:index];
// NSLog(#"Updating id: %# with id: %#",existingPerson.person_id,tempPerson.person_id);
// I have 60 attributes which I to update on each object, is there a quicker way other than overwriting existing?
index++;
}
} else {
NSLog(#"Number of people downloaded is different to number of players existing");
So now comes the slow part.
I end up using this (which is tooooo slow):
NSLog(#"Need people added to the league");
for (person *tempPerson in peopeArrayDownloaded) {
NSPredicate *predicate = [NSPredicate predicateWithFormat:#"person_id = %#",tempPerson.person_id];
// NSLog(#"Searching for existing person, person_id: %#",existingPerson.person_id);
NSArray *filteredArray = [peopleArrayExisting filteredArrayUsingPredicate:predicate];
if ([filteredArray count] == 0) {
NSLog(#"Couldn't find an existing person in the downloaded file. Adding..");
person *newPerson = [NSEntityDescription insertNewObjectForEntityForName:#"person" inManagedObjectContext:managedObjectContextPeople];
Is there a way to generate a new array of index items referring to the additional items in my downloaded file?
Incidentally, on my tableViews I'm using NSFetchedResultsController so updating attributes will call [cell setNeedsDisplay];
.. about 60 times per cell, not a good thing and it can crash the app.
Thanks for reading :)
I'll begin by saying that I'm still new to using the Core Data framework, but my guess is that your problem lies in the for loop you've posted.
If you look at your loop, each time it executes it creates a new NSPredicate object and then filters your existing array looking for matches. On a small data set this technique would work with seemingly small performance losses; however, with your large data set you will end up spending a lot of time creating NSPredicate objects that only differ in the name you've provided. I would suggest that you look at how to create a single predicate and then use variable substitution to perform the search. For information about variable use in predicates check out: http://developer.apple.com/iphone/library/documentation/Cocoa/Conceptual/CoreData/Articles/cdImporting.html#//apple_ref/doc/uid/TP40003174
As a side note, you may also consider how you've sorted your data and how you are performing the search operation. And another thing I noticed is that you don't release your NSPredicate object, so you're just tossing memory away too.