I have a huge NSMutableData object(approximately 1 MB in size) in memory and now I want to replace all the bytes in the object to 0 (not deallocate the memory). The resetBytesInRange method lets me do this. However how do I verify/check if the bytes are actually set to 0. I want to look at the memory addresses and confirm this. Is this possible?
I have the following code
NSMutableData *imgData = [NSMutableData dataWithCapacity:50000000];
imgData = (NSMutableData*)UIImageJPEGRepresentation(img, 1.0);
[imgData resetBytesInRange:NSMakeRange(0, [imgData length]) ];
Now when I look into the address pointed to by imgData (and the following few locations), before and after the resetBytes, I do not see any change in the values in the memory locations starting from the address pointd to by imgData (I expected to see zeros assigned). I assumed that the memory allocations are contiguous starting from the the address pointed to by imgData (upto [imgData length]). Is this assumption corerct (which seems like it is not)? If not where are the bytes in the NSMutableData object stored? Can I access them individually?
Thanks
Vivek
The problem is not that -resetBytesInRange: doesn't work. The problem is that you're sending that message to the wrong object.
imgData = (NSMutableData*)UIImageJPEGRepresentation(img, 1.0); doesn't assign the JPEG bytes to the mutable data you created in the first line. It replaces your reference to that object with a reference to a new object, and uses a cast to suppress the compiler warning about an invalid store. There are a few ways to do this properly:
NSMutableData *imgData = [NSMutableData data]; /* note, we do not specify a capacity--it's pointless for this use case */
[imgData setData: UIImageJPEGRepresentation(img, 1.0)];
[imgData resetBytesInRange: NSMakeRange(0, [imgData length])];
Or:
NSMutableData *imgData = [[UIImageJPEGRepresentation(img, 1.0) mutableCopy] autorelease];
[imgData resetBytesInRange: NSMakeRange(0, [imgData length])];
But both ignore the obvious: if you're immediately clearing the data to zero, why are you even bothering to create the JPEG data? What are you actually trying to do here?
imgData is a pointer (as indicated by the * symbol). So it points to a memory location and assignment will just change the location it points to. It's not really the intention of most programming languages to let you do byte-by-byte manipulation of actual memory locations.
Read this: http://www.cplusplus.com/forum/general/3644/
Related
Hello I would like to run a thread and check the current downloaded size of a file.
This is what I use
UIImage *image = [[UIImage alloc] initWithData:[NSData dataWithContentsOfURL:[NSURL URLWithString:#"http://lasp.colorado.edu/home/wp-content/uploads/2011/03/suncombo1.jpg"]]];
NSString *docDir = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
NSString *jpegFilePath = [NSString stringWithFormat:#"%#/test.jpeg",docDir];
NSData *data2 = [NSData dataWithData:UIImageJPEGRepresentation(image, 1.0f)];//1.0f = 100% quality
[data2 writeToFile:jpegFilePath atomically:YES];
downloadStatus.text =[NSString stringWithFormat:#"size: %zd", malloc_size(data2)];
[image release];
I have also tried to change malloc_size(data2) into image but again it is not the real result. I know this does not have thread and do not check during the download process but what am I supposed to use here to see the file size?
A couple of observations:
Your question presumed that your attempts to retrieve the size of the NSData were failing. They are not. The correct way to get the size of a NSData is via length.
Your confusion, though, stems from a faulty assumption that taking an externally generated JPEG on a roundtrip through UIImage and UIImageJPEGRepresentation would yield the identical NSData. This would have been extraordinarily unlikely. There are too many different JPG settings that could have changed (see the JPEG Wikipedia page). We certainly don't know what settings that original file used. I know that UIImage and/or UIImageJPEGRepresentation changed the color space of the file. I'd wager it's doing a lot of other things, too.
So your results are correct. The original file was 2.6mb and the resulting file was 4.5mb. If you change the compressionQuality from 1.0 to 0.99, the resulting file is only 1.4mb! But if you want the original file, just save it first (like I do below).
Consider the following code which downloads the image file, saves it, loads it into a UIImage, re-extracts it via UIImageJPEGRepresentation, and saves another copy of the image:
// let's make filenames where we'll store the files
NSString *documentsPath = [NSSearchPathForDirectoriesInDomains(NSDocumentDirectory, NSUserDomainMask, YES) objectAtIndex:0];
NSString *suncomboOrig = [documentsPath stringByAppendingPathExtension:#"suncombo1-orig.jpg"];
NSString *suncomboReprocessed = [documentsPath stringByAppendingPathExtension:#"suncombo1-reprocessed.jpg"];
// let's download the original suncombo1.jpg and save it in Documents and display the size
NSURL *url = [NSURL URLWithString:#"http://lasp.colorado.edu/home/wp-content/uploads/2011/03/suncombo1.jpg"];
NSData *data = [NSData dataWithContentsOfURL:url];
NSLog(#"original = %d", [data length]);
[data writeToFile:suncomboOrig atomically:NO];
// let's load that into a UIImage
UIImage *image = [UIImage imageWithData:data];
// let's extract data out of the image and write that to Documents, too, also logging the size of that
NSData *data2 = UIImageJPEGRepresentation(image, 1.0);
NSLog(#"reprocessed = %d", [data2 length]);
[data2 writeToFile:suncomboReprocessed atomically:NO];
What that does is it reports:
2012-12-13 22:30:39.576 imageapp[90647:c07] original = 2569128
2012-12-13 22:30:40.141 imageapp[90647:c07] reprocessed = 4382876
So the first file I saved (which I suspect is identical to what's on your server) was 2.5mb, and the file after doing a roundtrip to a UIImage and re-extracted via 4.3mb. If I look at the two files that the above code saved, I can confirm that these NSData sizes are correct.
My original answer was predicated on the presumption that the OP was either unable to retrieve the size of a NSData or that there was some subtle issue underlying the simple question (such as wanting to get the size before the download commenced). Anyway, I've expanded my answer above, but I'll keep my original answer for historical purposes:
Original Answer:
The NSData property length tells you how many bytes were downloaded. E.g. [data2 length].
If it's really big, you can use NSURLConnection to download it asynchronously, which, depending upon your web server, may provide total file size before the download commences in the method didReceiveResponse (with the expectedContentLength property in the NSHTTPURLResponse *response parameter).
The other nice thing about NSURLConnection downloading is that you don't have to load the entire file in memory as you're downloading it, but rather you can stream it directly to persistent storage, which is especially useful if you're downloading multiple large files at the same time. If you're downloading a reasonably sized file, using NSURLConnection to download is overkill, but it can be nice when downloading large files and you want a progress indicator (or want to get the file size before the download commences).
But if you just want to know how many bytes were downloaded to your NSData, use length.
You can just use the C FILE class to get the file size.
FILE * handle = fopen([jpegFilePath UTF8String], "r");
fseek(handle, EOF); // seek to end of file
int size = ftell(handle); // returns position in bytes
fclose();
Because you say "current" size and mention a thread, I'm guessing you're trying to determine the file size as it is received. In that case, you can get the thread for free from an NSURLConnection, and you can get the data size from the delegate methods as it's received...
Create an instance variable for the downloaded data:
#property (nonatomic, strong) NSMutableData *data;
Create and launch a connection:
NSURL *url = [NSURL URLWithString:#"http://lasp.colorado.edu/home/wp-content/uploads/2011/03/suncombo1.jpg"];
NSURLRequest *request = [NSURLRequest requestWithURL:url];
self.data = [NSMutableData data];
NSURLConnection *connection = [[NSURLConnection alloc] initWithRequest:request delegate:self];
Implement the required methods in NSURLConnectionDataDelegate. For your question, the special part is this:
- (void)connection:(NSURLConnection *)connection didReceiveData:(NSData *)data {
[self.data appendData:data];
NSUInteger bytesSoFar = [self.data length];
// you're on the app main thread here, so you can do something
// to the UI to indicate progress
downloadStatus.text = [NSString stringWithFormat#"size: %d", bytesSoFar];
}
A good doc on the rest of the protocol is here. When the connection is complete, you can create the image as you did with the dataWithContentsOfURL...
- (void)connectionDidFinishLoading:(NSURLConnection *)connection {
UIImage *image = [[UIImage alloc] initWithData:self.data];
downloadStatus.text = [NSString stringWithFormat#"size: %d", [self.data length]];
}
How to concatenate 3 NSData variables ?
NSData *iv;
NSData *salt;
NSData *encryptedData;
I need to join these to a single variable. Can any one show me a way.
use an NSMutableData object and the method -(void)appendData:(NSData *)otherData
Edited to add example :
NSMutableData *concatenatedData = [NSMutableData data];
[concatenatedData appendData:iv];
[concatenatedData appendData:salt];
[concatenatedData appendData:encryptedData];
// and now you have all of the data in the single variable "concatenatedData"
For those who coding for iOS5 and later.
I'd like to show some real good concatenation. Why are those answers aren't good enough? Because they are involves extra memory usage for copied data. Let's see the answer:
NSMutableData *concatenatedData = [NSMutableData data];
[concatenatedData appendData:iv];
[concatenatedData appendData:salt];
[concatenatedData appendData:encryptedData];
here we have memory allocated for iv, salt and encryptedData
also each time we append one of them to our mutable concatenation we are obviously copy it to mutable data again. Do we want this extra expenses while dealing with large data? Me not.
There is a way to avoid this unnecessary expense of memory - dispatch_data
I'm not going to explain how it works, you can google it if you want.
I just give you a code that works:
NSData *iv = [#"some data" dataUsingEncoding:NSUTF8StringEncoding];
NSData *salt = [#"even more data" dataUsingEncoding:NSUTF8StringEncoding];
NSData *encryptedData = [#"and one more" dataUsingEncoding:NSUTF8StringEncoding];
dispatch_queue_t queue = dispatch_get_global_queue(0, 0);
dispatch_data_t dispatch_data_iv = dispatch_data_create([iv bytes], [iv length], queue, DISPATCH_DATA_DESTRUCTOR_DEFAULT);
dispatch_data_t dispatch_data_salt = dispatch_data_create([salt bytes], [salt length], queue, DISPATCH_DATA_DESTRUCTOR_DEFAULT);
dispatch_data_t dispatch_data_encrypted = dispatch_data_create([encryptedData bytes], [encryptedData length], queue, DISPATCH_DATA_DESTRUCTOR_DEFAULT);
iv = nil; salt = nil; encryptedData = nil; // free all parts, we dont need it anymore
dispatch_data_t dispatch_data_concat = dispatch_data_create_concat( dispatch_data_create_concat(dispatch_data_iv, dispatch_data_salt), dispatch_data_encrypted);
NSData *concatenatedNSData = DataFromDispatchData(dispatch_data_concat);
// lets check now if the concatenation works properly
NSString *stringFromConcatenatedNSData = [[NSString alloc]initWithData:concatenatedNSData encoding:NSUTF8StringEncoding];
NSLog(#"%#",stringFromConcatenatedNSData);
don't forget about the helper-converter
NSData *DataFromDispatchData(dispatch_data_t data)
{
NSMutableData *result = [NSMutableData dataWithCapacity: dispatch_data_get_size(data)];
dispatch_data_apply(data, ^(dispatch_data_t region, size_t offset, const void *buffer, size_t size) {
[result appendBytes:buffer length:size];
return (_Bool)true;
});
return result;
}
You could use NSMutableData's -appendData method:
NSMutableData *result = [NSMutableData data];
[result appendData:iv];
[result appendData:salt];
[result appendData:encryptedData];
// result now has what you need.
This comes at the overhead of using mutable data, which can be slower & use more memory, so use with care. Generally speaking you don't want large NSData's floating around.
First create two NSObjects and use this method
-(void)appendData:(NSData *)otherData
and put in one NSData later with 3rd NSData also concatenate with the same method.
I'm currently working on an application that has to upload large files (mainly movies/videos) to the web. After reading what I can, I went the the approach of converting the movie to NSData and then including that as the NSURLConnection's HTTPBody. However, upon converting the movie (which was originally an ALAsset) into NSData, I receive a memory warning and then a subsequent crash.
I have no idea how I would go about uploading these types of large files, if that data just causes an instant crash. One solution that I was thinking of is writing to the filesystem and then uploading a file directly from there, but I have not been able to find any information on how one would accomplish this.
Here is the relevant code that I use. If there is something that I'm doing wrong right here, I'd love to know.
ALAssetRepresentation *representation = [asset defaultRepresentation];
Byte *buffer = (Byte *)malloc([representation size]);
NSUInteger buffered = [representation getBytes:buffer fromOffset:0.0 length:[representation size] error:nil];
uploadData = [NSData dataWithBytes:buffer length:buffered];
free(buffer);
Assuming that it makes sense to upload the movie in its native format, you can really make this easier using the BSD (ie Unix) section 3 interface:
given a filePath, open the file and get an int file descriptor (fd)
with fd, get the length of the file
keep track of how much you've loaded so you know where to get more data
use mmap(3) to map in JUST the data you want to upload at any time, and use the void * pointer returned by mmap as the location of the data
when the data has been sent, munmap the old data chunk and mmap a new chunk
after all data is sent, munmap the last chunk, the close(fd).
No temporary memory - no mallocs. I use mmap whenever I have to deal with huge files.
Edit: you can also use NSData dataWithContentsOfFile:options with options set to use mmap. You would then use the byte pointer to read small chunks as you need them.
In case anyone got here and couldn't solve your problems, I figured out a way to do this.
You have to firstly write your ALAssetRepresentation to disk (as described here):
NSUInteger chunkSize = 100 * 1024;
NSString *tempFile = [NSTemporaryDirectory() stringByAppendingPathComponent:#"temp.tmp"];
uint8_t *chunkBuffer = malloc(chunkSize * sizeof(uint8_t));
NSUInteger length = [rep size];
NSFileHandle *fileHandle = [[NSFileHandle fileHandleForWritingAtPath: tempFile] retain];
if(fileHandle == nil) {
[[NSFileManager defaultManager] createFileAtPath:tempFile contents:nil attributes:nil];
fileHandle = [[NSFileHandle fileHandleForWritingAtPath:tempFile] retain];
}
NSUInteger offset = 0;
do {
NSUInteger bytesCopied = [rep getBytes:chunkBuffer fromOffset:offset length:chunkSize error:nil];
offset += bytesCopied;
NSData *data = [[NSData alloc] initWithBytes:chunkBuffer length:bytesCopied];
[fileHandle writeData:data];
[data release];
} while (offset < length);
[fileHandle closeFile];
[fileHandle release];
free(chunkBuffer);
chunkBuffer = NULL;
Then you have to create an NSData object that can map the disk without using memory resources (kind of like David's answer, but inspired by this answer):
NSError *error;
NSData *fileData = [NSData dataWithContentsOfFile:tempFile options:NSDataReadingMappedIfSafe error:&error];
if (!fileData) {
NSLog(#"Error %# %#", error, [error description]);
NSLog(#"%#", tempFile);
//do what you need with the error
}
EDIT Although, if you are uploading the file somewhere, you should open a connection and send small buffers of the file, kind of like what I did above. I had to write a C++ class to handle the socket and the connection
You probably shouldn't be trying to read the whole asset in one shot:
Byte *buffer = (Byte *)malloc([representation size]);
NSUInteger buffered = [representation getBytes:buffer fromOffset:0.0 length:[representation size] error:nil];
Instead, set up a loop and read from the asset in chunks. I've outlined the basic approach. You'll need to fill in a few gaps, but it should solve the memory issue.
You might also want to consider running this in a thread so you don't lock up the UI.
NSError error;
int bufferSize = 1000;
float offset=0.0;
//TODO: Open Connection
while (1)
{
Byte *buffer = (Byte *)malloc(bufferSize);
NSUInteger buffered = [representation getBytes:buffer fromOffset:offset length:bufferSize error:&error];
//TODO: Write data
//TODO: Increment offset, check errors
free(buffer);
//if (done){
//break;
//}
}
//TODO close eonnection
I want to read a large file in objective-c, and used in iphone development
I used the follow code
NSData *data = [[NSData alloc] initWithContentsOfFile:file];
unsigned char *readData[2000];
[data getBytes:readData range:NSMakeRange(1000,3000)]
This maybe get the iphone app crash, because the size of the file is large.
I tried to use NSInputStream, but I can't find the range parameter in any method, can anyone give a help?
Thanks & Best Regards
Timmy
Use NSFileHandle: Seek to the offset where your data starts and start reading from there.
NSFileHandle *fileHandle = [NSFileHandle fileHandleForReadingAtPath:filePath];
[fileHandle seekToFileOffset:1000];
NSData *data = [fileHandle readDataOfLength:2000];
I use ABUnknownPersonViewController to display a contact view.
I try to set an image with:
NSData *dataRef = UIImagePNGRepresentation([UIImage imageNamed:#"contact3.png"]);
ABPersonSetImageData(newPersonViewController.displayedPerson, (CFDataRef)dataRef, nil);
It doesn't work and I don't know why. Any ideas?
You can't just cast an NSData object to a CFDataRef; as noted in the docs, a CFDataRef is a "reference to an immutable CFData object", which is not the same as an NSData instance:
typedef const struct __CFData *CFDataRef;
To create the CFDataRef from the NSData instance, you need to use the CFDataCreate method, passing the bytes and length:
NSData *dataRef = UIImagePNGRepresentation([UIImage imageNamed:#"contact3.png"]);
CFDataRef dr = CFDataCreate(NULL, [dataRef bytes], [dataRef length]);
Note also that since you create the object yourself, you must also release it, following the Core Foundation Ownership Policy; you use the CFRelease function to release ownership of the Core Foundation object:
CFRelease(dr);
This is similar to the Memory Management in Cocoa, and once the retain count of the Core Foundation object reaches zero it will be deallocated.
Edit: Stefan was completely right, in his comment, that NSData and CFData are also toll-free bridged on the iPhone with Cocoa-Touch as with Cocoa, so my original answer was wrong. My fault, should have edited it before.