In my MacOS app I am downloading an encrypted .zip file to the disk. I decrypt this file and keep the decrypted version in memory in the Data type. For security reasons the decrypted .zip will only be kept in memory.
I can successfully use ZIPFoundation's Closure based reading to extract the file contents in memory, but only by using an URL pointing to the (decrypted) .zip on disk:
guard let archive = Archive(url: url!, accessMode: .read) else { return }
Is there any way I can use the library with data only existing in memory? If not, can you point me towards a library that can handle this?
I have already tried DataCompression, but I couldn't make it work.
There's a (non-merged) Pull Request open that adds in-memory processing of ZIP archives to ZIP Foundation.
Sadly there are still some unresolved issues with in-memory writing of archives. The reading part is using fmemopen and should already work.
While the PR is not finished yet, you can have a look here: https://github.com/weichsel/ZIPFoundation/pull/78/
Related
I have a RAID disk which makes its files available over CIFS/SMB. If I copy a large file (600MB) from one location on the volume to another location on the same volume using Finder (i.e., option-drag the file), it only takes a second or two.
If I try to do the same operation programmatically using the code below, it takes upwards of a minute.
let source = URL(filePath: "/Volumes/media/tmp/bigfile.dat")
let dest = URL(filePath: "/Volumes/media/finaldest/file.dat")
try FileManager.default.copyItem(at: source, to: dest)
My theory is that Finder can detect that the source and destination are on the same network volume and uses a special SMB API to do the copying server side, without having to move a bunch of bytes back and forth over the network. FileManager does not have this same optimization.
Does anybody have suggestions on how I can make my program behave like the Finder does?
The file needs to be read and write later. On Android I used Room & RoomAssetHelper.
I understand the basic logic that I need to include it in the project assets and then copy it to somewhere accessible for the app at first start. But I want to avoid writing these things manually and risking making an error (I am not too confident with reading files & DBs).
All of the answers that I find are from people giving quick & dirty advice on how to manually code the logic for this. I would like to do it on a clean & professional level.
Is there a library that would do most of the "risky" work for me?
(Meaning import & copy the .sqlite file, so I can start using it in my code)
I found GRDB.swift, but I cannot figure out if it supports prepopulated files.
Please stop looking for a magical library that will do all of this for you automatically.
What you need to do yourself without any SQLite library?
Add your prepopulated database.sqlite as an asset to your project.
When the app launches, check if the database.sqlite file is present at the expected location (inside your app's documents directory for example). You can check this using FileManager APIs.
If the file exists at the expected path, you are fine, no need to copy any file.
If the database.sqlite file does not exist at the expected path, you need to copy your database.sqlite file at the path using FileManager APIs.
CAUTION :
Be aware that in this step, you may encounter an error while copying the file. This should not happen for most cases. In rare caes that it does happen, you should adjust your app accordingly - indicate to user somehow that initialization failed, free some space on your phone, restart app etc.
Steps 2-4 need to be checked on every app launch - put this logic somewhere close your app startup process. If all of above instructions are followed and you succeed either via step 3 OR 4, you now have the database.sqlite file where you want it to be.
Where the SQLite library comes in?
At this point, you can use any library that suits your purpose and you feel comfortable with.
As you mentioned GRDB.swift, it allows you to specify a custom path for your database file. Copy-pasting the current version minimal setup code here for reference.
import GRDB
// 1. Open a database connection
let dbQueue = try DatabaseQueue(path: "/path/to/database.sqlite")
// 2. Define the database schema
try dbQueue.write { db in
try db.create(table: "player") { t in
t.autoIncrementedPrimaryKey("id")
t.column("name", .text).notNull()
t.column("score", .integer).notNull()
}
}
// 3. Define a record type
struct Player: Codable, FetchableRecord, PersistableRecord {
var id: Int64
var name: String
var score: Int
}
// 4. Access the database
try dbQueue.write { db in
try Player(id: 1, name: "Arthur", score: 100).insert(db)
try Player(id: 2, name: "Barbara", score: 1000).insert(db)
}
let players: [Player] = try dbQueue.read { db in
try Player.fetchAll(db)
}
My code needs to parse heaps of JSON files, and those files are hosted on GitHub and only available bundled as 1 single ZIP file. Because the ZIP file is only about 80 MB, I want to keep the entire unzipping operation in memory.
I'm able to load the ZIP file into memory as a Data? variable, but I'm not able to find a way to unzip a Data variable in memory and then assign the unzipped file/data to some other variables. I've tried using ZIP Foundation, but its Archive type's initializers take only file URLs. I didn't try Zip, but its documentation shows that it takes file URLs as well.
Here is my code:
import Cocoa
import Alamofire
class ViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let zipURL = URL(string: "https://github.com/KSP-CKAN/CKAN-meta/archive/master.zip")!
AF.request(zipURL).validate().responseData { response in
var zipData: Data? = response.data
// Here I want to unzip `zipData` after unwrapping it.
}
}
}
I also looked into passing a Data variable off as a file, but failed to find a way to do it.
UPDATE (2019-12-01 05:00)
According to this pull request thread on ZIPFoundation, the feature I'm looking for will be included in the next release. I tried to use the feature's contributor's fork, but somehow Swift Package Manager wouldn't allow it.
Before finding this, I tried using Python's zipfile library through Swift-Python interoperability provided by PythonKit, but it didn't work out, because Foundation's Data in Swift can not be cast into a PythonObject type.
Apple's Compression framework also looked promising, but it seems to have a soft limit of 1 MB on compressed files. The compressed file I need is about 80 MB, way larger than 1 MB.
So far, ZIPFoundation is my most hopeful solution.
UPDATE (2019-12-01 06:00)
On another try, I was able to install microtherion's fork through Swift Package Manager. The following code should work:
import Cocoa
import Alamofire
import ZIPFoundation
... // ignoring irrelevant parts of the code
let zipURL = URL(string: "https://github.com/KSP-CKAN/CKAN-meta/archive/master.zip")!
AF.request(zipURL).validate().responseData { response in
// a Data variable that holds the raw bytes
var zipData: Data? = response.data
// an Archive instance created with the Data variable
var zipArchive = Archive(data: zipData!, accessMode: .read)
// iterate over the entries in the Archive instance, and extract each entry into a Data variable
for entry in zipArchive! {
var unzippedData: Data
do {
_ = try zipArchive?.extract(entry) {unzippedData($0)}
} catch {
...
}
...
}
}
I wrote the file transferring code as follows:
val fileContent: Enumerator[Array[Byte]] = Enumerator.fromFile(file)
val size = file.length.toString
file.delete // (1) THE FILE IS TEMPORARY SO SHOULD BE DELETED
SimpleResult(
header = ResponseHeader(200, Map(CONTENT_LENGTH -> size, CONTENT_TYPE -> "application/pdf")),
body = fileContent)
This code works successfully, even if the file size is rather large (2.6 MB),
but I'm confused because my understanding about .fromFile() is a wrapper of fromCallBack() and SimpleResult actually reads the file buffred,but the file is deleted before that.
MY easy assumption is that java.io.File.delete waits until the file gets released after the chunk reading completed, but I have never heard of that process of Java File class,
Or .fromFile() has already loaded all lines to the Enumerator instance, but it's against the fromCallBack() spec, I think.
Does anybody knows about this mechanism?
I'm guessing you are on some kind of a Unix system, OSX or Linux for example.
On a Unix:y system you can actually delete a file that is open, any filesystem entry is just a link to the actual file, and so is a file handle which you get when you open a file. The file contents won't become unreachable /deleted until the last link to it is removed.
So: it will no longer show up in the filesystem after you do file.delete but you can still read it using the InputStream that was created in Enumerator.fromFile(file) since that created a file handle. (On Linux you actually can find it through the special /proc filesystem which, among other things, contains the filehandles of each running process)
On windows I think you will get an error though, so if it is to run on multiple platforms you should probably check test your webapp on windows as well.
Obj-C or Monotouch.Net C# answers are fine.
I have a Base64 string that is a PDF document received over a web service. I can get the NSData.
How do I take the NSData and save it as a PDF?
-- I get the NSData this way --
byte[] encodedDataAsBytes = System.Convert.FromBase64String (myBase64String);
string decoded = System.Text.Encoding.Unicode.GetString (encodedDataAsBytes);
NSData data = NSData.FromString (decoded, NSStringEncoding.ASCIIStringEncoding);
The simplest way to save it is probably to use NSData's writeToFile:options:error: method.
I found that using the .NET framework works better than trying to use the iOS framework for this problem. This will take any file and convert it to it's original then save it to the iPhone/iPad device. "path" is just a folder on the dev ice.
using (var f = System.IO.File.Create (path))
{
byte[] encodedDataAsBytes = System.Convert.FromBase64String (Base64PDFString);
f.Write (encodedDataAsBytes, 0, encodedDataAsBytes.Length);
}
I'm working on a project where I recently had to accomplish the same thing you are describing. I get base64 encoded PDF files as strings from a .NET web service which need to be decoded to their original and saved as PDF files in the applications documents directory.
My solution was:
Use ASIHTTPRequest to communicate with the web service.
I then use TBXML to parse incoming xml and get the base64 as an NSString.
To decode the string I use a method from QSUtilities library called decodeBase64WithString.
Finally I save the result with NSData's writeToFile.
I have tested and successfully used this method with PDF files that are up to 25mb. I also had a couple of test runs with a 48mb file but that file made the decodeBase64WithString method take up too much memory and my app crashed. Haven't found a solution to this yet..
If you are working with multiple large files be sure to free up your memory once in a while. I got all my files in one loop in which I had to use my own nsautorelease pool and drain it at the end of the loop to free up any autoreleased objects.