I have encountered a very unusual problem with lazy initialization.
My application reads metadata from an mp4 file, edits some of the data, and writes it back to the file. In previous versions, I read all the data when the file was opened. In the current version, I have switched to lazy reading of the data to improve performance and lower the memory footprint. However, I have encountered a very sporadic but reproducible bug, in which a few bytes seem to be read incorrectly from the file during lazy initialization.
Here is the initialization code:
lazy var data: Data = { [unowned self] in self.readDataFromFile() }()
func readDataFromFile() -> Data {
guard let file = fileHandle else { return Data() }
file.seek(toFileOffset: dataFilePointer)
let data = file.readData(ofLength: dataCountInFile)
print(Pointer: \(dataFilePointer)" + " Data: " + data.hexBytes(length: 32))
return data
}
The print statement was added to debug this problem. The fileHandle is a FileHandle object which I create when the file is selected. The dataFilePointer and dataCountInFile variables are set when the file is opened, by parsing the file for the pointers to the data.
When I read from the file for the first time, this is what I see in the log:
Pointer: 6808113917 Data: 14600000 1EA70000 12FB0000 19260000 12E20000
If I stop app without writing any data to the file, and then restart, not doing anything else, this is what I see in the log:
Pointer: 6808113917 Data: CFCB378C CFCBCDDB 00015F90 4B4231AA 55C40000
This is actually the correct data. Subsequent running of the app consistently shows the correct data.
I have put breakpoints at all points where the file could be written to verify that no data is written to the file.
Additional observations:
This bug only occurs with a few selected files. The vast majority of
files are read correctly.
This bug only occurs with Release build configuration. It does not occur in Debug configuration. That is why
I have had to use the print statement to observe what is happening.
This bug does not occur if I do not initialize the data var lazily,
i.e. if I immediately read the data from the file when the object is
initialized.
I am having a difficult time understanding what could be causing this strange behavior.
Related
How to change the .xcdatamodeld file i.e. the data model?
Since the program has already been run and the Persistent Store Coordinator (PSC) contains a url to .sqlite, .sqlite-shm and .sqlite-wal files on disk I think the process is as follows but am unsure. Any input would be appreciated.
Run code below to delete the url from PSC.
Delete sqlite files from disk.
Change .xcdatamodeld file.
CodeGen is set to manual so create new managed object subclasses.
Make appropriate changes to code.
Run program which I assume will enter a url into the PSC and create the 3 sqlite files on disk but now based on the new .xcdatamodeld file.
func deletePersistentStore() {
guard let persistentStoreURL = container.persistentStoreCoordinator.persistentStores.first?.url
else {
print("URL Missing")
return
}
do {
try container.persistentStoreCoordinator.destroyPersistentStore(
at: persistentStoreURL,
ofType: "SQLite",
options: nil)
} catch {
print("Persistent Store Not Deleted: \(error) - \(error.localizedDescription)")
}
print("\(container.persistentStoreCoordinator.persistentStores.count)")
// prints 0
print("\(String(describing: container.persistentStoreCoordinator.persistentStores.first?.url) )")
// prints nil
}
It sounds like you’re still developing this app and that it’s not released yet. If that’s true, you could do something like this. But it would be easier to delete the app from your phone (or simulator), change the model, and then install a new copy of the app.
That would let you skip steps 1 and 2.
If your app is already released, you should look into Core Data model migration. It’s a process that lets you update the data model without deleting existing data. In most cases it’s nearly automatic, but it depends on how much your model is changing.
My code needs to parse heaps of JSON files, and those files are hosted on GitHub and only available bundled as 1 single ZIP file. Because the ZIP file is only about 80 MB, I want to keep the entire unzipping operation in memory.
I'm able to load the ZIP file into memory as a Data? variable, but I'm not able to find a way to unzip a Data variable in memory and then assign the unzipped file/data to some other variables. I've tried using ZIP Foundation, but its Archive type's initializers take only file URLs. I didn't try Zip, but its documentation shows that it takes file URLs as well.
Here is my code:
import Cocoa
import Alamofire
class ViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let zipURL = URL(string: "https://github.com/KSP-CKAN/CKAN-meta/archive/master.zip")!
AF.request(zipURL).validate().responseData { response in
var zipData: Data? = response.data
// Here I want to unzip `zipData` after unwrapping it.
}
}
}
I also looked into passing a Data variable off as a file, but failed to find a way to do it.
UPDATE (2019-12-01 05:00)
According to this pull request thread on ZIPFoundation, the feature I'm looking for will be included in the next release. I tried to use the feature's contributor's fork, but somehow Swift Package Manager wouldn't allow it.
Before finding this, I tried using Python's zipfile library through Swift-Python interoperability provided by PythonKit, but it didn't work out, because Foundation's Data in Swift can not be cast into a PythonObject type.
Apple's Compression framework also looked promising, but it seems to have a soft limit of 1 MB on compressed files. The compressed file I need is about 80 MB, way larger than 1 MB.
So far, ZIPFoundation is my most hopeful solution.
UPDATE (2019-12-01 06:00)
On another try, I was able to install microtherion's fork through Swift Package Manager. The following code should work:
import Cocoa
import Alamofire
import ZIPFoundation
... // ignoring irrelevant parts of the code
let zipURL = URL(string: "https://github.com/KSP-CKAN/CKAN-meta/archive/master.zip")!
AF.request(zipURL).validate().responseData { response in
// a Data variable that holds the raw bytes
var zipData: Data? = response.data
// an Archive instance created with the Data variable
var zipArchive = Archive(data: zipData!, accessMode: .read)
// iterate over the entries in the Archive instance, and extract each entry into a Data variable
for entry in zipArchive! {
var unzippedData: Data
do {
_ = try zipArchive?.extract(entry) {unzippedData($0)}
} catch {
...
}
...
}
}
I have an app which is inconsistently returning 'nil' when using FileHandle to open a file for Read. I'm on OSX (10.13.4), XCode 9.4, Swift 4.1
This OSX app uses the NSOpenPanel() to get a list of files selected by the user. My 'model' class code opens these files to build a collection of data structures The code which does this starts out like this and successfully gets a FileHandle EVERY TIME for any file and is able to read data from the file.
private func getFITHeader(filename: String) {
let file: FileHandle? = FileHandle(forReadingAtPath: filename)
if file == nil {
print("FITFile >>> File open failed for file \(filename)")
}
else {
var databuffer: Data
databuffer = (file?.readData(ofLength: 80))!
:
:
}
The files also contain a block of binary data which I process in another part of the app. While I develop the code for this I'm temporarily hard coding one of the same filenames as works above for test purposes. BUT this code (below) ALWAYS throws an exception 'Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value' when it gets to fileHandle?.seek() - for some reason the attempt to create a FileHandle is always returning 'nil' despite the code being functionally identical to tha above.
#IBAction func btnProcFile(_ sender: Any) {
var data: Data
let filename = "/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit"
let fileHandle: FileHandle? = FileHandle(forReadingAtPath: filename)
fileHandle?.seek(toFileOffset: 2880) //skip past the headers
let dataLenToRead = 1391 * 1039 * 2
data = (fileHandle?.readData(ofLength: dataLenToRead))!
:
:
}
The code in the second function works fine in a Playground (not attaching too much meaning to that) and, wierdly, has also worked when temporarily added to a different project. Probably also worth mentioning the length of the file path doesn't seem to matter - it behaves the same on short paths.
So the question is - why is this behaviour of FileHandle reliably inconsistent?
print()'ing the filenames presented to FileHandle() showed they were identical in each case (see below). So I'm stumped and frustrated by this - any perspectives or workarounds would be appreciated.
/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit
/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit
Found the answer - Sandboxing !!
Darren - coincidentally I did look at the URL based route and discovering it 'throws' put some proper error reporting in the catches. Low and behold they reported I didn't have permissions on the file (which initially surprised me since I'm obviously admin on my Mac's and all the files ar local and under my username.
I bit more research turned up. this article - https://forums.developer.apple.com/thread/96062 which quickly revealed its a sandboxing problem :-) Looks like recent versions of XCode have it turned on in 'Entitlements'. The post also points out that the NSOpenPanel FileOpen dialog returns 'Security scoped urls'. At first I thought this explained why the code in the first function worked but I'm not totally convinced because I was only feeding the url.path property to FileHandle.
However, turning off Sandbox in Entitlements makes everything work just fine. Yes, I know thats not the right thing to do longer term (or if I want this to go to the App Store) so I'll be checking out the right way to do this. At least I can get on now - thanks for the input.
The FileHandle initializers are not well named.
You should use FileHandle(forReadingFrom:URL) instead of FileHandle(forReadingAtPath:String). The former is newer API that throws an error instead of returning nil. You can use the thrown error to see why it is failing, and your variables are guaranteed to be non-nil.
For example:
#IBAction func btnProcFile(_ sender: Any) {
do {
let fileUrl = URL(fileURLWithPath:"/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit")
let fileHandle = try FileHandle(forReadingFrom: fileUrl)
fileHandle.seek(toFileOffset: 2880) //skip past the headers
let dataLenToRead = 1391 * 1039 * 2
let data: Data = fileHandle.readData(ofLength: dataLenToRead)
// etc...
} catch let error as NSError {
print("FITFile >>> File open failed: \(error)")
NSApp.presentError(error)
}
}
I'm running an mlmodel that is coming from keras on an iPhone 6. The predictions often fails with the error Error computing NN outputs. Does anyone know what could be the cause and if there is anything I can do about it?
do {
return try model.prediction(input1: input)
} catch let err {
fatalError(err.localizedDescription) // Error computing NN outputs error
}
EDIT: I tried apple's sample project and that one works in the background so it seems it's specific to either our project or model type.
I got the same error myself at similar "seemingly random" times. A bit of debug tracing established that it was caused by the app sometimes trying to load its coreml model when it was sent to background, then crashing or freezing when reloaded into foreground.
The message Error computing NN outputs error was preceded by:
Execution of the command buffer was aborted due to an error during execution. Insufficient Permission (to submit GPU work from background) (IOAF code 6)
I didn't need (or want) the model to be used when the app was in background, so I detected when the app was going in / out of background, set a flag and used a guard statement before attempting to call the model.
Detect when going into background using applicationWillResignActive within the AppDelegate.swift file and set a Bool flag e.g. appInBackground = true. See this for more info: Detect iOS app entering background
Detect when app re-enters foreground using applicationDidBecomeActive in the same AppDelegate.swift file, and reset flag appInBackground = false
Then in the function where you call the model, just before calling model, use a statement such as:
guard appInBackground == false else { return } // new line to add
guard let model = try? VNCoreMLModel(for modelName.model) else { fatalError("could not load model") // original line to load model
I doubt this is the most elegant solution, but it worked for me.
I haven't established why the attempt to load the model in background only happens sometimes.
In the Apple example you link to, it looks like their app only ever calls the model in response to a user input, so it will never try to load the model when in background. Hence the difference in my case ... and possibly yours as well?
In the end it was enough for us to set the usesCPUOnly flag. Using the GPU in the background seems prohibited in iOS. Apple actually wrote about this in their documentation as well. To specify this flag we couldn't use the generated model class anymore but had to call the raw coreml classes instead. I can imagine this changing in a future version however. The snippet below is taken from the generated model class, but with the added MLPredictionOptions specified.
let options = MLPredictionOptions()
options.usesCPUOnly = true // Can't use GPU in the background
// Copied from from the generated model class
let input = model_input(input: mlMultiArray)
let output = try generatedModel.model.prediction(from: input, options: options)
let result = model_output(output: output.featureValue(for: "output")!.multiArrayValue!).output
I was trying to add autosaving to my Core Data-based app and there was a line of code I added to textDidChange in a CollectionViewItem:
theNote?.updateChangeCount(NSDocumentChangeType.ChangeDone)
That gave me a lot of errors, so I commented out the line. I then went in to delete the CocoaAppCD.storedata persistent store file to make it cleaner (I'm still in early dev stages so all that goes in my persistent store is random test materials).
Now, I'm finding, Core Data commands crash my ViewController. Specifically this function:
func createNewNotebook(folderURL: NSURL)
{
let currentNotebook = Notebook(entity: sv.noteType, insertIntoManagedObjectContext:sv.context)
currentNotebook.folderURLstring = folderURL.absoluteString
let noteSet = currentNotebook.mutableSetValueForKey("contains")
print(String(noteSet))
intakeFilesFromFolder(noteSet, currentFolderURL: folderURL)
}
(Notebook and NoteEntity are the two entity types in my Core Data model.) When either the currentNotebook.folder... or let noteSet... commands run, I get these errors in my console:
Brouillon.NoteEntity folderURLstring]: unrecognized selector sent to instance 0x6080000a3a20
2016-07-24 19:27:02.622 Brouillon[8006:361665] Failed to set (contentViewController) user defined inspected property on (NSWindow): -[Brouillon.NoteEntity folderURLstring]: unrecognized selector sent to instance 0x6080000a3a20
and the WindowController is empty with no view filling it (even though these same statements had worked before I added the now-commented-out line). But if I bypass this function so the Core Data statements don't run, the views load. I would have thought that if I had anything dirty left in the database, deleting the CocoaAppCD.storedata file should have fixed it (and I keep deleting that file after every run). But it seems like something in Core Data is still mucked up for me - any ideas?
I figured this out. The problem was I had declared the wrong entity type: my sv.noteType should have been sv.notebookType, since it was a Notebook rather than a NoteEntity. (The error would have been introduced when I was creating the class instantiated as sv, as a class to hold my otherwise-duplicated code.)