I have been browsing the web, even this website... bu cannot find a good option to implement ffmpeg functionality in an iOS application made in swift.
Options looked at and reasons why they are not solutions:
SwiftFFmpeg - I am not sure it can run on iOS, plus I don't see an option to run my desired Ffmpeg command.
MobileFFmpeg - Not maintained anymore, and a new project by the same founders created a "better altenrative" called ffmpeg-kit.
ffmpeg-kit - looks amazing, but their API only allows for interaction in the Objective-C language.
Any solutions anyone can give me?
First...
Make sure you understand what "packages" are available. You can build it yourself, but to be honest, unless you have a very specific reason, I'd just use the pre-build packages. See "8. Packages" of the ffmpeg-kit README.MD to see what's on offer
Second
Go to the Releases and find the version you're interested (I used FFmpegKit Native v4.5.1), scroll down and expand the "Assets" list for the release you're interested in.
For this experiment I used ffmpeg-kit-full-4.5.1-macos-xcframework.zip. If you're doing this in iOS, the workflow is basically the same, you just need to take into account that your file access is sandboxed (and yes, I've done this, and yes, it takes a long time to transcode video, compared to a desktop)
Third
Create a new Xcode project. Again, for this experiment, I created a "MacOS App" using "Storyboards" as the UI (you could try using SwiftUI, but that's another layer of complexity this example didn't need right now)
Unzip the *xcframework.zip file you download from the last step.
In Xcode, select the "project" node, select the MacOS target (🤞 there's only one target).
Select "General", drag the *.xcframework folders from Finder to the "Frameworks, Libraries and Embedded Content" section of your project
Forth
For this experiment, I opened the ViewController class (which was automatically created by Xcode) and simply added...
func syncCommand() {
guard let session = FFmpegKit.execute("-i file1.mp4 -c:v file2.mp4") else {
print("!! Failed to create session")
return
}
let returnCode = session.getReturnCode()
if ReturnCode.isSuccess(returnCode) {
} else if ReturnCode.isCancel(returnCode) {
} else {
print("Command failed with state \(FFmpegKitConfig.sessionState(toString: session.getState()) ?? "Unknown") and rc \(returnCode?.description ?? "Unknown").\(session.getFailStackTrace() ?? "Unknown")")
}
}
func asyncCommand() {
FFmpegKit.executeAsync("-i file1.mp4 -c:v file2.mp4") { session in
guard let session = session else {
print("!! Invalid session")
return
}
guard let returnCode = session.getReturnCode() else {
print("!! Invalid return code")
return
}
print("FFmpeg process exited with state \(FFmpegKitConfig.sessionState(toString: session.getState()) ?? "Unknown") and rc \(returnCode).\(session.getFailStackTrace() ?? "Unknown")")
} withLogCallback: { logs in
guard let logs = logs else { return }
// CALLED WHEN SESSION PRINTS LOGS
} withStatisticsCallback: { stats in
guard let stats = stats else { return }
// CALLED WHEN SESSION GENERATES STATISTICS
}
}
The code above is basically the "2. Execute synchronous FFmpeg commands." and "4. Execute asynchronous FFmpeg commands by providing session specific execute/log/session callbacks." examples from the ffmpeg-kit/apple documentation
!! Important !! - don't forget to add import ffmpegkit to the start of the file!
At this point, this should now compile (you'll get a couple of warnings about logs and stats not been used, you can ignore those).
After thoughts...
You should realise by now that the code I've provided won't actually run, for two reasons.
I've not actually called either func from anywhere (I tested it by placing it in the viewDidLoad func of the ViewController class)
The input file, used in the execute command, doesn't exist. You will need to provide an actual reference to an actual file, preferably with an absolute path. This, how ever, may require you to change the "App Sandbox" settings under the targets "Signing and Capabilities"
Xcodes auto code suggestions aren't bad and I mostly filled out the above using it, and the Obj-c code as a starting point.
Also, beware, SO is not a "tutorial" site, Xcode is a complex beast and you may need to spend some time exploring other resources to overcome issues you encounter
I have an app which is inconsistently returning 'nil' when using FileHandle to open a file for Read. I'm on OSX (10.13.4), XCode 9.4, Swift 4.1
This OSX app uses the NSOpenPanel() to get a list of files selected by the user. My 'model' class code opens these files to build a collection of data structures The code which does this starts out like this and successfully gets a FileHandle EVERY TIME for any file and is able to read data from the file.
private func getFITHeader(filename: String) {
let file: FileHandle? = FileHandle(forReadingAtPath: filename)
if file == nil {
print("FITFile >>> File open failed for file \(filename)")
}
else {
var databuffer: Data
databuffer = (file?.readData(ofLength: 80))!
:
:
}
The files also contain a block of binary data which I process in another part of the app. While I develop the code for this I'm temporarily hard coding one of the same filenames as works above for test purposes. BUT this code (below) ALWAYS throws an exception 'Thread 1: Fatal error: Unexpectedly found nil while unwrapping an Optional value' when it gets to fileHandle?.seek() - for some reason the attempt to create a FileHandle is always returning 'nil' despite the code being functionally identical to tha above.
#IBAction func btnProcFile(_ sender: Any) {
var data: Data
let filename = "/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit"
let fileHandle: FileHandle? = FileHandle(forReadingAtPath: filename)
fileHandle?.seek(toFileOffset: 2880) //skip past the headers
let dataLenToRead = 1391 * 1039 * 2
data = (fileHandle?.readData(ofLength: dataLenToRead))!
:
:
}
The code in the second function works fine in a Playground (not attaching too much meaning to that) and, wierdly, has also worked when temporarily added to a different project. Probably also worth mentioning the length of the file path doesn't seem to matter - it behaves the same on short paths.
So the question is - why is this behaviour of FileHandle reliably inconsistent?
print()'ing the filenames presented to FileHandle() showed they were identical in each case (see below). So I'm stumped and frustrated by this - any perspectives or workarounds would be appreciated.
/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit
/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit
Found the answer - Sandboxing !!
Darren - coincidentally I did look at the URL based route and discovering it 'throws' put some proper error reporting in the catches. Low and behold they reported I didn't have permissions on the file (which initially surprised me since I'm obviously admin on my Mac's and all the files ar local and under my username.
I bit more research turned up. this article - https://forums.developer.apple.com/thread/96062 which quickly revealed its a sandboxing problem :-) Looks like recent versions of XCode have it turned on in 'Entitlements'. The post also points out that the NSOpenPanel FileOpen dialog returns 'Security scoped urls'. At first I thought this explained why the code in the first function worked but I'm not totally convinced because I was only feeding the url.path property to FileHandle.
However, turning off Sandbox in Entitlements makes everything work just fine. Yes, I know thats not the right thing to do longer term (or if I want this to go to the App Store) so I'll be checking out the right way to do this. At least I can get on now - thanks for the input.
The FileHandle initializers are not well named.
You should use FileHandle(forReadingFrom:URL) instead of FileHandle(forReadingAtPath:String). The former is newer API that throws an error instead of returning nil. You can use the thrown error to see why it is failing, and your variables are guaranteed to be non-nil.
For example:
#IBAction func btnProcFile(_ sender: Any) {
do {
let fileUrl = URL(fileURLWithPath:"/Users/johncneal/Dropbox/JN Astronomy/Astronomy/Spectroscopy/RSpec_Analyses/Gamma_Cas/20161203/Gamma Cas_065.fit")
let fileHandle = try FileHandle(forReadingFrom: fileUrl)
fileHandle.seek(toFileOffset: 2880) //skip past the headers
let dataLenToRead = 1391 * 1039 * 2
let data: Data = fileHandle.readData(ofLength: dataLenToRead)
// etc...
} catch let error as NSError {
print("FITFile >>> File open failed: \(error)")
NSApp.presentError(error)
}
}
I am trying to use some old IOKit functionality in a new Swift 4.0 Mac app (not iOS). I have created a bridging header to use an existing Objective C third party framework, DDHidLib, and I am current working in Xcode 9.
The code that attempts to create a plug in interface for a usb gamepad falls over on IOCreatePlugInInterfaceForService, returning a non-zero error.
The truly bizarre thing is I have an older app created in a previous version of Xcode that uses the same framework and works correctly after opening in the new Xcode 9. This previous project is still Swift using a bridging header for the same Obj-C framework. I have checked the build settings and tried to make everything match, but I get the same result; the old app works but any new apps do not.
Is there a way to either: find out the exact differences in build settings/compilers to see what the elusive difference may be, OR to step into the IOCreatePlugInInterfaceForService IOKit method to see what may be causing the error to be returned in one project but not another?
EDIT: Here is the method that is failing:
- (BOOL) createDeviceInterfaceWithError: (NSError **) error_; {
io_name_t className;
IOCFPlugInInterface ** plugInInterface = NULL;
SInt32 score = 0;
NSError * error = nil;
BOOL result = NO;
mDeviceInterface = NULL;
NSXReturnError(IOObjectGetClass(mHidDevice, className));
if (error)
goto done;
NSXReturnError(IOCreatePlugInInterfaceForService(mHidDevice, kIOHIDDeviceUserClientTypeID,kIOCFPlugInInterfaceID,&plugInInterface,&score));
if (error)
goto done;
//Call a method of the intermediate plug-in to create the device interface
NSXReturnError((*plugInInterface)->QueryInterface(plugInInterface, CFUUIDGetUUIDBytes(kIOHIDDeviceInterfaceID), (LPVOID) &mDeviceInterface));
if (error)
goto done;
result = YES;
done:
if (plugInInterface != NULL)
{
(*plugInInterface)->Release(plugInInterface);
}
if (error_)
*error_ = error;
return result;
}
In the old version that works, IOCreatePlugInInterfaceForService always returns a value of 0. In all the versions that don't work, the return value appears to always be -536870210. The mHidDevice in this function is the io_object_t handle for the device.
EDIT2: Here is the IORegistryExplorer page for the device
Finally managed to resolve this after weeks of head scratching. The new Xcode 9 uses app sandboxing to basically prevent access to USB, bluetooth, camera and microphone etc. by default in a new app. Once I switched this off it reverted to it's expected behaviour.
Glad it was such a simple answer in the end but disappointed Xcode does not provide more descriptive error messages or responses to let a user know they are essentially preventing themselves from accessing the parts of the system they need.
Looks like kIOReturnNoResources is returned if the loop at the end of IOCreatePlugInInterfaceForService completes with haveOne == false for whatever reason. Perhaps Start() is returning false because another process or driver already has exclusive access? I'd check what clients the device has in IORegistryExplorer.
This error also happens when an application is trying to access the camera or bluetooth on MacOS 10.14 and higher. Permission shall be granted either explicitly by user (pop-up window), or through the Security & Privacy. The application should check for permission as shown here.
Here is an example of what I want to do:
func application(application: UIApplication, didFailToRegisterForRemoteNotificationsWithError error: NSError)
{
let nm = NetworkModel()
nm.sendlog("file name :AppDelegate , line number : 288", info: " Failed to register: \(error)")
}
current scenario i done that hard coded value line number and file name . but is it possible to programatically pick line number and file name .
Literal Type Value
#file String The name of the file in which it appears.
#line Int The line number on which it appears.
#column Int The column number in which it begins.
#function String The name of the declaration in which it appears.
#dsohandle UnsafeMutablePointer The dso handle.
Example
print("Function: \(#function), line: \(#line)")
With default values in parameters you can also create a function
public func track(_ message: String, file: String = #file, function: String = #function, line: Int = #line ) {
print("\(message) called from \(function) \(file):\(line)")
}
which can be used like this
track("enters app")
In Swift 2.1
Literal Type Value
__FILE__ String The name of the file in which it appears.
__LINE__ Int The line number on which it appears.
__COLUMN__ Int The column number in which it begins.
__FUNCTION__ String The name of the declaration in which it appears.
for more info see the documentation
You can use #function, #file, #line
Here is the implementation of log method in swift : https://github.com/InderKumarRathore/SwiftLog
Below is the snippet
public func debugLog(object: Any, functionName: String = #function, fileName: String = #file, lineNumber: Int = #line) {
#if DEBUG
let className = (fileName as NSString).lastPathComponent
print("<\(className)> \(functionName) [#\(lineNumber)]| \(object)\n")
#endif
}
For swift 4 and swift 5:
func printLog(_ message: String, file: String = #file, function: String = #function, line: Int = #line) {
#if DEVELOPMENT
let className = file.components(separatedBy: "/").last
print(" ❌ Error ----> File: \(className ?? ""), Function: \(function), Line: \(line), Message: \(message)")
#endif
}
// "❌ Error ----> File: classNameViewController.swift, function: functionName(), Line: 123, Message: messageError"
Example Logging Code (TL;DR)
import os.log
#available(OSX 11.0, iOS 14.0, *)
extension Logger {
private static var subsystem = Bundle.main.bundleIdentifier!
/// Logs the payment flows like Apple Pay.
#available(OSX 11.0, iOS 14.0, *)
static let payments = Logger(subsystem: subsystem, category: "payments")
}
static func DLog(message: StaticString, file: StaticString = #file, function: StaticString = #function, line: UInt = #line, column: UInt = #column, category: String, type: OSLogType = .info, bundle: Bundle = .main) {
// This method is only for iOS 14+
if #available(OSX 11.0, iOS 14.0, *) {
Logger.payments.debug("\(file) : \(function) : \(line) : \(column) - \(message, privacy: .private)")
// This makes the message unreadable without a debugger attached.
} else {
// Fallback on earlier versions
let customLog = OSLog(subsystem: bundle.bundleIdentifier!,
category: category)
// IMPORTANT: I have assumed here that you only print out non-sensitive data! Using %{private}# or %{public}# in an interpolated string is not portable to `Logger`!
os_log(message, log: customLog, type: type)
// Unfortunately this legacy API doesn't support non-StaticString logs. :(
}
}
Note that you may need to rework this code for more flexibility over private/public access levels - os_log and Logger don’t handle privacy levels in the same way. This is just to show how to use the API if you don’t add os_log privacy levels in message.
Apple recommend using OS Logging which is why I have used this approach rather than print statements. I added this answer because of the new Logger API in iOS 14 that also enables string interpolation.
Motivation
I think the answers here solve the question for < iOS 14, but they can be improved further for memory efficiency since this Debug Logging function will likely be used all across a codebase in iOS 14+ (new Logger API). Apple recommend using OS Logging which is why I have used this approach rather than print statements (although these will still work).
These are minor improvements but I think they can help because even minor improvements add up. In fact, the Swift Standard Library uses these optimizations everywhere they can)! These optimisations are contributing factors to the incredible memory efficiency of Swift despite being a high-level language (and part of great API Design!). (-:
Because this function would probably fit well as part of a generalized (OS) Logging service, I have also included considerations I make when logging within apps. I think these could help you when logging for debugging purposes (I answered this question because it was probably for debug logging!).
Improvements
I prefer to use UInt in certain situations if I know the Int is going to be positive, and any edge case crashes are unlikely. Note, I don't use UInt when interfacing with Foundation classes that use Int types for this reason. This allocates less memory at runtime and I find being more specific also helps me understand the code better.
I prefer to use StaticString where I know the string is known at compile-time. From the docs, StaticString only provides low-level access to String contents and is immutable. Thus only use it where appropriate. Even concatenated String literals cannot be used to initialize a StaticString. Because its functionality is more restricted than String this means that it is lightweight - only requiring an address pointer and length under the hood. Using StaticString also provides a small performance boost with OS-level memory management because it is neither allocated nor deallocated (no need for reference counting).
Using Apple's recommended Logger API for logging is better than print statements for multiple reasons: performance, privacy and unified management. It can be better for debugging issues in released products (OS Logs provide more context).
I would recommend using StaticString where you can, for example with the function and file names, and UInt for the line and column numbers. This is only now possible with the Logger iOS 14+ API.
(Bonus) Logging Considerations
Should I do LOTS of logging?
I think logging is a balancing act. Lots of logging can be very helpful to debug issues when viewing crash reports if you have access to these. However, you need to balance Privacy, Size of the Compiled Binary and Overwhelming the Logging System.
1. Privacy: Redact sensitive info using "\(message, privacy: .private)" and "\(message, privacy: .public)" with Logger, and DON'T print sensitive information when using NSLog or print statements. I would recommend replacing print statements (for production) with OS Logs when they are for debugging purposes. Or use a preprocessor directive (#if DEBUG) inside the logging service to print only on a Debug scheme..
2. Size of the Compiled Binary: Logging statements are essentially more lines of code. The more code you add, the bigger your binary. This is usually not an issue, as long as you don't log everything under the sun since only a large increase in this metric would affect the binary size.
3. Overwhelming the logging system: If you log excessively, the device may have to discard some logs in order to keep running (limited RAM or swap space at runtime due to OS multitasking - worse for older devices). Apple is usually reliable in handling lots of user logging, and it's hard to achieve this in practice. Nevertheless, if you do encounter this, some more useful log events could be pushed out of the window.
OS Logs
The most important thing to get right is the log level. By default on iOS log entries at .info and below will be suppressed at the point of log generation, so the only real negative with those is the binary size. If you log at .log (.default for os_log) or higher, you need to be careful about not logging too much. Apple's general advice is that you look at each of these higher-level log entries to make sure they contain info that might be useful while debugging.
Finally, make sure to set the subsystem and category. The unified logging system processes a lot of log entries - the subsystem with category makes it much easier for you to focus on specific problems.
How to choose the category and subsystem?
This is a pretty opinionated topic, so I will focus on what I would think about when deciding. YMMV: there may be better options and if so, please let me know.
1. By feature (product-specific): For example, if you are a shopping app, maybe organise the subsystem by the payment, login or other app flows. If you aren't already using a modular codebase for organisation into features (or frameworks for shared code) then I can recommend this tutorial series.
2. By technology: What I mean by this is the domain of the code, is it for Push Notifications, Deep Links, User Defaults (or persistence) logic. This could be helpful for the category parameter.
3. By target: I like to also use the bundleIdentifier of the current Bundle if I can for the subsystem. This makes even more sense if you have Extension Targets (Push Notifications), multiple Targets, or different Platforms (like WatchOS, SiriKit extensions and similar). You can get the current bundle, using this code:
let myBundle = Bundle(for: MyClass.self)
Debugging user issues using OS device logs
See here for a detailed walkthrough (Access Device Console Logs). The main idea is to connect the device to your Mac (or use the Mac itself for Mac apps ;)), open Console.app, run the app and try to reproduce the crash. If you manage to get that far in the first place, then you can correlate the time of the crash with the logs for more context.
Any other reasons for logging less?
If you are an Apple Early-Adopter then you know how buggy some of the iOS Betas can be ;). When posting bug reports to Apple, the OS Logs get included in sysdiagnose files, so you may get a faster turnaround time because your bug reports have less noise.
Should I use a third-party logging service?
Depends if you can afford it and whether you really need it. I prefer to minimise the number of third-party dependencies I import in my code (if I can control this) and the native (Apple) Logging APIs worked fine for most of my use cases. These services charge extra per month, but the advantage is the user cannot view the logs (even if he wants to) by hooking his device up to Console.app on his Mac. Note that this is NOT an issue if you apply the .private log levels, so I don't usually use third-party services (YMMV). A potential reason to go for web-logging is more security against reverse-engineering though, as long as you trust the third-party vendor against data breaches (BUT also it's not as eco-friendly ;)).
static func DLog(message: String, file: String = #file, function: String = #function, line: Int = #line, column: Int = #column) {
print("\(file) : \(function) : \(line) : \(column) - \(message)")
}
add where ever you want in your code (inside a struct/var/closure/func...) the next line:
let _ = print("**** \(#file)" + String(describing: type(of: self)) + ".\(#function)[\(#line)]")
and it will print the next line:
**** /path/to/my/file.swift MyClass.someFunc()[17]
I am trying to instantiate a SB class using Swift but it doesn't seem to work:
if let messageClass = (mail as! SBApplication).class(forScriptingClass:"outgoing message") {
let message = (messageClass as! SBObject.Type).init(properties: ["subject": "message subjects"]) as MailOutgoingMessage
mail.outgoingMessages!().add(message)
print("Subject: \(message.subject)")
print("Outgoing messages: \(mail.outgoingMessages!().count)")
}
All I get in the output is:
Subject: nil
Outgoing messages: 0
I know that I should cast the message to a MailOutgoingMessage.type and not SBObject.type but I couldn't access the init method otherwise.
Someone has experience using Scripting Bridge with Swift? Clues?
Scripting Bridge is nasty, obfuscated and broken. If you have to use SB, Tony Ingraldi wrote some helper tools for Swift users. TBH, I recommend using AppleScript-ObjC over SB simply because AppleScript at least works.
For a native Swift-AE bridge, see SwiftAutomation. I'm still working on the documentation, but the code itself descends from appscript which is the only AppleScript alternative of the last 20 years that actually works right, so is just about beta-quality now.
Swift (and AppleScript) users who'd like to see Apple adopt SwiftAutomation in 10.3 are strongly encouraged to submit duplicate Radar tickets to bugreport.apple.com requesting it, e.g. here's one already copied to OpenRadar if you want to copy-paste it as-is (include the original rdar:// to help the Radar team resolve it quicker): openradar.appspot.com/29332915