Swift: Play a local video and apply CIFilter in realtime - swift

Question

I'm trying to play a local video and apply a CIFilter in realtime with no lag. How can I do that? I already know how to apply a CIFilter to a video in AVPlayer but the performance it's not as fast as I want.
This is my current code:
#objc func exposure(slider: UISlider, event: UIEvent) {
if let touchEvent = event.allTouches?.first {
switch touchEvent.phase {
case .moved:
player.currentItem?.videoComposition = AVVideoComposition(asset: player.currentItem!.asset, applyingCIFiltersWithHandler: { request in
let exposureFilter = CIFilter.exposureAdjust()
exposureFilter.inputImage = request.sourceImage.clampedToExtent()
exposureFilter.ev = slider.value
let output = self.exposureFilter.outputImage!.cropped(to: request.sourceImage.extent)
// Provide the filter output to the composition
request.finish(with: output, context: nil)
})
default:
break
}
}
}

Answer 1

The problem is that you re-create and re-assign the video composition to the player item every time the slider value changes. This is very costly and unnecessary. You can do the following instead:
Create the filter somewhere outside the composition block and keep a reference to it, for instance in a property.
Also, create the composition only once and let it apply the referenced filter (instead of creating a new one with every frame).
When the slider value changes, only set the corresponding parameter value of the filter. The next time the composition will render a frame, it will automatically use the new parameter value because it uses a reference to the just-changed filter.
Something like this:
let exposureFilter = CIFilter.exposureAdjust()
init() {
// set initial composition
self.updateComposition()
}
func updateComposition() {
player.currentItem?.videoComposition = AVVideoComposition(asset: player.currentItem!.asset, applyingCIFiltersWithHandler: { request in
self.exposureFilter.inputImage = request.sourceImage.clampedToExtent()
let output = self.exposureFilter.outputImage!.cropped(to: request.sourceImage.extent)
request.finish(with: output, context: nil)
})
}
#objc func exposureChanged(slider: UISlider) {
self.exposureFilter.ev = slider.value
// we need to re-set the composition if the player is paused to cause an update (see remark below)
if player.rate == 0.0 {
self.updateComposition()
}
}
(By the way, you can just do slider.addTarget(self, action:#selector(exposureChanged(slider:)), for: .valueChanged) to get notified when the slider value changes. No need to evaluate events.)
One final note: There actually is a use case when you want to re-assign the composition, which is when the player is currently paused but you still want to show a preview of the current frame with the filter values change. Please refer to this technical note from Apple on how to do that.