How to get frames from a local video file in Swift? - swift

Question

I need to get the frames from a local video file so i can process them before the video is played. I already tried using AVAssetReader and VideoOutput.
[EDIT] Here is the code i used from Accesing Individual Frames using AV Player
let asset = AVAsset(URL: inputUrl)
let reader = try! AVAssetReader(asset: asset)
let videoTrack = asset.tracksWithMediaType(AVMediaTypeVideo)[0]
// read video frames as BGRA
let trackReaderOutput = AVAssetReaderTrackOutput(track: videoTrack, outputSettings:[String(kCVPixelBufferPixelFormatTypeKey): NSNumber(unsignedInt: kCVPixelFormatType_32BGRA)])
reader.addOutput(trackReaderOutput)
reader.startReading()
while let sampleBuffer = trackReaderOutput.copyNextSampleBuffer() {
print("sample at time \(CMSampleBufferGetPresentationTimeStamp(sampleBuffer))")
if let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) {
// process each CVPixelBufferRef here
// see CVPixelBufferGetWidth, CVPixelBufferLockBaseAddress, CVPixelBufferGetBaseAddress, etc
}
}

Answer 1

I believe AVAssetReader should work. What did you try? Have you seen this sample code from Apple? https://developer.apple.com/library/content/samplecode/ReaderWriter/Introduction/Intro.html

Answer 2

I found out what the problem was! It was with my implementation. The code i posted is correct. Thank you all

Answer 3

You can have a look at VideoToolbox : https://developer.apple.com/documentation/videotoolbox
But beware: this is close to the hardware decompressor and sparsely documented terrain.

Answer 4

Depending on what processing you want to do, OpenCV may be a an option - in particular if you are detecting or tracking objets in your frames. If your needs are simpler, then the effort to use OpenCV with swift may be a little too much - see below.
You can open a video, read it frame by frame, do your work on the frames and then display then - bearing in mind the need to be efficient to avoid delaying the display.
The basic code structure is quite simple - this is a python example but the same principles apply across supported languages
import numpy as np
import cv2
cap = cv2.VideoCapture('vtest.avi')
while(cap.isOpened()):
ret, frame = cap.read()
//Do whatever work you want on the frame here - in this example
//from the tutorial the image is being converted from one colour
//space to another
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
//This displays the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
break
cap.release()
cv2.destroyAllWindows()
More info here: http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_gui/py_video_display/py_video_display.html
The one caveat is that using OpenCV with swift requires some additional effort - this is a good example, but it evolves constantly so it is worth searching for if you decide to go this way: https://medium.com/#yiweini/opencv-with-swift-step-by-step-c3cc1d1ee5f1