How to record a video and make it slow motion - iphone

I am working on an iPhone app for school and need some help. The app should record video, make it slow motion (about 2x), then save it to the photo library. So far I have everything except how to make the video slow motion. I know it can be done as there is already an app in the App Store that does it.
How can I take a video I've saved to a temp url and adjust the speed before saving it to the photo library?

If you need to export your video then you need to use the AVMutableComposition Class
Then add your video as an AVAsset to an AVMutableComposition and scale it with:
- (void)scaleTimeRange:(CMTimeRange)timeRange toDuration:(CMTime)duration
Finally you export it using AVAssetExportSession Class

I written a code that makes your video in "slow motion" and saves it in Photos Library. "Main Thing This Code Works In Swift 5". Creating "Slow motion" video in iOS swift is not easy, that I came across many "slow motion" that came to know not working or some of the codes in them are depreciated. And so I finally figured a way to make slow motion in Swift.
This code can be used for 120fps are greater than that too. Just add the url of your video and make it slow
Here is the "code snippet I created for achieving slow motion"
func slowMotion(pathUrl: URL) {
let videoAsset = AVURLAsset.init(url: pathUrl, options: nil)
let currentAsset = AVAsset.init(url: pathUrl)
let vdoTrack = currentAsset.tracks(withMediaType: .video)[0]
let mixComposition = AVMutableComposition()
let compositionVideoTrack = mixComposition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
let videoInsertError: Error? = nil
var videoInsertResult = false
do {
try compositionVideoTrack?.insertTimeRange(
CMTimeRangeMake(start: .zero, duration: videoAsset.duration),
of: videoAsset.tracks(withMediaType: .video)[0],
at: .zero)
videoInsertResult = true
} catch let videoInsertError {
}
if !videoInsertResult || videoInsertError != nil {
//handle error
return
}
var duration: CMTime = .zero
duration = CMTimeAdd(duration, currentAsset.duration)
//MARK: You see this constant (videoScaleFactor) this helps in achieving the slow motion that you wanted. This increases the time scale of the video that makes slow motion
// just increase the videoScaleFactor value in order to play video in higher frames rates(more slowly)
let videoScaleFactor = 2.0
let videoDuration = videoAsset.duration
compositionVideoTrack?.scaleTimeRange(
CMTimeRangeMake(start: .zero, duration: videoDuration),
toDuration: CMTimeMake(value: videoDuration.value * Int64(videoScaleFactor), timescale: videoDuration.timescale))
compositionVideoTrack?.preferredTransform = vdoTrack.preferredTransform
let dirPaths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).map(\.path)
let docsDir = dirPaths[0]
let outputFilePath = URL(fileURLWithPath: docsDir).appendingPathComponent("slowMotion\(UUID().uuidString).mp4").path
if FileManager.default.fileExists(atPath: outputFilePath) {
do {
try FileManager.default.removeItem(atPath: outputFilePath)
} catch {
}
}
let filePath = URL(fileURLWithPath: outputFilePath)
let assetExport = AVAssetExportSession(
asset: mixComposition,
presetName: AVAssetExportPresetHighestQuality)
assetExport?.outputURL = filePath
assetExport?.outputFileType = .mp4
assetExport?.exportAsynchronously(completionHandler: {
switch assetExport?.status {
case .failed:
print("asset output media url = \(String(describing: assetExport?.outputURL))")
print("Export session faiied with error: \(String(describing: assetExport?.error))")
DispatchQueue.main.async(execute: {
// completion(nil);
})
case .completed:
print("Successful")
let outputURL = assetExport!.outputURL
print("url path = \(String(describing: outputURL))")
PHPhotoLibrary.shared().performChanges({
PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: outputURL!)
}) { saved, error in
if saved {
print("video successfully saved in photos gallery view video in photos gallery")
}
if (error != nil) {
print("error in saing video \(String(describing: error?.localizedDescription))")
}
}
DispatchQueue.main.async(execute: {
// completion(_filePath);
})
case .none:
break
case .unknown:
break
case .waiting:
break
case .exporting:
break
case .cancelled:
break
case .some(_):
break
}
})
}

slowmoVideo is an OSS project which appears to do this very nicely, though I don't know that it would work on an iPhone.
It does not simply make your videos play at 0.01× speed. You can
smoothly slow down and speed up your footage, optionally with motion
blur. How does slow motion work? slowmoVideo tries to find out where
pixels move in the video (this information is called Optical Flow),
and then uses this information to calculate the additional frames.

Related

How to save AVMutableComposition (directly) without AVMutableVideoComposition to camera roll/gallery/photos - Swift 5

I am one step away from finishing my first original app so I'm quite desperate for an answer.
I have created an AVMutableComposition (not AVMutableVideoComposition) that combines multiple clips. All I want to do is save it to camera roll or gallery. The only way I know how to do so is using AVMutableVideoComposition but in my current situation that would take a really long time because of the way I structured my app. I have also used PHPhotoLibrary but that only applies to URL videos. (is there a way to convert AVMutableComposition's to URL's?)
Here's what I've tried that doesn't work:
var mixComposition = AVMutableComposition() //trying to save this
func saveToGallery() {
let savePathUrl: URL = URL(fileURLWithPath: NSHomeDirectory() + "/Documents/newVideo.mp4")
do {
try FileManager.default.removeItem(at: savePathUrl)
} catch { print(error.localizedDescription) }
let assetExport: AVAssetExportSession = AVAssetExportSession(asset: mixComposition, presetName: AVAssetExportPresetHighestQuality)!
assetExport.outputFileType = AVFileType.mp4
assetExport.outputURL = savePathUrl
assetExport.shouldOptimizeForNetworkUse = true
assetExport.exportAsynchronously { () -> Void in
switch assetExport.status {
case AVAssetExportSessionStatus.completed:
print("successfully saved") // but it still prints this
case AVAssetExportSessionStatus.failed:
print("failed")
case AVAssetExportSessionStatus.cancelled:
print("cancelled")
default:
print("complete")
}
}
}
#IBAction func SaveButtonDidTouch(_ sender: Any) {
saveToGallery() //??
}
It still prints "successfully saved" even though nothing is added to my gallery/camera roll.
So what is the most efficient way to save an AVMutableComposition? Where did I go wrong here?

swift: how to delete part of audio?

I'm creating a simple audio editing tool to trim and delete from an audio.
I implemented the trim function and it is working fine. However I searched and tried to implement the delete function and here is my code:
func deleteExportAsset(_ asset: AVAsset, fileName: String, completeAudioTime: CGFloat) -> URL {
print("\(#function)")
let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
let trimmedSoundFileURL = documentsDirectory.appendingPathComponent(fileName)
print("saving to \(trimmedSoundFileURL.absoluteString)")
if FileManager.default.fileExists(atPath: trimmedSoundFileURL.absoluteString) {
print("sound exists, removing \(trimmedSoundFileURL.absoluteString)")
do {
if try trimmedSoundFileURL.checkResourceIsReachable() {
print("is reachable")
}
try FileManager.default.removeItem(atPath: trimmedSoundFileURL.absoluteString)
} catch {
print("could not remove \(trimmedSoundFileURL)")
print(error.localizedDescription)
}
}
print("creating export session for \(asset)")
if let exporter = AVAssetExportSession(asset: asset, presetName: AVAssetExportPresetAppleM4A) {
exporter.outputFileType = AVFileType.m4a
exporter.outputURL = trimmedSoundFileURL
let timeRange1 = CMTimeRangeFromTimeToTime(CMTime(seconds: 0, preferredTimescale: 100), CMTime(seconds: endTimeOfRange1, preferredTimescale: 100))
let timeRange2 = CMTimeRangeFromTimeToTime(CMTime(seconds: startTimeOfRange2)), preferredTimescale: 100), CMTime(seconds: Double(completeAudioTime), preferredTimescale: 100))
exporter.timeRange = CMTimeRangeGetUnion(timeRange1, timeRange2)
// do it
exporter.exportAsynchronously(completionHandler: {
print("export complete \(exporter.status)")
switch exporter.status {
case AVAssetExportSessionStatus.failed:
if let e = exporter.error {
print("export failed \(e)")
}
case AVAssetExportSessionStatus.cancelled:
print("export cancelled \(String(describing: exporter.error))")
default:
print("export complete")
}
})
} else {
print("cannot create AVAssetExportSession for asset \(asset)")
}
return trimmedSoundFileURL
}
what I'm doing here is creating 2 Ranges. Range1 from 0 ->time1 and Range2 from time2->endOfAudio. (I want to delete from time1 -> time2)
then I'm creating the union between 2 ranges.
however, nothing is happening to the audio. It is saved exactly like it was before this function.
CMTimeRangeGetUnion returns another CMTimeRange, which is just a (start-)time and a duration. So there is nothing than can hold the two time ranges required to do what you are expecting. In extension, AVAssetExportSession has no API that takes a list of time ranges to export.
But there is a way to accomplish it. The idea is to create an editable copy of the asset, delete the time range, and then export the editable copy. AVMutableComposition does this:
// assuming 'asset', 'endTimeOfRange1' and 'startTimeOfRange2' from the question:
// create empty mutable composition
let composition: AVMutableComposition = AVMutableComposition()
// copy all of original asset into the mutable composition, effectively creating an editable copy
try composition.insertTimeRange( CMTimeRangeMake( kCMTimeZero, asset.duration), of: asset, at: kCMTimeZero)
// now edit as required, e.g. delete a time range
let startTime = CMTime(seconds: endTimeOfRange1, preferredTimescale: 100)
let endTime = CMTime(seconds: startTimeOfRange2, preferredTimescale: 100)
composition.removeTimeRange( CMTimeRangeFromTimeToTime( startTime, endTime))
// since AVMutableComposition is an AVAsset subclass, it can be exported with AVAssetExportSession (or played with an AVPlayer(Item))
if let exporter = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetAppleM4A)
{
// configure session and exportAsynchronously as above.
// You don't have to set the timeRange of the exportSession
}
Note that copying from the asset to the composition only modifies some in-memory structures defining which samples go where on the time line, but doesn't actually moves any media samples around. This is not done until exporting; as result, editing is (relatively) fast, and you have to keep the source file around at least until export is finished.

Can't able to get Video Tracks from AVURLAsset for HLS videos(.m3u8 format) for AVPlayer?

I am developing a custom video player to stream HLS videos from server. I can successfully play HLS videos using AVPlayerItem and AVPlayer.
After that I want to add subtitle track and audio tracks for my video player. So I used AVMutableComposition to do so. So now the issue is when I am creating AVURLAsset for HLS Videos, I can't able to get video tracks from AVURLAsset. It is giving me always 0 tracks. I tried "loadValuesAsynchronously" of AVURLAsset and I tried adding KVO for "tracks" of AVPlayerItem. But None of these producing me any positive result.
I am using the following code.
func playVideo() {
let videoAsset = AVURLAsset(url: videoURL!)
let composition = AVMutableComposition()
// Video
let videoTrack = composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
let tracks = videoAsset.tracks(withMediaType: .video)
guard let track = tracks.first else {
print("Can't get first video track")
return
}
try videoTrack?.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: track, at: kCMTimeZero)
} catch {
print(error)
return
}
guard let subtitlesUrl = Bundle.main.url(forResource: "en", withExtension: "vtt") else {
print("Can't load en.vtt from bundle")
return
}
//Subtitles
let subtitleAsset = AVURLAsset(url: subtitlesUrl)
let subtitleTrack = composition.addMutableTrack(withMediaType: .text, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
let subTracks = subtitleAsset.tracks(withMediaType: AVMediaType.text)
guard let subTrack = subTracks.first else {
print("Can't get first subtitles track")
return
}
try subtitleTrack?.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: subTrack, at: kCMTimeZero)
} catch {
print(error)
return
}
// Prepare item and play it
let item = AVPlayerItem(asset: composition)
self.player = AVPlayer(playerItem: item)
self.playerLayer = AVPlayerLayer.init()
self.playerLayer.frame = self.bounds
self.playerLayer.contentsGravity = kCAGravityResizeAspect
self.playerLayer.player = player
self.layer.addSublayer(self.playerLayer)
self.player.addObserver(self, forKeyPath: "currentItem.loadedTimeRanges", options: .new, context: nil)
self.player.play()
}
This procedure working well for .mp4 videos but not for HLS Videos(.m3u8). Anyone have some working solution for this?
or
How can we get tracks from HLS videos using AVURLAsset? If this is not possible then How can achieve similar result ?
Please let me know you feedback.
Many more thanks in advance.
For HLS video tracks(withMediaType: .video) will return an empty array.
Use this instead: player.currentItem.presentationSize.width and player.currentItem.presentationSize.height.
Pls let me know if it works.
I didn't have the exact same problem as you. But I got around a similar problem (querying for HDR) by instead of querying the tracks on the AVURLAsset, I queried the tracks on the AVPlayerItem.
Set up an observer on the item status:
player?.observe(\AVPlayer.currentItem?.status,
options: [.new, .initial], changeHandler: { [weak self] player, _ in
DispatchQueue.main.async {
self?.observedItemStatus(from: player)
}
})
Then query the AVMediaType of your choice (in your case text).
func observedItemStatus(from avPlayer: AVPlayer) {
guard let currentItem = avPlayer.currentItem else { return }
// ideally execute code based on currentItem.status...for the brevity of this example I won't.
let hasLegibleMedia = currentItem.tracks.first(where: {
$0.assetTrack?.mediaType == AVMediaType.text
})?.assetTrack.hasMediaCharacteristic(.legible)
}
Alternatively if you need more than just a Bool, you could do a loop to access the assetTrack you really want.

Why do I get this output when using AVPlayer in Swift?

I am trying to play an audio file using avplayer in swift, when I play a file i generated combining two files, i get this output
playing file:"file location".m4a -- file:///
however when I play another remade sound file it plays fine, and i don't get the -- file:/// in the output after playing it
this is how I am playing the audio
func play(url:NSURL) {
do {
soundPlayer = AVPlayer(url: url as URL)
soundPlayer.volume = 1.0
soundPlayer.play()
} catch let error as NSError {
print(error.localizedDescription)
} catch {
print("failed")
}
}
and this is what I am using to concatenate two audio files
func makeSounds(sounds: [NSURL], preName: String){
let composition = AVMutableComposition()
print(sounds)
for sound in sounds {
let compositionAudioTrack:AVMutableCompositionTrack = composition.addMutableTrack(withMediaType: AVMediaTypeAudio, preferredTrackID: CMPersistentTrackID())
let avAsset = AVURLAsset(url: sound as URL)
let track = avAsset.tracks(withMediaType: AVMediaTypeAudio)[0]
let timeRange = CMTimeRange(start: CMTimeMake(0, 600), duration: track.timeRange.duration)
try! compositionAudioTrack.insertTimeRange(timeRange, of: track, at: composition.duration)
}
let documentDirectoryURL = NSURL(fileURLWithPath: Urls.user)
var fileDestinationUrl = documentDirectoryURL.appendingPathComponent("\(SoundData.Name)\(preName).m4a")
let assetExport = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetAppleM4A)
assetExport?.outputFileType = AVFileTypeAppleM4A
assetExport?.outputURL = fileDestinationUrl
assetExport?.exportAsynchronously(completionHandler:
{
switch assetExport!.status
{
case AVAssetExportSessionStatus.failed:
print("failed \(assetExport?.error)")
case AVAssetExportSessionStatus.cancelled:
print("cancelled \(assetExport?.error)")
case AVAssetExportSessionStatus.unknown:
print("unknown\(assetExport?.error)")
case AVAssetExportSessionStatus.waiting:
print("waiting\(assetExport?.error)")
case AVAssetExportSessionStatus.exporting:
print("exporting\(assetExport?.error)")
default:
soundsToPlay.soundLocation = String(describing: fileDestinationUrl!)
print("Audio Concatenation Complete")
}
})
}
the audio file location appears correct to the url i am setting it to be exported to, but it doesn't play the sound file, i just get that error
AVAssetExportSession needs a path string in 'File' format, which is usually obtained through URL.relativeString, which includes 'file:///'

(Cocoa error -1) When attempting to save video created with AVCaptureSession

I'm generating a video using an AVCapture session and then using an AVVideoCompositionCoreAnimationTool to add a simple overlay. I then use an AVAssetExportSession to output the file. This all seems to work but then when I attempt to save it to the Photo Library using PHPhotoLibrary (because ALAssetsLibrary has been depreciated) is fails with the message:"Cant complete operation cocoa error -1". After extensive Google use and checking the docs I can't work out whats going wrong.
Any help would be great thanks.
func videoOutput() {
videoToMake = AVAsset(URL: videoToMakeURL!)
if (videoToMake == nil) {
return
}
//This holds the different tracks of the video like audio and the layers
let mixComposition = AVMutableComposition()
let videoTrack = mixComposition.addMutableTrackWithMediaType(AVMediaTypeVideo, preferredTrackID: Int32(kCMPersistentTrackID_Invalid))
print("The duration of the video to create is \(videoToMake!.duration.seconds)")
do{
try videoTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero,videoToMake!.duration), ofTrack: videoToMake!.tracksWithMediaType(AVMediaTypeVideo)[0], atTime: kCMTimeZero)
}catch let error as NSError{
print("Error inserting time range on video track \(error.localizedDescription)")
return
}catch{
print("An unknown error occured")
}
//Make the instructions for the other layers
let mainInstrucation = AVMutableVideoCompositionInstruction()
mainInstrucation.timeRange = CMTimeRangeMake(kCMTimeZero, videoToMake!.duration)
//Create the layer instructions
let videoLayerInstruction = AVMutableVideoCompositionLayerInstruction(assetTrack: videoTrack)
let videoAssetTrack = videoToMake!.tracksWithMediaType(AVMediaTypeVideo)[0]
let assetInfo = orientationFromTransform(videoAssetTrack.preferredTransform)
// sort size it in respect to the video orientation.
videoLayerInstruction.setTransform(videoAssetTrack.preferredTransform, atTime: kCMTimeZero)
videoLayerInstruction.setOpacity(0.0, atTime:videoToMake!.duration)
//Add the instructions
mainInstrucation.layerInstructions = [videoLayerInstruction]
let mainCompositionInst = AVMutableVideoComposition()
var naturalSize:CGSize
if assetInfo.isPortrait {
naturalSize = CGSizeMake(videoAssetTrack.naturalSize.height, videoAssetTrack.naturalSize.width);
}else{
naturalSize = videoAssetTrack.naturalSize
}
let renderWidth = naturalSize.width
let renderHeight = naturalSize.height
mainCompositionInst.renderSize = CGSizeMake(renderWidth, renderHeight)
mainCompositionInst.instructions = [mainInstrucation]
mainCompositionInst.frameDuration = CMTimeMake(1, 30);
//So now the main composition has been created add the video affects
applyVideoEffectsToComposition(mainCompositionInst, size: naturalSize)
let paths = NSSearchPathForDirectoriesInDomains(.DocumentDirectory,.UserDomainMask, true)
let documentsDirectory = paths[0]
let random = Int(arc4random_uniform(1000))
let url = NSURL(fileURLWithPath:documentsDirectory).URLByAppendingPathComponent("FinalVideo\(random)")
//Create the exporter
let exporter = AVAssetExportSession(asset: mixComposition, presetName:AVAssetExportPresetHighestQuality)
exporter!.outputURL = url
exporter!.outputFileType = AVFileTypeMPEG4
exporter!.shouldOptimizeForNetworkUse = true
exporter!.videoComposition = mainCompositionInst
//Perform the export
exporter!.exportAsynchronouslyWithCompletionHandler() {
dispatch_async(dispatch_get_main_queue(), { () -> Void in
self.exportDidFinish(exporter!)
})
}
}
Well it turns out I was missing the extension from the end of my movie name:
let url = NSURL(fileURLWithPath:documentsDirectory).URLByAppendingPathComponent("FinalVideo\(random)")
So it should have been "FinalVideo(random).mov"
Hope this helps somebody one day.