Switching subtitles of hls on tvOS/swift - swift

I am using AVplayer of swift for my tvOSApp.
When I use default player, subtitles of the video will be shown on "subtitle" bar on top and can easily switch subtitles.
So, how can I switch subtitles like that, without using the default playback controls?
I have buttons for switching subtitles on the screen and I'd like to switch during target method of these.
Here is my code for AVPlayer.
let asset = AVAsset(url: URL(string: videoUrl)!)
let playerItem = AVPlayerItem(asset: asset)
let player = AVPlayer(playerItem: playerItem)
player.play()
self.bgMovieLayer = AVPlayerLayer(player: player)
self.bgMovieLayer.frame = view.bounds
self.bgMovieLayer.videoGravity = .resizeAspectFill
self.bgMovieLayer.zPosition = -1
self.view.layer.insertSublayer(self.bgMovieLayer, at: 0)
I am using the video below for a test.
http://184.72.239.149/vod/smil:BigBuckBunny.smil/playlist.m3u8
I read the documents regarding AVPlayer/AVPlayerItem and came into no answer. I would appreciate if someone can help this.

You have to load availableMediaCharacteristics first and after that look for AVMediaCharacteristic.legible.
Something like this:
let asset:AVAsset
let key = #keyPath(AVAsset.availableMediaCharacteristicsWithMediaSelectionOptions)
asset.loadValuesAsynchronously(forKeys: [key]) {
var error: NSError? = nil
let status = asset.statusOfValue(forKey: key, error: &error)
switch status {
case .loaded:
// Sucessfully loaded, continue processing
case .failed:
// Examine NSError pointer to determine failure
case .cancelled:
// Loading cancelled
default:
// Handle all other cases
}
}
let subtitlesGroup = asset.mediaSelectionGroup(forMediaCharacteristic: AVMediaCharacteristic.legible)

Related

How do you fix the "found nil while unwrapping optional value" error when trying to play sound?

I'm making a function to play sound
func playSound(soundName: String) {
let url = Bundle.main.url(forResource: soundName, withExtension: "wav")
player = try! AVAudioPlayer(contentsOf: url!)
player.play()
}
Then call this function in an IBAction that contains all my buttons
#IBAction func buttonPiano(_ sender: UIButton) {
playSound(soundName: String(sender.currentTitle!))
sender.backgroundColor = UIColor.white
sender.alpha = 0.3
DispatchQueue.main.asyncAfter(deadline: .now() + .milliseconds(300), execute: {
sender.backgroundColor = UIColor.systemBackground
sender.alpha = 1
})
}
Running the app I can do. But whenever you press a button, it crashes and gives me this error:
Fatal error: Unexpectedly found nil while unwrapping an Optional value: file /Users/administrator/Desktop/Xcode Projects/pianoButtons/pianoButtons/ViewController.swift, line 37
The optional value seems to be url! from my sound function.
I've tried all I could, but no luck. How do I avoid this error and play the sound without crashes?
Make sure that your soundName.wave file is shown inside the copy bundle resources. You can find that by clicking on your project > selecting your target > Build Phases > Copy Bundle Resources. If you do not see it there, click the plus button to add it.
var soundPlayer: AVAudioPlayer?
func playSentSound() {
DispatchQueue.main.async{
let path = Bundle.main.path(forResource: "soundName.mp3", ofType: nil)!
let url = URL(fileURLWithPath: path)
do {
self.soundPlayer = try AVAudioPlayer(contentsOf: url)
print("Playing")
self.soundPlayer?.play()
} catch {
// couldn't load file :(
print("Cant Load File")
}
}
}
Your code is essentially calling this
try! AVAudioPlayer(contentsOf: Bundle.main.url(forResource: String(button.currentTitle!), withExtension: "wav")!)
Each ! is a potential crash. This is when they would occur
The button may not have a current title at the time the action is triggered.
The main bundle may not have a resource with that name/extension
The audio player may not be able to play the contents of that file
In your specific case it seems like it is failing at 2. The nicer way to handle this is like so
if let url = Bundle.main.url(forResource: soundName, withExtension: "wav") {
player = try! AVAudioPlayer(contentsOf: url)
} else {
print("No resouce named \(soundName).wav")
}
First of all your app will just not play a sound instead of crashing, second you will get a helpful log message which might show you why the resource isn't being found.
Ideally all of your ! should be replaced with similar constructs, to log errors or perform some fallback action instead of crashing.

AVPlayer not starting until cell is recycled

I have a UITableView that I am using to render out tweets that contain gifs. These are returned from Twitter as MP4's so I am using AVPlayer. Using HanekeSwift I am caching the item and then playing it in the tableview. The issue I am having is that any Tweets on screen when the tableview load, do not auto play, I must scroll them off screen and back on before they play.
I configure my AVPlayer in the following function
private func configureAVPlayer(url: URL) {
cache.fetch(URL: url).onSuccess { [weak self] stream in
guard let path = URL(string: DiskCache.basePath())?.appendingPathComponent("shared-data/original") else { return }
let cached = DiskCache(path: path.absoluteString).path(forKey: url.absoluteString)
let file = URL(fileURLWithPath: cached)
if !FileManager.default.fileExists(atPath: cached) {
try! stream.write(to: file, options: .atomicWrite)
}
self?.player = AVPlayer(url: file)
self?.player?.automaticallyWaitsToMinimizeStalling = false
self?.playerLayer = self?.avPlayerView.layer as? AVPlayerLayer
guard let player = self?.player else { return }
self?.playerLayer?.player = player
self?.player?.play()
}
}
Following something very similar as described here
However for me, the cell is essentially paused until I scroll.

How to fix 'appendingPathComponent' is unavailable: Use appendingPathComponent on URL error

I'm working on a old Swift tutorial (Swift 2.0) that's posted on Ray Wenderlich's web site (https://www.raywenderlich.com/2185-how-to-make-a-letter-word-game-with-uikit-and-swift-part-3-3) and I'm running into an error when I tried to re-setup a function called "preloadAudioEffects" in Swift 4.2 . The error? appendingPathComponent' is unavailable: Use appendingPathComponent on URL instead.
I've tried to rename the old Swift code [Ex: NSBundle to Bundle , stringByAppendingPathComponent to appendingPathComponent()], but I'm still running into some syntax issues due to my inexperience with Swift.
This is the original code:
func preloadAudioEffects(effectFileNames:[String]) {
for effect in AudioEffectFiles {
//1 get the file path URL
let soundPath = NSBundle.mainBundle().resourcePath!.stringByAppendingPathComponent(effect)
let soundURL = NSURL.fileURLWithPath(soundPath)
//2 load the file contents
var loadError:NSError?
let player = AVAudioPlayer(contentsOfURL: soundURL, error: &loadError)
assert(loadError == nil, "Load sound failed")
//3 prepare the play
player.numberOfLoops = 0
player.prepareToPlay()
//4 add to the audio dictionary
audio[effect] = player
}
}
And this is what I've tried to do via following the suggestions in Xcode:
func preloadAudioEffects(effectFileNames:[String]) {
for effect in AudioEffectFiles {
//1 get the file path URL
let soundPath = Bundle.main.resourcePath!.appendingPathComponent(effect)
let soundURL = NSURL.fileURL(withPath: soundPath)
//2 load the file contents
var loadError:NSError?
let player = AVAudioPlayer(contentsOfURL: soundURL, error: &loadError)
assert(loadError == nil, "Load sound failed")
//3 prepare the play
player.numberOfLoops = 0
player.prepareToPlay()
//4 add to the audio dictionary
audio[effect] = player
}
}
Get the full path to the sound file and convert it to a URL by using NSURL.fileURLWithPath().
Call AVAudioPlayer(contentsOfURL:error:) to load a sound file in an audio player.
Set the numberOfLoops to zero so that the sound won’t loop at all. Call prepareToPlay() to preload the audio buffer for that sound.
Finally, save the player object in the audio dictionary, using the name of the file as the dictionary key.
Just replace resourcePath with resourceURL
let soundURL = Bundle.main.resourceURL!.appendingPathComponent(effect)
and you have to wrap the AVAudioPlayer initializer in a try block
func preloadAudioEffects(effectFileNames:[String]) {
for effect in AudioEffectFiles {
let soundURL = Bundle.main.resourceURL!.appendingPathComponent(effect)
//2 load the file contents
do {
let player = try AVAudioPlayer(contentsOf: soundURL)
//3 prepare the play
player.numberOfLoops = 0
player.prepareToPlay()
//4 add to the audio dictionary
audio[effect] = player
} catch { print(error) }
}
}

Can't able to get Video Tracks from AVURLAsset for HLS videos(.m3u8 format) for AVPlayer?

I am developing a custom video player to stream HLS videos from server. I can successfully play HLS videos using AVPlayerItem and AVPlayer.
After that I want to add subtitle track and audio tracks for my video player. So I used AVMutableComposition to do so. So now the issue is when I am creating AVURLAsset for HLS Videos, I can't able to get video tracks from AVURLAsset. It is giving me always 0 tracks. I tried "loadValuesAsynchronously" of AVURLAsset and I tried adding KVO for "tracks" of AVPlayerItem. But None of these producing me any positive result.
I am using the following code.
func playVideo() {
let videoAsset = AVURLAsset(url: videoURL!)
let composition = AVMutableComposition()
// Video
let videoTrack = composition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
let tracks = videoAsset.tracks(withMediaType: .video)
guard let track = tracks.first else {
print("Can't get first video track")
return
}
try videoTrack?.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: track, at: kCMTimeZero)
} catch {
print(error)
return
}
guard let subtitlesUrl = Bundle.main.url(forResource: "en", withExtension: "vtt") else {
print("Can't load en.vtt from bundle")
return
}
//Subtitles
let subtitleAsset = AVURLAsset(url: subtitlesUrl)
let subtitleTrack = composition.addMutableTrack(withMediaType: .text, preferredTrackID: kCMPersistentTrackID_Invalid)
do {
let subTracks = subtitleAsset.tracks(withMediaType: AVMediaType.text)
guard let subTrack = subTracks.first else {
print("Can't get first subtitles track")
return
}
try subtitleTrack?.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: subTrack, at: kCMTimeZero)
} catch {
print(error)
return
}
// Prepare item and play it
let item = AVPlayerItem(asset: composition)
self.player = AVPlayer(playerItem: item)
self.playerLayer = AVPlayerLayer.init()
self.playerLayer.frame = self.bounds
self.playerLayer.contentsGravity = kCAGravityResizeAspect
self.playerLayer.player = player
self.layer.addSublayer(self.playerLayer)
self.player.addObserver(self, forKeyPath: "currentItem.loadedTimeRanges", options: .new, context: nil)
self.player.play()
}
This procedure working well for .mp4 videos but not for HLS Videos(.m3u8). Anyone have some working solution for this?
or
How can we get tracks from HLS videos using AVURLAsset? If this is not possible then How can achieve similar result ?
Please let me know you feedback.
Many more thanks in advance.
For HLS video tracks(withMediaType: .video) will return an empty array.
Use this instead: player.currentItem.presentationSize.width and player.currentItem.presentationSize.height.
Pls let me know if it works.
I didn't have the exact same problem as you. But I got around a similar problem (querying for HDR) by instead of querying the tracks on the AVURLAsset, I queried the tracks on the AVPlayerItem.
Set up an observer on the item status:
player?.observe(\AVPlayer.currentItem?.status,
options: [.new, .initial], changeHandler: { [weak self] player, _ in
DispatchQueue.main.async {
self?.observedItemStatus(from: player)
}
})
Then query the AVMediaType of your choice (in your case text).
func observedItemStatus(from avPlayer: AVPlayer) {
guard let currentItem = avPlayer.currentItem else { return }
// ideally execute code based on currentItem.status...for the brevity of this example I won't.
let hasLegibleMedia = currentItem.tracks.first(where: {
$0.assetTrack?.mediaType == AVMediaType.text
})?.assetTrack.hasMediaCharacteristic(.legible)
}
Alternatively if you need more than just a Bool, you could do a loop to access the assetTrack you really want.

How to record a video and make it slow motion

I am working on an iPhone app for school and need some help. The app should record video, make it slow motion (about 2x), then save it to the photo library. So far I have everything except how to make the video slow motion. I know it can be done as there is already an app in the App Store that does it.
How can I take a video I've saved to a temp url and adjust the speed before saving it to the photo library?
If you need to export your video then you need to use the AVMutableComposition Class
Then add your video as an AVAsset to an AVMutableComposition and scale it with:
- (void)scaleTimeRange:(CMTimeRange)timeRange toDuration:(CMTime)duration
Finally you export it using AVAssetExportSession Class
I written a code that makes your video in "slow motion" and saves it in Photos Library. "Main Thing This Code Works In Swift 5". Creating "Slow motion" video in iOS swift is not easy, that I came across many "slow motion" that came to know not working or some of the codes in them are depreciated. And so I finally figured a way to make slow motion in Swift.
This code can be used for 120fps are greater than that too. Just add the url of your video and make it slow
Here is the "code snippet I created for achieving slow motion"
func slowMotion(pathUrl: URL) {
let videoAsset = AVURLAsset.init(url: pathUrl, options: nil)
let currentAsset = AVAsset.init(url: pathUrl)
let vdoTrack = currentAsset.tracks(withMediaType: .video)[0]
let mixComposition = AVMutableComposition()
let compositionVideoTrack = mixComposition.addMutableTrack(withMediaType: .video, preferredTrackID: kCMPersistentTrackID_Invalid)
let videoInsertError: Error? = nil
var videoInsertResult = false
do {
try compositionVideoTrack?.insertTimeRange(
CMTimeRangeMake(start: .zero, duration: videoAsset.duration),
of: videoAsset.tracks(withMediaType: .video)[0],
at: .zero)
videoInsertResult = true
} catch let videoInsertError {
}
if !videoInsertResult || videoInsertError != nil {
//handle error
return
}
var duration: CMTime = .zero
duration = CMTimeAdd(duration, currentAsset.duration)
//MARK: You see this constant (videoScaleFactor) this helps in achieving the slow motion that you wanted. This increases the time scale of the video that makes slow motion
// just increase the videoScaleFactor value in order to play video in higher frames rates(more slowly)
let videoScaleFactor = 2.0
let videoDuration = videoAsset.duration
compositionVideoTrack?.scaleTimeRange(
CMTimeRangeMake(start: .zero, duration: videoDuration),
toDuration: CMTimeMake(value: videoDuration.value * Int64(videoScaleFactor), timescale: videoDuration.timescale))
compositionVideoTrack?.preferredTransform = vdoTrack.preferredTransform
let dirPaths = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).map(\.path)
let docsDir = dirPaths[0]
let outputFilePath = URL(fileURLWithPath: docsDir).appendingPathComponent("slowMotion\(UUID().uuidString).mp4").path
if FileManager.default.fileExists(atPath: outputFilePath) {
do {
try FileManager.default.removeItem(atPath: outputFilePath)
} catch {
}
}
let filePath = URL(fileURLWithPath: outputFilePath)
let assetExport = AVAssetExportSession(
asset: mixComposition,
presetName: AVAssetExportPresetHighestQuality)
assetExport?.outputURL = filePath
assetExport?.outputFileType = .mp4
assetExport?.exportAsynchronously(completionHandler: {
switch assetExport?.status {
case .failed:
print("asset output media url = \(String(describing: assetExport?.outputURL))")
print("Export session faiied with error: \(String(describing: assetExport?.error))")
DispatchQueue.main.async(execute: {
// completion(nil);
})
case .completed:
print("Successful")
let outputURL = assetExport!.outputURL
print("url path = \(String(describing: outputURL))")
PHPhotoLibrary.shared().performChanges({
PHAssetChangeRequest.creationRequestForAssetFromVideo(atFileURL: outputURL!)
}) { saved, error in
if saved {
print("video successfully saved in photos gallery view video in photos gallery")
}
if (error != nil) {
print("error in saing video \(String(describing: error?.localizedDescription))")
}
}
DispatchQueue.main.async(execute: {
// completion(_filePath);
})
case .none:
break
case .unknown:
break
case .waiting:
break
case .exporting:
break
case .cancelled:
break
case .some(_):
break
}
})
}
slowmoVideo is an OSS project which appears to do this very nicely, though I don't know that it would work on an iPhone.
It does not simply make your videos play at 0.01× speed. You can
smoothly slow down and speed up your footage, optionally with motion
blur. How does slow motion work? slowmoVideo tries to find out where
pixels move in the video (this information is called Optical Flow),
and then uses this information to calculate the additional frames.