How do I load a Data object into a SCNScene? - swift

I want to load a 3d usdz blob into a view, but since I only have the data object, I'm trying to initialize the scene with that with no luck.
To that, I initialize the SCNSceneSource() and then open it using .scene().
Now what I don't understand:
If I use a URL and load the scene directly - it works.
If I use a Data object on the same URL it doesn't.
Apple docs says, the data should be of type NSData but that seems wrong.
import SceneKit
let url = URL(string: "file:///Users/thilo/Desktop/Input/UU2.usdz")!
// working
let src_ok = SCNSceneSource(url: url)
let scn_ok = src_ok?.scene(options: nil, statusHandler: {
a,b,c,d in print("OK: \(a) \(b) \(String(describing: c)) \(d) ")
})
print("Ok: \(scn_ok)")
// Not working?
let data = try! Data(contentsOf: url)
let src_bad = SCNSceneSource(data: data)
let scn_bad = src_bad?.scene(options: nil, status handler: {
a,b,c,d in print("BAD: \(a) \(b) \(String(describing: c)) \(d) ")
})
print("Failed: \(scn_bad)")
running on Playground says:
Ok: Optional(<SCNScene: 0x6000038e1200>)
BAD: 0.0 SCNSceneSourceStatus(rawValue: 4) nil 0x000000016fa948bf
BAD: 0.0 SCNSceneSourceStatus(rawValue: 4) nil 0x000000016fa942af
BAD: 0.0 SCNSceneSourceStatus(rawValue: -1) Optional(Error Domain=NSCocoaErrorDomain Code=260 "Could not load the scene" UserInfo={NSLocalizedDescription=Could not load the scene, NSLocalizedRecoverySuggestion=An error occurred while parsing the COLLADA file. Please check that it has not been corrupted.}) 0x000000016fa942af
Failed: nil
What am I missing?

SCNSceneSource doesn't support .usdz in Data context
Official documentation says that SCNSceneSource object supports only .scn, .dae and .abc file formats. But it turns out that SceneKit doesn't support URL-loading of .usdz only in the context of working with Data. Thus, when working with Data, use files in the .scn format.
import SceneKit
import Cocoa
class GameViewController : NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
if let url = URL(string: "file:///Users/swift/Desktop/ship.scn") {
let data = try! Data(contentsOf: url)
let source = SCNSceneSource(data: data)
let sceneView = self.view as! SCNView
sceneView.scene = source?.scene()
}
}
}
To load .usdz using URL, try SCNSceneSource.init?(url: URL)
class GameViewController : NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
if let url = URL(string: "file:///Users/swift/Desktop/ship.usdz") {
let source = SCNSceneSource(url: url)
let sceneView = self.view as! SCNView
sceneView.scene = source?.scene()
}
}
}
Or use SCNScene object to load .usdz model
class GameViewController : NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let url = URL(fileURLWithPath: "/Users/swift/Desktop/ship.usdz")
do {
let scene = try SCNScene(url: url)
let sceneView = self.view as! SCNView
sceneView.scene = scene
sceneView.autoenablesDefaultLighting = true
} catch {
print(error.localizedDescription)
}
}
}

Gathering from the comment "does not support usdz" my solution is:
to create a temporary file ( .usdz) seems to be required by the API...
and then manually remove the temporary file after loading.
First extend FileManager with the below code:
public extension FileManager {
func temporaryFileURL(fileName: String = UUID().uuidString,ext: String) -> URL? {
return URL(fileURLWithPath: NSTemporaryDirectory(), isDirectory: true)
.appendingPathComponent(fileName + ext)
}
}
For a limited hard-coded use case:
let fm = FileManager.default
let tempusdz = fm.temporaryFileURL(ext:".usdz")!
fm.createFile(atPath: tempusdz.path(), contents: sceneData)
let src = SCNSceneSource(url: tempusdz)
if let scene = src?.scene(options: nil) {
....
}
try? fm.removeItem(at: tempusdz)
of course this is a hack, because it will only work if the data is in usdz format.
Since usdz is a ZIP archive, maybe testing for a zip and then just doing the below is a better option:
let sceneData:Data? = data
var sceneSrc: SCNSceneSource? = nil
var tempURL:URL? = nil
if let dataStart = sceneData?.subdata(in: 0..<4),
let dataMagic = String(data: dataStart, encoding: String.Encoding.utf8) as String?,
dataMagic == "PK\u{3}\u{4}" {
let fm = FileManager.default
tempURL = fm.temporaryFileURL(ext: ".usdz")
if let tempURL {
fm.createFile(atPath: tempURL.path(), contents: sceneData)
sceneSrc = SCNSceneSource(url: tempURL)
}
} else {
sceneSrc = SCNSceneSource(data: sceneData!)
}
let scene = sceneSrc?.scene()
if let tempURL {
try? FileManager.default.removeItem(at: tempURL)
}
Does anyone knows a better solution?
Is there an easy way to check the type of the Data ?

potential solution could be to verify the format of the data object and ensure that it is a valid COLLADA file.
import Foundation
let url = URL(string: "file:///Users/thilo/Desktop/Input/UU2.usdz")!
let data = try! Data(contentsOf: url)
print("Data size: \(data.count)")
print("Data format: \(data.description)")
you usually get these types of errors when the data wasn't properly formatted

Related

How to create ARQuickLookPreviewItem with a ModelEntity with RealityKit?

I'm trying to Load a model from a usdz file, change the material colour of it and then display the updated model with AR QuickLook. I have got my code to the point that it loads the original model in the quick look preview. How do i use the updated ModelEntity?
guard let path = Bundle.main.path(forResource: "atm", ofType: "usdz") else {
fatalError("Couldn't find the supported asset file.")
}
let url = URL(fileURLWithPath: path)
guard let modelEntity = try? Entity.loadModel(contentsOf: url) else {
fatalError("Entity not found")
}
var newMaterial = SimpleMaterial()
newMaterial.color.tint = UIColor.cyan
for i in 0...(modelEntity.model?.materials.count ?? 1) - 1 {
modelEntity.model?.materials[i] = newMaterial
}
//let temporaryDirectory = FileManager.default.temporaryDirectory
//let temporaryFileURL = temporaryDirectory.appendingPathComponent("model.usdz")
let previewItem = ARQuickLookPreviewItem(fileAt: url) // how do i use modelEntity here?
previewItem.allowsContentScaling = allowsContentScaling
previewItem.canonicalWebPageURL = nil
return previewItem
}
I just need to change the colour of the model at runtime and show it. TIA

[Swift]I want to instantly save the sound with AVAudioEngine's effect as a file

I'm creating a process to read an existing audio file, add an effect using AVAudioEngine, and then save it as another audio file.
However, with the following method using an AVAudioPlayerNode, the save process must wait until the end of playback.
import UIKit
import AVFoundation
class ViewController: UIViewController {
let engine = AVAudioEngine()
let playerNode = AVAudioPlayerNode()
let reverbNode = AVAudioUnitReverb()
override func viewDidLoad() {
super.viewDidLoad()
do {
let url = URL(fileURLWithPath: Bundle.main.path(forResource: "original", ofType: "mp3")!)
let file = try AVAudioFile(forReading: url)
// playerNode
engine.attach(playerNode)
// reverbNode
reverbNode.loadFactoryPreset(.largeChamber)
reverbNode.wetDryMix = 5.0
engine.attach(reverbNode)
engine.connect(playerNode, to: reverbNode, format: file.processingFormat)
engine.connect(reverbNode, to: engine.mainMixerNode, format: file.processingFormat)
playerNode.scheduleFile(file, at: nil, completionCallbackType: .dataPlayedBack){ [self] _ in
reverbNode.removeTap(onBus: 0)
}
// start
try engine.start()
playerNode.play()
let url2 = URL(fileURLWithPath: fileInDocumentsDirectory(filename: "changed.wav"))
let outputFile = try! AVAudioFile(forWriting: url2, settings: playerNode.outputFormat(forBus: 0).settings)
reverbNode.installTap(onBus: 0, bufferSize: AVAudioFrameCount(reverbNode.outputFormat(forBus: 0).sampleRate), format: reverbNode.outputFormat(forBus: 0)) { (buffer, when) in
do {
try outputFile.write(from: buffer)
} catch let error {
print(error)
}
}
} catch {
print(error.localizedDescription)
}
}
func getDocumentsURL() -> NSURL {
let documentsURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0] as NSURL
return documentsURL
}
func fileInDocumentsDirectory(filename: String) -> String {
let fileURL = getDocumentsURL().appendingPathComponent(filename)
return fileURL!.path
}
}
Is there a way to complete the writing without waiting for the playback to complete? My ideal is to complete the write in the time required by CPU and storage performance.
It seems that
reverbNode.installTap(...) { (buffer, when) in ...}
in the code is processed in parallel with the current playback position. But I would like to dramatically improve the processing speed.
Best regards.

RealityKit – Get a model entity from USDZ file

I haave a file (exists in main bundle with target membership checked) named matrix.usdz and need to load it with
do {
let path = Bundle.main.path(forResource: "Matrix", ofType: "usdz")!
let url = URL(fileURLWithPath: path)
let assetsLoader = try Entity.load(contentsOf: url)
}
catch {
print(error)
}
But it crashes with
Thread 1: signal SIGABRT
on this line
let assetsLoader = try Entity.load(contentsOf: url)
Preview
You have to create an anchor if you need to load an entity into your scene. In order to get a ModelEntity, you need to grab it from the scene hierarchy using .children[X] subscript.
import RealityKit
class ViewController: UIViewController {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
do {
let path = Bundle.main.path(forResource: "Matrix", ofType: "usdz")!
let url = URL(fileURLWithPath: path)
// Scene
let scene = try Entity.load(contentsOf: url)
print(scene)
// Entity
let entity = scene.children[0].........children[0] as! ModelEntity
entity.model?.materials[0] = UnlitMaterial(color: .red)
let anchor = AnchorEntity(plane: .any)
anchor.addChild(scene)
arView.scene.anchors.append(anchor)
} catch {
print(error)
}
}
}
You can also get a model this way:
let modelEntity = try Entity.loadModel(contentsOf: url)
modelEntity.model?.materials[0] = UnlitMaterial(color: .red)
P.S.
I should say that you have an obvious naming error – "Matrix" vs "matrix". Also Matrix.rcproject and Matrix.usdz are not the same. To load Matrix.rcproject (Reality Composer project) use the following approach:
// .rcproject
let scene = try! Matrix.loadCircle()
let circleEntity = scene.children[0]...........children[0] as! ModelEntity
to load a USDZ model use this one:
// .usdz
let model = try! Entity.loadModel(named: "Matrix", in: nil)
But as far as I know, you do not need an RC project, so export USDZ from Reality Composer.
To load .reality file use the following approach:
// .reality
let carModel = try! Entity.loadAnchor(named: "car")
print(carModel)
arView.scene.addAnchor(carModel)
Here's your USDZ model on iOS simulator:

Mac - Swift 3 - queuing audio files and playing

I would like to write an app in swift 3 in order to play queued audio files without any gap, crack or noise when passing from one to another.
My first try was using AvAudioPlayer and AvAudioDelegate (AVAudioPlayer using array to queue audio files - Swift), but I don't know how to preload the next song to avoid gap. Even if I know how to do it, I am not certain it is the best way to achieve my goal.
AVQueuePlayer seems to be a better candidate for the job, it is made for that purpose, but I don't find any example to help me out.
Maybe it is only a problem of preloading or buffering? I am a bit lost in this ocean of possibilities.
Any suggestion is welcomed.
It is far to be perfect, specially if you want to do it twice or more ("file exist" error), but it can serve as a base.
What it does is taking two files (mines are aif samples of ap. 4 sec.), encode them in one file and play the resulting files. If you have hundreds of them, assembled aleatory or not, it can make great fun.
All credits for the mergeAudioFiles function goes to #Peyman and #Pigeon_39. Concatenate two audio files in Swift and play them
Swift 3
import Cocoa
import AVFoundation
var action = AVAudioPlayer()
let path = Bundle.main.path(forResource: "audiofile1.aif", ofType:nil)!
let url = URL(fileURLWithPath: path)
let path2 = Bundle.main.path(forResource: "audiofile2.aif", ofType:nil)!
let url2 = URL(fileURLWithPath: path2)
let array1 = NSMutableArray(array: [url, url2])
class ViewController: NSViewController, AVAudioPlayerDelegate
{
#IBOutlet weak var LanceStop: NSButton!
override func viewDidLoad()
{
super.viewDidLoad()
}
override var representedObject: Any?
{
didSet
{
// Update the view, if already loaded.
}
}
#IBAction func Lancer(_ sender: NSButton)
{
mergeAudioFiles(audioFileUrls: array1)
let url3 = NSURL(string: "/Users/ADDUSERNAMEHERE/Documents/FinalAudio.m4a")
do
{
action = try AVAudioPlayer(contentsOf: url3 as! URL)
action.delegate = self
action.numberOfLoops = 0
action.prepareToPlay()
action.volume = 1
action.play()
}
catch{print("error")}
}
func audioPlayerDidFinishPlaying(_ player: AVAudioPlayer, successfully flag: Bool)
{
if flag == true
{
}
}
var mergeAudioURL = NSURL()
func mergeAudioFiles(audioFileUrls: NSArray) {
//audioFileUrls.adding(url)
//audioFileUrls.adding(url2)
let composition = AVMutableComposition()
for i in 0 ..< audioFileUrls.count {
let compositionAudioTrack :AVMutableCompositionTrack = composition.addMutableTrack(withMediaType: AVMediaTypeAudio, preferredTrackID: CMPersistentTrackID())
let asset = AVURLAsset(url: (audioFileUrls[i] as! NSURL) as URL)
let track = asset.tracks(withMediaType: AVMediaTypeAudio)[0]
let timeRange = CMTimeRange(start: CMTimeMake(0, 600), duration: track.timeRange.duration)
try! compositionAudioTrack.insertTimeRange(timeRange, of: track, at: composition.duration)
}
let documentDirectoryURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first! as NSURL
self.mergeAudioURL = documentDirectoryURL.appendingPathComponent("FinalAudio.m4a")! as URL as NSURL
let assetExport = AVAssetExportSession(asset: composition, presetName: AVAssetExportPresetAppleM4A)
assetExport?.outputFileType = AVFileTypeAppleM4A
assetExport?.outputURL = mergeAudioURL as URL
assetExport?.exportAsynchronously(completionHandler:
{
switch assetExport!.status
{
case AVAssetExportSessionStatus.failed:
print("failed \(assetExport?.error)")
case AVAssetExportSessionStatus.cancelled:
print("cancelled \(assetExport?.error)")
case AVAssetExportSessionStatus.unknown:
print("unknown\(assetExport?.error)")
case AVAssetExportSessionStatus.waiting:
print("waiting\(assetExport?.error)")
case AVAssetExportSessionStatus.exporting:
print("exporting\(assetExport?.error)")
default:
print("Audio Concatenation Complete")
}
})
}
}

Impossible to stream video after downloading from Parse

It seems that I can't stream my videos once I downloaded them from Parse backend. However I know that the videos are working well as you can play it from Chrome but not from Safari ( I don't know why ...).
My code to play the video on the AVPlayer is also right since I test it for multiple NSUrl from local database.
Several posts about it confirm that there is a problem going on but no one has an answer.
iOS - Can't stream video from Parse Backend
Cant stream video (PFFile) from parse server
Thanks in advance if someone can help me.
The secret is to get the PFFIle data, save it locally with the proper extention (mov) and treaming the local file.
look at this entry [Cant stream video (PFFile) from parse server][1], the second anwser.
The PFFIle was created with the following code:
let videoPath = info[UIImagePickerControllerMediaURL] as! NSURL
let imageData = NSData (contentsOfURL:videoPath)! as NSData
do {
let file = try PFFile(name: "_mov", data: imageData, contentType: "video/quicktime")
videoSaveSelected(file )
} catch let error as NSError {
print("Error generating PFFile: \(error)")
}
This is the code for the player
import UIKit
import AVKit
import MobileCoreServices
import AssetsLibrary
import AVFoundation
import Parse
import ParseUI
class VideoPlayerViewController: UIViewController {
var videoUrl: NSURL!
var file: PFFile!
override func viewDidLoad() {
super.viewDidLoad()
// navigation
let backButtonCustom = UIButton( frame: CGRectMake(0, 0, 22, 22))
let backIcon = UIImage(named: "0836-arrow-left")?.imageWithRenderingMode(UIImageRenderingMode.AlwaysTemplate)
backButtonCustom.setImage(backIcon, forState: UIControlState.Normal)
backButtonCustom.addTarget(self, action: #selector(ReqDataEntryLgViewController.doneButtonTapped), forControlEvents: UIControlEvents.TouchUpInside)
navigationItem.leftBarButtonItem = UIBarButtonItem(customView: backButtonCustom)
let doneButton = UIBarButtonItem(title: "Done", style: .Done, target: self, action: #selector(ReqDataEntryLgViewController.doneButtonTapped))
self.navigationItem.setRightBarButtonItems([doneButton], animated: true)
// video player
let playerController = AVPlayerViewController()
self.addChildViewController(playerController)
self.view.addSubview(playerController.view)
playerController.view.frame = self.view.frame
// get data from PFFile in database
file!.getDataInBackgroundWithBlock({
(movieData: NSData?, error: NSError?) -> Void in
if (error == nil) {
let documentsPath : AnyObject = NSSearchPathForDirectoriesInDomains(.DocumentDirectory,.UserDomainMask,true)[0]
let destinationPath:NSString = documentsPath.stringByAppendingString("/file.mov")
let filemanager = NSFileManager.defaultManager()
do {
try filemanager.removeItemAtPath(destinationPath as String)
} catch {
print("Ooops! Something went wrong: \(error)")
}
// write to local file
movieData!.writeToFile ( destinationPath as String, atomically:true)
// do {
// let attr : NSDictionary? = try NSFileManager.defaultManager().attributesOfItemAtPath(destinationPath as String)
//
// if let _attr = attr {
// let fileSize = _attr.fileSize()
// print ("Movie Size \(fileSize)" )
// }
// } catch {
// print("Ooops! Something went wrong getting size: \(error)")
// }
let playerItem = AVPlayerItem(asset: AVAsset(URL: NSURL(fileURLWithPath: destinationPath as String)))
let player = AVPlayer(playerItem: playerItem)
playerController.player = player
player.play()
} else {
print ("error on getting movie data \(error?.localizedDescription)")
}
})
}
func doneButtonTapped(){
navigationController?.popViewControllerAnimated(true)
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
}