I have a Reality Composer scene and I want to extract it as usdz file or any files that can be used in ARQuickLook?
is it possible?
At build time, Xcode compiles your .rcproject into a .reality file, and AR Quick Look accepts preview items of type .reality. Here's an example that uses AR Quick Look to preview the Experience.rcproject taken from Apple's SwiftStrike TableTop sample code:
import UIKit
import QuickLook
import ARKit
class ViewController: UIViewController, QLPreviewControllerDataSource {
override func viewDidAppear(_ animated: Bool) {
let previewController = QLPreviewController()
previewController.dataSource = self
present(previewController, animated: true, completion: nil)
}
func numberOfPreviewItems(in controller: QLPreviewController) -> Int { return 1 }
func previewController(_ controller: QLPreviewController, previewItemAt index: Int) -> QLPreviewItem {
guard let path = Bundle.main.path(forResource: "Experience", ofType: "reality") else { fatalError("couldn't find the rcproject file.") }
let url = URL(fileURLWithPath: path)
let item = ARQuickLookPreviewItem(fileAt: url)
return item
}
}
From Apple's Creating 3D Content with Reality Composer
document:
You can also save your composition to a .reality file for use as a
lightweight AR Quick Look experience in your app or on the web. This
allows users to place and preview content in the real world to get a
quick sense of what it’s like.
To create a Reality file, choose File > Export > Export Project in the
Reality Composer menu, and provide a name for the file. You use the
Reality file that gets stored to disk just like you would use a USDZ
file, as described in Previewing a Model with AR Quick Look.
Related
The way to display animated gif image in swiftUI
because of Image
Image("fall-leaves")
does not support gifs
answer below
Easiest and fastest way to display gif image in swiftUI - is to use Preview / QuickLook (QL) / QLPreviewView
Quartz available only in macOS 10.4+ https://developer.apple.com/documentation/quartz
import SwiftUI
import Quartz
struct QLImage: NSViewRepresentable {
var url: URL
func makeNSView(context: NSViewRepresentableContext<QLImage>) -> QLPreviewView {
let preview = QLPreviewView(frame: .zero, style: .normal)
preview?.autostarts = true
preview?.previewItem = url as QLPreviewItem
return preview ?? QLPreviewView()
}
func updateNSView(_ nsView: QLPreviewView, context: NSViewRepresentableContext<QLImage>) {
nsView.previewItem = url as QLPreviewItem
}
typealias NSViewType = QLPreviewView
}
I was having a hard time using this in my macOS app because I try to load my gif file from the local assets rather than a URL. After looking around for answers, the best answer seems to be putting the gif file into the project folder rather than into the Assets.xcassets and then load the resource into the URL. Somehow Bundle.main.url(forResource: myFile, withExtension: "gif") will return nil if the file isn't in the main project folder. I checked the ProjectName -> Target -> Build Phases -> Copy Bundle Resources and the gif file is there automatically when I add to the project. The weird thing is somehow the gif file is not available if it's in the Assets.xcassets.
I updated the QLImage struct from #Andrew to load the gif file by name (make sure to add the gif directly to your project as mentioned above). Here's the updated code below. You need to also provide a sample gif image called preview-gif or call it whatever you want and match the name in the preview section.
import SwiftUI
import Quartz
struct QLImage: NSViewRepresentable {
private let name: String
init(_ name: String) {
self.name = name
}
func makeNSView(context: NSViewRepresentableContext<QLImage>) -> QLPreviewView {
guard let url = Bundle.main.url(forResource: name, withExtension: "gif")
else {
let _ = print("Cannot get image \(name)")
return QLPreviewView()
}
let preview = QLPreviewView(frame: .zero, style: .normal)
preview?.autostarts = true
preview?.previewItem = url as QLPreviewItem
return preview ?? QLPreviewView()
}
func updateNSView(_ nsView: QLPreviewView, context: NSViewRepresentableContext<QLImage>) {
guard let url = Bundle.main.url(forResource: name, withExtension: "gif")
else {
let _ = print("Cannot get image \(name)")
return
}
nsView.previewItem = url as QLPreviewItem
}
typealias NSViewType = QLPreviewView
}
struct QLImage_Previews: PreviewProvider {
static var previews: some View {
QLImage("preview-gif")
}
}
This will return a view, but you need to specify the frame size or you can't see anything. Use it like this it a view:
QLImage("myGif").frame(width: 500, height: 300, alignment: .center)
Set whatever frame size is appropriate. All the other view-related properties are also available with this.
Hope this helps somebody else who is trying to display GIF images from local resources on macOS. Apple should really make it easier to display animated GIF since it's being used everywhere these days.
I am trying to create a native package for flutter Swift/Kotlin, I built the UIViewController in a XCode project and moving the code to the flutter ios plugin therefore I added the MediaPickerController.swift file (as seen in the screenshot below).
In my SwiftMediaFilePickerPlugin.swift, I am trying to present the MediaPickerController but I am getting the "error: cannot find 'MediaPickerController' in scope"
The snippet of how I am presenting the MediaPickerController in SwiftMediaFilePickerPlugin.swift:-
let mediaPickerController = MediaPickerController(mediaType: mediaType, limit: limit)
UIViewController.topViewController().present(mediaPickerController, animated: true, completion: nil)
fileprivate extension UIViewController {
class func topViewController(base: UIViewController? = UIApplication.shared.keyWindow?.rootViewController) -> UIViewController? {
if let presented = base?.presentedViewController {
return topViewController(base: presented)
}
return base
}
}
The code is working fine on a seperate swift project
Open example/ios/Runner.xcworkspace in Xcode and add additional files to Pods/Development Pods/.../ios/Classes.
Is it possible to programmatically export 3D mesh as .usdz file format using ModelIO and MetalKit frameworks?
Here's a code:
import ARKit
import RealityKit
import MetalKit
import ModelIO
let asset = MDLAsset(bufferAllocator: allocator)
asset.add(mesh)
let filePath = FileManager.default.urls(for: .documentDirectory,
in: .userDomainMask).first!
let usdz: URL = filePath.appendingPathComponent("model.usdz")
do {
try asset.export(to: usdz)
let controller = UIActivityViewController(activityItems: [usdz],
applicationActivities: nil)
controller.popoverPresentationController?.sourceView = sender
self.present(controller, animated: true, completion: nil)
} catch let error {
fatalError(error.localizedDescription)
}
When I press a Save button I get an error.
The Andy Jazz answer is correct, but needs modification in order to work in a SwiftUI Sandboxed app:
First, the SCNScene needs to be rendered in order to export correctly. You can't create a bunch of nodes, stuff them into the scene's root node and call write() and get a correctly rendered usdz. It must first be put on screen in a SwiftUI SceneView, which causes all the assets to load, etc. I suppose you could instantiate a SCNRenderer and call prepare() on the root node, but that has some extra complications.
Second, the Sandbox prevents a direct export to a URL provided by .fileExporter(). This is because Scene.write() works in two steps: it first creates a .usdc export, and zips the resulting files into a single .usdz. The intermediate files don't have the write privileges the URL provided by .fileExporter() does (assuming you've set the Sandbox "User Selected File" privilege to "Read/Write"), so Scene.write() fails, even if the target URL is writeable, if the target directory is outside the Sandbox.
My solution was to write a custom FileWrapper, which I return if the WriteConfiguration UTType is .usdz:
public class USDZExportFileWrapper: FileWrapper {
var exportScene: SCNScene
public init(scene: SCNScene) {
exportScene = scene
super.init(regularFileWithContents: Data())
}
required init?(coder inCoder: NSCoder) {
fatalError("init(coder:) has not been implemented")
}
override public func write(to url: URL,
options: FileWrapper.WritingOptions = [],
originalContentsURL: URL?) throws {
let tempFilePath = NSTemporaryDirectory() + UUID().uuidString + ".usdz"
let tempURL = URL(fileURLWithPath: tempFilePath)
exportScene.write(to: tempURL, delegate: nil)
try FileManager.default.moveItem(at: tempURL, to: url)
}
}
Usage in a ReferenceFileDocument:
public func fileWrapper(snapshot: Data, configuration: WriteConfiguration) throws -> FileWrapper {
if configuration.contentType == .usdz {
return USDZExportFileWrapper(scene: scene)
}
return .init(regularFileWithContents: snapshot)
}
08th January 2023
At the moment iOS developers still can export only .usd, .usda and .usdc files; you can check this using canExportFileExtension(_:) type method:
let usd = MDLAsset.canExportFileExtension("usd")
let usda = MDLAsset.canExportFileExtension("usda")
let usdc = MDLAsset.canExportFileExtension("usdc")
let usdz = MDLAsset.canExportFileExtension("usdz")
print(usd, usda, usdc, usdz)
It prints:
true true true false
However, you can easily export SceneKit's scenes as .usdz files using instance method called: write(to:options:delegate:progressHandler:).
let path = FileManager.default.urls(for: .documentDirectory,
in: .userDomainMask)[0]
.appendingPathComponent("file.usdz")
sceneKitScene.write(to: path,
options: nil,
delegate: nil,
progressHandler: nil)
I'm new to swift development and building a simple app for the apple watch. I'd like a short sound to play when a button is tapped. Is this possible?
From what I understand the best way to do this is with AVFoundation and AVAudioPlayer. There seem to have been a lot of updates for this in the last few releases and I'm finding conflicting advice. Based on a few tutorials I've put this simple test together, but I'm getting a "Thread 1:Fatal error: Unexpectedly found nil when unwrapping an Optional value" Here is my code:
import WatchKit
import Foundation
import AVFoundation
var dogSound = AVAudioPlayer()
class InterfaceController: WKInterfaceController {
override func awake(withContext context: Any?) {
super.awake(withContext: context)
// Configure interface objects here.
}
override func willActivate() {
// This method is called when watch view controller is about to be visible to user
super.willActivate()
}
override func didDeactivate() {
// This method is called when watch view controller is no longer visible
super.didDeactivate()
}
#IBAction func playTapped() {
let path = Bundle.main.path(forResource: "Dog", ofType: ".mp3")!
let url = URL(fileURLWithPath: path)
do {
dogSound = try AVAudioPlayer(contentsOf: url)
dogSound.play()
} catch {
//couldn't load file :(
}
}
}
My audio file is an mp3 named Dog in the Assets.xcassets folder in WatchKit Extension and I have added AVFoundation.framework to the Link Binary with Libraries
What am I doing wrong and is there a tutorial for the right way to implement this? Thanks a lot!
I am trying to get my macOS Cocoa Application Xcode project to play some video when I feed it a specified URL.
To simplify the question lets just use an empty Xcode project.
I want to achieve the same level of control as I am able to get using AVPlayerViewController() in my iOS Single View Application. Within that app I am currently using the solution offered here.
However, only the second example works in my macOS Cocoa Application when adapted like so:
import Cocoa
import AVFoundation
import AVKit
class ViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let videoURL = URL(string: "https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4")
let player = AVPlayer(url: videoURL!)
let playerLayer = AVPlayerLayer(player: player)
playerLayer.frame = self.view.bounds
self.view.layer?.addSublayer(playerLayer)
player.play()
}
override var representedObject: Any? {
didSet {
// Update the view, if already loaded.
}
}
}
This option however has 2 downsides:
It creates a repetitive error message in the console:
2017-06-27 18:22:06.833441+0200 testVideoOnmacOS[24456:1216773] initWithSessionInfo: XPC connection interrupted
2017-06-27 18:22:06.834830+0200 testVideoOnmacOS[24456:1216773] startConfigurationWithCompletionHandler: Failed to get remote object proxy: Error Domain=NSCocoaErrorDomain Code=4097 "connection to service named com.apple.rtcreportingd" UserInfo={NSDebugDescription=connection to service named com.apple.rtcreportingd}
This AVPlayer solution does not offer the same control as the AVPlayerViewController() solution.
So I attempted to get the following AVPlayerViewController() solution to work. But it fails:
import Cocoa
import AVFoundation
import AVKit
class ViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let videoURL = URL(string: "https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4")
let player = AVPlayer(url: videoURL!)
let playerViewController = AVPlayerViewController()
playerViewController.player = player
self.present(playerViewController, animated: true) {
playerViewController.player!.play()
}
}
override var representedObject: Any? {
didSet {
// Update the view, if already loaded.
}
}
}
The error I am getting is:
Use of unresolved identifier 'AVPlayerViewController' - Did you mean
'AVPlayerViewControlsStyle'?
I have tried to work around this in many different ways. It will be confusing if I spell them all out. My prominent problem seems to be that the vast majority of online examples are iOS based.
Thus, the main question:
How would I be able to successfully use AVPlayerViewController within my Xcode macOS Cocoa Application?
Yes, got it working!
Thnx to ninjaproger's comment.
This seems to work in order to get a video from the web playing / streaming on a macOS Cocoa Application in Xcode.
First, create a AVKitPlayerView on your storyboard and link it as an IBOutlet to your NSViewController.
Then, simply apply this code:
import Cocoa
import AVFoundation
import AVKit
class ViewController: NSViewController {
#IBOutlet var videoView: AVPlayerView!
override func viewDidLoad() {
super.viewDidLoad()
let videoURL = URL(string: "https://clips.vorwaerts-gmbh.de/big_buck_bunny.mp4")
let player = AVPlayer(url: videoURL!)
let playerViewController = videoView
playerViewController?.player = player
playerViewController?.player!.play()
}
override var representedObject: Any? {
didSet {
// Update the view, if already loaded.
}
}
}
Update august 2021
Ok, it's been a while since this answer has been up and I am no longer invested in it. But it seems others still end up here and the solution needs a bit more work to work properly at times.
With thanks to #SouthernYankee65:
What is needed sometimes is to go to the project file and on the Signing & Capabilities tab check Outgoing Connection (Client). Then in the info.plist add App Transport Security Settings dictionary property, then add Allows Arbitrary Loads boolean property and set it to YES.
The video now plays. However, some errors in the console might occur.