I’ve been experimenting with ARKit on Swift Playgrounds. I’ve written the starter code, but when I run it nothing happens. Instead of evaluating the code, it displays the pop up that shows ay issues in the code.
I know the code I’m using works, because I’ve used the same code on an iPad running an older version of Swift Playgrounds and the code works perfectly. It seems to be a problem with either Swift Playgrounds 3 or Swift 5.
Here’s the interesting part. When I remove the line of code that runs the ARWorldTrackingConfiguration initializer, and the code that makes the view controller the delegate of the session and scene, the code runs just fine. When I put it back, it does the same error again. I don’t know what’s going wrong.
I’m running Swift Playgrounds 3.0 on and iPad 6th Generation. The playground uses ARKit, UIKit, SceneKit, and PlaygroundSupport.
Lastly, here’s some code.
// Code inside modules can be shared between pages and other source files.
import ARKit
import SceneKit
import UIKit
extension ARSCNView {
public func setup(){
antialiasingMode = .multisampling4X
automaticallyUpdatesLighting = false
preferredFramesPerSecond = 60
contentScaleFactor = 1.0
if let camera = pointOfView?.camera {
camera.wantsHDR = true
camera.wantsExposureAdaptation = true
camera.exposureOffset = -1
camera.minimumExposure = -1
camera.maximumExposure = 3
}
}
}
public class vc : UIViewController, ARSessionDelegate, ARSCNViewDelegate {
var arscn : ARSCNView!
var scene : SCNScene!
public override func loadView() {
arscn = ARSCNView(frame: CGRect(x: 0, y: 0, width: 768, height: 1024))
arscn.delegate = self
arscn.setup()
scene = SCNScene()
arscn.scene = scene
var config = ARWorldTrackingConfiguration()
config.planeDetection = .horizontal
arscn.session.delegate = self
self.view = arscn
arscn.session.run(configåå)
}
public func session(_ session: ARSession, didFailWithError error: Error) {
// Present an error message to the user
}
public func sessionWasInterrupted(_ session: ARSession) {
// Inform the user that the session has been interrupted, for example, by presenting an overlay
}
public func sessionInterruptionEnded(_ session: ARSession) {
// Reset tracking and/or remove existing anchors if consistent tracking is required
}
}
Lastly, please note that I’m presenting the live view in the main playground page and putting the class in the shared code.
I’ve figured out a way to make this work. All I had to do was assign the view controller to a variable and then present the variable. I’m not exactly sure why this works, I just know it does.
import ARKit
import SceneKit
import UIKit
import PlaygroundSupport
public class LiveVC: UIViewController, ARSessionDelegate, ARSCNViewDelegate {
let scene = SCNScene()
public var arscn = ARSCNView(frame: CGRect(x: 0,y: 0,width: 640,height: 360))
override public func viewDidLoad() {
super.viewDidLoad()
arscn.delegate = self
arscn.session.delegate = self
arscn.scene = scene
let config = ARWorldTrackingConfiguration()
config.planeDetection = [.horizontal]
arscn.session.run(config)
view.addSubview(arscn)
}
public func session(_ session: ARSession, didFailWithError error: Error) {}
public func sessionWasInterrupted(_ session: ARSession) {}
public func sessionInterruptionEnded(_ session: ARSession) {}
}
var vc = LiveVC()
PlaygroundPage.current.liveView = vc
PlaygroundPage.current.needsIndefiniteExecution = true
Use UpperCamelCasing for classes' names and add two strings of code in the bottom.
This code is suitable for macOS Xcode Playground and iPad Swift Playgrounds:
import ARKit
import PlaygroundSupport
class LiveVC: UIViewController, ARSessionDelegate, ARSCNViewDelegate {
let scene = SCNScene()
var arscn = ARSCNView(frame: CGRect(x: 0,
y: 0,
width: 640,
height: 360))
override func viewDidLoad() {
super.viewDidLoad()
arscn.delegate = self
arscn.session.delegate = self
arscn.scene = scene
let config = ARWorldTrackingConfiguration()
config.planeDetection = [.horizontal]
arscn.session.run(config)
}
func session(_ session: ARSession, didFailWithError error: Error) {}
func sessionWasInterrupted(_ session: ARSession) {}
func sessionInterruptionEnded(_ session: ARSession) {}
}
PlaygroundPage.current.liveView = LiveVC().arscn
PlaygroundPage.current.needsIndefiniteExecution = true
P.S. Tip for Playground on macOS (although it doesn't have much sense when using ARKit module):
To turn on Live View in Xcode Playground 11.0 and higher use the following shortcut:
Command+Option+Return
Related
Trying to use my RealityKit project as the foundation for an on screen app (VR) instead of projecting onto the real-world (AR) out the back camera.
Anyone know how to load a RealityKit project asynchronously with the .nonAR camera option, so it project in an app instead of leveraging the rear facing camera?
Do I create position information in the Swift code or the Reality Composer project?
Here's how you can asynchronously load .usdz VR-model with a help of RealityKit's .loadModelAsync() instance method and Combine's AnyCancellable type.
import UIKit
import RealityKit
import Combine
class VRViewController: UIViewController {
#IBOutlet var arView: ARView!
var anyCancellable: AnyCancellable? = nil
let anchorEntity = AnchorEntity(world: [0, 0,-2])
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
arView.backgroundColor = .black
arView.cameraMode = .nonAR
anyCancellable = ModelEntity.loadModelAsync(named: "horse").sink(
receiveCompletion: { _ in
self.anyCancellable?.cancel()
},
receiveValue: { [self] (object: Entity) in
if let model = object as? ModelEntity {
self.anchorEntity.addChild(model)
self.arView.scene.anchors.append(self.anchorEntity)
} else {
print("Can't load a model due to some issues")
}
}
)
}
}
However, if you wanna move inside 3D environment, instead of using .nonAR camera mode use:
arView.environment.background = .color(.black)
I am using RealityKit + SwiftUI + ARSessionDelegate to render 3D content on top of an ARReferenceObject. I want to remove the 3D content once the camera pans away from the object and it is no longer in the frame.
Currently I render the 3D content when the object is detected, which is what I want. But I have multiple identical objects that I want to identify separately using the same ARReferenceObject. So in order to do this I need to remove the original anchoring.
This is my wrapper for SWiftUI:
struct ARViewWrapper: UIViewRepresentable {
#ObservedObject var arManager: ARManager
// cretae alias for our wrapper
typealias UIViewType = ARView
// delegate for view representable
func makeCoordinator() -> Coordinator {
return Coordinator(arManager: self.arManager)
}
func makeUIView(context: Context) -> ARView {
// create ARView
let arView = ARView(frame: .zero, cameraMode: .ar, automaticallyConfigureSession: true)
// assign delegate
arView.session.delegate = context.coordinator
return arView
}
func updateUIView(_ uiView: ARView, context: Context) {
print("Updating View")
// create anchor using an image and add it to the ARView
let target = AnchorEntity(.object(group: "AR Resources", name: "bj"))
target.name = "obj_anchor"
// add anchor to AR world
if(uiView.scene.anchors.count == 0){
uiView.scene.anchors.append(target)
}else{
uiView.scene.anchors[0] = target
}
// add plane and title to anchor
addARObjs(anchor: target, arObj: arManager.currARObj)
return()
}
}
This is my Delegate:
class Coordinator: NSObject, ARSessionDelegate {
#ObservedObject var arManager: ARManager
init(arManager: ARManager) {
self.arManager = arManager
}
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
return
}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]){
return
}
func session(_ session: ARSession, didUpdate frame: ARFrame) {
return
}
}
SceneKit
You can do it in SceneKit. All you need is to use isNode(_:insideFrustumOf:) instance method that returns a Boolean value indicating whether a node might be visible from a specified point of view or not. This method is also implemented in ARKit (as a part of SceneKit).
func isNode(_ node: SCNNode, insideFrustumOf pointOfView: SCNNode) -> Bool
Sample code:
var allYourNodes = [SCNNode]()
allYourNodes.append(node001)
allYourNodes.append(node002)
guard let pointOfView = arSCNView.pointOfView
else { return }
for yourNode in allYourNodes {
if !arView.isNode(yourNode, insideFrustumOf: pointOfView) {
arSCNView.session.remove(anchor: yourARAnchor)
}
}
However, I haven't found a similar method in RealityKit 2.0. Hope it'll be added by Cupertino engineers in the near future.
RealityKit
Here's what we have in RealityKit 2.0 at the moment:
Apple's documentation says: During an AR session, RealityKit automatically uses the device’s camera to define the perspective from which to render the scene. When rendering a scene outside of an AR session – with the view’s cameraMode property set to
ARView.CameraMode.nonAR
RealityKit uses a PerspectiveCamera instead. You can add a perspective camera anywhere in your scene to control the point of view. If you don't explicitly provide one, RealityKit creates a default camera for you.
So, the only available parameters of a PerspectiveCameraComponent at the moment are:
init(near: Float, far: Float, fieldOfViewInDegrees: Float)
Can I see an example using a RealityKit ARView with ARImageTrackingConfiguration including the ARSessionDelegate delegate methods?
Here is an example of a RealityKit ARView using ARImageTrackingConfiguration and the ARSessionDelegate delegate methods. I didn't see a complete example of exactly this on Stack Overflow so thought I would ask/answer it myself.
import ARKit
import RealityKit
class ViewController: UIViewController, ARSessionDelegate {
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
// There must be a set of reference images in project's assets
guard let referenceImages = ARReferenceImage.referenceImages(inGroupNamed: "AR Resources", bundle: nil) else { fatalError("Missing expected asset catalog resources.") }
// Set ARView delegate so we can define delegate methods in this controller
arView.session.delegate = self
// Forgo automatic configuration to do it manually instead
arView.automaticallyConfigureSession = false
// Show statistics if desired
arView.debugOptions = [.showStatistics]
// Disable any unneeded rendering options
arView.renderOptions = [.disableCameraGrain, .disableHDR, .disableMotionBlur, .disableDepthOfField, .disableFaceOcclusions, .disablePersonOcclusion, .disableGroundingShadows, .disableAREnvironmentLighting]
// Instantiate configuration object
let configuration = ARImageTrackingConfiguration()
// Both trackingImages and maximumNumberOfTrackedImages are required
// This example assumes there is only one reference image named "target"
configuration.maximumNumberOfTrackedImages = 1
configuration.trackingImages = referenceImages
// Note that this config option is different than in world tracking, where it is
// configuration.detectionImages
// Run an ARView session with the defined configuration object
arView.session.run(configuration)
}
func session(_ session: ARSession, didAdd anchors: [ARAnchor]) {
// This example assumes only one reference image of interest
// A for-in loop could work for more targets
// Ensure the first anchor in the list of added anchors can be downcast to an ARImageAnchor
guard let imageAnchor = anchors[0] as? ARImageAnchor else { return }
// If the added anchor is named "target", do something with it
if let imageName = imageAnchor.name, imageName == "target" {
// An example of something to do: Attach a ball marker to the added reference image.
// Create an AnchorEntity, create a virtual object, add object to AnchorEntity
let refImageAnchor = AnchorEntity(anchor: imageAnchor)
let refImageMarker = generateBallMarker(radius: 0.02, color: .systemPink)
refImageMarker.position.y = 0.04
refImageAnchor.addChild(refImageMarker)
// Add new AnchorEntity and its children to ARView's scene's anchor collection
arView.scene.addAnchor(refImageAnchor)
// There is now RealityKit content anchored to the target reference image!
}
}
func session(_ session: ARSession, didUpdate anchors: [ARAnchor]) {
guard let imageAnchor = anchors[0] as? ARImageAnchor else { return }
// Assuming only one reference image. A for-in loop could work for more targets
if let imageName = imageAnchor.name, imageName == "target" {
// If anything needs to be done as the ref image anchor is updated frame-to-frame, do it here
// E.g., to check if the reference image is still being tracked:
// (https://developer.apple.com/documentation/arkit/artrackable/2928210-istracked)
if imageAnchor.isTracked {
print("\(imageName) is tracked and has a valid transform")
} else {
print("The anchor for \(imageName) is not guaranteed to match the movement of its corresponding real-world feature, even if it remains in the visible scene.")
}
}
}
// Convenience method to create colored spheres
func generateBallMarker(radius: Float, color: UIColor) -> ModelEntity {
let ball = ModelEntity(mesh: .generateSphere(radius: radius), materials: [SimpleMaterial(color: color, isMetallic: false)])
return ball
}
}
I'm a beginner on TVOS.
I'd like to create an hybrid app on AppleTV using a native app and TVMLKIT.
My native application is just a simple native app with buttons (using swift).
When we click on a button, I launch a a javascript app using TVLMKIT and TVJS.
My TVJS as uses the Player to display a video.
When the video is over, I want to close the TVJS app and back to the native ViewController.
My problem is that when I back to native app, I loose the focus on my native View (the app is frozen).
native ViewController:
import UIKit
import TVMLKit
class ViewController: UIViewController, TVApplicationControllerDelegate {
var window: UIWindow?
var appController: TVApplicationController?
var appControllerContext = TVApplicationControllerContext();
static let TVBaseURL = "http://localhost:9001/"
static let TVBootURL = "\(ViewController.TVBaseURL)/client/js/application.js"
override func viewDidLoad() {
super.viewDidLoad()
// Do any additional setup after loading the view, typically from a nib.
}
override func didReceiveMemoryWarning() {
super.didReceiveMemoryWarning()
// Dispose of any resources that can be recreated.
}
#IBOutlet weak var label: UILabel!
#IBOutlet weak var viewAd: UIView!
#IBAction func clickOnlaunchAd(sender: AnyObject) {
window = UIWindow(frame: UIScreen.mainScreen().bounds)
guard let javaScriptURL = NSURL(string: ViewController.TVBootURL) else {
fatalError("unable to create NSURL")
}
appControllerContext.javaScriptApplicationURL = javaScriptURL
appControllerContext.launchOptions["BASEURL"] = ViewController.TVBaseURL
appController = TVApplicationController(context: appControllerContext, window: window,delegate: self)
}
#IBAction func clickOnChangeText(sender: AnyObject) {
label.text = "changed";
}
func appController(appController: TVApplicationController, didStopWithOptions options: [String : AnyObject]?) {
self.setNeedsFocusUpdate()
self.updateFocusIfNeeded()
}
func appController(appController: TVApplicationController, evaluateAppJavaScriptInContext jsContext: JSContext){
let notifyEventToNative : #convention(block) (NSString!) -> Void = {
(string : NSString!) -> Void in
print("[log]: \(string)\n")
self.appController?.stop()
}
jsContext.setObject(unsafeBitCast(notifyEventToNative, AnyObject.self), forKeyedSubscript: "notifyEventToNative")
}
}
Just before calling "notifyEventToNative" from my TVJS, I call "navigationDocument.clear();" to clear the TVML view.
I can see my native app but I can't interact with it.
Any ideas?
Thanks.
I also had the same problem. I was opened a TVML document from the UIViewController. And I also lost the focus. So, first of all I can advice you to override var called preferredFocusedView in your ViewController. In this method you can return reference to viewAd. But the better solution would be to wrap your ViewController into the TVML-item (with the TVMLKit framework). In that case I hope that you will have no problems with focus because you will use TVML during the whole application.
I am a beginning Swift programmer.
The following code seems to compile fine in Xcode 7.0 Playground (no visible errors):
//: Playground - noun: a place where people can play
//#!/usr/bin/env xcrun swift
import WebKit
let application = NSApplication.sharedApplication()
application.setActivationPolicy(NSApplicationActivationPolicy.Regular)
let window = NSWindow()
window.setContentSize(NSSize(width:800, height:600))
window.styleMask = NSTitledWindowMask | NSClosableWindowMask | NSMiniaturizableWindowMask
window.center()
window.title = "Minimal Swift WebKit Browser"
window.makeKeyAndOrderFront(window)
class WindowDelegate: NSObject, NSWindowDelegate {
func windowWillClose(notification: NSNotification) {
NSApplication.sharedApplication().terminate(0)
}
}
let windowDelegate = WindowDelegate()
window.delegate = windowDelegate
class ApplicationDelegate: NSObject, NSApplicationDelegate {
var _window: NSWindow
init(window: NSWindow) {
self._window = window
}
func applicationDidFinishLaunching(notification: NSNotification) {
let webView = WebView(frame: self._window.contentView!.frame)
self._window.contentView!.addSubview(webView)
webView.mainFrame.loadRequest(NSURLRequest(URL: NSURL(string: "http://www.apple.com")!))
}
}
When pasting that exact same code into the "AppDelegate.swift" file of a new Cocoa application for OSX, I get 7 errors, all exactly the same: "Expressions are not allowed at the top level".
Through searching I've deduced that the Playground allows things that normal projects do not and the errors are occurring because the expressions are "outside of a class or instance method".
However I'm not sure how the program could be modified in order to build correctly.
Yes, normal projects do not allow code at the top level, because there is no obvious time for it to run. You need to decide when your activation policy and window/delegate code should run (that is, move that code inside of a method). I suggest applicationDidFinishLaunching(_:), as it is called when your app is finished launching and is a common place to do this kind of setup. The finished code would read:
import WebKit
class WindowDelegate: NSObject, NSWindowDelegate {
func windowWillClose(notification: NSNotification) {
NSApplication.sharedApplication().terminate(0)
}
}
class ApplicationDelegate: NSObject, NSApplicationDelegate {
var _window: NSWindow
init(window: NSWindow) {
self._window = window
}
func applicationDidFinishLaunching(notification: NSNotification) {
let webView = WebView(frame: self._window.contentView!.frame)
self._window.contentView!.addSubview(webView)
webView.mainFrame.loadRequest(NSURLRequest(URL: NSURL(string: "http://www.apple.com")!))
let application = NSApplication.sharedApplication()
application.setActivationPolicy(NSApplicationActivationPolicy.Regular)
let window = NSWindow()
window.setContentSize(NSSize(width:800, height:600))
window.styleMask = NSTitledWindowMask | NSClosableWindowMask | NSMiniaturizableWindowMask
window.center()
window.title = "Minimal Swift WebKit Browser"
window.makeKeyAndOrderFront(window)
let windowDelegate = WindowDelegate()
window.delegate = windowDelegate
}
}