Adding 3D object to ARGeoAnchor - swift

Please forgive me if this question is not that great. I've hit a bit of a road block on Apple's documentation of ARGeoAnchor.
Currently ARGeoAnchor just shows a blue dot in the AR Scene View. I'm trying to show any 3d rendereing or object instead.
My code:
let coordinate = CLLocationCoordinate2D(latitude: lat, longitude: lng)
let geoAnchor = ARGeoAnchor(name: "Point 1", coordinate: coordinate)
let boxGeometry = SCNBox(width: 0.1, height: 0.1, length: 0.1, chamferRadius: 0)
let cube = SCNNode(geometry: boxGeometry)
geoAnchor.scene.rootNode.addChildNode(cube)
self.addGeoAnchor(geoAnchor)
The error i'm getting: Value of type 'ARGeoAnchor' has no member 'scene'
I have multiple ARGeoAnchors, they are all currently showing blue dots. How do I get them to show custom 3d objects instead?
Thanks for taking a look!

At first you have to check if ARGeoTrackingConfiguration is supported on your device.
Requirements: You need a device with A12+ chip and cellular (GPS) support.
import ARKit
#main class AppDelegate: UIResponder, UIApplicationDelegate {
var window: UIWindow?
func application(_ application: UIApplication,
didFinishLaunchingWithOptions launchOptions:
[UIApplication.LaunchOptionsKey: Any]?) -> Bool {
if !ARGeoTrackingConfiguration.isSupported {
let sb = UIStoryboard(name: "Main", bundle: nil)
window?.rootViewController = sb.instantiateViewController(withIdentifier:
"unsupportedConfiguration")
}
return true
}
}
...then check whether Geo Tracking is available at your location:
ARGeoTrackingConfiguration.checkAvailability { (available, error) in
if !available {
let errorDescription = error?.localizedDescription ?? ""
let recommendation = "You need a place where geo tracking is supported."
let restart = UIAlertAction(title: "Restart", style: .default) { (_) in
self.restartSession()
}
self.alertUser(withTitle: "Geo tracking unavailable",
message: "\(errorDescription)\n\(recommendation)",
actions: [restart])
}
}
After that supply Info.plist with camera and location permission. Here's a comprehensive information on Authorization Request for Location Services what includes:
NSLocationWhenInUseUsageDescription
NSLocationAlwaysAndWhenInUseUsageDescription
NSLocationUsageDescription
NSLocationAlwaysUsageDescription
Only several cities in US, UK, Australia, Canada, Japan and Singapore are supported at the moment...
A list of cities that support ARGeoTrackingConfiguration.
Then you must run Geo Configuration:
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let config = ARGeoTrackingConfiguration()
sceneView.session.run(config)
}
Then add parametrised anchor to the session:
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
sceneView.scene = SCNScene()
let coordinate = CLLocationCoordinate2D(latitude: 40.730610,
longitude: -73.935242)
let geoAnchor = ARGeoAnchor(name: "Geo Anchor",
coordinate: coordinate,
altitude: 33.0)
sceneView.session.add(anchor: geoAnchor)
}
And after that you can add a model with the help of ARGeoAnchor:
extension ViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer,
didAdd node: SCNNode,
for anchor: ARAnchor) {
guard let geoAnchor = anchor as? ARGeoAnchor,
geoAnchor.name == "Geo Anchor"
else { return }
print(geoAnchor.coordinate)
let boxGeometry = SCNBox(width: 1.0,
height: 1.0,
length: 1.0,
chamferRadius: 0.1)
boxGeometry.firstMaterial?.diffuse.contents = UIColor.red
let cube = SCNNode(geometry: boxGeometry)
node.addChildNode(cube)
}
}
P.S.
If you're interested in how a similar feature works in ARCore, read my post about Geospatial API.

Related

Errors: 'Argument passed to call that takes no arguments' & 'Type of expression is ambiguous without more context'

Trying to render a 2D image to show on plane instead of the white/colored plane I have have currently. The following code renders the plane as colored. Tried the following to
the 2D image appear over the colored plane instead. The image I am using is in Assets folder and is: PNG image - 966 KB. Looked/read/researched Apple Docs, StackOflow searching for how.
import UIKit
import SceneKit
import ARKit
import RealityKit
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
// Set the view's delegate
sceneView.delegate = self
// Show statistics such as fps and timing information
sceneView.showsStatistics = true
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
// Create a session configuration
let configuration = ARImageTrackingConfiguration()
if let imageToTrack = ARReferenceImage.referenceImages(inGroupNamed: "BdayCardImages", bundle: Bundle.main) {
configuration.trackingImages = imageToTrack
configuration.maximumNumberOfTrackedImages = 1
}
// Run the view's session
sceneView.session.run(configuration)
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
// Pause the view's session
sceneView.session.pause()
}
// MARK: - ARSCNViewDelegatemake
//this method finds and triggers image plane
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: SCNNode) -> ARAnchor?
{
let node = SCNNode()
if let imageAnchor = anchor as? ARImageAnchor
{
let plane = SCNPlane(
width: imageAnchor.referenceImage.physicalSize.width,
height: imageAnchor.referenceImage.physicalSize.height
)
plane.firstMaterial?.diffuse.contents =
UIColor(red: 1, green: 1.3, blue: 0.5, alpha: 0.8)
let planeNode = SCNNode(geometry: plane)
if imageAnchor.referenceImage.name == "TamBDYCArd3"
{
if let bdayImage = ARImageAnchor(named: "assets/TamBDYCArd3") //Error here: 'Argument passed to call that takes no arguments'
{
if let imageNode = bdayImage.rootNode.childNode.first //Error here: Type of expression is ambiguous without more context'
{
imageNode.update(with: imageNode)
planeNode.addChildNode(imageNode)
if imageAnchor.referenceImage.name == "TamBDYCArd3" //the png image
{
if let bdayImage = ARImageAnchor(named: "assets/TamBDYCArd3")
{
if let imageNode = bdayImage.rootNode.childNode.first
{
imageNode.update(with: imageNode)
planeNode.addChildNode(imageNode)
planeNode.addChildNode(bdayImage)
}
}
}
}
return node
}
}
Any help is appreciated. I am a newb so thanks for your consideration and time.
I was expecting to render the 2D image over the plane. Thanks

"unrecognized selector sent to instance" error when adding ViewController

I've recently attempted to add a welcome screen to my AR app that works as a Home screen. When the app loads, the user can tap the button and then the app freezes, crashes and displays the code
"Thread 1: "-[_0_2_2020_2.WelcomeViewController letsGo:]: unrecognized selector sent to instance 0x13ec05e00"
I've tried a few of the solutions available, but I haven't been able to come up with a solution. I think it has something to do with my *IBAction connection. Any assistance is greatly appreciated!
import UIKit
import RealityKit
import ARKit
class WelcomeViewController: UIViewController {
#IBAction func gotPressed(_ sender: Any) {let storyboard = UIStoryboard(name: "Main",
bundle: nil)
if let viewController = storyboard.instantiateViewController(withIdentifier:
"ViewController") as? ViewController {
self.present(viewController, animated: true, completion: nil) /// present the view
controller (the one with the ARKit)!
} }
}
class ViewController: UIViewController, ARSessionDelegate {
//delay app launch to show splash screen
func application(_ application: UIApplication, didFinishLaunchingWithOptions
launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
Thread.sleep(forTimeInterval: 3.0)
// Override point for customization after application launch.
return true
}
//end splash screen delay
#IBOutlet var arView: ARView!
override func viewDidLoad() {
super.viewDidLoad()
arView.session.delegate = self
showModel()
overlayCoachingView()
setupARView()
arView.addGestureRecognizer(UITapGestureRecognizer(target: self, action: #selector(handleTap(recognizer:))))
}
func showModel(){
let anchorEntity = AnchorEntity(plane: .horizontal, minimumBounds:[0.7, 0.7])
anchorEntity.scale = [0.2, 0.2, 0.2]
let entity = try! Entity.loadModel(named: "COW_ANIMATIONS")
entity.setParent(anchorEntity)
arView.scene.addAnchor(anchorEntity)
}
//Overlay coaching view "adjust iphone scan"
func overlayCoachingView () {
let coachingView = ARCoachingOverlayView(frame: CGRect(x: 0, y: 0, width: arView.frame.width, height: arView.frame.height))
coachingView.session = arView.session
coachingView.activatesAutomatically = true
coachingView.goal = .horizontalPlane
view.addSubview(coachingView)
}//end overlay
func setupARView(){
arView.automaticallyConfigureSession = false
let configuration = ARWorldTrackingConfiguration()
configuration.planeDetection = [.horizontal, .vertical]
configuration.environmentTexturing = .automatic
arView.session.run(configuration)
}
//object placement
#objc
func handleTap(recognizer: UITapGestureRecognizer){
let location = recognizer.location(in:arView)
let results = arView.raycast(from: location, allowing: .estimatedPlane, alignment: .horizontal)
if let firstResult = results.first {
let brownCowAnchor = ARAnchor(name: "COW_ANIMATIONS", transform: firstResult.worldTransform)
arView.session.add(anchor: brownCowAnchor)
} else {
print("Object placement failed - couldn't find surface.")
//cow animations
//let robot = try! ModelEntity.load(named: "COW_ANIMATIONS")
let brownCowAnchor = AnchorEntity()
let blackCowAnchor = AnchorEntity()
//anchor.children.append(robot)
//arView.scene.anchors.append(anchor)
//robot.playAnimation(robot.availableAnimations[0].repeat(duration: .infinity),
//transitionDuration: 0.5,
//startsPaused: false)
//start cow animation
let brownCow = try! ModelEntity.load(named: "COW_ANIMATIONS")
let blackCow = try! ModelEntity.load(named: "Cow")
brownCow.position.x = -1.0
blackCow.position.x = 1.0
brownCowAnchor.position.z = -2.0
blackCowAnchor.position.z = -2.0
brownCow.setParent(brownCowAnchor)
blackCow.setParent(blackCowAnchor)
arView.scene.anchors.append(brownCowAnchor)
arView.scene.anchors.append(blackCowAnchor)
let cowAnimationResource = brownCow.availableAnimations[0]
let horseAnimationResource = blackCow.availableAnimations[0]
brownCow.playAnimation(cowAnimationResource.repeat(duration: .infinity),
transitionDuration: 1.25,
startsPaused: false)
blackCow.playAnimation(horseAnimationResource.repeat(duration: .infinity),
transitionDuration: 0.75,
startsPaused: false)
//end cow animations
}
}
func placeObject(named entityName: String, for anchor: ARAnchor) {
let entity = try! ModelEntity.loadModel(named: entityName)
entity.generateCollisionShapes(recursive: true)
arView.installGestures([.rotation, .translation], for: entity)
let anchorEntity = AnchorEntity(anchor: anchor)
anchorEntity.addChild(entity)
arView.scene.addAnchor(anchorEntity)
}
}
The button is triggering a function letsGo which doesn't appear anywhere on the WelcomeViewController you posted. Check interface builder and make sure that you've removed the old connection from the button. Should be the final tab.

Distinguish between multiple tracked images at the same time?

I currently have a group of 5 reference images that my app is tracking. However, I want it to be able distinguish between the reference images (i.e. if the image is of image A, image B, or image C. Tracking real photos I print out on postcards).
Currently, my code is able to detect the image and put a plane (i.e. a simple rectangle) over it. However, my question is, is it possible to distinguish whether the app has detected a picture of picture A vs picture B? If so, how?
I know there's an option to use ML, but wanted to see if there were any easier options in SceneKit/ARKit that I haven't considered; especially since I'm using the exact image and not trying to have the app guess an object.
class ViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
}
//MARK-: WHERE I CONFIGURE MY APP TO DETECT AN IMAGE
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARImageTrackingConfiguration()
guard let trackedImages = ARReferenceImage.referenceImages(inGroupNamed: "Photos", bundle: Bundle.main) else {
print("No images available")
return
}
configuration.trackingImages = trackedImages
configuration.maximumNumberOfTrackedImages = 7
sceneView.session.run(configuration)
}
//MARK-: WHERE I PLACE A PLANE OVER THE DETECTED IMAGE
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
if let imageAnchor = anchor as? ARImageAnchor {
let plane = SCNPlane(width: imageAnchor.referenceImage.physicalSize.width, height: imageAnchor.referenceImage.physicalSize.height)
plane.firstMaterial?.diffuse.contents = UIColor(white: 1, alpha: 0.8)
let planeNode = SCNNode(geometry: plane)
planeNode.eulerAngles.x = -.pi / 2
node.addChildNode(planeNode)
}
return node
}
}
You can easily do it using such instance properties as referenceImage and name.
// The detected image referenced by the image anchor.
var referenceImage: ARReferenceImage { get }
and:
// A descriptive name for your reference image.
var name: String? { get set }
Here's how they look like in code:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor,
let _ = imageAnchor.referenceImage.name
else { return }
anchorsArray.append(imageAnchor)
if imageAnchor.referenceImage.name == "apple" {
print("Image with apple is successfully detected...")
}
}
As per Andy's solution, I used the code below, and used the variable imageName as a reference.
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
guard let imageAnchor = anchor as? ARImageAnchor else { return }
let referenceImage = imageAnchor.referenceImage
let imageName = referenceImage.name ?? "no name"
}

Is it possible to detect object using CoreML model and find measurement of that object?

I want to detect object categories like door, window using CoreML and ARKit and I want to find measurements (like height, width and area) of a door.
How can I detect objects and add some overlay shape on that object so I could find real world position and measurement of that object?
Use ARKit's built-in object detection algorithm for that task. It's simple and power.
With ARKit's object detection you can detect your door (preliminary scanned or shot on smartphone).
The following code helps you detect real world objects (like door) and place 3D object or 3D text at ARObjectAnchor position:
import ARKit
extension ViewController: ARSCNViewDelegate {
func renderer(_ renderer: SCNSceneRenderer,
didAdd node: SCNNode,
for anchor: ARAnchor) {
if let _ = anchor as? ARObjectAnchor {
let text = SCNText(string: "SIZE OF THIS OBJECT IS...",
extrusionDepth: 0.05)
text.flatness = 0.5
text.font = UIFont.boldSystemFont(ofSize: 10)
let textNode = SCNNode(geometry: text)
textNode.geometry?.firstMaterial?.diffuse.contents = UIColor.white
textNode.scale = SCNVector3(0.01, 0.01, 0.01)
node.addChildNode(textNode)
}
}
}
And supply an Xcode's folder Resources with images of your real-life objects.
class ViewController: UIViewController {
#IBOutlet var sceneView: ARSCNView!
let configuration = ARWorldTrackingConfiguration()
override func viewDidLoad() {
super.viewDidLoad()
sceneView.debugOptions = .showFeaturePoints
sceneView.delegate = self
guard let dObj = ARReferenceObject.referenceObjects(inGroupNamed: "Resources",
bundle: nil)
else {
fatalError("There's no reference image")
return
}
configuration.detectionObjects = dObj
sceneView.session.run(configuration)
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
sceneView.session.pause()
}
}

Q: How to add multiple objects AR dectection

I added an ar function into my app to dectect our products. One object is working but i want to add multiple arobject files. I scanned some objects and added them into my ar recource group.
I created a product.sks and added a label + background.
My first question: How can i fix the label to one object? I have two arobjects at the moment so i need 2 different labels. How can i differ the correct label to an object?
My second question: At the moment my HU label is fixed and doesnt move when i'm moving my iPhone. I tried to change some positions etc. but its always fixed.
How can i make it move?
Thanks in advance. I hope my concern is detailed enough
import UIKit
import SceneKit
import ARKit
class ARViewController: UIViewController, ARSCNViewDelegate {
#IBOutlet var sceneView: ARSCNView!
override func viewDidLoad() {
super.viewDidLoad()
sceneView.delegate = self
sceneView.showsStatistics = true
let scene = SCNScene(named: "art.scnassets/scene.scn")!
sceneView.scene = scene
}
override func viewWillAppear(_ animated: Bool) {
super.viewWillAppear(animated)
let configuration = ARWorldTrackingConfiguration()
configuration.detectionObjects = ARReferenceObject.referenceObjects(inGroupNamed: "Module", bundle: Bundle.main)!
sceneView.session.run(configuration)
}
override func viewWillDisappear(_ animated: Bool) {
super.viewWillDisappear(animated)
sceneView.session.pause()
}
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
return node
}
func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
let node = SCNNode()
if let objectAnchor = anchor as? ARObjectAnchor {
let plane = SCNPlane(width: CGFloat(objectAnchor.referenceObject.extent.x * 0.8), height: CGFloat(objectAnchor.referenceObject.extent.y * 0.5))
plane.cornerRadius = plane.width * 0.125
let displayScene = SKScene(fileNamed: "product")
plane.firstMaterial?.diffuse.contents = displayScene
plane.firstMaterial?.isDoubleSided = true
plane.firstMaterial?.diffuse.contentsTransform = SCNMatrix4Translate(SCNMatrix4MakeScale(1, -1, 1), 0, 1, 0)
let planeNode = SCNNode(geometry: plane)
planeNode.position = SCNVector3Make(objectAnchor.referenceObject.center.x, objectAnchor.referenceObject.center.y + 0.12, objectAnchor.referenceObject.center.z)
node.addChildNode(planeNode)
}
return node
}
func session(_ session: ARSession, didFailWithError error: Error) {
// Present an error message to the user
}
func sessionWasInterrupted(_ session: ARSession) {
// Inform the user that the session has been interrupted, for example, by presenting an overlay
}
func sessionInterruptionEnded(_ session: ARSession) {
// Reset tracking and/or remove existing anchors if consistent tracking is required
}
}
Use anchor.referenceObject.name?
Move how? If you want it to always face the phone, apply billboard constraint.