ARKit, detect when tracking is available - swift

Question

I'm looking for a way to detect when spacial tracking is "working/not working" in ARKit, i.e when ARKit has enough visual information to start the 3d spacial tracking.
In other apps i've tried, the user gets prompted to look around with the phone/camera to resume space tracking if ARKit doesn't get enough information from the camera. I have even seen apps with a progress bar showing how much more the user needs to move the device around to resume tracking.
Would a good way to detect if tracking is available to check how many rawFeaturePoints the ARSessions current frame has? E.g if the current frame has more than say 100 rawFeaturePoints, we can assume that spacial tracking is working.
Would this be a good approach, or is there built in functionality or better way in ARKit to detect if spacial tracking is working what i don't know about?

Answer 1

You could use feature points but I think that is probably overkill, as such something like this might be a good start:
Using the currentFrame of an ARSession you can get the current tracking state from like so:
//------------------------------------------------
//MARK: ARSession Extension To Log Tracking States
//------------------------------------------------
extension ARSession{
/// Returns The Status Of The Current ARSession
///
/// - Returns: String
func sessionStatus() -> String? {
//1. Get The Current Frame
guard let frame = self.currentFrame else { return nil }
var status = "Preparing Device.."
//1. Return The Current Tracking State & Lighting Conditions
switch frame.camera.trackingState {
case .normal: status = ""
case .notAvailable: status = "Tracking Unavailable"
case .limited(.excessiveMotion): status = "Please Slow Your Movement"
case .limited(.insufficientFeatures): status = "Try To Point At A Flat Surface"
case .limited(.initializing): status = "Initializing"
case .limited(.relocalizing): status = "Relocalizing"
}
guard let lightEstimate = frame.lightEstimate?.ambientIntensity else { return nil }
if lightEstimate < 100 { status = "Lighting Is Too Dark" }
return status
}
}
Which you would call something like this in the ARSCNViewDelegate callback:
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
DispatchQueue.main.async {
//1. Update The Tracking Status
print(self.augmentedRealitySession.sessionStatus())
}
}
There are also other delegate callbacks you can use as well e.g:
func session(_ session: ARSession, didFailWithError error: Error) {
print("The ARSession Failed")
}
func sessionWasInterrupted(_ session: ARSession) {
print("ARSession Was Interupted")
}
These ARKit Guidelines also provide some useful information as how to handle these states: Apple Guidelines
If you do actually want to track the number of featurePoints however you can do something like this:
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
guard let currentFrame = self.augmentedRealitySession.currentFrame,
let featurePointCount = currentFrame.rawFeaturePoints?.points.count else { return }
print("Number Of Feature Points In Current Session = \(featurePointCount)")
}
And if you want to see an example you can have a look here: Feature Points Example
Hope it helps...