How to get a vertex position in ARKit's reconstructed mesh? - swift

Question

Recently, I make it about lidar scan project.
It is very difficult. And I need to manipulate vertex data.
So I tried by this code
guard let meshAnchors = arView.session.currentFrame?.anchors.compactMap { $0 as? ARMeshAnchor }
else { return }
meshAnchors.first?.geometry.vertices // I want to get vertex position
There is no position of vertex, only buffer data
How can I do that? is it change from buffer data to array?
Plz help me.

Answer 1

Just went through this myself so I figured I'd drop ya my solution.
First grab this extension from Apple's Documentation to get a vertex at a specific index:
extension ARMeshGeometry {
func vertex(at index: UInt32) -> SIMD3<Float> {
assert(vertices.format == MTLVertexFormat.float3, "Expected three floats (twelve bytes) per vertex.")
let vertexPointer = vertices.buffer.contents().advanced(by: vertices.offset + (vertices.stride * Int(index)))
let vertex = vertexPointer.assumingMemoryBound(to: SIMD3<Float>.self).pointee
return vertex
}
}
Then, to get the positions in ARKit world space, you can do something like this:
func getVertexWorldPositions(frame: ARFrame) {
let anchors = frame.anchors.filter { $0 is ARMeshAnchor } as! [ARMeshAnchor]
// Each mesh geometry lives in its own anchor
for anchor in anchors {
// Anchor's transform in world space
let aTrans = SCNMatrix4(anchor.transform)
let meshGeometry = anchor.geometry
let vertices: ARGeometrySource = meshGeometry.vertices
for vIndex in 0..<vertices.count {
// This will give you a vertex in local (anchor) space
let vertex = meshGeometry.vertex(at: UInt32(vIndex))
// Create a new matrix with the vertex coordinates
let vTrans = SCNMatrix4MakeTranslation(vertex[0], vertex[1], vertex[2])
// Multiply it by the anchors's transform to get it into world space
let wTrans = SCNMatrix4Mult(vTrans, aTrans)
// Use the coordinates for something!
let vPos = SCNVector3(wTrans.m41, wTrans.m42, wTrans.m43)
print(vPos)
}
}
}