How to create a SceneKit SCNSkinner object in code? - swift

Question

I have a Swift app using SceneKit for iOS 8. I load a scene from a .dae file that contains a mesh controlled by a skeleton.
At runtime, I need to modify the texture coordinates. Using a transform is not an option -- I need to compute a different, completely new UV for each vertex in the mesh.
I know geometry is immutable in SceneKit, and I've read that the suggested approach is to make a copy manually. I'm trying to do that, but I always end up crashing when trying to re-create the SCNSkinner in code. The crash is an EXC_BAD_ACCESS inside C3DSourceAccessorGetMutableValuePtrAtIndex. Unfortunately, there is no source code for this, so I'm not sure why exactly it's crashing. I've narrowed it down to the SCNSkinner object attached to the mesh node. If I do not set that, I don't get a crash and things appear to be working.
EDIT: Here is a more complete call stack of the crash:
C3DSourceAccessorGetMutableValuePtrAtIndex
C3DSkinPrepareMeshForGPUIfNeeded
C3DSkinnerMakeCurrentMesh
C3DSkinnerUpdateCurrentMesh
__CFSetApplyFunction_block_invoke
CFBasicHashApply
CFSetApplyFunction
C3DAppleEngineRenderScene
...
I've not found any documentation or example code about how to create an SCNSkinner object manually. Since I'm just creating it based on a previously working mesh, it shouldn't be too difficult. I'm creating the SCNSkinner according to the Swift documentation, passing all of the correct things into the init. However, there is a skeleton property in the SCNSkinner that I'm not sure how to set. I set it to the skeleton that was on the original SCNSkinner of the mesh I'm copying, which I think
should work... but it doesn't. When setting the skeleton property, it does not appear to be assigning. Checking it immediately after the assignment shows that it is still nil. As a test, I tried to set the original mesh's skeleton property to something else, and after the assignment it was left untouched as well.
Can anyone shed any light on what is happening? Or how to correctly create and set up an SCNSkinner object manually?
Here is the code I'm using to manually clone a mesh and replace it with a new one (I have not modified any of the source data here -- I'm simply trying to make sure I can create a copy at this point):
// This is at the start of the app, just so you can see how the scene is set up.
// I add the .dae contents into its own node in the scene. This seems to be the
// standard way to put multiple .dae models into the same scene. This doesn't seem to
// have any impact on the problem I'm having -- I've tried without this indirection
// and the same problem exists.
let scene = SCNScene()
let modelNode = SCNNode()
modelNode.name = "ModelNode"
scene.rootNode.addChildNode(modelNode)
let modelScene = SCNScene(named: "model.dae")
if modelScene != nil {
if let childNodes = modelScene?.rootNode.childNodes {
for childNode in childNodes {
modelNode.addChildNode(childNode as SCNNode)
}
}
}
// This happens later in the app after a tap from the user.
let modelNode = scnView.scene!.rootNode.childNodeWithName("ModelNode", recursively: true)
let modelMesh = modelNode?.childNodeWithName("MeshName", recursively: true)
let verts = modelMesh?.geometry!.geometrySourcesForSemantic(SCNGeometrySourceSemanticVertex)
let normals = modelMesh?.geometry!.geometrySourcesForSemantic(SCNGeometrySourceSemanticNormal)
let texcoords = modelMesh?.geometry!.geometrySourcesForSemantic(SCNGeometrySourceSemanticTexcoord)
let boneWeights = modelMesh?.geometry!.geometrySourcesForSemantic(SCNGeometrySourceSemanticBoneWeights)
let boneIndices = modelMesh?.geometry!.geometrySourcesForSemantic(SCNGeometrySourceSemanticBoneIndices)
let geometry = modelMesh?.geometry!.geometryElementAtIndex(0)
// Note: the vertex and normal data is shared.
let vertsData = NSData(data: verts![0].data)
let texcoordsData = NSData(data: texcoords![0].data)
let boneWeightsData = NSData(data: boneWeights![0].data)
let boneIndicesData = NSData(data: boneIndices![0].data)
let geometryData = NSData(data: geometry!.data!)
let newVerts = SCNGeometrySource(data: vertsData, semantic: SCNGeometrySourceSemanticVertex, vectorCount: verts![0].vectorCount, floatComponents: verts![0].floatComponents, componentsPerVector: verts![0].componentsPerVector, bytesPerComponent: verts![0].bytesPerComponent, dataOffset: verts![0].dataOffset, dataStride: verts![0].dataStride)
let newNormals = SCNGeometrySource(data: vertsData, semantic: SCNGeometrySourceSemanticNormal, vectorCount: normals![0].vectorCount, floatComponents: normals![0].floatComponents, componentsPerVector: normals![0].componentsPerVector, bytesPerComponent: normals![0].bytesPerComponent, dataOffset: normals![0].dataOffset, dataStride: normals![0].dataStride)
let newTexcoords = SCNGeometrySource(data: texcoordsData, semantic: SCNGeometrySourceSemanticTexcoord, vectorCount: texcoords![0].vectorCount, floatComponents: texcoords![0].floatComponents, componentsPerVector: texcoords![0].componentsPerVector, bytesPerComponent: texcoords![0].bytesPerComponent, dataOffset: texcoords![0].dataOffset, dataStride: texcoords![0].dataStride)
let newBoneWeights = SCNGeometrySource(data: boneWeightsData, semantic: SCNGeometrySourceSemanticBoneWeights, vectorCount: boneWeights![0].vectorCount, floatComponents: boneWeights![0].floatComponents, componentsPerVector: boneWeights![0].componentsPerVector, bytesPerComponent: boneWeights![0].bytesPerComponent, dataOffset: boneWeights![0].dataOffset, dataStride: boneWeights![0].dataStride)
let newBoneIndices = SCNGeometrySource(data: boneIndicesData, semantic: SCNGeometrySourceSemanticBoneIndices, vectorCount: boneIndices![0].vectorCount, floatComponents: boneIndices![0].floatComponents, componentsPerVector: boneIndices![0].componentsPerVector, bytesPerComponent: boneIndices![0].bytesPerComponent, dataOffset: boneIndices![0].dataOffset, dataStride: boneIndices![0].dataStride)
let newGeometry = SCNGeometryElement(data: geometryData, primitiveType: geometry!.primitiveType, primitiveCount: geometry!.primitiveCount, bytesPerIndex: geometry!.bytesPerIndex)
let newMeshGeometry = SCNGeometry(sources: [newVerts, newNormals, newTexcoords, newBoneWeights, newBoneIndices], elements: [newGeometry])
newMeshGeometry.firstMaterial = modelMesh?.geometry!.firstMaterial
let newModelMesh = SCNNode(geometry: newMeshGeometry)
let bones = modelMesh?.skinner?.bones
let boneInverseBindTransforms = modelMesh?.skinner?.boneInverseBindTransforms
let skeleton = modelMesh!.skinner!.skeleton!
let baseGeometryBindTransform = modelMesh!.skinner!.baseGeometryBindTransform
newModelMesh.skinner = SCNSkinner(baseGeometry: newMeshGeometry, bones: bones, boneInverseBindTransforms: boneInverseBindTransforms, boneWeights: newBoneWeights, boneIndices: newBoneIndices)
newModelMesh.skinner?.baseGeometryBindTransform = baseGeometryBindTransform
// Before this assignment, newModelMesh.skinner?.skeleton is nil.
newModelMesh.skinner?.skeleton = skeleton
// After, it is still nil... however, skeleton itself is completely valid.
modelMesh?.removeFromParentNode()
newModelMesh.name = "MeshName"
let meshParentNode = modelNode?.childNodeWithName("MeshParentNode", recursively: true)
meshParentNode?.addChildNode(newModelMesh)

Answer 1

This three methods may help you to find the solution:
SCNNode *hero = [SCNScene sceneNamed:#"Hero"].rootNode;
SCNNode *hat = [SCNScene sceneNamed:#"FancyFedora"].rootNode;
hat.skinner.skeleton = hero.skinner.skeleton;
[Export ("initWithFrame:")]
public UIView (System.Drawing.RectangleF frame) : base (NSObjectFlag.Empty)
{
// Invoke the init method now.
var initWithFrame = new Selector ("initWithFrame:").Handle;
if (IsDirectBinding)
Handle = ObjCRuntime.Messaging.IntPtr_objc_msgSend_RectangleF (this.Handle, initWithFrame, frame);
else
Handle = ObjCRuntime.Messaging.IntPtr_objc_msgSendSuper_RectangleF (this.SuperHandle, initWithFrame, frame);
}
See this link as well.

Answer 2

I don't specifically know what causes your code to crash but here is a way of generating a mesh, bones, and skinning that mesh -- all from code. Swift4 and iOS 12.
In the example, there is mesh representing the concatenation of two cylinders, with one of the cylinders branching off at a 45 degree angle, like so:
\
|
The cylinders are just extruded triangles, i.e., radialSegmentCount = 3. (Note that there are 12 vertices, not 9, since the two cylinders aren't really conjoined. The triangles are ordered like this:
v5
^
v3 /__|__\ v1
| | |
| v4 |
v2 |/___\| v0
There are 3 bones, corresponding to the heads and feet of the cylinders, where the middle bone corresponds to the head of the bottom cylinder and simultaneously the foot of the top cylinder. So for example, vertices v0, v2, and v4 correspond to bone0; v1, v3, v5 correspond to bone1, and so forth. That explains why boneIndices (see below) has the value that it does.
The resting positions of the bones corresponds to the resting positions of the cylinders in the geometry (bone2 sprouts off at a 45 degree angle from bone1, just like the cylinder geometry).
With that as context, the following code creates everything needed to skin the geometry:
let vertices = [float3(0.17841241, 0.0, 0.0), float3(0.17841241, 1.0, 0.0), float3(-0.089206174, 0.0, 0.1545097), float3(-0.089206174, 1.0, 0.1545097), float3(-0.089206256, 0.0, -0.15450965), float3(-0.089206256, 1.0, -0.15450965), float3(0.12615661, 1.1261566, 0.0), float3(-0.58094996, 1.8332633, 0.0), float3(-0.063078284, 0.9369217, 0.1545097), float3(-0.7701849, 1.6440284, 0.1545097), float3(-0.063078344, 0.93692166, -0.15450965), float3(-0.77018493, 1.6440284, -0.15450965)]
let indices: [UInt8] = [0, 1, 2, 3, 4, 5, 0, 1, 1, 6, 6, 7, 8, 9, 10, 11, 6, 7]
let geometrySource = SCNGeometrySource(vertices: vertices.map { SCNVector3($0) })
let geometryElement = SCNGeometryElement(indices: indices, primitiveType: .triangleStrip)
let geometry = SCNGeometry(sources: [geometrySource], elements: [geometryElement])
let bone0 = SCNNode()
bone0.simdPosition = float3(0,0,0)
let bone1 = SCNNode()
bone1.simdPosition = float3(0,1,0)
let bone2 = SCNNode()
bone2.simdPosition = float3(0,1,0) + normalize(float3(-1,1,0))
let bones = [bone0, bone1, bone2]
let boneInverseBindTransforms: [NSValue]? = bones.map { NSValue(scnMatrix4: SCNMatrix4Invert($0.transform)) }
var boneWeights: [Float] = vertices.map { _ in 1.0 }
var boneIndices: [UInt8] = [
0, 1, 0, 1, 0, 1,
1, 2, 1, 2, 1, 2,
]
let boneWeightsData = Data(bytesNoCopy: &boneWeights, count: boneWeights.count * MemoryLayout<Float>.size, deallocator: .none)
let boneIndicesData = Data(bytesNoCopy: &boneIndices, count: boneWeights.count * MemoryLayout<UInt8>.size, deallocator: .none)
let boneWeightsGeometrySource = SCNGeometrySource(data: boneWeightsData, semantic: .boneWeights, vectorCount: boneWeights.count, usesFloatComponents: true, componentsPerVector: 1, bytesPerComponent: MemoryLayout<Float>.size, dataOffset: 0, dataStride: MemoryLayout<Float>.size)
let boneIndicesGeometrySource = SCNGeometrySource(data: boneIndicesData, semantic: .boneIndices, vectorCount: boneIndices.count, usesFloatComponents: false, componentsPerVector: 1, bytesPerComponent: MemoryLayout<UInt8>.size, dataOffset: 0, dataStride: MemoryLayout<UInt8>.size)
let skinner = SCNSkinner(baseGeometry: geometry, bones: bones, boneInverseBindTransforms: boneInverseBindTransforms, boneWeights: boneWeightsGeometrySource, boneIndices: boneIndicesGeometrySource)
let node = SCNNode(geometry: geometry)
node.skinner = skinner
Note: In most cases, you should use UInt16 not UInt8.