I tried to draw simple graph using Metal API.
I've done similar job using OpenGL in Android and it takes 0.2~3 in old machine. But using Metal API, it takes more than 2~3 seconds in iPhone6s.
I call render() after assignVertices(...) in every 1 second.
And vertexData has more than 25000 vertices.
(For your reference 5000 vertices works well in 0.2 seconds).
How can I speed up following code?
Thanks!
var commandBuffer : MTLCommandBuffer!
var renderEncoder : MTLRenderCommandEncoder!
func assignVertices(vertices: Array<Vertex>, device: MTLDevice) {
var vertexData = Array<Float>()
for vertex in vertices{
vertexData += vertex.floatBuffer()
}
// 2
let dataSize = vertexData.count * MemoryLayout.size(ofValue: vertexData[0])
vertexBuffer = device.makeBuffer(bytes: vertexData, length: dataSize, options: [])!
// 3
vertexCount = vertices.count
}
func render(commandQueue: MTLCommandQueue, pipelineState: MTLRenderPipelineState, drawable: CAMetalDrawable, parentModelViewMatrix: Matrix4, projectionMatrix: Matrix4, clearColor: MTLClearColor?) {
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].texture = drawable.texture
renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(red: 255.0/255.0, green: 255.0/255.0, blue: 255.0/255.0, alpha: 1.0)
renderPassDescriptor.colorAttachments[0].storeAction = .store
renderEncoder = commandBuffer.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.setRenderPipelineState(pipelineState)
renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
let nodeModelMatrix = self.modelMatrix()
nodeModelMatrix.multiplyLeft(parentModelViewMatrix)
let uniformBuffer = device.makeBuffer(length: MemoryLayout<Float>.size * Matrix4.numberOfElements() * 2, options: [])
let bufferPointer = uniformBuffer!.contents()
memcpy(bufferPointer, nodeModelMatrix.raw(), MemoryLayout<Float>.size * Matrix4.numberOfElements())
renderEncoder.setVertexBuffer(uniformBuffer, offset: 0, index: 1)
memcpy(bufferPointer + MemoryLayout<Float>.size * Matrix4.numberOfElements(), projectionMatrix.raw(), MemoryLayout<Float>.size * Matrix4.numberOfElements())
renderEncoder.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: vertexCount, instanceCount: vertexCount / 3)
renderEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
}
Related
Using Swift 5.5, iOS 14
Trying to create a simple 3D bar chart and I immediately find myself in trouble.
I wrote this code...
var virgin = true
for i in stride(from: 0, to: 6, by: 0.5) {
let rnd = CGFloat.random(in: 1.0...4.0)
let targetGeometry = SCNBox(width: 0.5,
height: rnd,
length: 0.5,
chamferRadius: 0.2)
targetGeometry.firstMaterial?.fillMode = .lines
targetGeometry.firstMaterial?.diffuse.contents = UIColor.blue
let box = SCNNode(geometry: targetGeometry)
box.simdPosition = SIMD3(x: 0, y: 0, z: 0)
coreNode.addChildNode(box)
}
This works well, but all the bars a centred around their centre. But how can I ask SceneKit to change the alignment?
Almost got this working with this code...
box.simdPosition = SIMD3(x: Float(i) - 3,
y: Float(rnd * 0.5),
z: 0)
But the result isn't right... I want/need the bar to grow from the base.
https://youtu.be/KJgvdBFBfyc
How can I make this grow from the base?
--Updated with working solution--
Tried Andy Jazz suggestion replacing simdposition with the following formulae.
box.simdPosition = SIMD3(x:box.simdPosition.x, y:0, z:0)
box.simdPivot.columns.3.y = box.boundingBox.max.y - Float(rnd * 0.5)
Worked well, to which I added some animation! Thanks Andy.
changer = changeling.sink(receiveValue: { [self] _ in
var rnds:[CGFloat] = []
for i in 0..<12 {
let rnd = CGFloat.random(in: 1.0...2.0)
let targetGeometry = SCNBox(width: 0.45, height: rnd, length: 0.45, chamferRadius: 0.01)
let newNode = SCNNode(geometry: targetGeometry)
sourceNodes[i].simdPivot.columns.3.y = newNode.boundingBox.min.y
sourceNodes[i].simdPosition = SIMD3(x: sourceNodes[i].simdPosition.x, y: 0, z: 0)
rnds.append(rnd)
}
for k in 0..<12 {
if virgin {
coreNode.addChildNode(sourceNodes[k])
}
let targetGeometry = SCNBox(width: 0.45, height: rnds[k], length: 0.45, chamferRadius: 0.01)
targetGeometry.firstMaterial?.fillMode = .lines
targetGeometry.firstMaterial?.diffuse.contents = UIColor.blue
let morpher = SCNMorpher()
morpher.targets = [targetGeometry]
sourceNodes[k].morpher = morpher
let animation = CABasicAnimation(keyPath: "morpher.weights[0]")
animation.toValue = 1.0
animation.repeatCount = 0.0
animation.duration = 1.0
animation.fillMode = .forwards
animation.isRemovedOnCompletion = false
sourceNodes[k].addAnimation(animation, forKey: nil)
}
virgin = false
})
You have to position a pivot point of each bar to its base:
boxNode.simdPivot.columns.3.y = someFloatNumber
To reposition a pivot to bar's base use bounding box property:
boxNode.simdPivot.columns.3.y += (boxNode.boundingBox.min.y as? simd_float1)!
After pivot's offset, reposition boxNode towards negative direction of Y-axis.
boxNode.position.y = 0
I'm applying a LUT (from a .png - Example LUT Image) to an image using CIColorCube. It works well. The only problem I'm facing is that when I create the buttons thumbnails the app stops for a few seconds.
The buttons look like this -> Buttons Example Image
This is my code:
#IBOutlet weak var filtersScrollView: UIScrollView!
var filters = ["-", "Filter1", "Filter2", "Filter3", "Filter4"]
override func viewDidLoad() {
createFilters()
}
func createFilters() {
var x: CGFloat = 10
let y: CGFloat = 0
let width: CGFloat = 60
let height: CGFloat = 83
let gap: CGFloat = 2
for i in 0..<filters.count {
let filterButton = UIButton(type: .custom)
filterButton.frame = CGRect(x: x, y: y, width: width, height: height)
filterButton.imageView?.contentMode = .scaleAspectFill
filterButton.setTitleColor(#colorLiteral(red: 1, green: 1, blue: 1, alpha: 0), for: .normal)
let text = UILabel()
text.frame = CGRect(x: 0, y: height - 21, width: filterButton.frame.width, height: 21)
text.textAlignment = .center
text.backgroundColor = #colorLiteral(red: 0.9372549057, green: 0.3490196168, blue: 0.1921568662, alpha: 1)
text.textColor = .white
text.font = .systemFont(ofSize: 8.5, weight: .medium)
filterButton.addSubview(text)
filtersScrollView.insertSubview(filterButton, at: 1)
x += width + gap
if i == 0 {
filterButton.setImage(originalImage, for: .normal)
text.text = "-"
text.backgroundColor = #colorLiteral(red: 0.1215686275, green: 0.1215686275, blue: 0.1215686275, alpha: 1)
}
else {
// THIS LINE MAKES THE APP STOP FOR A FEW SECONDS
let filteredImage = filterFromLUT(inputImage: originalCIImage, lut: "\(filters[i])")?.outputImage
filterButton.setImage(UIImage(ciImage: filteredImage!), for: .normal)
text.text = "\(filters[i])"
}
}
filtersScrollView.contentSize = CGSize(width: x, height: height)
}
func filterFromLUT(inputImage: CIImage, lut: String) -> CIFilter? {
let dimension = 64
let lutImage = UIImage(named: lut)!.cgImage
let width = lutImage!.width
let height = lutImage!.height
let rowNum = width / dimension
let columnNum = height / dimension
let bitmap = createBitmap(image: lutImage!)
let dataSize = dimension * dimension * dimension * MemoryLayout<Float>.size * 4
var array = Array<Float>(repeating: 0, count: dataSize)
var bitmapOffest: Int = 0
var z: Int = 0
for _ in stride(from: 0, to: rowNum, by: 1) {
for y in stride(from: 0, to: dimension, by: 1) {
let tmp = z
for _ in stride(from: 0, to: columnNum, by: 1) {
for x in stride(from: 0, to: dimension, by: 1) {
let dataOffset = (z * dimension * dimension + y * dimension + x) * 4
let position = bitmap!
.advanced(by: bitmapOffest)
array[dataOffset + 0] = Float(position
.advanced(by: 0)
.pointee) / 255
array[dataOffset + 1] = Float(position
.advanced(by: 1)
.pointee) / 255
array[dataOffset + 2] = Float(position
.advanced(by: 2)
.pointee) / 255
array[dataOffset + 3] = Float(position
.advanced(by: 3)
.pointee) / 255
bitmapOffest += 4
}
z += 1
}
z = tmp
}
z += columnNum
}
free(bitmap)
let data = Data.init(bytes: array, count: dataSize)
// Create CIColorCube filter
let filter = CIFilter.colorCube()
filter.inputImage = inputImage
filter.cubeData = data
filter.cubeDimension = Float(dimension)
return filter
}
func createBitmap(image: CGImage) -> UnsafeMutablePointer<UInt8>? {
let width = image.width
let height = image.height
let bitsPerComponent = 8
let bytesPerRow = width * 4
let bitmapSize = bytesPerRow * height
guard let data = malloc(bitmapSize) else {
return nil
}
let context = CGContext(
data: data,
width: width,
height: height,
bitsPerComponent: bitsPerComponent,
bytesPerRow: bytesPerRow,
space: CGColorSpaceCreateDeviceRGB(),
bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue,
releaseCallback: nil,
releaseInfo: nil)
context!.draw(image, in: CGRect(x: 0, y: 0, width: width, height: height))
return data.bindMemory(to: UInt8.self, capacity: bitmapSize)
}
I think that maybe the CGContext inside the createBitmap() function is causing this. Does anyone know how to solve this?
There are a few things you can do to improve performance:
Currently, you are processing the original input image (which I assume is pretty large) just to display the result in a 60 x 83 button. Consider scaling the image down first before putting it through the filters.
You can avoid blocking the UI by making the image processing code asynchronous. Just create the buttons in their proper size and DispatchQueue.global().async { ... } the image processing.
Don't use .setImage(UIImage(ciImage: filteredImage). In my experience creating a UIImage from a CIImage this way to very unpredictable. Rather use a CIContext to render the filtered image into a CGImage and convert that into a UIImage afterward. Also try to re-use a single CIContext instead of re-creating it again for each image.
The code for converting the LUT image into a float data array can be sped-up by using vDSP (see below).
Using vDSP for creating the LUT data:
let lutImage = UIImage(named: lut)!.cgImage
let dimension = lutImage.height
// get data from image
let lutImageData = lutImage.dataProvider?.data
let lutImageDataPtr = CFDataGetBytePtr(lutImageData)!
// convert to float and divide by 255
let numElements = dimension * dimension * dimension * 4
let inDataFloat = UnsafeMutablePointer<Float>.allocate(capacity: numElements)
vDSP_vfltu8(lutImageDataPtr, 1, inDataFloat, 1, vDSP_Length(numElements))
var div: Float = 255.0
vDSP_vsdiv(inDataFloat, 1, &div, inDataFloat, 1, vDSP_Length(numElements))
// convert buffer pointer to data
let lutData = NSData(bytesNoCopy: inDataFloat, length: numElements * MemoryLayout<Float>.size, freeWhenDone: true)
I'm following a tutorial - https://www.raywenderlich.com/7475-metal-tutorial-getting-started - to learn how to use metal. I've done exactly what the tutorial said to do and no errors pop up before I try to build it, then it says build failed, along with the error - cannot load module 'metal' as 'Metal'.
I can't find an answer anywhere else, so can someone help me fix this? I'm new to coding and I'm expecting this to have a straightforward solution.
Edit: So I just discovered that there was indeed a very simple solution. I hadn't downloaded the materials for the tutorial. But now that I have, another error has shown up, saying Use of unresolved identifier 'vertexBuffer'
Here's my entire code, just to resolve any confusion -
viewControllerer.swift -
import Metal
var device: MTLDevice!
var metalLayer: CAMetalLayer!
var pipelineState: MTLRenderPipelineState!
var commandQueue: MTLCommandQueue!
var timer: CADisplayLink!
class ViewController: UIViewController {
override func viewDidLoad() {
super.viewDidLoad()
metalLayer = CAMetalLayer() // 1
metalLayer.device = device // 2
metalLayer.pixelFormat = .bgra8Unorm // 3
metalLayer.framebufferOnly = true // 4
metalLayer.frame = view.layer.frame // 5
view.layer.addSublayer(metalLayer) // 6
let dataSize = vertexData.count * MemoryLayout.size(ofValue: vertexData[0]) // 1
vertexBuffer = device.makeBuffer(bytes: vertexData, length: dataSize, options: []) // 2
// 1
let defaultLibrary = device.makeDefaultLibrary()!
let fragmentProgram = defaultLibrary.makeFunction(name: "basic_fragment")
let vertexProgram = defaultLibrary.makeFunction(name: "basic_vertex")
// 2
let pipelineStateDescriptor = MTLRenderPipelineDescriptor()
pipelineStateDescriptor.vertexFunction = vertexProgram
pipelineStateDescriptor.fragmentFunction = fragmentProgram
pipelineStateDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
// 3
pipelineState = try! device.makeRenderPipelineState(descriptor: pipelineStateDescriptor)
device = MTLCreateSystemDefaultDevice()
commandQueue = device.makeCommandQueue()
timer = CADisplayLink(target: self, selector: #selector(gameloop))
timer.add(to: RunLoop.main, forMode: .default)
}
let vertexData: [Float] = [
0.0, 1.0, 0.0,
-1.0, -1.0, 0.0,
1.0, -1.0, 0.0
]
func render() {
guard let drawable = metalLayer?.nextDrawable() else { return }
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].texture = drawable.texture
renderPassDescriptor.colorAttachments[0].loadAction = .clear
renderPassDescriptor.colorAttachments[0].clearColor = MTLClearColor(
red: 0.0,
green: 104.0/255.0,
blue: 55.0/255.0,
alpha: 1.0)
let commandBuffer = commandQueue.makeCommandBuffer()!
let renderEncoder = commandBuffer
.makeRenderCommandEncoder(descriptor: renderPassDescriptor)!
renderEncoder.setRenderPipelineState(pipelineState)
renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, index: 0)
renderEncoder
.drawPrimitives(type: .triangle, vertexStart: 0, vertexCount: 3, instanceCount: 1)
renderEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()
}
#objc func gameloop() {
autoreleasepool {
self.render()
}
}
}
Shaders.metal
#include <metal_stdlib>
using namespace metal;
vertex float4 basic_vertex( // 1
const device packed_float3* vertex_array [[ buffer(0) ]], // 2
unsigned int vid [[ vertex_id ]]) { // 3
return float4(vertex_array[vid], 1.0); // 4
}
fragment half4 basic_fragment() { // 1
return half4(1.0); // 2
}
Note - Shaders.metal is a file that the tutorial says to create
The compiler detected a similar name to Metal, which is metal. Sometimes libraries change name in their own different versions, that's why it can be different from the tutorial you're following.
try doing as the error suggest and replacing the import with: import metal
You need to import Metal framework as below (mentioned under Creating an MTLDevice),
import Metal
Currently you are doing as below which is wrong,
import metal
You need to have a var vertexBuffer: MTLBuffer variable
I'm creating multiple nodes automatically and I want to arrange them around me, because at the moment I'm just increasing of 0.1 the current X location.
capsuleNode.geometry?.firstMaterial?.diffuse.contents = imageView
capsuleNode.position = SCNVector3(self.counterX, self.counterY, self.counterZ)
capsuleNode.name = topic.name
self.sceneLocationView.scene.rootNode.addChildNode(capsuleNode)
self.counterX += 0.1
So the question is, how can I have all of them around me instead of just in one line?
Did someone of you have some math function for this? Thank you!
Use this code (macOS version) to test it:
import SceneKit
class GameViewController: NSViewController {
override func viewDidLoad() {
super.viewDidLoad()
let scene = SCNScene()
let scnView = self.view as! SCNView
scnView.scene = scene
scnView.allowsCameraControl = true
scnView.backgroundColor = NSColor.black
for i in 1...12 { // HERE ARE 12 SPHERES
let sphereNode = SCNNode(geometry: SCNSphere(radius: 1))
sphereNode.position = SCNVector3(0, 0, 0)
// ROTATE ABOUT THIS OFFSET PIVOT POINT
sphereNode.simdPivot.columns.3.x = 5
sphereNode.geometry?.firstMaterial?.diffuse.contents = NSColor(calibratedHue: CGFloat(i)/12,
saturation: 1,
brightness: 1,
alpha: 1)
// ROTATE ABOUT Y AXIS (STEP is 30 DEGREES EXPRESSED IN RADIANS)
sphereNode.rotation = SCNVector4(0, 1, 0, (-CGFloat.pi * CGFloat(i))/6)
scene.rootNode.addChildNode(sphereNode)
}
}
}
P.S. Here's a code for creating 90 spheres:
for i in 1...90 {
let sphereNode = SCNNode(geometry: SCNSphere(radius: 0.1))
sphereNode.position = SCNVector3(0, 0, 0)
sphereNode.simdPivot.columns.3.x = 5
sphereNode.geometry?.firstMaterial?.diffuse.contents = NSColor(calibratedHue: CGFloat(i)/90, saturation: 1, brightness: 1, alpha: 1)
sphereNode.rotation = SCNVector4(0, 1, 0, (-CGFloat.pi * (CGFloat(i))/6)/7.5)
scene.rootNode.addChildNode(sphereNode)
}
You need some math here
How I prepare the circle nodes
var nodes = [SCNNode]()
for i in 1...20 {
let node = createSphereNode(withRadius: 0.05, color: .yellow)
nodes.append(node)
node.position = SCNVector3(0,0,-1 * 1 / i)
scene.rootNode.addChildNode(node)
}
DispatchQueue.main.asyncAfter(deadline: .now() + 5) {
self.arrangeNode(nodes: nodes)
}
func createSphereNode(withRadius radius: CGFloat, color: UIColor) -> SCNNode {
let geometry = SCNSphere(radius: radius)
geometry.firstMaterial?.diffuse.contents = color
let sphereNode = SCNNode(geometry: geometry)
return sphereNode
}
Math behind arrange the view into circle
func arrangeNode(nodes:[SCNNode]) {
let radius:CGFloat = 1;
let angleStep = 2.0 * CGFloat.pi / CGFloat(nodes.count)
var count:Int = 0
for node in nodes {
let xPos:CGFloat = CGFloat(self.sceneView.pointOfView?.position.x ?? 0) + CGFloat(cosf(Float(angleStep) * Float(count))) * (radius - 0)
let zPos:CGFloat = CGFloat(self.sceneView.pointOfView?.position.z ?? 0) + CGFloat(sinf(Float(angleStep) * Float(count))) * (radius - 0)
node.position = SCNVector3(xPos, 0, zPos)
count = count + 1
}
}
Note: In third image I have set.
let xPos:CGFloat = -1 + CGFloat(cosf(Float(angleStep) * Float(count))) * (radius - 0)
let zPos:CGFloat = -1 + CGFloat(sinf(Float(angleStep) * Float(count))) * (radius - 0)
That means if you need view around the camera then
use CGFloat(self.sceneView.pointOfView?.position.x ?? 0) or at the random place then provide the value
Output
I would like to know how I can animate an SCNNode's color using Swift.
For example: I would like the node to constantly be changing color or I would like the node to fade from black to blue.
Do I use the SCNAction fadeIn or fadeOut?
You can create a custom action.
If you have a red sphere in your scene
let sphereNode = scene.rootNode.childNode(withName: "sphere", recursively: false)!
sphereNode.geometry!.firstMaterial!.diffuse.contents = UIColor.red
This is how you build the custom action
let changeColor = SCNAction.customAction(duration: 10) { (node, elapsedTime) -> () in
let percentage = elapsedTime / 5
let color = UIColor(red: 1 - percentage, green: percentage, blue: 0, alpha: 1)
node.geometry!.firstMaterial!.diffuse.contents = color
}
Finally you just need to run the action on the sphere
sphereNode.runAction(changeColor)
Result
Got idea from #Luca Angeletti, I write the code so we can animate between any colors, include their alphas:
func aniColor(from: UIColor, to: UIColor, percentage: CGFloat) -> UIColor {
let fromComponents = from.cgColor.components!
let toComponents = to.cgColor.components!
let color = UIColor(red: fromComponents[0] + (toComponents[0] - fromComponents[0]) * percentage,
green: fromComponents[1] + (toComponents[1] - fromComponents[1]) * percentage,
blue: fromComponents[2] + (toComponents[2] - fromComponents[2]) * percentage,
alpha: fromComponents[3] + (toComponents[3] - fromComponents[3]) * percentage)
return color
}
Use:
let oldColor = UIColor.red
let newColor = UIColor(colorLiteralRed: 0.0, green: 0.0, blue: 1.0, alpha: 0.5)
let duration: TimeInterval = 1
let act0 = SCNAction.customAction(duration: duration, action: { (node, elapsedTime) in
let percentage = elapsedTime / CGFloat(duration)
node.geometry?.firstMaterial?.diffuse.contents = self.aniColor(from: newColor, to: oldColor, percentage: percentage)
})
let act1 = SCNAction.customAction(duration: duration, action: { (node, elapsedTime) in
let percentage = elapsedTime / CGFloat(duration)
node.geometry?.firstMaterial?.diffuse.contents = self.aniColor(from: oldColor, to: newColor, percentage: percentage)
})
let act = SCNAction.repeatForever(SCNAction.sequence([act0, act1]))
node.runAction(act)
I am using this function;
You just give name of material, how long animation will take and current color and
the which color will be reached.
You give color as arrays (red, green, blue and alpha);
let startColor: [CGFloat] = [0.5, 0.5, 1, 1]
let targetColor: [CGFloat] = [1, 1, 1, 1]
This is how you call function;
changeColorWithAnimation(duration : 5,
materialName: myCubeMaterial,
start : startColor,
end : targetColor)
func chageColorWithAnimation(duration: CGFloat,
materialName: SCNMaterial,
start: [CGFloat],
end: [CGFloat]){
let rs = (end[0] - start[0]) / duration
let gs = (end[1] - start[1]) / duration
let bs = (end[2] - start[2]) / duration
let alphas = (end[3] - start[3]) / duration
let changeColor = SCNAction.customAction(duration: TimeInterval(duration)) { (node, elapsedTime) -> () in
let red = start[0] + rs * elapsedTime
let green = start[1] + gs * elapsedTime
let blue = start[2] + bs * elapsedTime
let alpha = start[3] + alphas * elapsedTime
materialName.diffuse.contents = UIColor(displayP3Red:red,
green: green,
blue: blue,
alpha: alpha)
}
rootNode.runAction(changeColor)
}