I am following this WWDC lecture.
In the lecture he mentions a filter named "CIEdgePreserveUpsampleFilter" that makes the edges more preserved and upsampled.
I am trying to apply this on my CIImage and I get an uninitialized result for the Image and crashes.
This is the code I am using and an example of how I try to apply the filter (which is obviously wrong). I just cannot find any related instructions for applying this filter, I just know I want its results on my image.
I comment next to where I try to apply the filter, and what happens when I do it.
func createMask(for depthImage: CIImage, withFocus focus: CGFloat, andScale scale: CGFloat, andSlope slope: CGFloat = 4.0, andWidth width: CGFloat = 0.1) -> CIImage {
let s1 = slope
let s2 = -slope
let filterWidth = 2 / slope + width
let b1 = -s1 * (focus - filterWidth / 2)
let b2 = -s2 * (focus + filterWidth / 2)
let mask0 = depthImage
.applyingFilter("CIColorMatrix", withInputParameters: [
"inputRVector": CIVector(x: s1, y: 0, z: 0, w: 0),
"inputGVector": CIVector(x: 0, y: s1, z: 0, w: 0),
"inputBVector": CIVector(x: 0, y: 0, z: s1, w: 0),
"inputBiasVector": CIVector(x: b1, y: b1, z: b1, w: 0)])
.applyingFilter("CIColorClamp").applyingFilter("CIEdgePreserveUpsampleFilter") //returns uninitialized image
let mask1 = depthImage
.applyingFilter("CIColorMatrix", withInputParameters: [
"inputRVector": CIVector(x: s2, y: 0, z: 0, w: 0),
"inputGVector": CIVector(x: 0, y: s2, z: 0, w: 0),
"inputBVector": CIVector(x: 0, y: 0, z: s2, w: 0),
"inputBiasVector": CIVector(x: b2, y: b2, z: b2, w: 0)])
.applyingFilter("CIColorClamp")
var combinedMask = mask0.applyingFilter("CIEdgePreserveUpsampleFilter", withInputParameters: ["inputBackgroundImage" : mask1]) //complete crash
if PortraitModel.sharedInstance.filterArea == .front {
combinedMask = combinedMask.applyingFilter("CIColorInvert")
}
let mask = combinedMask.applyingFilter("CIBicubicScaleTransform", withInputParameters: [kCIInputScaleKey: scale])
return mask
}
The runtime headers and some usage code I've found seems to suggest that CIEdgePreserveUpsampleFilter does not take a inputBackgroundImage parameter, but rather inputSmallImage.
See https://gist.github.com/HarshilShah/ca0e18db01ce250fd308ab5acc99a9d0
Related
I have a PointCloud which have points with position(x, y, z) and color(r, g, b)
But points lays in big distance from coordinates canter:
Question is: what algorithm can be used to place all points to coordinates center? My guess is to create translation matrix and multiply all pointCloud points to it, but I can't determine what this matrix should contain
Just found an answer. Need to find center of mass of PointCloud with something like this:
var summX: Float = 0
var summY: Float = 0
var summZ: Float = 0
for point in points {
summX += point.x
summY += point.y
summZ += point.z
}
let middleX = summX / Float(points.count)
let middleY = summY / Float(points.count)
let middleZ = summZ / Float(points.count)
let centerOfMass = Float3(x: middleX, y: middleY, z: middleZ)
Then create translation matrix
And finally multiply all points to this matrix
let translationMatrix = float4x4(simd_float4(x: 1, y: 0, z: 0, w: -centerOfMass.x),
simd_float4(x: 0, y: 1, z: 0, w: -centerOfMass.y),
simd_float4(x: 0, y: 0, z: 1, w: -centerOfMass.z),
simd_float4(x: 0, y: 0, z: 0, w: 1))
let translatedPoints = points.map { point in
return point * translationMatrix
}
I'm following this guide about working with matrices with the accelerate framework.
There they using something similar to this to rotate a vector:
func makeRotationMatrix(angle: Float) -> simd_float3x3 {
let rows = [
simd_float3(cos(angle), -sin(angle), 0),
simd_float3(sin(angle), cos(angle), 0),
simd_float3(0, 0, 1)
]
return float3x3(rows: rows)
}
let vector = simd_float3(x: 1, y: 1, z: 1)
let angle = Measurement(value: 180, unit: UnitAngle.degrees)
let radians = Float(angle.converted(to: .radians).value)
let rotationMatrix = makeRotationMatrix(angle: radians)
let rotatedVector = rotationMatrix * vector
print("vector:", vector) // SIMD3<Float>(1.0, 1.0, 1.0)
print("angle:", angle) // 180.0 °
print("radians:", radians) // 3.1415927
print("rotatedVector:", rotatedVector) // SIMD3<Float>(-0.99999994, -1.0000001, 1.0)
I expected the x of the rotated vector to be -1 instead of -0.99999994. I guess this is caused by the radians being a float? We could correct for this by rounding by hand:
let correctedVector = simd_float3(
x: rotatedVector.x.rounded(),
y: rotatedVector.y.rounded(),
z: rotatedVector.z.rounded()
)
print("correctedVector:", correctedVector) // SIMD3<Float>(-1.0, -1.0, 1.0)
But I'm wondering if there is a way to rotate this vector without rounding errors?
In simd_float4x4, columns is the only property, however, this won't work solely because I'm not calling self.init(). Everything would have been initialize anyway. Why is the compiler complaining? I saw something similar in this video, and it was working. Why can't I do it?
extension simd_float4x4 {
init(ProjectionFrame: CGSize) {
let Y = FOV(), FarZ = Float((Settings.VisibilityRange+1)*2), Z = FarZ / (NearZ - FarZ)
columns = (vector_float4(Y / Float(ProjectionFrame.width / ProjectionFrame.height), 0, 0, 0), vector_float4(0, Y, 0, 0), vector_float4(0, 0, Z, -1), vector_float4(0, 0, Z * NearZ, 0))
}
}
In the video I noticed this.
extension simd_float4x4 {
init(translationX x: Float, x: Float, x: Float) {
columns = (
vector_float4(x, 0, 0, 0),
vector_float4(0, x, 0, 0),
vector_float4(0, 0, x, 0),
vector_float4(0, 0, 0, 1)
)
}
}
And the compiler wasn't complaining. How come it's complaining for me?
Instead of trying to set columns from your init, you should just call self.init(_ columns:) and pass in the 4 vector_float4s as an Array rather than as a tuple.
extension simd_float4x4 {
init(ProjectionFrame: CGSize) {
let Y = FOV(), FarZ = Float((Settings.VisibilityRange+1)*2), Z = FarZ / (NearZ - FarZ)
self.init([vector_float4(Y / Float(ProjectionFrame.width / ProjectionFrame.height), 0, 0, 0), vector_float4(0, Y, 0, 0), vector_float4(0, 0, Z, -1), vector_float4(0, 0, Z * NearZ, 0)])
}
}
That video is 2 years old and hence uses an older Swift version. The code you link from that video also doesn't compile in Swift 5.
Unrelated to your question, but variable names in Swift should be lowerCamelCase, so projectionFrame is the correct naming.
Alternatively I could bridge it to C
#include <simd/simd.h>
matrix_float4x4 ProjectPerspective(const float Ratio) {
const float Y = 1/tanf(Rad(Settings.FOV+15)), FarZ = (Settings.VisibilityRange+1)*32, Z = FarZ/(NearZ-FarZ);
return (matrix_float4x4){.columns = {{Y/Ratio, 0, 0, 0}, {0, Y, 0, 0}, {0, 0, Z, -1}, {0, 0, Z*NearZ, 0}}};
}
I'm stuck with interpolation in Swift. Can anyone help me with that?
I want to interpolate the float array (say [0, 0, 100, 25, 0, 0, 0, 25, 0, 0, 0]) into another array with some given size (for example 128). I found an article (Use Linear Interpolation to Construct New Data Points) that shows, how to achieve this stuff.
There are two ways (you can see the results below, how they perform):
Linear Interpolation using vDSP_vgenp and
Smoother (but not for my purposes) Interpolation using vDSP_vlint
The problem is both techniques don't realize my expectations, which illustrated in Screenshot 3. How can I make my interpolated distribution smoother? I want to see a cube-like curve.
Initial Plot:
Linear Interpolation:
import Accelerate
let n = vDSP_Length(128)
let stride = vDSP_Stride(1)
let values: [Float] = [0, 0, 100, 25, 0, 0, 0, 25, 0, 0, 0]
let indices: [Float] = [0, 11, 23, 34, 46, 58, 69, 81, 93, 104, 116]
var result = [Float](repeating: 0, count: Int(n))
vDSP_vgenp(values, stride, indices, stride, &result, stride, n, vDSP_Length(values.count))
Smooth Interpolation:
import Accelerate
import AVFoundation
let n = vDSP_Length(1024)
let stride = vDSP_Stride(1)
let values: [Float] = [0, 0, 100, 25, 0, 0, 0, 25, 0, 0, 0]
let denominator = Float(n) / Float(values.count - 1)
let control: [Float] = (0 ... n).map {
let x = Float($0) / denominator
return floor(x) + simd_smoothstep(0, 1, simd_fract(x))
}
var result = [Float](repeating: 0, count: Int(n))
vDSP_vlint(values, control, stride, &result, stride, n, vDSP_Length(values.count))
It seems to me that the vDSP_vqint quadratic interpolation functions would solve the problem. See the discussion at https://developer.apple.com/documentation/accelerate/1449942-vdsp_vqint.
Given a CATransform3D transform, I want to extract the scale, translation and rotation as separate transforms. From some digging, I was able to accomplish this for CGAffineTransform in Swift, like so:
extension CGAffineTransform {
var scaleDelta:CGAffineTransform {
let xScale = sqrt(a * a + c * c)
let yScale = sqrt(b * b + d * d)
return CGAffineTransform(scaleX: xScale, y: yScale)
}
var rotationDelta:CGAffineTransform {
let rotation = CGFloat(atan2f(Float(b), Float(a)))
return CGAffineTransform(rotationAngle: rotation)
}
var translationDelta:CGAffineTransform {
return CGAffineTransform(translationX: tx, y: ty)
}
}
How would one do something similar for CATransform3D using math? (I am looking for a solution that doesn't use keypaths.)
(implementation or math-only answers at your discretion)
If you're starting from a proper affine matrix that can be decomposed correctly (if not unambiguously) into a sequence of scale, rotate, translate, this method will perform the decomposition into a tuple of vectors representing the translation, rotation (Euler angles), and scale components:
extension CATransform3D {
func decomposeTRS() -> (float3, float3, float3) {
let m0 = float3(Float(self.m11), Float(self.m12), Float(self.m13))
let m1 = float3(Float(self.m21), Float(self.m22), Float(self.m23))
let m2 = float3(Float(self.m31), Float(self.m32), Float(self.m33))
let m3 = float3(Float(self.m41), Float(self.m42), Float(self.m43))
let t = m3
let sx = length(m0)
let sy = length(m1)
let sz = length(m2)
let s = float3(sx, sy, sz)
let rx = m0 / sx
let ry = m1 / sy
let rz = m2 / sz
let pitch = atan2(ry.z, rz.z)
let yaw = atan2(-rx.z, hypot(ry.z, rz.z))
let roll = atan2(rx.y, rx.x)
let r = float3(pitch, yaw, roll)
return (t, r, s)
}
}
To show that this routine correctly extracts the various components, construct a transform and ensure that it decomposes as expected:
let rotationX = CATransform3DMakeRotation(.pi / 2, 1, 0, 0)
let rotationY = CATransform3DMakeRotation(.pi / 3, 0, 1, 0)
let rotationZ = CATransform3DMakeRotation(.pi / 4, 0, 0, 1)
let translation = CATransform3DMakeTranslation(1, 2, 3)
let scale = CATransform3DMakeScale(0.1, 0.2, 0.3)
let transform = CATransform3DConcat(CATransform3DConcat(CATransform3DConcat(CATransform3DConcat(scale, rotationX), rotationY), rotationZ), translation)
let (T, R, S) = transform.decomposeTRS()
print("\(T), \(R), \(S))")
This produces:
float3(1.0, 2.0, 3.0), float3(1.5708, 1.0472, 0.785398), float3(0.1, 0.2, 0.3))
Note that this decomposition assumes an Euler multiplication order of XYZ, which is only one of several possible orderings.
Caveat: There are certainly values for which this method is not numerically stable. I haven't tested it extensively enough to know where these pitfalls lie, so caveat emptor.
For symmetry with the CGAffineTransform extension in my question, here is the CATransform3D extension that provides the "deltas" for scale, translation and rotation, based on Warren's decomposeTRS, which I have marked as the accepted answer.
extension CATransform3D {
var scaleDelta:CATransform3D {
let s = decomposeTRS().2
return CATransform3DMakeScale(CGFloat(s.x), CGFloat(s.y), CGFloat(s.z))
}
var rotationDelta:CATransform3D {
let r = decomposeTRS().1
let rx = CATransform3DMakeRotation(CGFloat(r.x), 1, 0, 0)
let ry = CATransform3DMakeRotation(CGFloat(r.y), 0, 1, 0)
let rz = CATransform3DMakeRotation(CGFloat(r.z), 0, 0, 1)
return CATransform3DConcat(CATransform3DConcat(rx, ry), rz)
}
var translationDelta:CATransform3D {
let t = decomposeTRS().0
return CATransform3DMakeTranslation(CGFloat(t.x), CGFloat(t.y), CGFloat(t.z))
}
}