Resulting MTLTexture lighter than CGImage - swift

Question

I have kernel func which must convert Y and CbCr textures created from pixelBuffer(ARFrame.capturedImage) to RGB texture like in apple guide https://developer.apple.com/documentation/arkit/displaying_an_ar_experience_with_metal
But I get over lighted texture
kernel void renderTexture(texture2d<float, access::sample> capturedImageTextureY [[ texture(0) ]],
texture2d<float, access::sample> capturedImageTextureCbCr [[ texture(1) ]],
texture2d<float, access::read_write> outTextue [[texture(2)]],
uint2 size [[threads_per_grid]],
uint2 pid [[thread_position_in_grid]]){
constexpr sampler colorSampler(mip_filter::linear,
mag_filter::linear,
min_filter::linear);
const float4x4 ycbcrToRGBTransform = float4x4(
float4(+1.0000f, +1.0000f, +1.0000f, +0.0000f),
float4(+0.0000f, -0.3441f, +1.7720f, +0.0000f),
float4(+1.4020f, -0.7141f, +0.0000f, +0.0000f),
float4(-0.7010f, +0.5291f, -0.8860f, +1.0000f)
);
float2 texCoord;
texCoord.x = float(pid.x) / size.x;
texCoord.y = float(pid.y) / size.y;
// Sample Y and CbCr textures to get the YCbCr color at the given texture coordinate
float4 ycbcr = float4(capturedImageTextureY.sample(colorSampler, texCoord).r,
capturedImageTextureCbCr.sample(colorSampler, texCoord).rg, 1.0);
float4 color = ycbcrToRGBTransform * ycbcr;
outTextue.write(color, pid);
}
I create CGImage with this code:
var cgImage: CGImage?
VTCreateCGImageFromCVPixelBuffer(pixelBuffer, options: nil, imageOut: &cgImage)
cgImage has normal lightning
when I try to create texture from cgImage with MTKTextureLoader I get over lighted texture too
How to get MTLTexture with normal light like in cgImage
cgImage: (expected result)
kernel func:
create texture with this code:
let descriptor = MTLTextureDescriptor()
descriptor.width = Int(Self.maxTextureSize.width)
descriptor.height = Int(Self.maxTextureSize.height)
descriptor.usage = [.shaderWrite, .shaderRead]
let texture = MTLCreateSystemDefaultDevice()?.makeTexture(descriptor: descriptor)
and write pixels with kernel func.
already tried different pixelFormats of MTLTextureDescriptor
textureLoader:
let textureLoader = MTKTextureLoader(device: MTLCreateSystemDefaultDevice()!)
let texturee = try! textureLoader.newTexture(cgImage: cgImage!, options: [.SRGB : (false as NSNumber)])
already tried different MTKTextureLoader.Options
GitHub project demonstrating issue: PixelBufferToMTLTexture

Answer 1

Problem was solved Thanks 0xBFE1A8, by adding gamma correction
by replacing
outTextue.write(color, pid);
with:
outTextue.write(float4(pow(color.rgb, float3(2,2,2)), color.a), pid);

Answer 2

when I try to create texture from cgImage with MTKTextureLoader I get
over lighted texture
It's because metal applies gamma correction to your texture.
MTKTextureLoader has an SRGB key that is used to specify whether the texture data is stored as sRGB image data.
If the value is false, the image data is treated as linear pixel data.
If the value is true, the image data is treated as sRGB pixel data. If
this key is not specified and the image being loaded has been
gamma-corrected, the image data uses the specified sRGB information.
let path = Bundle.main.path(forResource: "yourTexture", ofType: "png")!
let data = NSData(contentsOfFile: path) as! Data
let texture = try! textureLoader.newTexture(with: data, options: [MTKTextureLoaderOptionSRGB : (false as NSNumber)])
You can also solve this problem by adding a gamma correcting equation to your shader.
Linear to sRGB and vice versa:
rgb = mix(rgb.0.0774, pow(rgb*0.9479 + 0.05213, 2.4), step(0.04045, rgb))
rgb = mix(rgb12.92, pow(rgb*0.4167) * 1.055 - 0.055, step(0.00313, rgb))