Unity - use GetRawTextureData to change underlying RGB bytes without copying - unity3d

Question

I am trying to use OpenCV with Unity for image processing, and I am trying to make the data transfers between OpenCV and Unity code as efficient as possible.
Currently, I am able to create a new byte[] in C#, then load an image into these bytes in OpenCV, and then use texture.LoadRawTextureData(array) and texture.Apply() to show this texture in Unity.
However, the Unity documentation recommends to use texture.GetRawTextureData() to get a reference to the NativeArray (the version of function that returns byte[] makes a copy of the raw data) and then write the data directly into this buffer (+ call Apply()).
Unfortunately, the documentation on NativeArrays is rather scarce - how exactly do NativeArrays look in the memory? They do have an ToArray() function, but this again makes a copy of the data. What I need is a byte[] array, which can be either RGB24 or RGBA32 (RGBA seems to be preferred, as even though it is memory-inefficient if the texture is opaque, the modern GPUs apparently do not support RGB24).
Is there any way to pass the pointer to the beginning of the buffer in the texture without making copies and calling LoadRawTextureData()? Or are the data in the Texture in a completely different format?

Answer 1

I had the same confusion over NativeArray. It looks like the CopyTo() method doesn't seem to alloc memory. There is a ToArray() method, which I'm certain allocates.
I was able to work out this utility method which is working fine for a webcam feed.
private byte[] m_byteCache = null;
public byte[] GetRawTextureData(Texture2D texture)
{
NativeArray<byte> nativeByteArray = texture.GetRawTextureData<byte>();
if (m_byteCache?.Length != nativeByteArray.Length)
{
m_byteCache = new byte[nativeByteArray.Length];
}
nativeByteArray.CopyTo(m_byteCache);
return m_byteCache;
}

Answer 2

ToArray() allocates a new array. CopyTo() doesn't alloc memory but of course it copies the data.
But what I gather from the documentation is that you should be able to just access the NativeArray like a normal array to modify the memory and then call .Apply() on the corresponding Texture object. If you can make your C# OpenCV code write to it, that should do the trick.
The issue with NativeArray is I guess that you directly get the memory of whatever implementation your code runs on, so the exact byte representation could differ depending on the platform. Also the memory will be invalid as soon as the texture is gone.