After reading the book "Real Time Rendering 4. Edition", I've decided to give online pbr a try and i chose the GGX algorithm for the normal distribution function. The equation shown in this book looks like in this image:
Now, h is the half vector created from the light and view directions L and V respectively.
X(nDotH)+ is 1 if nDotH is greater than 0, else 0.
alpha-g is the GGX roughness value between 0 and 1.
My question now is the following: As far as I understood the concept of NDF and roughness, a high alpha value would mean that the (micro)surface is very rough, and a low value smooth. So, if I want to render a smooth metallic surface such as the body of a car, I would set my alpha to a low value such as 0,1. By doing so, the result of my D(h) is so low that the object cant even be seen.
Am I missing something or did I not fully understand the value of alpha?
I implemented the NDF in MATLAB to analyse my results. I tried it with the coordinates of a cube placed at origin without transformations.
Given 2 coordinates (world space):
N = [0.0 1.0 0.0; 0.0 0.0 1.0] P = [-1.0 1.0 1.0; -1.0 1.0 1.0] L-Direction = [0.0 1.0 1.0] C-Position = [0.0 3.0 4.0] alpha = 0.1
Results:
D(h) for N1 = 8.6212e-03 D(h) for N2 = 1.7998e-02
As you can see, the values are so low, they aren't visible, specially for the first coordinate whose normal vector point straight up.
The root problem is that simple point lights often don't suffice for full PBR rendering. Consider the following two renderings of a smooth metallic sphere:
This is the top-left sphere from a glTF sample model rendered in Babylon Sandbox.
On the left side, the sphere is placed in a dark environment against a gray background, and a single point light illuminates the scene. The light is quite bright, but because the sphere is so smooth, and because the "point" nature of the light gives it essentially no radius, the reflection of this light is barely a few pixels, regardless of how bright it may be. The remainder of the sphere has the low D(h) values you mentioned, and is almost black.
On the right side, the same sphere again in the same rendering engine, but this time the engine is using its default environment, which comes from an HDR image. In the case of smooth metal, the resulting render is mostly a mirror reflection of the environment, but rougher and non-metallic surfaces can also have their appearance greatly influenced by colors and intensities in the surrounding environment. With a good quality environment, there's often no need to add point lights at all, and indeed there are no point lights in the right image.
In general, PBR, and particularly metallic PBR, looks best with a full HDRI environment, not just point lights. For some sample code and shaders showing some of this math in action, the Khronos glTF Sample Viewer might be a good place to start. [Disclaimer, I'm a contributor.]
As far as I understood the concept of NDF and roughness, a high alpha
value would mean that the (micro)surface is very rough, and a low
value smooth. So, if I want to render a smooth metallic surface such
as the body of a car, I would set my alpha to a low value such as 0,1.
By doing so, the result of my D(h) is so low that the object cant even
be seen. Am I missing something or did I not fully understand the
value of alpha?
It's true that the numerator of the equation goes to zero.
But denominator too. And it does so more rapidly.
Taking, as an example, n = h -> dot(n, h) will be one. And if alpha is 0.1:
0.1^2 / (3.141593 * (1 + (0.1^2 - 1))^2)
If you plug that into your calculator you will get ~32.83.
So, as you can see, the whole equation doesn't go to zero.
Actually, if you calculate the limit of the equation as alpha goes to zero, the equation goes to infinity. Which makes sense, because when roughness is zero, all the normals are concentrated in a single direction.
Related
I have a image:
The upper part of this image, which alpha value is 1 (or 255 in RGBA)
The lower part of this image, which alpha value is 0.3, I used it for shadow in game.
So When I import it to Unity ShaderGraph as a _MainTex, when I split it alpha, it looks like this:
imported alpha
My first questions is:
"alpha" is actually a VECTOR 1 type in Unity Documention, but as I could see from the preview, there are three colors, black indicates alpha's value 0, hard white for alpha's value 1 and soft white for alpha's value 0.3, how can one single value transfer so much messages?
My first understanding is:
each pixel's alpha value is stored in the images already, the "alpha" in the shadergraph is just
like a global parameter to control them based every pixel.[I dont know if this is correct]
but when I give alpha a smoothstep node, I
am going to set the pixels's alpha under 0.3 to 0, I found it worked like this:
smoothstep added to the alpha, as you can see, 0.3<0.99, so
the translucent of the image is removed!
So here comes my second question:
Since "alpha" in the input works like a global parameter, how does it affect a picture separately?
My second understanding is:
"alpha" is just like an one-dimensional array, it stores transparency likes this:
{1,1,1,0.3,0.3,0.3}
and when it calculated by smoothstep,its value will be changed like this:
{1,1,1,0,0,0}
But it comes to my first question, ALPHA IS A VECTOR1 TYPE, it only has one value to edit
in the node, it can not be an array!
So, How does an image'alpha transfer so much information to other nodes in Unity Shadergraph?
https://docs.unity3d.com/Packages/com.unity.shadergraph#6.9/manual/Data-Types.html
https://docs.unity3d.com/Packages/com.unity.shadergraph#6.9/manual/Smoothstep-Node.html
Someone who can help me really appreciated!
Shaders work in parallel: for any given vertex or pixel you only get data local to this element. Also critically here 'pixel' (or 'fragment') is a screen pixel, not a texel, which relates a texture's pixel.
In this context, the output of the texture node is a single rgba Vector4 (4 scalar values) at the provided coordinate. This is disconnected from how textures are stored: filtering, compression and mipmapping will come into play (and the control over this comes from the sampler, which you can also provide to the node even though it's most of the time implicit).
Smoothstep is a function that can remap a value - a vector (like the rgba output of the tex node), or a scalar (like the alpha) - into another range. More specifically it does it with smoothing both ends of the spectrum so that the slope is 0 at min and max. The linear equivalent is inverse lerp (which doesn't have a built in instruction in hlsl). You can read about the breakdown on the wikipedia page: https://www.wikiwand.com/en/Smoothstep
I'm making a water simulation, and I'm trying to visualize the velocity field. In this demo I continuously add water in the centre of the image, where the blue 'raindrop' things are.
I have a texture where rg is the X and Y direction of the velocity, and ba is the total movement of water through it (ie: every step ba = ba + rg * delta_time).
I'm working in Unity Shader Graph.
I want to rotate a 'ripple' texture in the direction of the velocity, and then translate in that direction as well. To prevent the shader from jumping around when the velocity changes I thought of using the ba channels (which were previously unused) to keep like a total velocity like described above.
However, both the rotation (based on velocity alone), and the translation (based on the 'total velocity') work fine on their own. But when I sum them together it looks like the translation is also rotated. I'm not sure why this happens.
Here's what I do:
First part: rotating my water texture in the direction of the velocity, and that looks fine:
The shader itself looks like this:
So basically I discretize the uv (custom function on the right), get the angle of the velocity (using arctan2), and then rotate each discrete block using the Rotate block. This works as expected.
Second part: translating the texture based on the total velocity (in the ba channels), also works as expected:
The shader itself looks like this:
Again I used the discretized uv, now I translate each block based on the ba channels, which contain the total of the velocity (ba = ba + rg * delta_time each time step). As you can see this shows the textures flowing away from the centre (where water is added constantly). This is what I would expect to happen.
Now, when I combine them, it goes wrong:
The one I circled in red shows the problem the best (though all block seem to have it to some degree, depending on how much they were rotated). The arrow point to the bottom-right, which seems to be correct, however it flows to the top now.
The shader:
So here I add the rotated discrete block to the translation. But it looks like the translation part now also rotated, even though I add them together after the rotation block. So while the translation isn't rotated, it looks like it is.
Why is this happening? And how can I fix it.
I hope I explained it adequately, since it's not easy to show in just pictures and gifs.
Thanks!
So I fixed my problem by rather than storing the x and y of the offset in the b & a channels, to just storing the total distance moved in the b channel (thus b += length(rg)).
Then I'm using float2(0, b)` as the offset.
This is then also rotated for some reason and visually works as I wanted it.
However, I still don't really see why, sometimes I think I get it, and then I think some more and I don't any more.
So if anyone knows why this happens and can explain, I'm happy to accept that answer.
However, for now it is solved.
I have been looking at the lens correction shader code from the Google GVR SDK for Unity and have been scratching my head as to the use of the z component of the view space position (UNITY_MATRIX_MV, without the perspective transform of UNITY_MATRIX_MVP) in the undistort() functions (this one of the simpler variants):
float r2 = clamp(dot(pos.xy, pos.xy) / (pos.z*pos.z), 0, _MaxRadSq);
pos.xy *= 1 + (_Undistortion.x + _Undistortion.y*r2)*r2;
Given my understand that we want to warp the rendered image in 2d screenspace to counteract distortion that will be applied by lens the screen is viewed through, what on earth are we doing dividing our radius(?) by the linear depth (pos.z) squared? I can conceive that this is in lieu of dividing by w for perspective, but then why would we want to divide by the square of the z component (how would that ever be more correct than simply dividing by z or w) ?
Felt a bit silly in hind sight, as this is just the result of a regular optimisation.
The division is regular perspective division (but leaving the z coord used for depth buffer/culling as linear, and presumably w should thus be 1.0 to ensure proper depth interpolation). Reorganising the computation presumably was found to save shader cycles and/or accuracy.
This code is equivalent to foreshortening pos.xy by dividing it by pos.z first, then doing taking the dot product of pos.xy with itself to get its length squared in 2D screenspace (and then clamping it, etc.)
I am developing a project of detecting vehicles' headlights in night scene. I am working on a demo on MATLAB. My problem is that I need to find region of interest (ROI) to get low computing requirement. I have researched in many papers and they just use a fixed ROI like this one, the upper part is ignored and the bottom is used to analysed later.
However, if the camera is not stable, I think this approach is inappropriate. I want to find a more flexible one, which alternates in each frame. My experiments images are shown here:
If anyone has any idea, plz give me some suggestions.
I would turn the problem around and say that we are looking for headlights
ABOVE a certain line rather than saying that the headlights are below a certain line i.e. the horizon,
Your images have a very high reflection onto the tarmac and we can use that to our advantage. We know that the maximum amount of light in the image is somewhere around the reflection and headlights. We therefore look for the row with the maximum light and use that as our floor. Then look for headlights above this floor.
The idea here is that we look at the profile of the intensities on a row-by-row basis and finding the row with the maximum value.
This will only work with dark images (i.e. night) and where the reflection of the headlights onto the tarmac is large.
It will NOT work with images taking in daylight.
I have written this in Python and OpenCV but I'm sure you can translate it to a language of your choice.
import matplotlib.pylab as pl
import cv2
# Load the image
im = cv2.imread('headlights_at_night2.jpg')
# Convert to grey.
grey_image = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)
Smooth the image heavily to mask out any local peaks or valleys
We are trying to smooth the headlights and the reflection so that there will be a nice peak. Ideally, the headlights and the reflection would merge into one area
grey_image = cv2.blur(grey_image, (15,15))
Sum the intensities row-by-row
intensity_profile = []
for r in range(0, grey_image.shape[0]):
intensity_profile.append(pl.sum(grey_image[r,:]))
Smooth the profile and convert it to a numpy array for easy handling of the data
window = 10
weights = pl.repeat(1.0, window)/window
profile = pl.convolve(pl.asarray(intensity_profile), weights, 'same')
Find the maximum value of the profile. That represents the y coordinate of the headlights and the reflection area. The heat map on the left show you the distribution. The right graph shows you the total intensity value per row.
We can clearly see that the sum of the intensities has a peak.The y-coordinate is 371 and indicated by a red dot in the heat map and a red dashed line in the graph.
max_value = profile.max()
max_value_location = pl.where(profile==max_value)[0]
horizon = max_value_location
The blue curve in the right-most figure represents the variable profile
The row where we find the maximum value is our floor. We then know that the headlights are above that line. We also know that most of the upper part of the image will be that of the sky and therefore dark.
I display the result below.
I know that the line in both images are on almost the same coordinates but I think that is just a coincidence.
You may try downsampling the image.
Does any body know how to find average luminosity for a texture in a fragment shader? I have access to both RGB and YUV textures the Y component in YUV is an array and I want to get an average number from this array.
I recently had to do this myself for input images and video frames that I had as OpenGL ES textures. I didn't go with generating mipmaps for these due to the fact that I was working with non-power-of-two textures, and you can't generate mipmaps for NPOT textures in OpenGL ES 2.0 on iOS.
Instead, I did a multistage reduction similar to mipmap generation, but with some slight tweaks. Each step down reduced the size of the image by a factor of four in both width and height, rather than the normal factor of two used for mipmaps. I did this by sampling from four texture locations that were in the middle of the four squares of four pixels each that made up a 4x4 area in the higher-level image. This takes advantage of hardware texture interpolation to average the four sets of four pixels, then I just had to average those four pixels to yield a 16X reduction in pixels in a single step.
I converted the image to luminance at the very first stage using a dot product of the RGB values with a vec3 of (0.2125, 0.7154, 0.0721). This allowed me to just read the red channel for each subsequent reduction stage, which really helps on iOS hardware. Note that you don't need this if you are starting with a Y channel luminance texture already, but I was dealing with RGB images.
Once the image had been reduced to a sufficiently small size, I read the pixels from that back onto the CPU and did a last quick iteration over the remaining few to arrive at the final luminosity value.
For a 640x480 video frame, this process yields a luminosity value in ~6 ms on an iPhone 4, and I think I can squeeze out a 1-2 ms reduction in that processing time with a little tuning. In my experience, that seems faster than the iOS devices normally generate mipmaps for power-of-two images at around that size, but I don't have solid numbers to back that up.
If you wish to see this in action, check out the code for the GPUImageLuminosity class in my open source GPUImage framework (and the GPUImageAverageColor superclass). The FilterShowcase example demonstrates this luminosity extractor in action.
You generally don't do this just with a shader.
One of the more common methods is to create a buffer texture with full mip-maps (down to 1x1, this is important). When you want to find luminosity, you copy the backbuffer to this buffer, then regenerate mips with a nearest neighbor algorithm. The bottom pixel will then have the average color of the entire surface and can be used to find average lum through something like (c.r * 0.6) + (c.g * 0.3) + (c.b * 0.1) (edit: if you have a YUV, then do similar and use the Y; the trick is just averaging the texture down to a single value, which is what mips do).
This isn't a precise technique, but is reasonably fast, especially on hardware that can generate mipmaps internally.
I'm presenting a solution for the RGB texture here as I'm not sure mip map generation would work with a YUV texture.
The first step is to create mipmaps for the texture, if not already present:
glGenerateMipmapOES(GL_TEXTURE_2D);
Now we can access the RGB value of the smallest mipmap level from the fragment shader by using the optional third argument of the sampler function texture2D, the "bias":
vec4 color = texture2D(sampler, vec2(0.5, 0.5), 8.0);
This will shift the mipmap level up eight levels, resulting in sampling a far smaller level.
If you have a 256x256 texture and render it with a scale of 1, a bias of 8.0 will effectively reduce the picked mipmap to the smallest 1x1 level (256 / 2^8 == 1). Of course you have to adjust the bias for your conditions to sample the smallest level.
OK, now we have the average RGB value of the whole image. The third step is to reduce RGB to a luminosity:
float lum = dot(vec3(0.30, 0.59, 0.11), color.xyz);
The dot product is just a fancy (and fast) way of calculating a weighted sum.