I'm looking for a quick and dirty, very efficient edge detection shader or edge mesh for a video application. Since this will be done on a mobile device, I need something that places performance over accuracy. I'll be blurring the output anyway, and all edge detection algorithms I've seen tend to be done by comparing against a certain threshold an original image and a blurred one. I think it's the blur that tends to cause the most performance issues.
I have a function like this working:
vec4 edge()
{
float K00 = -1.0;
float K01 = -2.0;
float K02 = -1.0;
float K10 = 0.0;
float K11 = 0.0;
float K12 = 0.0;
float K20 = 1.0;
float K21 = 2.0;
float K22 = 1.0;
vec2 ox = vec2 (0.0,0.0);
ox[0] = width;
vec2 oy = vec2 (0.0,0.0);
oy[1] = height;
float g00, g01, g02;
float g10, g11, g12;
float g20, g21, g22;
vec4 CC;
vec2 PP = TextureCoord - oy;
CC = texture2D(blurredFrame, vec2(PP-ox));
g00 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP));
g01 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP+ox));
g02 = getGrey(CC);
PP = TextureCoord;
CC = texture2D(blurredFrame, vec2(PP-ox));
g10 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP));
g11 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP+ox));
g12 = getGrey(CC);
PP = TextureCoord + oy;
CC = texture2D(blurredFrame, vec2(PP-ox));
g20 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP));
g21 = getGrey(CC);
CC = texture2D(blurredFrame, vec2(PP+ox));
g22 = getGrey(CC);
float sx = 0.0, sy = 0.0;
sx = sx + g00 * K00;
sx = sx + g01 * K01;
sx = sx + g02 * K02;
sx = sx + g10 * K10;
sx = sx + g11 * K11;
sx = sx + g12 * K12;
sx = sx + g20 * K20;
sx = sx + g21 * K21;
sx = sx + g22 * K22;
sy = sy + g00 * K00;
sy = sy + g01 * K10;
sy = sy + g02 * K20;
sy = sy + g10 * K01;
sy = sy + g11 * K11;
sy = sy + g12 * K21;
sy = sy + g20 * K02;
sy = sy + g21 * K12;
sy = sy + g22 * K22;
float dist = sqrt(sx * sx + sy * sy);
return dist > threshold ? vec4 (0,0,0,1) : vec4 (1,1,1,1);
}
All examples I have seen are like this and seem to focus on a desktop platform--too involved and costly to get a decent framerate on an iPhone or Android device. This will be for a 2d application only, and speed is key.
Any ideas to make this more efficient, or perhaps a better alternative? Thanks everyone.
Not sure if I know of a different algorithm.
But, some suggestions come to mind:
Don't take the sqrt() before doing the compare to dist, instead compare to dist^2
See if you can optimize the access pattern of texture loads. The texture memory access pattern can have a big impact on the performance. You want to keep your memory access as contigious as possible (i.e. 0,1,2,3,...), instead of random.
Turn mip mapping off, or use texture2DLod where you manually specify the mip map level.
I have a couple of ideas about optimizing the texture samples:
No need to sample where the corresponding coefficient is zero
(K1*).
Use texture2DOffset instead of texture2D. It accepts constant integer
offsets, allowing the driver to
predict your access pattern more
effectively.
You are weighting the samples. You can use built-in linear filtering
mechanics to do that. For example, to
get a sum of samples in two neighbour
texels you can sample linearly (only
once) between them and multiply the
result by 2. This variant excludes
previous suggestion.
Related
I'm using the following library:
https://github.com/tengbao/vanta/blob/master/src/vanta.halo.js
A demo can be found here:
https://www.vantajs.com/?effect=halo
If I'm using a bright (or even white) background color, the effect is not visible anymore.
With my limited WebGL knowledge, my guess is that this is because of the subtraction of the background color (mixedColor = texture2D(...) - backgroundColor) (but I could be wrong).
void main() {
vec2 res2 = iResolution.xy * iDpr;
vec2 uv = gl_FragCoord.xy / res2; // 0 to 1
vec4 oldImage = texture2D(iBuffer, uv);
vec3 mixedColor = oldImage.rgb - backgroundColor;
float cropDist = 0.01;
float cropXOffset = 0.2;
float cropYOffset = 0.2;
vec2 offset = uv + vec2((mixedColor.g - cropXOffset) * cropDist, (mixedColor.r - cropYOffset) * cropDist);
float spinDist = 0.001;
float spinSpeed = 0.2 + 0.15 * cos(iTime * 0.5);
float timeFrac = mod(iTime, 6.5);
vec2 offset2 = uvBig + vec2(cos(timeFrac * spinSpeed) * spinDist, sin(timeFrac * spinSpeed) * spinDist);
mixedColor = texture2D(iBuffer, offset).rgb * 0.4
+ texture2D(iBuffer, offset2).rgb * 0.6
- backgroundColor;
float fadeAmt = 0.0015; // fade this amount each frame // 0.002
mixedColor = (mixedColor - fadeAmt) * .995;
vec4 spectrum = abs( abs( .95*atan(uv.x, uv.y) -vec4(0,2,4,0) ) -3. )-1.;
float angle = atan(pixel.x, pixel.y);
float dist = length(pixel - mouse2*0.15) * 8. + sin(iTime) * .01;
float flowerPeaks = .05 * amplitudeFactor * size;
float flowerPetals = 7.;
float edge = abs((dist + sin(angle * flowerPetals + iTime * 0.5) * sin(iTime * 1.5) * flowerPeaks) * 0.65 / size);
float colorChangeSpeed = 0.75 + 0.05 * sin(iTime) * 1.5;
float rainbowInput = timeFrac * colorChangeSpeed;
float brightness = 0.7;
vec4 rainbow = sqrt(j2hue(cos(rainbowInput))) + vec4(baseColor,0) - 1.0 + brightness;
float factor = smoothstep(1., .9, edge) * pow(edge, 2.);
vec3 color = rainbow.rgb * smoothstep(1., .9, edge) * pow(edge, 20.);
vec4 ring = vec4(
backgroundColor + clamp( mixedColor + color, 0., 1.)
, 1.0);
gl_FragColor = ring;
}
However I'm not able to figure out, how to adapt the behavior, so I can use a bright background.
If I remove the subtraction (and also remove the addition of the same at the end (vec4 ring = vec4(clamp(...))), I get the correct effect but with a black background.
Does anyone have an idea how to adapt the shader?
The problem is likely that backgroundColor is being added to the color to calculate the ring value. This will blow out your final color if backgroundColor is too bright.
I have a video stream coming from a 180 degree fisheye camera. I want to do some image-processing to convert the fisheye view into a normal view.
After some research and lots of read articles I found this paper.
They describe an algorithm (and some formulas) to solve this problem.
I used tried to implement this method in a Matlab. Unfortunately it doesn't work, and I failed to make it work. The "corrected" image looks exactly like the original photograph and there's no any removal of distortion and secondly I am just receiving top left side of the image, not the complete image but changing the value of 'K' to 1.9 gives mw the whole image, but its exactly the same image.
Input image:
Result:
When the value of K is 1.15 as mentioned in the article
When the value of K is 1.9
Here is my code:
image = imread('image2.png');
[Cx, Cy, channel] = size(image);
k = 1.5;
f = (Cx * Cy)/3;
opw = fix(f * tan(asin(sin(atan((Cx/2)/f)) * k)));
oph = fix(f * tan(asin(sin(atan((Cy/2)/f)) * k)));
image_new = zeros(opw, oph,channel);
for i = 1: opw
for j = 1: oph
[theta,rho] = cart2pol(i,j);
R = f * tan(asin(sin(atan(rho/f)) * k));
r = f * tan(asin(sin(atan(R/f))/k));
X = ceil(r * cos(theta));
Y = ceil(r * sin(theta));
for k = 1: 3
image_new(i,j,k) = image(X,Y,k);
end
end
end
image_new = uint8(image_new);
warning('off', 'Images:initSize:adjustingMag');
imshow(image_new);
This is what solved my problem.
input:
strength as floating point >= 0. 0 = no change, high numbers equal stronger correction.
zoom as floating point >= 1. (1 = no change in zoom)
algorithm:
set halfWidth = imageWidth / 2
set halfHeight = imageHeight / 2
if strength = 0 then strength = 0.00001
set correctionRadius = squareroot(imageWidth ^ 2 + imageHeight ^ 2) / strength
for each pixel (x,y) in destinationImage
set newX = x - halfWidth
set newY = y - halfHeight
set distance = squareroot(newX ^ 2 + newY ^ 2)
set r = distance / correctionRadius
if r = 0 then
set theta = 1
else
set theta = arctangent(r) / r
set sourceX = halfWidth + theta * newX * zoom
set sourceY = halfHeight + theta * newY * zoom
set color of pixel (x, y) to color of source image pixel at (sourceX, sourceY)
I want to convert from cube map [figure1] into an equirectangular panorama [figure2].
Figure1
Figure2
It is possible to go from Spherical to Cubic (by following: Convert 2:1 equirectangular panorama to cube map ), but lost on how to reverse it.
Figure2 is to be rendered into a sphere using Unity.
Assuming the input image is in the following cubemap format:
The goal is to project the image to the equirectangular format like so:
The conversion algorithm is rather straightforward.
In order to calculate the best estimate of the color at each pixel in the equirectangular image given a cubemap with 6 faces:
Firstly, calculate polar coordinates that correspond to each pixel in
the spherical image.
Secondly, using the polar coordinates form a vector and determine on
which face of the cubemap and which pixel of that face the vector
lies; just like a raycast from the center of a cube would hit one of
its sides and a specific point on that side.
Keep in mind that there are multiple methods to estimate the color of a pixel in the equirectangular image given a normalized coordinate (u,v) on a specific face of a cubemap. The most basic method, which is a very raw approximation and will be used in this answer for simplicity's sake, is to round the coordinates to a specific pixel and use that pixel. Other more advanced methods could calculate an average of a few neighbouring pixels.
The implementation of the algorithm will vary depending on the context. I did a quick implementation in Unity3D C# that shows how to implement the algorithm in a real world scenario. It runs on the CPU, there is a lot room for improvement but it is easy to understand.
using UnityEngine;
public static class CubemapConverter
{
public static byte[] ConvertToEquirectangular(Texture2D sourceTexture, int outputWidth, int outputHeight)
{
Texture2D equiTexture = new Texture2D(outputWidth, outputHeight, TextureFormat.ARGB32, false);
float u, v; //Normalised texture coordinates, from 0 to 1, starting at lower left corner
float phi, theta; //Polar coordinates
int cubeFaceWidth, cubeFaceHeight;
cubeFaceWidth = sourceTexture.width / 4; //4 horizontal faces
cubeFaceHeight = sourceTexture.height / 3; //3 vertical faces
for (int j = 0; j < equiTexture.height; j++)
{
//Rows start from the bottom
v = 1 - ((float)j / equiTexture.height);
theta = v * Mathf.PI;
for (int i = 0; i < equiTexture.width; i++)
{
//Columns start from the left
u = ((float)i / equiTexture.width);
phi = u * 2 * Mathf.PI;
float x, y, z; //Unit vector
x = Mathf.Sin(phi) * Mathf.Sin(theta) * -1;
y = Mathf.Cos(theta);
z = Mathf.Cos(phi) * Mathf.Sin(theta) * -1;
float xa, ya, za;
float a;
a = Mathf.Max(new float[3] { Mathf.Abs(x), Mathf.Abs(y), Mathf.Abs(z) });
//Vector Parallel to the unit vector that lies on one of the cube faces
xa = x / a;
ya = y / a;
za = z / a;
Color color;
int xPixel, yPixel;
int xOffset, yOffset;
if (xa == 1)
{
//Right
xPixel = (int)((((za + 1f) / 2f) - 1f) * cubeFaceWidth);
xOffset = 2 * cubeFaceWidth; //Offset
yPixel = (int)((((ya + 1f) / 2f)) * cubeFaceHeight);
yOffset = cubeFaceHeight; //Offset
}
else if (xa == -1)
{
//Left
xPixel = (int)((((za + 1f) / 2f)) * cubeFaceWidth);
xOffset = 0;
yPixel = (int)((((ya + 1f) / 2f)) * cubeFaceHeight);
yOffset = cubeFaceHeight;
}
else if (ya == 1)
{
//Up
xPixel = (int)((((xa + 1f) / 2f)) * cubeFaceWidth);
xOffset = cubeFaceWidth;
yPixel = (int)((((za + 1f) / 2f) - 1f) * cubeFaceHeight);
yOffset = 2 * cubeFaceHeight;
}
else if (ya == -1)
{
//Down
xPixel = (int)((((xa + 1f) / 2f)) * cubeFaceWidth);
xOffset = cubeFaceWidth;
yPixel = (int)((((za + 1f) / 2f)) * cubeFaceHeight);
yOffset = 0;
}
else if (za == 1)
{
//Front
xPixel = (int)((((xa + 1f) / 2f)) * cubeFaceWidth);
xOffset = cubeFaceWidth;
yPixel = (int)((((ya + 1f) / 2f)) * cubeFaceHeight);
yOffset = cubeFaceHeight;
}
else if (za == -1)
{
//Back
xPixel = (int)((((xa + 1f) / 2f) - 1f) * cubeFaceWidth);
xOffset = 3 * cubeFaceWidth;
yPixel = (int)((((ya + 1f) / 2f)) * cubeFaceHeight);
yOffset = cubeFaceHeight;
}
else
{
Debug.LogWarning("Unknown face, something went wrong");
xPixel = 0;
yPixel = 0;
xOffset = 0;
yOffset = 0;
}
xPixel = Mathf.Abs(xPixel);
yPixel = Mathf.Abs(yPixel);
xPixel += xOffset;
yPixel += yOffset;
color = sourceTexture.GetPixel(xPixel, yPixel);
equiTexture.SetPixel(i, j, color);
}
}
equiTexture.Apply();
var bytes = equiTexture.EncodeToPNG();
Object.DestroyImmediate(equiTexture);
return bytes;
}
}
In order to utilize the GPU I created a shader that does the same conversion. It is much faster than running the conversion pixel by pixel on the CPU but unfortunately Unity imposes resolution limitations on cubemaps so it's usefulness is limited in scenarios when high resolution input image is to be used.
Shader "Conversion/CubemapToEquirectangular" {
Properties {
_MainTex ("Cubemap (RGB)", CUBE) = "" {}
}
Subshader {
Pass {
ZTest Always Cull Off ZWrite Off
Fog { Mode off }
CGPROGRAM
#pragma vertex vert
#pragma fragment frag
#pragma fragmentoption ARB_precision_hint_fastest
//#pragma fragmentoption ARB_precision_hint_nicest
#include "UnityCG.cginc"
#define PI 3.141592653589793
#define TWOPI 6.283185307179587
struct v2f {
float4 pos : POSITION;
float2 uv : TEXCOORD0;
};
samplerCUBE _MainTex;
v2f vert( appdata_img v )
{
v2f o;
o.pos = mul(UNITY_MATRIX_MVP, v.vertex);
o.uv = v.texcoord.xy * float2(TWOPI, PI);
return o;
}
fixed4 frag(v2f i) : COLOR
{
float theta = i.uv.y;
float phi = i.uv.x;
float3 unit = float3(0,0,0);
unit.x = sin(phi) * sin(theta) * -1;
unit.y = cos(theta) * -1;
unit.z = cos(phi) * sin(theta) * -1;
return texCUBE(_MainTex, unit);
}
ENDCG
}
}
Fallback Off
}
The quality of the resulting images can be greatly improved by either employing a more sophisticated method to estimate the color of a pixel during the conversion or by post processing the resulting image (or both, actually). For example an image of bigger size could be generated to apply a blur filter and then downsample it to the desired size.
I created a simple Unity project with two editor wizards that show how to properly utilize either the C# code or the shader shown above. Get it here:
https://github.com/Mapiarz/CubemapToEquirectangular
Remember to set proper import settings in Unity for your input images:
Point filtering
Truecolor format
Disable mipmaps
Non Power of 2: None (only for 2DTextures)
Enable Read/Write (only for 2DTextures)
cube2sphere automates the entire process. Example:
$ cube2sphere front.jpg back.jpg right.jpg left.jpg top.jpg bottom.jpg -r 2048 1024 -fTGA -ostitched
I am following the quaternion tutorial: http://www.raywenderlich.com/12667/how-to-rotate-a-3d-object-using-touches-with-opengl and am trying to rotate a globe to some XYZ location. I have an initial quaternion and generate a random XYZ location on the surface of the globe. I pass that XYZ location into the following function. The idea was to generate a lookAt vector with GLKMatrix4MakeLookAt and define the end Quaternion for the slerp step from the lookAt matrix.
- (void)rotateToLocationX:(float)x andY:(float)y andZ:(float)z {
// Turn on the interpolation for smooth rotation
_slerping = YES; // Begin auto rotating to this location
_slerpCur = 0;
_slerpMax = 1.0;
_slerpStart = _quat;
// The eye location is defined by the look at location multiplied by this modifier
float modifier = 1.0;
// Create a look at vector for which we will create a GLK4Matrix from
float xEye = x;
float yEye = y;
float zEye = z;
//NSLog(#"%f %f %f %f %f %f",xEye, yEye, zEye, x, y, z);
_currentSatelliteLocation = GLKMatrix4MakeLookAt(xEye, yEye, zEye, 0, 0, 0, 0, 1, 0);
_currentSatelliteLocation = GLKMatrix4Multiply(_currentSatelliteLocation,self.effect.transform.modelviewMatrix);
// Turn our 4x4 matrix into a quat and use it to mark the end point of our interpolation
//_currentSatelliteLocation = GLKMatrix4Translate(_currentSatelliteLocation, 0.0f, 0.0f, GLOBAL_EARTH_Z_LOCATION);
_slerpEnd = GLKQuaternionMakeWithMatrix4(_currentSatelliteLocation);
// Print info on the quat
GLKVector3 vec = GLKQuaternionAxis(_slerpEnd);
float angle = GLKQuaternionAngle(_slerpEnd);
//NSLog(#"%f %f %f %f",vec.x,vec.y,vec.z,angle);
NSLog(#"Quat end:");
[self printMatrix:_currentSatelliteLocation];
//[self printMatrix:self.effect.transform.modelviewMatrix];
}
The interpolation works, I get a smooth rotation, however the ending location is never the XYZ I input - I know this because my globe is a sphere and I am calculating XYZ from Lat Lon. I want to look directly down the 'lookAt' vector toward the center of the earth from that lat/lon location on the surface of the globe after the rotation. I think it may have something to do with the up vector but I've tried everything that made sense.
What am I doing wrong - How can I define a final quaternion that when I finish rotating, looks down a vector to the XYZ on the surface of the globe? Thanks!
Is the following your meaning:
Your globe center is (0, 0, 0), radius is R, the start position is (0, 0, R), your final position is (0, R, 0), so rotate the globe 90 degrees around X-asix?
If so, just set lookat function eye position to your final position, the look at parameters to the globe center.
m_target.x = 0.0f;
m_target.y = 0.0f;
m_target.z = 1.0f;
m_right.x = 1.0f;
m_right.y = 0.0f;
m_right.z = 0.0f;
m_up.x = 0.0f;
m_up.y = 1.0f;
m_up.z = 0.0f;
void CCamera::RotateX( float amount )
{
Point3D target = m_target;
Point3D up = m_up;
amount = amount / 180 * PI;
m_target.x = (cos(PI / 2 - amount) * up.x) + (cos(amount) * target.x);
m_target.y = (cos(PI / 2 - amount) * up.y) + (cos(amount) * target.y);
m_target.z = (cos(PI / 2 - amount) * up.z) + (cos(amount) * target.z);
m_up.x = (cos(amount) * up.x) + (cos(PI / 2 + amount) * target.x);
m_up.y = (cos(amount) * up.y) + (cos(PI / 2 + amount) * target.y);
m_up.z = (cos(amount) * up.z) + (cos(PI / 2 + amount) * target.z);
Normalize(m_target);
Normalize(m_up);
}
void CCamera::RotateY( float amount )
{
Point3D target = m_target;
Point3D right = m_right;
amount = amount / 180 * PI;
m_target.x = (cos(PI / 2 + amount) * right.x) + (cos(amount) * target.x);
m_target.y = (cos(PI / 2 + amount) * right.y) + (cos(amount) * target.y);
m_target.z = (cos(PI / 2 + amount) * right.z) + (cos(amount) * target.z);
m_right.x = (cos(amount) * right.x) + (cos(PI / 2 - amount) * target.x);
m_right.y = (cos(amount) * right.y) + (cos(PI / 2 - amount) * target.y);
m_right.z = (cos(amount) * right.z) + (cos(PI / 2 - amount) * target.z);
Normalize(m_target);
Normalize(m_right);
}
void CCamera::RotateZ( float amount )
{
Point3D right = m_right;
Point3D up = m_up;
amount = amount / 180 * PI;
m_up.x = (cos(amount) * up.x) + (cos(PI / 2 - amount) * right.x);
m_up.y = (cos(amount) * up.y) + (cos(PI / 2 - amount) * right.y);
m_up.z = (cos(amount) * up.z) + (cos(PI / 2 - amount) * right.z);
m_right.x = (cos(PI / 2 + amount) * up.x) + (cos(amount) * right.x);
m_right.y = (cos(PI / 2 + amount) * up.y) + (cos(amount) * right.y);
m_right.z = (cos(PI / 2 + amount) * up.z) + (cos(amount) * right.z);
Normalize(m_right);
Normalize(m_up);
}
void CCamera::Normalize( Point3D &p )
{
float length = sqrt(p.x * p.x + p.y * p.y + p.z * p.z);
if (1 == length || 0 == length)
{
return;
}
float scaleFactor = 1.0 / length;
p.x *= scaleFactor;
p.y *= scaleFactor;
p.z *= scaleFactor;
}
The answer to this question is a combination of the following rotateTo function and a change to the code from Ray's tutorial at ( http://www.raywenderlich.com/12667/how-to-rotate-a-3d-object-using-touches-with-opengl ). As one of the comments on that article says there is an arbitrary factor of 2.0 being multiplied in GLKQuaternion Q_rot = GLKQuaternionMakeWithAngleAndVector3Axis(angle * 2.0, axis);. Remove that "2" and use the following function to create the _slerpEnd - after that the globe will rotate smoothly to XYZ specified.
// Rotate the globe using Slerp interpolation to an XYZ coordinate
- (void)rotateToLocationX:(float)x andY:(float)y andZ:(float)z {
// Turn on the interpolation for smooth rotation
_slerping = YES; // Begin auto rotating to this location
_slerpCur = 0;
_slerpMax = 1.0;
_slerpStart = _quat;
// Create a look at vector for which we will create a GLK4Matrix from
float xEye = x;
float yEye = y;
float zEye = z;
_currentSatelliteLocation = GLKMatrix4MakeLookAt(xEye, yEye, zEye, 0, 0, 0, 0, 1, 0);
// Turn our 4x4 matrix into a quat and use it to mark the end point of our interpolation
_slerpEnd = GLKQuaternionMakeWithMatrix4(_currentSatelliteLocation);
}
Will this GLSL code create compile-time constants out of "oneSixth" and "twoThirds"?
// GLSL TESSELLATION EVALUATION SHADER
#version 410
layout (isolines, equal_spacing) in;
in vec4 tessColor[];
out vec4 pointColor;
const float oneSixth = 1. / 6.;
const float twoThirds = 2. / 3.;
void main ()
{
float s2 = gl_TessCoord.s * gl_TessCoord.s;
float s3 = s2 * gl_TessCoord.s;
float w0 = oneSixth - .5 * gl_TessCoord.s + .5 * s2 - oneSixth * s3;
float w1 = twoThirds - s2 + .5 * s3;
float w2 = oneSixth + .5 * gl_TessCoord.s + .5 * s2 - .5 * s3;
float w3 = oneSixth * s3;
gl_Position = w0 * gl_in[0].gl_Position + w1 * gl_in[1].gl_Position +
w2 * gl_in[2].gl_Position + w3 * gl_in[3].gl_Position;
pointColor = w0 * tessColor[0] + w1 * tessColor[1] +
w2 * tessColor[2] + w3 * tessColor[3];
}
A colleague of mine thinks this code is inefficient and says I should hard-code the division or it will happen at run-time.
const float oneSixth = .1666666667;
const float twoThirds = .6666666667;
I'm new to GLSL but I'm skeptical that this is necessary. Any thoughts? Is it vendor dependent?
It will happen at compile-time. No need to hardcode trivialities like this. However, this is not mentioned in the GLSL specification.
When in doubt, measure, but I would consider any implementation that didn't do this at compile time broken.