Just in theory: How is the alpha component premultiplied into the other components of an PNG in iPhone OS, and how can it be unpremultiplied properly? - iphone

Actually, I thought that there would be an easy way to achieve that. What I need is pure alpha value information. For testing, I have a 50 x 55 px PNG, where on every edge a 5x5 pixel rectangle is fully transparent. In these areas alpha has to be 0.Everywhere else it has to be 255. I made very sure that my PNG is created correctly, and it also looks correctly.
Please tell me if this is theoretically correct:
I created an CGImageRef that has only the alpha channel and nothing else. This is done with CGBitmapContextCreate and kCGImageAlphaOnly as param.
CGBitmapContextGetBitsPerPixel(context) returns me 1, so it indicates me that I really have only one component per pixel: The desired alpha value.
I've been reading that CGBitmapContextCreate will handle all the conversion from the given image to the new created context. My image was previously PNG-24 with transparency, but pngcrunch from Xcode seems to convert them somehow.
So, just in theory: Do I have any chance to get to the correct, unpremultiplied alpha at this point? The values I get seem to almost match, but in a big 5x5 transparent square I get values like 19, 197, 210, 0, 0, 0, 98 and so on. If they were true, I would have to see something from the image. The image itself is solid blue.

Premultiplication doesn't affect the alpha channel, it affects the color channels.
The formula for raster compositing (putting one raster image over another) is:
dst.r = src.r * src.a + dst.r * (1.0 - src.a);
dst.g = src.g * src.a + dst.g * (1.0 - src.a);
dst.b = src.b * src.a + dst.b * (1.0 - src.a);
Premultiplication cuts out the first multiplication expression:
dst.r = src.r′ + dst.r * (1.0 - src.a);
dst.g = src.g′ + dst.g * (1.0 - src.a);
dst.b = src.b′ + dst.b * (1.0 - src.a);
This works because the source color components are already multiplied by the alpha component—hence the name “premultiplied”. It doesn't need to multiply them now, because it already has the results.
unpremultiplied alpha
The alpha component itself is never premultiplied: What would you multiply it by? The color components are premultiplied by the alpha.

Since premultiplying color values is a simple as:
r = (r * a) / 255;
g = (g * a) / 255;
b = (b * a) / 255;
Getting the inverse would be:
if (a > 0) {
r = (r * 255) / a;
g = (g * 255) / a;
b = (b * 255) / a;
}

This formular is not correct. The goal is to find unmultiplexed (r, g, b) that would latter result to the same multiplexed values (it is not possible to find the original r, g, b values, though.
However with the formular above we find for the example of
alpha = 100
r_premulti = 1
a reconstructed r = 2;
Latter if this r is multiplexed again we find 2 * 100 / 255 = 0 but we wanted r_premulti == 1 instead!!
The correct formular needs to round up. Example for r-component:
reconstruced r = ceiling(r_premulti * 255 / alpha)

Related

refresh drawing area in gtk

I have a bunch of drawing areas (they are actually cairo surfaces, but I don't think it matters too much) in a scrolled window, and I would like to refresh the drawings. However, when I redraw the images, they are not shown till I scroll the window up and down. After that the figures are correct, so I have to conclude that the drawing routine itself is proper. I have also included a
while Gtk.events_pending():
Gtk.main_iteration()
loop to wait for all pending operations, but that does not solve the problem. Could someone point out to me what else is missing?
Thanks,
v923z
OK, so the larger chunks of the code. First, a class defining the a drawing area onto which I am going to paint (note that the body is not indented properly! I don't know how to indent larger pieces of code here):
class Preview:
def __init__(self):
self.frame = Gtk.Frame()
self.frame.set_shadow_type(Gtk.ShadowType.IN)
self.frame.show()
self.da = Gtk.DrawingArea()
self.da.set_size_request(200, 300)
self.da.connect('configure-event', self.configure_event)
self.da.connect('draw', self.on_draw)
self.frame.add(self.da)
self.da.show()
def configure_event(self, da, event):
allocation = da.get_allocation()
self.surface = da.get_window().create_similar_surface(cairo.CONTENT_COLOR,
allocation.width,
allocation.height)
cairo_ctx = cairo.Context(self.surface)
cairo_ctx.set_source_rgb(1, 1, 1)
cairo_ctx.paint()
return True
def on_draw(self, da, cairo_ctx):
cairo_ctx.set_source_surface(self.surface, 0, 0)
cairo_ctx.paint()
return True
pass
Next, the point where I actually create the drawing area. viewport_preview is a viewport created in glade.
self.previews = []
self.widget('viewport_preview').remove(self.vbox_preview)
self.vbox_preview = Gtk.VBox(homogeneous=False, spacing=8)
self.widget('viewport_preview').add(self.vbox_preview)
self.vbox_preview.show()
for page in self.pages:
preview = Preview()
self.vbox_preview.pack_start(preview.frame, False, False, 10)
self.previews.append(preview)
while Gtk.events_pending():
Gtk.main_iteration()
self.draw_preview(None)
return True
Then the function drawing the previews. This is really just a wrapper for the next function, and I needed this only because if I delete one entry in the previews, then I have to handle that case. I believe, the while loop at the end of this function is not necessary, for it will be at the end of the next one anyway.
def draw_preview(self, counter=None):
if counter is not None:
self.vbox_preview.remove(self.previews[counter].frame)
self.previews.pop(counter)
self.pages.pop(counter)
self.vbox_preview.show()
while Gtk.events_pending():
Gtk.main_iteration()
for i in range(len(self.pages)):
self.draw_note(self.previews[i].da, self.previews[i].surface, self.pages[i])
while Gtk.events_pending():
Gtk.main_iteration()
Finally, the drawing function itself:
def draw_note(self, widget, surface, page):
list_pos = '%d/%d'%(self.page + 1, len(self.pages))
self.widget('label_status').set_text(list_pos)
cairo_ctx = cairo.Context(surface)
cairo_ctx.set_source_rgb(page.background[0], page.background[1], page.background[2])
cairo_ctx.paint()
width, height = widget.get_size_request()
xmin, xmax, ymin, ymax = fujitsu.page_size(page)
factor = min(height / (2.0 * self.margin + ymax - ymin), width / (2.0 * self.margin + xmax - xmin))
factor *= 0.8
page.scale = factor
value = self.widget('adjustment_smooth').get_value()
#print value
for pen in page.pagecontent:
x = self.margin + pen.path[0][0] - xmin
y = self.margin + pen.path[0][1] - ymin
cairo_ctx.move_to(x * factor, y * factor)
if self.widget('checkbutton_smooth').get_active() == False:
[cairo_ctx.line_to((self.margin + x - xmin) * factor,
(self.margin + y - ymin) * factor) for x, y in pen.path]
else:
bezier_curve = bezier.expand_coords(pen.path, value)
x = self.margin + bezier_curve[0][0][0] - xmin
y = self.margin + bezier_curve[0][0][1] - ymin
cairo_ctx.move_to(x * factor, y * factor)
[cairo_ctx.curve_to((self.margin + control[1][0] - xmin) * factor,
(self.margin + control[1][1] - ymin) * factor,
(self.margin + control[2][0] - xmin) * factor,
(self.margin + control[2][1] - ymin) * factor,
(self.margin + control[3][0] - xmin) * factor,
(self.margin + control[3][1] - ymin) * factor)
for control in bezier_curve]
cairo_ctx.set_line_width(pen.thickness * self.zoom_factor)
cairo_ctx.set_source_rgba(pen.colour[0], pen.colour[1], pen.colour[2], pen.colour[3])
cairo_ctx.stroke()
cairo_ctx.rectangle(0, height * 0.96, width, height)
cairo_ctx.set_source_rgba(page.banner_text[0][0], page.banner_text[0][1], page.banner_text[0][2], page.banner_text[0][3])
cairo_ctx.fill()
cairo_ctx.move_to(width * 0.05, height * 0.99)
cairo_ctx.show_text(self.filename + ' ' + list_pos)
cairo_ctx.set_font_size(self.zoom_factor * 10.0)
xbearing, ybearing, twidth, theight, xadvance, yadvance = (cairo_ctx.text_extents(page.banner_text[3]))
cairo_ctx.move_to(width - 1.03 * twidth, height * 0.99)
cairo_ctx.show_text(page.banner_text[3])
cairo_ctx.set_source_rgba(0, 0, 0.9, 0.90)
cairo_ctx.stroke()
rect = widget.get_allocation()
widget.get_window().invalidate_rect(rect, False)
while Gtk.events_pending():
Gtk.main_iteration()
I think that's about it.
You could use gtk_widget_queue_draw_area or gdk_window_invalidate_rect.This will mark the widget (or rectangle) as dirty and once the main loop is idle expose event will be received where in you can redraw. From you description it appears the updates are happening on expose event so these APIs might be of use. Also you can check this sample from the cairo site where in you can see the usage of gtk_widget_queue_draw_area.
I have not used pygtk but from Google I found that the corresponding call for gtk_widget_queue_draw_area is gtk.Widget.queue_draw_area & for gdk_window_invalidate_rect is gtk.gdk.Window.invalidate_rect
Hope this helps!

Determining what frequencies correspond to the x axis in aurioTouch sample application

I'm looking at the aurioTouch sample application for the iPhone SDK. It has a basic spectrum analyzer implemented when you choose the "FFT" option. One of the things the app is lacking is X axis labels (i.e. the frequency labels).
In the aurioTouchAppDelegate.mm file, in the function - (void)drawOscilloscope at line 652, it has the following code:
if (displayMode == aurioTouchDisplayModeOscilloscopeFFT)
{
if (fftBufferManager->HasNewAudioData())
{
if (fftBufferManager->ComputeFFT(l_fftData))
[self setFFTData:l_fftData length:fftBufferManager->GetNumberFrames() / 2];
else
hasNewFFTData = NO;
}
if (hasNewFFTData)
{
int y, maxY;
maxY = drawBufferLen;
for (y=0; y<maxY; y++)
{
CGFloat yFract = (CGFloat)y / (CGFloat)(maxY - 1);
CGFloat fftIdx = yFract * ((CGFloat)fftLength);
double fftIdx_i, fftIdx_f;
fftIdx_f = modf(fftIdx, &fftIdx_i);
SInt8 fft_l, fft_r;
CGFloat fft_l_fl, fft_r_fl;
CGFloat interpVal;
fft_l = (fftData[(int)fftIdx_i] & 0xFF000000) >> 24;
fft_r = (fftData[(int)fftIdx_i + 1] & 0xFF000000) >> 24;
fft_l_fl = (CGFloat)(fft_l + 80) / 64.;
fft_r_fl = (CGFloat)(fft_r + 80) / 64.;
interpVal = fft_l_fl * (1. - fftIdx_f) + fft_r_fl * fftIdx_f;
interpVal = CLAMP(0., interpVal, 1.);
drawBuffers[0][y] = (interpVal * 120);
}
cycleOscilloscopeLines();
}
}
From my understanding, this part of the code is what is used to decide which magnitude to draw for each frequency in the UI. My question is how can I determine what frequency each iteration (or y value) represents inside the for loop.
For example, if I want to know what the magnitude is for 6kHz, I'm thinking of adding a line similar to the following:
if (yValueRepresentskHz(y, 6))
NSLog(#"The magnitude for 6kHz is %f", (interpVal * 120));
Please note that although they chose to use the variable name y, from what I understand, it actually represents the x-axis in the visual graph of the spectrum analyzer, and the value of the drawBuffers[0][y] represents the y-axis.
I believe that the frequency of each bin it is using is given by
yFract * hwSampleRate * .5
I'm fairly certain that you need the .5 because yFract is a fraction of the total fftLength and the last bin of the FFT corresponds with half of the sampling rate. Thus, you could do something like
NSLog(#"The magnitude for %f Hz is %f.", (yFract * hwSampleRate * .5), (interpVal * 120));
Hopefully that helps to point you in the right direction at least.

Looking for some help working with premultiplied alpha

I am trying to update a source image with the contents of multiple destination images. From what I can tell using premultiplied alpha is the way to go with this, but I think I am doing something wrong (function below). the image I am starting with is initialized with all ARGB values set to 0. When I run the function once the resulting image looks great, but when I start compositing on any others all the pixels that have alpha information get really messed up. Does anyone know if I am doing something glaringly wrong or if there is something extra I need to do to modify the color values?
void CompositeImage(unsigned char *src, unsigned char *dest, int srcW, int srcH){
int w = srcW;
int h = srcH;
int px0;
int px1;
int px2;
int px3;
int inverseAlpha;
int r;
int g;
int b;
int a;
int y;
int x;
for (y = 0; y < h; y++) {
for (x= 0; x< w*4; x+=4) {
// pixel number
px0 = (y*w*4) + x;
px1 = (y*w*4) + (x+1);
px2 = (y*w*4) + (x+2);
px3 = (y*w*4) + (x+3);
inverseAlpha = 1 - src[px3];
// create new values
r = src[px0] + inverseAlpha * dest[px0];
g = src[px1] + inverseAlpha * dest[px1];
b = src[px2] + inverseAlpha * dest[px2];
a = src[px3] + inverseAlpha * dest[px3];
// update destination image
dest[px0] = r;
dest[px1] = g;
dest[px2] = b;
dest[px3] = a;
}
}
}
I'm not clear on what data you are working with. Do your source images already have the alpha values pre-multiplied as they are stored? If not, then pre-multiplied alpha does not apply here and you would need to do normal alpha blending.
Anyway, the big problem in your code is that you're not keeping track of the value ranges that you're dealing with.
inverseAlpha = 1 - src[px3];
This needs to be changed to:
inverseAlpha = 255 - src[px3];
You have all integral value types here, so the normal incoming 0..255 value range will result in an inverseAlpha range of -254..1, which will give you some truly wacky results.
After changing the 1 to 255, you also need to divide your results for each channel by 255 to scale them back down to the appropriate range. The alternative is to do the intermediate calculations using floats instead of integers and divide the initial channel values by 255.0 (instead of these other changes) to get values in the 0..1 range.
If your source data really does already have pre-multiplied alpha, then your result lines should look like this.
r = src[px0] + inverseAlpha * dest[px0] / 255;
If your source data does not have pre-multiplied alpha, then it should be:
r = src[px0] * src[px3] / 255 + inverseAlpha * dest[px0] / 255;
There's nothing special about blending the alpha channel. Use the same calculation as for r, g, and b.

iphone compass tilt compensation

has anybody already programmed a iphone compass heading tilt compensation?
i have got some approaches, but some help or a better solution would be cool!
FIRST
i define a vector Ev, calculated out of the cross product of Gv and Hv. Gv is a gravity vector i build out of the accelerometer values and Hv is an heading vector built out the magnetometer values.
Ev stands perpendicular on Gv and Hv, so it is heading to horizonatl East.
SECOND
i define a vector Rv, calculated out of the cross product Bv and Gv. Bv is my looking vector and it is defined as [0,0,-1]. Rv is perpendicular to Gv and Bv and shows always to the right.
THIRD
the angle between these two vectors, Ev and Rv, should be my corrected heading. to calculate the angle i build the dot product and thereof the arcos.
phi = arcos ( Ev * Rv / |Ev| * |Rv| )
Theoretically it should work, but maybe i have to normalize the vectors?!
Has anybody got a solution for this?
Thanks, m01d
Yep. You DEFINITELY have to normalize.
This is from my code that I use to extract the orientation of the device.
Gravity is obtained as the x,y,z of the accelerometer
and compass is obtained from the x,y,z of the heading function
gravity.normalize();
compass.normalize();
compassEast=gravity.cross(compass);
compassEast.normalize();
compassNorth=compassEast.cross(gravity);
compassNorth.normalize();
Let me know if you need the full code.
Also, for those who havnt yet seen the iphone 4s gyroscope in action: its amazing! I swapped the above input to gravity and compass for the equivalents from the gyro and the result is stable and smooth and awesome :) Go Apple.
I didn't receive the source code but I set up my own example. You can see the project and code here: http://www.sundh.com/blog/2011/09/stabalize-compass-of-iphone-with-gyroscope/
yes, i did it like described above. but the result is not very accurate. i think with smoother accelerometer values it should work that way. because of this i have choosen to do the tilt compensation by adding/subtracting the accelermoter values of the corresponding axis to/from the compass values.
Here iss my code for the solution above, but its not a final working solution:
- (void)locationManager:(CLLocationManager *)manager didUpdateHeading:(CLHeading *)newHeading{
if (newHeading != nil) {
float Ax = accelerationValueX;
float Ay = accelerationValueY;
float Az = accelerationValueZ;
float filterFactor = 0.2;
Mx = [newHeading x] * filterFactor + (Mx * (1.0 - filterFactor));
My = [newHeading y] * filterFactor + (My * (1.0 - filterFactor));
Mz = [newHeading z] * filterFactor + (Mz * (1.0 - filterFactor));
float counter = ( -pow(Ax, 2)*Mz + Ax*Az*Mx - pow(Ay, 2)*Mz + Ay*Az*My );
float denominator = ( sqrt( pow((My*Az-Mz*Ay), 2) + pow((Mz*Ax-Mx*Az), 2) + pow((Mx*Ay-My*Ax), 2) ) * sqrt(pow(Ay, 2)+pow(-Ax, 2)) );
headingCorrected = (acos(counter/denominator)* (180.0 / M_PI)) * filterFactor + (headingCorrected * (1.0 - filterFactor));
}
...
}

How can I obtain clean alpha channel information from an PNG image in iPhone OS?

I have created an context like this (simplified):
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate (bitmapData,
pixWide,
pixHeigh,
8, // bits per component
bitmapBytesPerRow,
colorSpace,
kCGImageAlphaPremultipliedFirst);
Now, when I try to extract the data for the first pixel in my PNG with Alphatransparency, it has very weird alpha values. I have an simple PNG that's a square. On each edge I cut off 10x10 pixel and made them totally transparent. Alpha shouldn't be something like 153 there.
There's an kCGImageAlphaOnly declared in CGImage.h. The doc says:
kCGImageAlphaOnly There is no color
data, only an alpha channel.
Ok, so that actually sounds good, because I only need Alpha data, and nothing else. But this raises some question marks in my head. If I do habe a fully equipped PNG with a bunch of colors + alpha: Would this constant make sure that my PNG is converted to match that color space? Or would I have to provide an PNG that matches that specified color space?
Edit: I tried using kCGImageAlphaOnly, but I get this error:
<Error>: CGBitmapContextCreate: unsupported parameter combination: 8 integer bits/component; 24 bits/pixel; 0-component colorspace; kCGImageAlphaOnly; 55 bytes/row.
What may be the problem here? I specified this before:
size_t pixelsWide = CGImageGetWidth(inImage);
size_t pixelsHigh = CGImageGetHeight(inImage);
bitmapBytesPerRow = (pixelsWide * 1); // not * 4, because I just want alpha
bitmapByteCount = (bitmapBytesPerRow * pixelsHigh);
Edit: I've been reading this a minute ago:
PNG's which are added to XCode are
optimized by 'pngcrush' during
compilation. This does some
byte-swapping (from RGBA to BRGA) and
pre-multiplication of alpha.
I assume that this pre-multiplication of alpha makes trouble.
Edit: The alpha channel keeps intact after pngcrunch did the byte-swapping stuff to the PNG. Since I don't care about colors, just alpha, that pre-multiplication shouldn't be a too big problem, I think.
My PNG's have been 24bit PNG bevor I added them to Xcode.
You can't do this:
bitmapBytesPerRow = (pixelsWide * 1); // not * 4, because I just want alpha
The function you're calling will always return all the image data. The kCGImageAlphaOnly constant is used to tell YOU that an image only contains an alpha channel, no colour information.
You'll need to use pixelsWide * 4 for the bytesPerRow. Also note that the bitmapData argument to CGBitmapContextCreate() is used to provide storage space explicitly, rather than having it drawn for you.
Possibly what you want to do is this (untested code, just typed from memory):
CGImageRef image = GetMyImageFromWhereverItIs();
CGColorSpaceRef space = CGColorSpaceCreateDeviceRGB();
CGContextRef ctx = CGBitmapContextCreate( NULL, CGImageGetWidth(image),
CGImageGetHeight(image), CGImageGetBitsPerComponent(image),
CGImageGetBytesPerRow(image), space,
kCGBitmapByteOrderDefault | kCGImageAlphaLast );
CGColorSpaceRelease( space );
// now draw the image into the context
CGRect rect = CGRectMake( 0, 0, CGImageGetWidth(image), CGImageGetHeight(image) );
CGContextDrawImage( ctx, rect, image );
UInt32 * pixels = CGBitmapContextGetData( ctx );
// now we can iterate through the data & read the alpha values
int i, count = CGBitmapContextGetBytesPerRow(ctx) * CGBitmapContextGetHeight(ctx);
for ( i = 0; i < count; i++ )
{
UInt8 alpha = pixels[i] & 0x000000ff;
// do with the alpha what you will
}
Are you sure you're looking only at alpha values?
If you're expecting all the alpha components to come first, then all the red components, etc.: That's planar layout, and I don't think Quartz supports it natively—it only supports all the components together in each pixel (ARGBARGBARGBARGB…, not AAAA…RRRR…GGGG…BBBB…). So if you're just marching straight into the data treating every byte as alpha, that's your problem: you're looking at red, green, and blue components and treating them as alpha.
As for premultiplication, that doesn't affect the alpha channel, it affects the color channels. The formula for raster compositing (putting one raster image over another) is:
dst.r = src.r * src.a + dst.r * (1.0 - src.a);
dst.g = src.g * src.a + dst.g * (1.0 - src.a);
dst.b = src.b * src.a + dst.b * (1.0 - src.a);
Premultiplication cuts out the first multiplication expression:
dst.r = src.r′ + dst.r * (1.0 - src.a);
dst.g = src.g′ + dst.g * (1.0 - src.a);
dst.b = src.b′ + dst.b * (1.0 - src.a);
This works because the source color components are already multiplied by the alpha component—hence the name “premultiplied”. It doesn't need to multiply them now, because it already has the results.
It's an optimization, and presumably an important one on the iPhone (all those multiplication operations add up when you do a million or two of them). But it doesn't affect the layout of the components: interleaved remains interleaved, whether the RGB components are premultiplied or not.