I'm working on a little project that does "scan" sheets of paper using a camera (specifically the iPhone camera).
Workflow:
Lets users take pictures of sheets of paper while I will afterwards detect corners/the outline so I know the 4 corner points of the original sheet of paper in the photograph. It is then easy to inverse the perspective transformation of the camera to gain a "flat" image of the photographed sheet of paper.
However, my question is: how do I maintain or even calculate the correct (original) aspect ratio?
As you can see above I have the coordinates for the corner points and I also know how to construct a matrix to map the top left corner to the point (0,0), the top right corner to (width,0), the bottom right corner to (width, height) and the bottom left corner to (0,height).
The question now is: How do I get the correct width/height aspect ratio the sheet of paper originally has. I'm pretty sure I can figure it out using the angles to determine the perspective transformation but I'm stuck finding the right formula. I also have a hunch that I also would need the focal length of the camera but it would be great not to need that.
Any ideas would be appreciated.
This paper
http://research.microsoft.com/en-us/um/people/zhang/papers/tr03-39.pdf
describes the process.
I did some experiments and although the results for a 'synthetic' images were good, with real images (or when I added some noise to the 'detected' corners of synthetic image) the result were unstable - aspect ratio varied considerably.
Related
I'm new to Unity and to game development in general.
I would like to make a text-based game.
I'm looking to reproduce the behavior of an instant messenger like messenger or whatapp.
I made the choice to use the Unity UI system for the pre-made components like the rect scroll.
But this choice led me to the following problem:
I have "bubbles" of dialogs, which must be able to grow in width as well as in height with the size of the text. Fig.1
I immediately tried to use VectorGraphics to import .svg with the idea to move runtime the points of my curves of Beziers.
But I did not find how to access these points and edit them runtime.
I then found the "Sprite shapes" but they are not part of the "UI",
so if I went with such a solution, I would have to reimplement
scroll, buttons etc...
I thought of cutting my speech bubble in 7 parts Fig.2 and scaling it according to the text size. But I have the feeling that this is very heavy for not much.
Finally I wonder if a hybrid solution would not be the best, use the
UI for scrolling, get transforms and inject them into Shape sprites
(outside the Canvas).
If it is possible to do 1. and then I would be very grateful for an example.
If not 2. 3. 4. seem feasible, I would like to have your opinion on the most relevant of the 3.
Thanks in advance.
There is a simpler and quite elegant solution to your problem that uses nothing but the sprite itself (or rather the design of the sprite).
Take a look at 9-slicing Sprites from the official unity documentation.
With the Sprite Editor you can create borders around the "core" of your speech bubble. Since these speech bubbles are usually colored in a single color and contain nothing else, the ImageType: Sliced would be the perfect solution for what you have in mind. I've created a small Example Sprite to explain in more detail how to approach this:
The sprite itself is 512 pixels wide and 512 pixels high. Each of the cubes missing from the edges is 8x8 pixels, so the top, bottom, and left borders are 3x8=24 pixels deep. The right side has an extra 16 pixels of space to represent a small "tail" on the bubble (bottom right corner). So, we have 4 borders: top=24, bottom=24, left=24 and right=40 pixels. After importing such a sprite, we just have to set its MeshType to FullRect, click Apply and set the 4 borders using the Sprite Editor (don't forget to Apply them too). The last thing to do is to use the sprite in an Image Component on the Canvas and set the ImageType of this Component to Sliced. Now you can scale/warp the Image as much as you like - the border will always keep its original size without deforming. And since your bubble has a solid "core", the Sliced option will stretch this core unnoticed.
Edit: When scaling the Image you must use its Width and Height instead of the (1,1,1)-based Scale, because the Scale might still distort your Image. Also, here is another screenshot showing the results in different sizes.
I am wanting to layer multiple image overlays over each other on a map (minimum will be 3, maximum 10). These are weather radar images with each image being a higher elevation in the storm. I have attached a screenshot. Just imagine several layers over that image in the same spot.
I am hoping there's an easy image style like "HeightAboveGround" or something like that that will let me do this.
If this is possible, is there also a way to tilt/rotate image overlays? This would be such a nice feature in the map.
I am wanting to layer multiple image overlays over each other...with each image being a higher elevation in the storm.
There is no way to render image overlays at any height other than zero.
If this is possible, is there also a way to tilt/rotate image overlays?
If by "tilt", you mean, along an axis parallel to the ground, such that one end of the image is higher than the other, then there's no way to do that.
If, by "rotate" you mean, along an axis perpendicular to the ground, so that the image no longer aligns with north, then there's no way to do that either.
Sorry this couldn't be more helpful. :) If 3D is important to your application, you might want to consider using a true 3D library (as opposed to 2.5D) such as Cesium.
I am trying to get 'picking' working in a 3D scene, where the view is rotated such that the iPhone is being held in a landscape mode. I'm using OpenGL ES 2.0 (so all shaders, no fixed-function pipeline).
I'm performing the unproject from within the rendering code and immediately drawing the resulting ray using GL_LINES (ray only gets calculated the 1st time that I touch the screen, so afterwards I can move the camera around to observe the resulting line from various angles).
My unproject code/call is fine (lots of examples of gluUnproject online). My matrix-inversion code is fine (even compared with excel for a few matrices). However, the resulting ray is off by at least 5-15 degrees from where I actually 'clicked' (in the Simulator it really is a click, so I'm expecting a lot more precision from the unproject).
My view is rotated to landscape (after I create the perspective-projection matrix, I rotate it around the Z by -90 degrees; the aspect ratio remains at a portrait one). I believe that the problem with the math being off lies here.
Does anyone have any experience doing picking/unprojection with specifically a landscape view?
Is it possible you simply have the field of view off? Assuming you've stuck to something a lot like the traditional pipeline, if you were inverting your modelview matrix then using generic unproject code (ie, code that assumes a 90 degree field of view in both directions to fill eye space) then that would explain it.
A quick diagnostic test is to see how far off it is for different touches. Touches nearer the centre of projection should be closer to the correct answer.
On a screen with square pixels like the iPhone, the aspect ratio is just the proportion of the horizontal field compared to the vertical. So if you want to be unscientific about it, find the field of view you're using, say f, and try multiplying your results by 90/f or f/90. If that doesn't work, try also throwing a factor of 480/320 or 320/480 in there.
A better solution is to follow your code through and figure out what your actual horizontal and vertical fields of view are. And multiply your results by that over 90.
I am adding some functionality to an iPhone app, and could use some help in picking the fastest / most efficient / best practice approach for solving this problem:
At the upper-half of my screen, I have speech bubbles (think comic book) that are UIImageViews translating across the screen (dynamic x & y position). It is a UIImageView because there is an image as the background of the speech bubble.
Each speech bubble has a matching image moving around the bottom of the screen (elsewhere in the layer tree)
I would like to draw a tail (that triangle bit from a speech bubble) so the point of the triangle is tracking the lower image, with the base of the triangle being attached to the bottom of the upper UIImageView. (technically the base doesn't have to be butted against, it can overlap as long as I can match the color of my background image to the triangle).
I have already done all the tracking & drawn a line with CGContextStrokePath methods, and now I am stuck on how to replace the line with a triangle.
I have looked at drawing a triangle in Quartz and filling it. My concern is the speech bubbles are repositioned every 1/10th of a second, and it looks like drawing just the line used for proof of concept had a pretty severe performance / visual smoothness impact.
One idea I have is to do the trigonometry myself, and stretch & rotate an image of a triangle to connect each of these speech bubbles with the lower spot. Something is telling me there is a more efficient / more elegant solution, but I am not able to see it looking through the documentation. Any help on how you have or would approach this issue is appreciated. Thanks.
If the speech bubbles are fixed in size, just use a static UIImage. Set the image view's layer.position property at the point of the triangle. Then you can use view animation to move the bubbles around.
If you need the speech bubbles to be different sizes, I'd create a resizeable image using resizableImageWithCapInsets. Then I'd do the same as above to position it.
If there was something special about the speech bubble that I could't achieve with either a static image or a resizable image, I'd probably create a custom CA Layer or layers to get the effect I wanted (Like a gradient layer with a shape layer as it's mask layer)
I'm writing a small program to convert a standard definition 4:3 video to a hi-def video 16:9 and I'm experiencing a serious stretching effect, as expected I suppose (though I didn't think about it until my code started working). Anyhow, the only way I can think of getting around this stretching effect and still fill the whole 16:9 screen is to cut off the top and bottom of the image.
1) So my question is, when converting from SD to HD, do I have to lose image parts of the image in order to fill the whole screen without any stretching effects?
2) Same question for converting from HD to SD.
I'm new image processing, are there any popular approaches to reducing the stretching in these kinds of operations? Is there a smarter approach to this problem than just cutting off parts of the image or introducing black bars to the image?
Thanks in advance for all your help!
Other than the obvious methods of cropping, letterboxing, and pillarboxing, which either lose image data or necessitate potentially-undesirable black bars, there is also adaptive image resizing. Basically, the intent of these techniques is to be able to create a version of an image with an arbitrary aspect ratio, without losing the essential characteristics of the image or distorting it. One technique is called seam carving, and can be seen here.
If you'd like to test the technique out on some images of your own, the functionality is included in recent versions of ImageMagick, as explained here.
Reduction of quality or loss of content is always a problem in resizing images or video. Generally you scale the image in one direction, and either trim or pad the other direction.
On TV it is common to cut off the left and ride side of a 16:9 frame to put is on 4:3 screen, and to add black side bars to go from 4:3 to 16:9. TV editors don't cut off the top and bottom of a 4:3 frame to fit it on 16:9 because there's almost always important parts of the scene there. The far left and far right of a 16:9 frame don't usually have important elements, although in some cinematic scenes losing the sides makes a huge difference.