Detecting vertical planes is possible now with ios 11.3 and apple arkit 1.5 (a good example: ARKit Vertical Plane Detection ).
But there is one condition; you need to have some color differences or structure on your wall in order to get detected.
Is it also possible to detect blank walls or walls that have 1 color?
There is a natural tension here that imposes some inherent design constraints.
For ARKit to even “see” a surface for purposes of world tracking — before even detecting it as a plane — the surface needs to have some texture. Variation in color, relief, points of high contrast, something that causes it to have some visual features.
That’s okay for a lot of horizontal plane detection use cases, since people like to buy tables made of wood, install floors made of tile, take countertops for granite, etc. But a lot of walls in home and office environments are lightly textured or featureless. You probably can’t get your customers to change their walls. (If you do, though, I can refer the guy who did great textured paint on my house...)
So instead you need to think about how this fits into your AR experience at a basic design level...
For horizontal planes, you could make experiences where a small stretch of floor/table near the viewer becomes the play field for a game or whatever, but you can’t just flip that on its side for a vertical plane experience.
Vertical planes detect better at larger distances — you can find a wall when you see its edges, or the furniture backed against it, etc.
Use estimated plane hit tests to place content on a wall, and refine your placement when plane detection kicks in later.
Don’t use vertical planes the same way you would horizontal planes. They can be boundaries or background scenery instead of the focus of an experience.
Related
I am currently working on a 3d game in Unity, and I am working on the level design using ProBuilder. I basically created a huge cube which I "flipped normal", and a second one way smaller
which, as the other one, I flipped normal, but as soon as I crossed it while turning backward I could weirdly see through the small cube, which makes it feel unfinished. How could I fix this issue?
Normally in 3d games triangles can only be seen from one side. Usually this is not a problem because walls have some thickness. Since you clearly have paper thin walls made of only one layer of triangles this is exactly the result you should expect.
In short add the other side to your walls. The simples way is to duplicate the existing triangles and flipping them. Walls will still be paper thin of course. Later you should probably make them thicker.
Via ARKit, I want to place indoor map on floor.
Currently I tried 2 things:
I've placed large Plane below camera and above floor, But it causes quite drift. Does not move well when we walk, and overall experience is not overwhelming.
Saw a solution where you can identify horizontal plane, but it has its own issues.
So is it really possible with good results?
Devices with LiDAR
The LiDAR scanner has its advantages and disadvantages. The main advantage of LiDAR is its ability to almost instantly reconstruct floor and walls, then you can easily attach any 3D model to the resulted surface – a model will be stable, it will not drift, so a user's AR experience will be overwhelming, as you said. Also, an important advantage of LiDAR is the excellent performance in environment with poor lighting and with poor textures.
Here you can read about Occlusion feature and some of the LiDAR peculiarities. Good news: LiDAR perfectly works in conjunction with the Plane Detection option.
ARKit subdivides the reconstructed scene into ARMeshAnchors which give you access to polygonal geometry and surface classification.
ARMeshAnchor().geometry.classification
ARMeshAnchor().geometry.faces
ARMeshAnchor().geometry.vertices
ARMeshAnchor().transform.columns.3
Devices without LiDAR
In the absence of a LiDAR scanner, we can only detect horizontal and vertical surfaces using the Plane Detection feature. I can say that all AR frameworks (including ARKit and RealityKit) are much better and faster in defining horizontal surfaces, as opposed to vertical ones.
However, Detected Planes are less stable compared to Reconstructed Surfaces, and therefore, a slight drifting is possible sometimes. To successfully complete the Plane Detection stage, you need a well-lit room and good-for-tracking surrounding objects' textures.
ARKit calls your delegate's renderer(_:didAdd:for:) with a ARPlaneAnchor for each unique vertical and/or horizontal surface. And each plane anchor provides details about the surface – its world position, dimensions and real-world surfaces' classification.
In addition to the above, the delegate method called renderer(_:didUpdate:for:) is required to merge multiple coplanar Detected Planes into bigger resulting Detected Plane (a surface of a floor, for example).
ARPlaneAnchor().classification
ARPlaneAnchor().extent
ARPlaneAnchor().alignment
ARPlaneAnchor().center
Is it really possible with good results?
Yes, in both cases, it's possible to attach a map without drifting – whether you're using Plane Detection or Scene Reconstruction.
I built an apk using the HelloAR scene (which is provided with ARcore package). The app is only detecting Horizontal surface like table and creates it's own semi-transparent plane over it. When I moved my phone around a bottle, the app again, only created a horizontal plane cutting through the bottle. I expected ARCore to create planes along the bottle as I move my phone around, like polygons in a mesh.
Another scenario is, I placed 2 books on the floor, and each of them have different thickness. But the HelloAR app creates only one semi-transparent horizontal surface over the thicker book, instead of creating two surfaces (one for each book).
What is going wrong here? How can I fix it and make the HelloAR app work more precisely? Please help.
Software: Unity v2018.2,
ARcore v1.11.0
ARCore generates an approximate point cloud using a soft movement of the device to identify the featured points, this points are detected by contrast in the different shapes, if you use your application in test mode in unity you can see how the points are placed in your empty scene.
Once the program has enough points at the "same height" (I don't know the exact precision), it generates the plane that you can see, but it won't detect planes separated by a difference of 5cm or even more distance.
If you want to know the approximate accuracy of the app, test it with unity and make a script to capture the generated points that have been used to generate the planes, then check the Y difference to see which is the tolerance distance.
Okay so Vuforia is currently one of the leading SDKs for augmented reality providing a wide area of detection options (Images, Ground, Point, 3D objects, ...)
So regarding your question about detecting a bottle I would most certainly use the 3D model detection feature. You can read the official docs here.
You need to first generate an approximate of the object in a 3d modeling software and the use their program to generate the detection model. Then you put this in Unity and setup the detection. (no coding needed)
I have some experience with this kind of detection. I used it to detect a large 2mx2m scale model of an electric vehicle. It works great, you can walk around it and it tracks it through and through. You can see a short official demo here
Hope it helped to explain this in short!
I'd like to create a game that has levels such as this: http://img169.imageshack.us/img169/7294/picdq.png
The Player moves "flies" through the level and mustn't collide with the walls. How can I create such levels?
I found that piece of software: http://www.sapusmedia.com/levelsvg/
It's not that cheap, so I wonder whether there is another way to create such a level as shown in the picture above...?
You can do that pretty easy by reading the color value of pixels at specific places of the level. Take for instance that your level background is white and the walls are black. In order to perform collision detection, whether your character had hit the wall, you would do the following:
-take your character's position
-look at the color values of the pixels of your map that overlap with character's bounding box or sphere at that position
-if any of those contain black color you have yourself a collision :)
Now if your level is all colourful, you would want to build a black and white mask texture that would reflect the wall surfaces of your actual map. Then use the coloured map for drawing and the bw map for collision detection.
I'd spend a good solid couple weeks getting caught up on Objective-C, Xcode, Interface Builder, and Apple iOS documentation. There are many good tutorials out there and sample Xcode projects to download and run on the iPhone/iPad simulator.
If just starting out, some of those quick startup libraries can rob you of the intimate knowledge you'll need to create the intricacies and nuances you'll need when your application starts to reach outside the boundaries of the code sandbox. Not bad to use as learning tools or to speed up development time, but I'd advise against using them as a crutch until you strengthen your developer legs. Crawl. Walk. Run!
I'm attempting to build a Lunar Lander style game on the iPhone. I've got Cocos2D and I'm going to use Box2D. I'm wondering what the best way is to build the floor for the game. I need to be able to create both the visual aspect of the floor and the data for the physics engine.
Oh, did I mention I'm terrible at graphics editing?
I haven't used Box2D before (but I have used other 2D physics engines), so I can give you a general answer but not a Box2D-specific answer. You can easily just use a single static (stationary) Box if you want a flat plane as the floor. If you want a more complicated lunar surface (lots of craters, the sea of tranquility, whatever), you can construct it by creating a variety of different physics objects - boxes will almost always do the trick. You just want to make sure that all your boxes are static. If you do that, they won't move at all (which you don't want, of course) and they can overlap without and problems (to simulate a single surface).
Making an image to match your collision data is also easy. Effectively what you need to do is just draw a single image that more or less matches where you placed boxes. Leave any spots that don't have boxes transparent in your image. Then draw it at the bottom of the screen. No problem.
The method I ended up going with (you can see from my other questions) is to dynamically create the floor at runtime and then draw it to the screen.