I am working on a project where I need to take high quality photos( ReplayKit quality is not enough) and combine them with positions from ARKit frame. I need to take about 10 photos with positions and each of those photos should be in the same coordinate space.
Since it's impossible to use ARKit and AVCatpureSession simultaneously, I'm thinking about getting position from ArKit, pausing AR, taking a photo via new AVCaptureSession, and playing back again the AR session.
The question though is if it's possible to resume AR session, without having too big drift to origin point of first session ?
It would be great to confirm, before implementing this experiment.
Thanks !
I think this is generally difficult (some say you can, some say you can't).
In theory though if you have a camera on a tripod and can keep it really still then you could save the position and rotation of the camera when you stop the session, then create a new session and load these parameters to use as offset.
I.e. in the second session your position is:
new_position = position + old_position
This will obviously only work if you can really minimise any movement between sessions (like with a tripod and remote trigger).
Related
I created an Object to Play Animation through the HelloAR Example of ARCore. Then he covered Camera with his hand and caused a tracking loss.
And if you shine the space again, the object you create will return, but the Animation will start from the beginning.
If space is recognized again after the tracking loss occurs, sometimes the object is returned but not returned. Is there a way of distinguishing?
If you recognize space again after a tracking loss occurs, why does Animation start all over again when the object returns? Are you deleting and recreating the object?
ARCore uses a techniques called Visual Inertial Odometry. It is a hybrid techniques which combine computer vision and sensor fusion.
So what VIO does is it combines data extracted from feature points(corners, blobs, edges, etc) with data acquired from mobile device IMU unit. It is crucial in ARCore you know the position of your device. Because you estimate every trackable position based on this information(triangulation using device pose).
Also another aspect is ARCore builds a sparse map of the environment while you move in the room. So those extracted feature points are stored in the memory based on a confidence level and used later to localize device.
At last, what happens when tracking is lost is you can not extract feature points due to a while wall for example. When you can not extract feature points you can not localize the device. Therefore, device does not know where it is in this Sparse map i mentioned above. Sometimes you recover because you go back to the places which are already scanned and kept in this Sparse map.
Now for your questions:
If you anchor your objects. Your objects will return but there can be drifts because ARCore can accumulate errors during this process especially if you move during device tracking is lost. So probably they return but they are not at the same physical position anymore because of the drifts.
As in animation restarting since those anchors can not be tracked they deactivated. Also since you anchor your objects they are child of the anchor so your objects are deactivated as well. That is why your animation restart.
You can test both issues using instant preview and see what happens to anchors when you lose tracking. Good luck!
We have the new Vive Focus headset, which has markerless inside out tracking. By default these headsets can only do relative tracking from their initial position. But this isn't totally accurate, you get some drift, and the position in the virtual world and the real world go out of sync. For the y position, this can mean ending up at the wrong height in the virtual world as well. Also, you don't know the user's absolute position, which you would need for a mulitplayer game for instance, so your players don't run into each other.
Now the ZED camera from Stereolabs has an option to use a reference frame (which I assume is a pointmap), which it will then use to do absolute positional tracking by calculating your position relative to the reference frame, instead of to the last frame (which I assume normal markerless inside out tracking does). Of course the ZED code is in a dll, so my question is, how difficult is it to code this system using a reference frame for the Vive Focus or another markerless inside out tracked headset. Preferably in C#, preferably using the Unity plugin, but any example would help.
And what I'm wondering about this reference frame system is would one reference frame be enough? The ZED documentation says you need to look at approximately the same scene as you were when you first made the reference frame. This makes sense, otherwise how would the system find its reference. But doesn't that mean you would need more references, for the other sides of your room as well? The ZED documentation also says that using the reference frame setting can cause jumps in VR, when syncing to the reference. Would this be a big problem? Because if it would jump all the time, that would only increase motion sickness, which is a big enough problem as it is in VR. And finally, would it require a lot of processing power to track using a reference frame? Because we're dealing with standalone headsets here powered by mobile processors, they have a hard enough time of it as it is.
Or would it be feasible to make something using markers and maybe Vuforia and do absolute positional tracking that way?
Thanks for any input!
I'm using the motion tracking but I would like to "pause" it. The room where I use the app is small, so I would like to pause/stop motion tracking then move (physically) into the center of the room and then restart motion tracking.
I tried to return if the app is paused into "OnTangoPoseAvailable" but the origin of the scene is still at the position where I launched the app.
Is it possible to redifine the origine of the scene ?
(I don't need a code, just the function to use or an idea)
Thanks
I have not seen anything in the API that allows you to redefine the origin. However, I have had such a need. Here's what I did:
Capture a few points to define the users origin. If you don't care about the rotation you could use just one point to redefine the origin.
Use those points to translate and rotate pose data to the new coordinate system
The biggest problem being that it depends on the user creating their own origin. Which might be simple for you to visualize but not the end user.
I'm currently developing an iPhone game where the player needs to tilt the device to do something. The game is somewhat of a memory game with the four corners of the screen being possible targets. The object of the game is to remember the order and then move the device to the right place.
My question is more about the design of the moving mechanic. The two options that I thought of were to get the values from the accelerometer directly and when the are greater than a specific value return whether that was the correct place to go (ie the right corner for the given instruction). My second idea is that each corner would have its own CGrect and the accelerometer would move an other CGrect and when the two intersect it would return whether the move was right or wrong.
In your opinion which one would be best? I think that the accelerometer data would be quicker but it might be affected by sudden movements while the other way might be slower but more accurate. Let me know what you think.
I think you should try both, test each of them on players, and see which works better. Drive game design decisions from user testing whenever possible.
My speculation is that you are going to need to damp or accumulate the accelerometer data somehow, since it is noisy; and if you are integrating that data into a moving average, then you should show where that moving average is with eg an onscreen sprite.
You probably don't even need to use the CGRect's intersection -- if you're just trying to determine whether <x0,y0> is within r units of <x1,y1> then you can do it via a simple Pythagorean distance. But the important thing is that if there is some internal state in the algorithm calculating what the accelerometer data has integrated to, then you need to show that state onscreen to feel responsive.
I'm currently using box2d with cocos2d on iPhone. I have quite a complex scene set up, and I want the end user to be able to record it as video as part of the app. I have implemented a recorder using the AVAssetWriter etc. and have managed to get it recording frames grabbed from OpenGL pixel data.
However, this video recording seems to a) slow down the app a bit, but more importantly b) only record a few frames per second at best.
This led me to the idea of rendering a Box2D scene, manually firing ticks and grabbing an image every tick. However, dt could be an issue here.
Just wondering if anyone has already done this, or if anyone has any better ideas?
A good solution I guess would be to use a screen recorder solution like ScreenFlow or similar...
I think your box2d is a good idea... however, you would want to used a fixed-time step. if you use dt the steps in the physics simulation will be to big, and box2d will be unstable and jittery.
http://gafferongames.com/game-physics/fix-your-timestep/
The frame rate will take a hit, but you'll get every frame. I don't think you'll be able to record every frame and still maintain a steady frame rate - that seems to be asking a lot of the hardware.