Facebook released a demo video of their Surround 360 technology a few days ago, called "Here and Now": https://www.facebook.com/facebook/videos/10154659446236729/
Apparently they are using their proposed cubic mapping perspective for this. Can someone familiar verify that?
When on my Gear VR and I rotate my head, I notice a slight quality improvement. So also, does anybody know if they are using an adaptive view-aware streaming such as DASH or something for that (which will be impressive)? I am assuming it is not first downloaded and played, so maybe this is not due to the rendering.
Facebook uses pyramid encoding. They put a sphere inside a pyramid so that the base of the pyramid is the full-resolution FOV and the sides of the pyramid gradually decrease in quality until they reach a point directly opposite from the viewport, behind the viewer. That explains why, when you turned your head with the GearVR on, you noticed a quality change. They don't use MPEG-DASH, yet.
https://code.facebook.com/posts/1126354007399553/next-generation-video-encoding-techniques-for-360-video-and-vr/
Related
I want to make an ocean simulation that is physically accurate.
The height and speed of the waves should be controlled by the keyboard at runtime.
In the ocean, there needs to be a boat that either moves along a path or is controlled by the keyboard.
So far I have made this simulation in Blender:
https://youtu.be/LJ6ncxv-k7w
The problems are as follows:
1. There is no collision with the ocean
2. There are no controllers for the boat's movement
3. I am able to control the waves, but not at runtime
I thought about switching to Unity because the user interface is obviously better, as it is a game engine. I do not want to use Blender's game engine as its future is uncertain at this point.
After reviewing the various Unity water simulation plugins, I came to these conclusions:
1. the buoyancy is great in most of them, such as in Aquas and SUIMONO
2. None of them seems to offer a physically realistic collision with the boat.
3. they do offer wave height control, but not much else as far as wave properties go.
4. Some of the plugins can be combined to get closer to satisfactory results.
My question is:
Should I go with Unity completely?
It seems perfect for my user control needs, but the plugins are lacking in the collision aspect. I came across this video, but no tutorial: https://www.youtube.com/watch?v=T0D_vrYm4FQ
Even if there was one, how could I combine it with the plugins?
Is there a way to build the scene in Blender and then import it into Unity?
Would I be able to control the waves and boat after importing them?
Thank you very much for your time and knowledge.
if you really means an ocean, i suggest you to check out NVIDIA WaveWorks. it's a C library and doesn't have an officially integration with Unity3D, but since you go this far for it, i guess maybe you'll have enough courage to trying make it into a useable plugin yourself.
I'm working on a project for an exhibition where an AR scene is supposed to be layered on top of a 3D printed object. Visitors will be given a device with the application pre-installed. Various objects should be seen around / on top of the exhibit, so the precision of tracking is quite important.
We're using Unity to render the scene, this is not something that can be changed as we're already well into development. However, we're somewhat flexible on the technology we use to recognize the 3D object to position the AR camera.
So far we've been using Vuforia. The 3D target feature didn't scan our object very well, so we're resorting to printing 2D markers and placing them on the table that the exhibit sits on. The tracking is precise enough, the downside is that the scene disappears whenever the marker is lost, e.g. when the user tries to get a closer look at something.
Now we've recently gotten our hands on a Lenovo Phab 2 pro and are trying to figure out if Tango can improve on this solution. If I understand correctly, the advantage of Tango is that we can use its internal sensors and motion tracking to estimate its trajectory, so even when the marker is lost it will continue to render the scene very accurately, and then do some drift correction once the marker is reacquired. Unfortunately, I can't find any tutorials on how to localize the marker in the first place.
Has anyone used Tango for 3D marker tracking before? I had a look at the Area Learning example included in the Unity plugin, by letting it scan our exhibit and table in a mostly featureless room. It does recognize the object in the correct orientation even when it is moved to a different location, however the scene it always off by a few centimeters, which is not precise enough for our purposes. There is also a 2D marker detection API for Tango, but it looks like it only works with QR codes or AR tags (like this one), not arbitrary images like Vuforia.
Is what we're trying to achieve possible with Tango? Thanks in advance for any suggestions.
Option A) Sticking with Vuforia.
As Hristo points out, Your marker loss problem should be fixable with Extended Tracking. This sounds definitely worth testing.
Option B) Tango
Tango doesn't natively support other markers than the ARTags and QRCodes.
It also doesn't support the Area Learnt scene moving (much). If your 3DPrinted objects stayed stationary you could scan an ADF and should have good quality tracking. If all the objects stay still you should have a little but not too much drift.
However, if you are moving those 3D Printed objects, it will definitely throw that tracking off. So moving objects shouldn't be part of the scanned scene.
You could make an ADF Scan without the 3D objects present to track the users position, and then track the 3D printed objects with ARMarkers using Tangos ARMarker detection. (unsure - is that what you tried already?) . If that approach doesn't work, I think your only Tango option is to add more features/lighting etc.. to the space to make the tracking more solid.
Overall, Natural Feature tracking by Vuforia (or Marker tracking for robustness) sounds more suited to what I think your project is doing, as users will mostly be looking at the ARTag/NFT objects. However, if it's robustness is not up to scratch, Tango could provide a similar solution.
I am experimenting with overlaying augmented reality objects over a pass-through image from the rear camera in Unity.
Has anyone experimented with overlaying objects with accurate tracking? I've tweaked the movement scale to get somewhat decent results but rotation is still not accurate and drift is a big issue.
I've had good luck with the augmented reality sample that ships with the latest tango. in my experience it does work the way you speculated where if you add items to the unity scene they are synced to motion detected by the device.
I believe the tracking and syncing function have improved since you asked this question originally because I've noticed an improvement since I got my tango devkit a month or so ago. there was an update a week or so later, with an immediate improvement.
I have found that some scenes track better than others, it seems to help for there to be additional scenery for it to track. in my workspace, a fairly cluttered apartment, it tracks well but in the neighboring identical apartment unit which is currently vacant and empty, it does not track as well. that could also be a product of the blinds hanging up in my unit that are not hanging up in the vacant unit, filtering out additional infrared.
I'm experimenting with placing 3D objects over the real time input from the Tango color camera.
One problem here is that the hardware color camera 'point' in a (strange) direction. I wasn't able to get the direction vector from the api until now. Your virtual camera for rendering the scene needs this rotation to render 3D objects properly.
There are augmented reality examples of Tango's Unity plugin:
https://developers.google.com/tango/apis/unity/unity-simple-ar
They solve this problem with a matrix that rotates the 3d camera.
It can be found in the Unity script "TangoARPoseController" (C#) that, when attached to a unity camera, rotates it so that it looks at the scene in the right direction. The matrix is obtained in the method "SetCameraExtrinsics" of that script.
Unfortunately, when I apply the matrix to my unity scene it does not produce a perfect overlay (actually it's quiet bad). But I have other sources of position input which may be the problem here.
However, until now I'm not sure if the matrix used in the examples is good enough for accurate ar overlays. Maybe it is just suitable for demonstration purposes. But it should be a good starting point for further investigation.
Are we talking about displaying the 'webcam' in the background as opposed to a skybox ?
Take a look at my GhostHunter repo. It includes a shader and a script for displaying the rear facing camera 'behind' the gameplay objects (like the skybox). It should be useable with Tango and it is better than the 'display on a mesh' technique I`ve seen others used.
https://github.com/NVentimiglia/Augmented-Reality-Ghost-Hunter
I am learning cocos2d for iPhone and came across Munch Time (see also the company website page), notably game of the week this week :). I am an indie developer, so won't be able to achieve the same great quality, but would like to understand how they managed to do the light effect of the moon and start that can be viewed in this video.
What do you think? I am planning to stick with cocos2d 1.x as it is compatible with a wider range of iPhones rather than cocos2d 2.x unless using the latter is the only way to get those effects (or unless you reckon that achieving the quality of Munch time is not possible with Cocos2d).
Thanks for reading..
You can use the stable release on Cocos2d which is 1.0.1 as of this post.
The lighting effects you are referring to are likely done with the images. The moon is solid, but has alpha fade on the 'light' around it, meaning it starts mostly white near the edge of the moon, but fades in transparency as it gets away from the moon edge to give the glow effect.
The clouds also look to be transparent somewhat, to allow the moonlight to slightly seep through.
However, all the animations, effects, etc, should all be reproducible in Cocos2d.
Is there a decent eye tracking package to replace the mouse for code editing?
I want to free up the mouse, but keep using my keyboard for editing code.
Having done some research on it, I concluded that proper eye tracking hardware is expensive. Using a webcam or high resolution video camera seems to be the most viable option.
Unfortunately, image-based tracking (as opposed to infra-red tracking) restricts the accuracy, and so not all features might be practical.
Desired eye-tracking IDE features:
Page scrolling
Tab selection
Setting cursor position
Selecting gaze-focused text with keyboard
A similar question recommends Opengazer for webcams, but I am particularly interested in speeding up basic text-editing. Any recommendations are appreciated, especially if you have experience with eye tracking and practical use cases.
The kind of accuracy you're looking for is pretty difficult to achieve (Since text tends to be pretty small).
IR tracking is actually pretty easy to accomplish. A few IR LEDs and an IR camera (which is really just a normal camera with different filters) and your pupil lights up (This can be done with under $100, more if you want a better camera though).
It's the head tracking that might be more of an issue.
You end up with quite a few degrees of freedoms that you need to track and your inaccuracies will just build up.
I'm pretty sure there is no out-of-the-box solution for problem, but on eyewriter.org there are really nice instructions how to build your own eye-tracker. It's accurate enought to let someone "draw" graphities using only his eyes. so it should be possible to convert the eye-movements into mouse-events.
It can be done reasonably accurately (al la this article on how people read code) but i've never seen a commercial product that does what you're asking for
Maybe take a look at Emotiv's headsets, they use thought patterns to perform tasks. They're designed for games but you can probably repurpose it for normal tasks
Re text cursor placement, Lightning (While I have not worked on this particular feature, I have previously contributed to the Text 2.0 project as a student) which is described in this paper:
Universal eye-tracking based text cursor warping
will place the text cursor at the most salient target in the neighborhood of the gaze position reported by the eye tracker.
However, you need a Tobii eye tracker that supports the TET API. You might want to contact Tobii to verify that the Tobii X2-30 eye tracker which costs < $10k is compatible.
Just use vim. Do more with the keyboard, less with the mouse.
Personally I had an issue always having to reach for a normal mouse, looked at various option eyetracking/voice/touchscreen and ended up changing the keyboard to a IBM Trackpoint end result being my hands never leave the Keyboard and my typing speed and accuracy improved due to not having to reposition my right hand.
Eye Tribe has a $99 consumer-level eye tracker that is available now.
“Using a webcam or high resolution video camera seems to be the most
viable option.”
Eye Tribe is a spinoff of Gaze Group, a research group located at the IT University of Copenaghen. The people of Gaze Group developed the open-source ITU GazeTracker software, which allows people to turn low-cost webcams into eye trackers.
http://www.gazegroup.org/downloads
Upon looking at the “downloads” section for Gaze Group, it seems that there are already some eye tracking applications to do some basic actions.
melhosseiny mentioned the Text 2.0 framework for creating eye tracking apps using HTML, CSS and JavaScript, and the Universal eye-tracking based text cursor warping feature for placing the text cursor at the most salient target.
Eye Tribe has its own SDK, but those things above could help if they work with Eye Tribe.