I'm very new to this. During my research for my PhD thesis I found a way to solve a problem and for that I need to move my lab testing in the virtual environment. Anyway, I have an Oculus Rift and an OPTOTRAK system that allows me to motion capture a full body for VR (in theory). What my question is, can someone point me in the right direction, of what materials do I need to check out to start working on a project. I have a background in programming, so it's just that I need a nudge in the right direction (or if someone knows a similar project)
https://www.researchgate.net/publication/301721674_Insert_Your_Own_Body_in_the_Oculus_Rift_to_Improve_Proprioception - I want to make something like this :)
Tnx a lot
Nice challenge too.. how accurate and how real time is the image of your body in the Oculus Rift world ? my two - or three - cents
A selfie-based approach would be the most comfortable to the user.. there's an external camera somewhere and the software transforms your image to reflect the correct perspective, as you would see your body, through the oculus, at any moment. This is not trivial and quite expensive vision software. To let it work 360 degrees there should be more than 1 camera, watching all individual oculus users in a room !
An indirect approach could be easier.. model your body, only show dynamics. There's WII style electronics in bracelets and on/in special user clothing, involving multiple tilt and acceleration sensors. They form a cluster of "body state" sensor information, to be accessed by the modeller in the software. No camera is needed, and the software is not that complicated when you'd use a skeleton model.
Combine. Use the camera for the rendering texture and drive the skeleton model via dynamics drive by the clothing sensors. Maybe deep learning could be applied, in conjunction with a large number of tilt sensors in the clothing, a variety of body movement patterns are to be trained and connected to the rendering in the oculus. This would need the same hardware as the previous solution, but the software could be easier and your body looks properly textured and it moves less "mechanistic". There will be some research needed to find the correct deep learning strategy..
Related
I would like to make an AR iPhone app in unity that places an object in the real world which you can then interact with it on your iPhone. like you have a bar at the bottom of your screen and you can drag the objects into the ar world and interact with them with the ability of hand tracking. This will work kind of like the meta 2 interface https://www.youtube.com/watch?v=m7ZDaiDwnxY which you can grab things and drag them. it uses hand tracking to do this.
I have done some research on this but, I need some help doing this because I don't know where to start and how to accomplish what I am trying to do.
I don't have any code.
You can email me at jaredmiller219#gmail.com for any comments and questions. also, you can email me to help me with this. thanks so much for your support!
To get started in mobile AR in Unity, I would recommend starting with Unity's resources:
https://unity.com/solutions/mobile-ar
Here's a tutorial resource for learning ARKit:
https://unity3d.com/learn/learn-arkit
As for hand tracking, obviously the Meta 2 has specialized hardware to execute its features... you shouldn't necessarily be expecting to achieve the same feature set with only a phone driving your experience. Leap Motion is the most common hand tracker I've seen integrated into VR and AR setups and it works well, but if you really need hand tracking with just a phone, you could check out ManoMotion which seeks to bring hand tracking and gesture recognition to ARKit, although I haven't personally worked with it.
i'm developing VR using google cardboard SDK..
i want to move on virtual environment when i walk on real world, like this : https://www.youtube.com/watch?v=sZG5__Z9pzs&feature=youtu.be&t=48
is it possible to make VR application like that for android...? maybe using accelerometer sensor ? how can i implement this using unity...?
i try to record accelerometer sensor while i walk with smartphone, here are the result : https://www.youtube.com/watch?v=ltPwS7-3nOI [i think the accelerometer value is so random -___- ]
Actually it is not possible with only mobile:
You're up against a fundamental limitation of the humble IMU (the primary motion sensor in a smartphone).
I won't go into detail, but basically you need an external reference frame when trying to extract positional data from acceleration data. This is the topic of a lot of research right now, and it's why VR headsets that track position like the Oculus Rift have external tracking cameras.
Unfortunately, what you're trying to do is impossible without using the camera on your phone to track visual features in the scene and use those as the external reference point, which is a hell of a task better suited to a lab full of computer vision experts.
One another possible but difficult way is:
This may be possible if you connect device to internet then watch it's position from satelite(google maps or something like that)but that is a very hard thing to do.
Apologies if this question has been asked before, and apologies too if it is obvious to those with knowledge - I'm completely tech illiterate especially when it comes to gaming etc so bear with me!
I'm wondering whether it is possible to record gameplay (any console/platform) but be able to play this back in a 360/VR format?
The usecase is this:
I want to watch and follow a game but rather than having 1st person PoV, I'd love to be able to use either a VR headset device (most ideal) or a 360 viewer (tablet or smartphone) to move perspective beyond forward facing field of vision.
Ideally the PoV would follow players (think specatator mode) and not necessarily be a static camera - although not necessarily a deal breaker.
Is this possible?
How would this be done with existing tools etc or would new tools need to be developed?
Would it be 'recorded' client side or serverside - and would this matter?
Huge thanks in advance - also very very happy to be pointed in the direction of sources of info around this subject to consume if readily available.
Thanks
S
You need to connect the gameobject(character) in your game that has the camera to your VR display (wherever you are coding the display at) and write a code that takes the image that it displaces in that camera under that gameobject and make it so it is continuously updating, making it seem like you are in the game.
look here http://docs.unity3d.com/Manual/VROverview.html
I have a couple ideas for some 3D games/apps and need to know which 3D engine to go with. I like the high-level approach of Unity and UDK. I have done some 2D games using cocos2d before, but I'm entirely new to the 3D world, which is why i think Unity or UDK are a good choice. I know about the differences in licensing, and i am more concerned with the learning curve instead of the licensing cost.
Plans:
A 3D "side scroller" that goes forwards (up) instead of to the side. Third person space ship. This would primarily be for learning. Tilt to steer, tap to jump. Very simple graphics, vertex coloring would be enough.
A 2.5D "side scroller" like the above one but with a car. This game would generate the levels randomly out of a couple prefab blocks of a certain length that fit together seamlessly.
A 3D augmented reality display for pilots with a terrain mesh loaded from DEM data. Accelerometer and GPS access required.
Other important points:
Must be able to tie in to In-App purchases.
The more community content like tutorials and forums the better.
Ability to add third party libraries like Flurry Analytics is a big plus! But i guess this isn't possible?
Which engine would you recommend for these projects, and why? Preferably, i'd like to pick one and stick with it.
You’re going to have a way, way better time developing with Unity. UDK’s got a fantastic, incredibly capable engine, but its tools don’t have the ease-of-use of Unity’s, its developer documentation leaves a lot to be desired, and the community hasn’t been using it for long enough for there to be much help to be found there. Some quick Googling suggests you can write your own Objective-C plug-ins for Unity games, so in-app purchases and third-party libraries are definitely a possibility. I think Unity’s your best bet.
What about cryengine? it free for non commercial use and also provides mono c#.
Check it out CryEngine
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Am used to developing business and reference apps. Now I have to do an iPhone game. What tips can you share to help me with:
understanding the scope of the project
defining the deliverables
specifying the game play and other parameters
estimating the development effort
testing the game
A big question, I know. Thanks!
My answer is not following the waterfall methodology response you were looking for because I think you will need to learn the skill of game programming before you can plan, design, implement and test it. Game programming is VERY different to business programming and a whole other field in and of itself.
Due to your lack of experience in programming games there are a few concepts that you will need to learn before you can program a game period, let alone one on the iPhone.
Some of these things will depend on whether you are programming a 3D game or going for the 2D platform style.
OpenGL ES
The first thing you will need to learn is the OpenGL ES programming language. This is basically a 3D API which enables you to do drawing of 3D primitives. You will still need to use this if you are coding a 2D game as it is quick due to using the GPU for acceleration.
There are some good tutorials on the Google that you should begin with.
Vector Math
If you are doing anything 3D, you will need to learn about 3D vector math, vectors are basically used for everything in games, camera look direction, position of characters, speed, collision detection, etc. 2D vectors (x,y) minus the z component are still needed for 2D games programming.
Collision Detection
How do I know when my ball hits the wall? The answer is collision detection. There are many forms of collision detection such as Sphere to Sphere, AABB, OOBB, Convex Hulls, Triangle Mesh, etc.
AI
How do I get the enemy to attack my player character? Artificial intelligence is another large field essential to give NPCs/Enemies the ability to make intelligent decisions. AI can be simple such as if else statements but usually requires Finite State Machines or Fuzzy logic to be effective.
Pathfinding
If you want to move a character from Point A to Point B while avoiding enemies and moving obstacles, you will need to use a pathfinding algorithm. A Star (A*) is one of the most popular.
Scene Graph
If you wish to have 10-20+ enemies on screen at a time, you will need to code a scene graph to manage the dynamic drawing, logic and creation and deletion of resources. If you don't know what polymorphism is you will need to know it as it is essential for your game objects to adhere to and it ties in with the scene graph.
Physics
Position, Speed, Acceleration, Gravity and Rays are all represented using vectors and you may need to brush up on your physics math in order to code any game. Start with Newton's Second law of motion F=MA (Force = Mass * Acceleration). An open source physics engine such as Bullet, ODE, Newton, Tokamak will make things easier, meaning you won't need to write these physics rules yourself.
Objective-C++
This is optional although recommended. If you don't know C++ this is essentially a mixture of C++ and Objective-C. I tend to use C++ for the core game engine and programming because of the speed of C++ and availability of third party libraries in C++.
Sound
If you need sound you can just go ahead and use the simple audio frameworks that Apple provide, however 3D positional audio is going to require something better. I would recommend learning the FMOD SDK for iPhone. As #Stowelly mentioned, FMOD requires a license for commercial distribution but there are others you can look for which are royalty free.
Use a Game Engine
There are game engines available for the iPhone at the moment which will make it much easier for you to get a game going, in your case this will be faster although you will still need to learn the concepts I mentioned above.
Here are some game engines I know of:
Unity3D
This probably the most popular one that I know of. Unity is a PC/Mac game engine that lets you write code on the Mac and compile for Windows/Linux/Mac OS X. I doubt the iPhone building is compatible directly with other platforms, I would imagine you'd be restricted to iPhone if you started a new project. This engine does however have a commercial deployment cost of $199-$399.
Cocos2D
This one is an open source 2D game engine that might be useful for a lot of games. Worth taking a look at. Hosted on Google code.
Here are some others to check out:
Ston3D for iPhone
OOlong Engine
SIO2Engine
iTGB for 2D Games
The main difference between business apps and games, especially on mobile devices, is the importance of performance. An app that puts up a form and waits for user input probably isn't doing anything in the meantime. A game loop, on the other hand is going all the time and probably doing alot. Business programmers are not used to thinking in these terms, but games draw down battery power and, believe it or not, how you implement your game will have a great effect on how quickly that battery is drawn down.
So, one question is, what is this game that you "have to" develop? If it is sudoku, no worries. If it's a real-time 3D space battle, that's another story.
If your business apps were on the iPhone, then you probably used 100% Objective-C. (If you were doing C# or Java apps on desktops, then welcome to managing your own memory.) There are those who will tell you that the runtime type management of Objective-C is too slow for complex games. People certainly make OK games using it with Cocos2D and other engines, but again it depends on the game. There are professional developers who will only work with C++ or even straight C.