When I saw the Google Cardboard for Unity, I assumed this meant that you would be able to make a Unity PC game and use your phone as a screen/controller. All I can see is it wanting me to make an android app which is all well and good, but it doesn't allow for input from the keyboard.
Is there a way to stream the Unity PC project to the device and retrieve input (i.e. Headtracking, NFC magnet)?
The problem with such a solution is latency. In VR latency is a big deal. The overall latency from input to photons reaching your eyes should be 20ms or lower. Regular games have 30-60 ms latency by themselves. Add to that the gyro latency, the phone display latency... If you want to add another 25ms or more ping to your VR experience, that's gonna be painful and may even make you sick. If you want to read more on why latency is such a big deal in VR, Michael Abrash wrote an excellend blogpost about it: post on latency
If you want to necessairly use a keyboard for navigation, consider using a bluetooth keyboard that can be used with android devices. Also keep in mind that with the current technology, especially without a dedicated headset, really dynamic vr experiences probably won't work very well and can make some people uncomfortable or sick. For a good read on designign virtual reality experiences, please refer to this guide from the Oculus Rift: http://static.oculus.com/sdk-downloads/documents/Oculus_Best_Practices_Guide.pdf
There's nothing in the Cardboard SDK for talking with a PC-hosted Unity game. You could adapt the code from the Unity Remote 4 project:
https://www.assetstore.unity3d.com/en/#!/content/18106
We are developing the app what you want except it uses GearVR instead of Cardboard. Please check the link below.
http://challengepost.com/software/airvr
Streaming from your PC to your phone's Cardboard is possible using third-party apps, such as Trinus VR (the client app on your phone) and Vireio (the streaming app on your computer). The two apps will then communicate via your home network (Wi-Fi or other) to stream the images.
Related
I need to share my desktop on Hololens2 in real time.
All the methods I have found deal with synchronizing the HoloLens2 screen to the PC side, instead of the PC side to HoloLens2. I want to try the Unity plug-in FMETP STREAM, but it doesn't seem to meet my requirements. What should I do to synchronize the PC screen to HoloLens2 in real time?
Please excuse my poor English. I would be grateful if you could answer my question.
FMETP STREAM should satisfy this use case, I find this post the developer replied, this package supports the case desktop Unity app stream the content to the VR headset. I believe it should also work for HoloLens.
Besides, the MixedReality-WebRTC release from Microsoft can help you enable real-time audio/video/data communication with a remote peer. It also meets your requirements. You can get it starts with this doc:Unity library overview
You can modify Hololens Remoting player. It streams desktop app to Hololens in real time.
https://learn.microsoft.com/en-us/windows/mixed-reality/develop/platform-capabilities-and-apis/holographic-remoting-player
I am working in a vr game for Android in unity and it has a quiet large map and many objects to render and my phone doesn't has enough power to run my game so i was thinking that is it possible to stream my game made for desktop pc to my android device inside unity without using any other external software. And also i need to use my phone's gyroscope to control the game.
What you are talking about is called remote rendering. As you said you don't want to use any other external software you will have to decide and implement everything by yourself. The trickiest part in this will be the streaming of the game view to the phone: there are implementations that send every single frame over a TCP connection. If you want to use something external you could investigate into WebRTC implementations like this.
By the way, what you are trying to achieve is already available as a product, e.g. here
For a project at my university I need to create a Unity3D application on my laptop, in which the camera is stationairy and can be controlled to rotate in any direction using the gyroscope of my Android smartphone (Nexus 5), wirelessly or through usbcable.
I've looked at the possibility of OSC or the Unity Remote 5 App, but up till now haven't found a way that works in order to obtain this result.
Any help or advice would be hugely appreciated - I don't have much experience yet with all this.
Thanks!
If i was going to do this then I would use UNET (Unitys built in multiplayer networking API) and have the rotation sync over LAN.
On the camera I would have a Network Transform and a script to control its rotation based on accelerometer input.
The version on the phone would be the authority and sync it's rotation over the network to the client on the laptop.
Pros: Wireless, fast (over wifi), very little code required to make it work, lots of documentation available online.
Cons: Relies totally on your network situation, you will have to do a lot of trial and error to get a smooth experience (not jerky) i think.
As for getting the tilt input on the device, Unity have a great tutorial here: https://unity3d.com/learn/tutorials/topics/mobile-touch/accelerometer-input
It's pretty straight forward.
Sound like a fun project, good luck!
It's possible to do this via cable, WiFi, Bluetooth 2.0 or Bluetooth 4.0 (BLE). I have implemented what you need in WiFi and Bluetooth 2.0 for my current work.
It's pretty easy to get rotation values and stream them from Android but I don't think you will need to do anything because you can just use this!
https://play.google.com/store/apps/details?id=de.lorenz_fenster.sensorstreamgps&hl=en
So the question is how do you receive the data this is sending on Unity's side. The answer is the UdpClient class.
If you need more reliability because every second student in your uni library is torrenting Mr. Robot and your getting huge lag then I can tell you how to implement the same thing in Bluetooth, which is not super trivial as .NET 2.0 (which unity uses) doesn't support bluetooth libraries, but there are solutions...
I can't figure out the wording to research the following idea.
With the Holograms app, I can set the hologram and see the other apps front window or use web browser at the same time.
How can I create an app that does not occupy the whole system but only run on the desktop with browser and others?
EDIT: I am trying to run an hologram within the shell.
https://developer.microsoft.com/en-us/windows/holographic/hololens_shell_overview
Mostly, this is for 2D apps, but the Holograms app runs 3D holograms, so is it possible to duplicate this?
3D Hololens apps do not currently support running side by side with other applications. As of the May release you are able to run multiple "flat apps" (UWP apps) side by side:
https://developer.microsoft.com/en-us/windows/holographic/release_notes_-_may_2016
Currently there has not been any announcement about running multiple 3d apps side by side. I optimistically hope that this is coming in a future OS release.
If I understand you correctly, you are interested in creating an application for the Hololens that rather than having it "appear in the world around you", would run in a Window (Desktop) that would appear in the spatial world (like Edge). If this is correct, any UWP application can run Windowed and be placed in you virtual space. You don't need a specific Hololens project for that. If however you want to pull in spatial data, you would need to wire-up the required events so that spatial data callbacks handle the incoming data. I'm not 100% sure Unity is a best fit for this problem (at least currently) since they are focusing attention on Spatial hologram applications rather than desktop UWP applications that consume spatial data.
Just to note, it is worth stating that a single 3D app (ie Unity developed) takes over the whole HoloLens device and experience. If you bloom (windows key in emulator) you can change to the UWP app view but you can't see other apps at the same time.
Is it possible to build an Iphone/Ipad app (and Android app) that can do two things: stream an interface and the respective content (particularly video) to a TV and then let me use the phone itself as a remote control for this interface?
Basically the idea is, you don’t need a smart TV anymore or some kind of set-top box or other connected device, just the smart phone which you carry around all the time anyway and which is connected to your local wireless connection. Maybe a docking station with a HDMR connection to the TV, so you are not emptying your battery.
Do you know any comparable implementation or use?
If it is theoretically possible, can you anticipate any performance problems, bottlenecks and how those could be resolved?
If this it’s not possible, which links are missing, what technology would have to be developed first?
Thank you for your thoughts on this!
Jacob
The iPhone/iPad would work for this. It allows you to output to a second screen. You can stream video, audio, whatever. A cool example I saw was using the TV as the primary display and the phone as a controller for a game.
There are two ways to do it. You can use an hdmi output or a vga output. There is also a AirPlay, which will let you do it wirelessly. You would need an AirPlay capable device (like an AppleTV) for it to work though.