I'd like the Hololens to take in through the camera and project an image over tracked images and I can't seem to find a concrete way as to how online. I'd like to avoid using Vuforia etc for this.
I'm currently using AR Foundation Tracked Image manager (https://docs.unity3d.com/Packages/com.unity.xr.arfoundation#2.1/manual/tracked-image-manager.html) in order to achieve the same functionality on mobile, however it doesn't seem to work very well on hololens.
Any help would be very appreciated, thanks!
AR Foundation is a Unity tool and 2D Image tracking feature of AR Foundation is not supported on HoloLens platforms for now, you can refer to this link to learn more about feature support per platform:Platform Support
Currently, Microsoft does not provide an official library to support image tracking for Hololens. But that sort of thing is possible with OpenCV, you can implement it yourself, or refer to some third-party libraries.
Besides, if you are using HoloLens2 and making the QR code as a tracking image is an acceptable option in your project, recommand using Microsoft.MixedReality.QR to detect an QR code in the environment and get the coordinate system for the QR code, more information please see: QR code tracking
Related
How does one go about creating a Face Swap mechanism in Flutter?
Can anyone point me in the right direction?
Thank you
You’ll probably need a good plugin to do all the hard work for you. I recommend Google’s ML Kit on Flutter, as it is the most popular way to run on-device ML with Flutter.
The face detection plugin is what you want. You would basically get the face oval shape with face contour detection and swap those shapes. And this can be done real-time with a given video input.
But you should keep in mind that the plugin is on v0.0.1. If you’re aiming for production, you’d better do that with Swift or Kotlin.
There are multiple ways to archive this thing in Flutter. It might be in real-time or with some delay of seconds.
You can use one of these packages.
Open CV
TensorFlow
Google's ML Kit
It might be possible you will not get good support from openCV and TensorFlow in a flutter. But you can integrate the OpenCV/TensorFlow native libs or SDK for both Android and IOS and invoke them through platform channels
There is also one more possible solution but it will definitely have a delay. For this kind of ML project python have great support of library and projects.
You can set up a python project which is responsible for face swapping it takes input from the flutter app (using rest API or socket) and return the output image after face-swapping.
Some great face swap projects are available on GitHub you can look into it.
I wanted to know if anybody managed to use ARFoundation with Vuforia.
The reason we want to do this is that we find the Vuforia ground plane not flexible and we found out the ground plane is less accurate than the native capabilities but we still want to keep the image tracking (without the extended tracking because since 7.5 we really don't like the fact that targets are extended by default).
If you want to use Vuforia and AR foundation in the same project , The answer is yes it is possible.
Only Thing we need to make sure is please disable the "Include AR core Library" settings under Vuforia configuration.
i have to build an app like this:
https://www.youtube.com/watch?v=vetDCkbQGM4
It should simply detect the cockpit of a car and should show informations. For example "this is air conditioning", "this is switch button for the radio". The targets will be pre defined. Basically the app should detect everything and should show information.
Can I realize this with Vuforia? Which framework is suitable for this task?
I hope you guys can help me.
Cheers!
Since your targets are pre-defined, the simplest solution would be to use aruco markers to get 3D world positions/rotations through your user's camera feed.
See the AR Marker Detector in the Unity Asset Store for an example. Vuforia uses 'VuMarks' that are more intricate versions of this.
If you can't add computer-readable labels to the real world for your project, then you are talking about real-time object recognition. That is a much harder problem and not yet easily solvable in Unity as far as I know. It would require something like Google's Cloud Vision API. There is a Unity Cloud Vision project on GitHub, but I have no idea how well it works or what it's capabilities are.
Yes it is possible, you were first require to google. There are different SDK/Framework and Unity Asset store packages available.
You can use Free Vuforia AR Starter Kit from asset store to up and run your logic. Or You can also use Free AR Toolkit. There are different kind of tut available which can show you how to implement these pacakges.
I want to build a cross-platform mobile app that can identify QR-codes and will render a 3d model on it using AR.
I found that Unity in combination with Vuforia will do the trick on the AR part, but is it possible here to download and use 3D models dynamically?
Thanks
I guess what you're looking for is called AssetBundle be aware that downloading a large model (+texture) at run time can be heavy and will highly depend on the internet connection of the device.
Hope this helps.
I'm already make an augmented reality app that can read images marker, but I wonder if i can make augmented reality without marker in unity.
Can anyone tell me how?
Maybe What you need is SLAM, I think. Simultaneous Localization And Mapping, and is markerless, just recognite and track the environment.
These are 2 videos about slam:
https://www.youtube.com/watch?v=HbaEw5-YvA0
https://www.youtube.com/watch?v=_YLzcWX-gWU
One is from kudan, one is from wikitude. If this feature is what you want. Then I am sure that what you need is SLAM.
You can get more from:
wikitude slam
kudan
And, vuforia is not adviced. In fact, Smart Terrain does recoginate the environment, it still need a marker.
You can use Google ARCore, Vuforia or 8thWall SDKs. All of them have motion tracking or extended tracking. So you do not have to use markers. You can take a look at ARCore HelloAR example:
https://developers.google.com/ar/develop/unity/tutorials/hello-ar-sample
you can using ARUnity. ARUnity is the Unity plugin for ARToolKit. (Well the marker would be some kind of image which is used for tracking. ARToolKit calls it NFT.)
You can download it here:
http://www.artoolkit.org/download-artoolkit-sdk
(Scroll down for the ARUnity download link)
Documentation is available here:
http://www.artoolkit.org/documentation/doku.php?id=6_Unity:unity_getting_started
Best is, it is free and open-source :).