For an academy project I have to find parts of a object (location / angle relative to the object ). The object is marked with a QR-code.
Currently I'm stuck on the basics. I scanned a room and load this room with "Spatial Object Mesh Observer".
But this observer gives no relevant information:
The observer does not attempt to find 3D model LODs when sending the meshes to the application.
Someone a hint where I could start?
Scanned Room with a box ( object to find ):
var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
// Loop through all known Meshes
foreach (SpatialAwarenessMeshObject meshObject in observer.Meshes.Values)
{
Mesh mesh = meshObject.Filter.mesh;
var vertices = mesh.vertices;
// mesh.vertexCount -> 15978
// mesh.vertices -> empty
// mesh.triangles -> empty
// Do something with the Mesh object
}
Edit: 29.04.2021
Unity: Unity 2019.4.21f1
MRTK: 2.6.1
Everything seems to be loaded correctly and the triangles are recognized in an external tool.
However, this information is not available in Unity.
From our understanding, you can't obtain the vertices and triangles property of Spatial Mesh at the runtime. So, we try to make some modifications based on the SpatialAwarenessMeshDemo scene (Assets/MRTK/Examples/Demos/SpatialAwareness/Scenes) to reproduce this issue. We added the following code to the ToggleObservers() method and made it to be invoked when the sphere is clicked.
var observer = CoreServices.GetSpatialAwarenessSystemDataProvider<IMixedRealitySpatialAwarenessMeshObserver>();
foreach (SpatialAwarenessMeshObject meshObject in observer.Meshes.Values)
{
Mesh mesh = meshObject.Filter.mesh;
Debug.Log(mesh.vertexCount);
Debug.Log(mesh.vertices);
Debug.Log(mesh.triangles);
}
After our test, everything works well both in the HoloLens2 Device and Unity Holographic Remoting, it always outputs the expected value. So we recommend that you check for updates in Settings to see If there is a system update available for HoloLens 2. Then follow our steps to make a simple test to see if that works. This may help you locate the problem in your project.
I moved the code to "void OnBecameVisible()", now the information are available
Related
I'm trying to load a mesh with scripts to replace another mesh. I'm not doing a Unity project but rather a visual studio project with unity references.
My code for loading the asset bundle and getting the mesh is:
var vshojoBundle = AssetBundle.LoadFromFile("vshojo");
if (vshojoBundle == null) meshNames.Add("bundle not read");
GameObject silverObject = vshojoBundle.LoadAsset("SilverAssets.obj");
MeshFilter silverNewMesh = silverObject.GetComponent<MeshFilter>();
meshNames.Add(silverNewMesh.mesh.name);
MeshNames is just a list that later gets written in a log file to help me figure out things. When trying to get the name of the mesh in the last line, I get the following error: NullReferenceExpection: System.NullReferenceException: Object reference not set to an instance of an Object. The same exception occurs if I don't use that line to write on log and skip to the mesh replacement, when I replace the mesh I get the same error.
Overview
I wanted to have a cube, that I can drag around the scene with the components Collider, Rigidbody and ObjectManipulator. In play mode everything works fine, but running it on the hololens, the cube starts flying around after dragging it a few time.
Steps to reproduce (All components are set via editor, not via code)
Create new project and set it up for AR/HoloLens
Import MRTK 2.4
Create cube with a Box Collider. Set Is Trigger = true
Add Rigidbody to cube. Set Use Gravity = false and Is Kinematic = true
Add Object Manipulator to cube. I have a method getting called after On Manipulation Ended, but don't know if thats important.
Expected behavior
The rigidbody is set to Is Kinematic = true and Use Gravity = false, so that the cube stays still/stops moving after releasing dragging it. This works while trying it inside the unity play mode. But running it on the hololens, the cube behaves like Is Kinematic = false and starts flying around after interacting with it. Sometimes after the second drag and sometimes after the third time drag.
Error
Before updating this post, I didnt noticed the development console in left corner of my hololens. At the beginng of the scene I get the message [Physics.PhysX] BV4 midphase only supported on intel platforms but at that moment everything is fine. As the cube begins to fly around I get the a NullReferenceExeption: Object reference not set to an instance of an object.
I fixed my issue. I know the approximate cause, but I do not fully understand it. The method, getting called after OnManipulationEnded caused that.
I have a list, getting filled and drained by OnTriggerEnter/-Exit (exit looks the same, except add→remove):
private void OnTriggerEnter(Collider other){
if (other.gameObject.layer != 31) return;
_objectsCollidingWith.Add(other.gameObject);}
OnManipulationEnded triggered this method:
private int GetMeshes(List<KeyValuePair<Transform, Mesh>> transMeshes){
foreach (GameObject go in _objectsCollidingWith)
{
transMeshes.Add(new KeyValuePair<Transform, Mesh>(go.transform , go.GetComponent<MeshFilter>().mesh));
}
return transMeshes.Count;}
So I got alot of nullreferences from GetMeshes, because some gameobject in the list _objectsCollidingWith were null. Thats because the mesh is getting updated every once in a while. That caused a lot of nullreferences until the cube just flew away.
I used the whole time the logging provider via the device portal and couldnt see what is causing this errors. But after running the project via holographic emulation I could see in the console where they were coming from.
How did I fixed my problem?
I found this post because I realized that my OnTriggerExit didn't get called and cased having null objects and some spatial meshes with the same name triggered OnTriggerEnter very often. Also I added this line in the foreach loop in GetMeshes because once in a while there is still a null object:
if (go == null)
continue;
PS: Please forgive the strange code formatting, somehow the editor here on so does not allow me to place the brackets somewhere else
I am using the A* graph package that I found here.
https://arongranberg.com/astar/download
it all works well in scene view and I was able to set the graph to treat walls like obstacles.
However once I start the game the canvas scales and the graph's nodes no longer align with the walls.
this is really messing up my path finding. If anyone has any ideas how to fix this that would be much appreciated. I tried parenting the graph to the canvas but it still doesn't scale.
Kind regards
For anyone struggling with this what we did is edited the script of the A* code, in the update function we just got it to rescan once. This means that once the game started and all the scaling had taken place the graph re adjusted its bounds. This is probably not the most proper way but it only took four lines and worked for us.
private bool scanAgain = true;
private void Update () {
// This class uses the [ExecuteInEditMode] attribute
// So Update is called even when not playing
// Don't do anything when not in play mode
if (!Application.isPlaying) return;
navmeshUpdates.Update();
// Execute blocking actions such as graph updates
// when not scanning
if (!isScanning) {
PerformBlockingActions();
}
// Calculates paths when not using multithreading
pathProcessor.TickNonMultithreaded();
// Return calculated paths
pathReturnQueue.ReturnPaths(true);
if (scanAgain)
{
Scan();
scanAgain = false;
}
Working in Godot 3.2, I have a scene, Player.tscn. Near the top of Player.tscn, I have "class_name Player"
Now, when instantiating the Player, I have, as far as I see it, two options:
player = Player.new()
or
player = load("res://Player.tscn").instance() as Player
Now, the first version seems best to me...but it clearly isn't. If I use .new(), it claims that it has no children, and any method calls that attempt to get to its children (.get_texture() on a Sprite, e.g.), produces things like "Attempt to call function 'get_texture' in base 'null instance' on a null instance", because apparently Player has a no children.
Of course, doing it the second way, everything works fine. But why? Why can't I just use .new() if I've registered it as a class using class_name?
I'm new to Godot myself so I may be wrong, but I think it's because the .new() keyword is a language feature for loading single classes/nodes,
whereas .instance() is more of an engine feature that takes a packaged scene which is a combination of many classes/nodes, and restores their relative position in the tree, attaches scripts and resources, etc
So if your player is a tscn scene, you'd do something like:
var player = load("res://etc/Player.tscn").instance()
get_tree().root.add_child(player)
player.global_transform = etc ...
which would load the root node of Player.tscn and all of its children (rigidbodies, collidershapes etc) into the scene tree, set their relative positions, connect the scripts and what have you
Or if your Player is a class or a GDScript, you could do
var player = Player.new()
get_tree().root.add_child(player)
player.global_transform = etc ...
Which would add the single-node root player object only into the scene tree, but not any children
Once it's in the scene tree and _ready()'s triggered though, you can instantiate and attach the child component nodes (.new() > parent.add_child() > set transform) from the script itself
I am trying to persist markers in an augmented reality game. Here is the gist of what I am doing:
I have my users recording and saving an area to an ADF. Then they drop marker’s into the scene and save out their position data in Unity World coordinates to a text file. I then restart the app, load and localize to the ADF and load the markers.
In order to get this working, I've modified the ARPoseController.cs file in the Unity demo package to use the Area Description as it's base frame. In the _UpdateTransformation method I've swapped out the frame pairs
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_START_OF_SERVICE;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
for
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_AREA_DESCRIPTION;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
I've also added some code confirming that I'm successfully localizing to the ADF, but I'm noticing that my markers position in Unity World Space do not position properly relative to real environment.
I can confirm that my markers save and load properly based on START_OF_SERVICE origin so I assume that they are properly serializing and deserializing. What could be causing this? Am I wrong in assuming this should just work by switching the base framepair to Area_Description instead of START_OF_SERVICE?
I had a similar problem getting the AR and ADF integrated, I had to modify the TangoPointCloud to check if you're using an AreaDescription in OnTangoDepthAvailable() and adjust the baseFrame target as required.
i.e.:
if (m_tangoDeltaPoseController.m_useAreaDescriptionPose)
{
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_AREA_DESCRIPTION;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
}
else
{
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_START_OF_SERVICE;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
}
That way, the geometry of the point cloud adjusts itself based on the ADF offset instead of from device start.
After that change, when I'm using the sample code for AR to drop markers, it registers the surface properly so I'm placing the markers in the correct spots and orientation. I'm still encountering some flakiness with the markers not adjusting when relocalized though, have to look into the AreaLearningInGameController for loop closure events.
Hope that helps!