I would like to use three kinects v2 running on three computers and then gather their data in one computer (real time 3d reconstruction using unity3d). Is it possible to do so and how ? Thank you.
So what you're asking is very do-able, it just takes a lot of work.
For reference I'm referring to the frames of the 3D point cloud gathered by the kinect as your image.
All you need is to set up a program on each of your kinect-computers that runs them as a client. With the other computer you can run that as a server and have the clients sending packets of images with some other data attached.
The data you'll need at a minimum will be angle and position from 'origin'.
For this to work properly you need to be able to reference the data in all your kinects to each other. The easiest way to do this is to have a known point and measure the distance from that point and angle the kinects are facing vs North and sea level.
Once you have all that data you can take each image from each computer and rotate the bit clouds using trigonometry, then combine all the data. Combing the data is something you'll have to play with as there are loads of different ways to do it and it will depend on your application.
Related
I am building a game on Unity with two Azure Kinects. How do I calibrate them to have get the positional data of body so solve occlusion?
Currently I get two bodies for each person. How can I map the 2 virtual bodies (from each camera) to each individual person?
Your idea is great as Multiple-camera setups offer a solution to increase the coverage of the captured human body and to minimize the occlusions.
Please go through the document: Benefits of using multiple Azure Kinect DK devices to read more on Fill in occlusions. Although the Azure Kinect DK data transformations produce a single image, the two cameras (depth and RGB) are actually a small distance apart. The offset makes occlusions possible. Use the Kinect SDK to capture the depth data from both devices and store it in separate matrices. Align the two matrices using a 3D registration algorithm. This will help you to map the data from one device to the other, taking into account the relative position and orientation of each device.
Please refer to this article published by: Nadav Eichler
Spatio-Temporal Calibration of Multiple Kinect Cameras Using 3D Human Pose
Quoted:
When using multiple cameras, two main requirements must be fulfilled
in order to fuse the data across cameras:
Camera Synchronization (alignment between the cameras’ clocks).
Multi-Camera Calibration (calculating the mapping between cameras’
coordinate systems).
I am working on taking two HoloLens 2 users' gaze data, and comparing them to verify they are tracking the same hologram's trajectory. I have access to the GazeProvider data, no issues there. However, the GazeProvider.GazeDirection data throws me. For instance, I've referenced the API at:
GazeDirection API Data
But, I dont really understand what the Vector 3 it returns means. Are the X,Y,Z relative motion? If not, can I use Vector3.angle to compute relative motion vectors between two points?
The vector returned by the GazeDirection property leveraging three coordinate parameters to point the direction that the user's eyes are looking towards. The origin is located between the user's eyes. The Vector3.angle method you mentioned can help you compute the angle between the two eye gaze directions.
I have just started to dig into gaze from a different scenario, but one suggestion I would make is that you also take a look at the gaze origin api.
Each user occupies a different location in space and is gazing into the world in a "gaze direction" from their location in space which would be their "gaze origin".
Basically you need to reconcile the different spatial coordinate systems.
I'm a complete illiterate when it comes to working with geographical data, so bear with me.
For our application we will be tracking a fairly large amount of rapidly changing points on a map. It would be nice to be able to cache the location of these points in some kind of map-tile structure so it would be easy to find all points currently in the same tile or neighbouring tiles, making it easier to quickly determine the nearest neigbours and have special logic for specific tiles, etc.
Although we're working for one specific (but already large) location, it would be nice if a solution would scale to other locations as well. Since we would only cache tiles that concern the system, would just tiling the enitre planet be the best option? The dimensions of a tile would then be measured in arc seconds/minutes, or is that a bad idea?
We already work with Postgres and this seems like something that could be done with PostGIS (is this what rasters are?), but jumping in to the documentation/tutorials without knowing what exactly I'm looking for is proving difficult. Any ideas?
PostGIS is all that you need. It can store your points in any coordinate reference system, but you'll probably be using longitude/latitude. Are your points coming from a GPS device?
PostGIS uses GIST indexing, making the search for points close to a given point quite efficient. One option you might want to look at, seeing that you are interested in tiling, is to "geohash" your points. Basically, this turns an (X,Y) coordinate pair into a single "string" with a length depending on the level of partitioning. Nearby points will have the same geohash value (= 1 tile) and are then easily identified with standard database search tools. See this answer and related question for some more considerations and an example in PostgreSQL.
You do not want to look at rasters. These are gridded data, think aerial photography or satellite images, weather maps, etc.
But if you want a more specific answer you should give some more details:
How many points? How are they collected?
Do you have large clusters?
Local? Regional? Global?
What other data does this relate to?
Pseudo table structure? Data layout?
etc
More info = better answer
Cheers, hope you get your face back
I am doing research on touch screens and I couldnot find a good source except for this image below which could explain how multitouch IR systems work. basically the single touch IR systems are pretty simple as on two sides of the panel, lets say left and top are the IR transmitters and on the right and bottom are the receivers. So if a user touches somewhere in the middle, the path of IR will be disrupted and the ray will not reach the receiving end, therefore the processor can pick up the coordinates. but this will not work for multitouch systems as there is an issue of ghost points with this approach.
Below I have an image of 'PQ labs' multitouch IR system working, but as there is no explanation given, therefore I am not able to understand its working, Any help will be greatly appreciated.
I consider that they have a special algorithm to avoid the point caused by the inner cross of emitter light. But this algorithm will not work for every time, so sometime if you put your finger very close to each other. The ghost point may will show up.
My guess:
The sensors are analog (there must be an Analog to digital converter to read each of the opto transistor (IR receiver).
LEDa and LEDb are not on at the same time
The opto transistor are running in a linear range (not in saturation) when no object is present.
One object:
4. When an One object is placed on the surface. There will be less light accessing some of the opto transistors. This will be reflected by a reading that is lower then the read when no object is present.
The reading of the photo transistor array (it is an array reflecting the read from each opto transistor) will provide information about:
4.1. How many opto transistors are completely shaded (off)
4.2. What opto transistor are effected
Please note: A reading from one LED is not sufficient to know the object position
To get object location we need two reading (one from LEDa and one from LEDb). Now, we can calculate the object Position and Size since we know the geometry of the screen.
Two Objects:
Now each array may have "holes" (there will be two groups) in the shaded area. These holes will indicate that there is an additional object.
If the objects are closed to each other the holes may not be seen. However, there are many LEDs. So there will be multiple arrays (one for each LED) and based on the presented geometry these holes may be seen by some of the LEDs.
For more information please see US patent#: US7932899
Charles Bibas
I have already asked this question on scientific computing and wondered if this forum could offer alternatives.
I need to simulate the movement of a large number of agents undergoing soft body deformation. The processes that govern the agents' movement are complex and so the entire process requires parallelisation.
The simulation needs to be visualised in 3D. As I will be running this simulation across many different nodes (MPI or even MPI+GPGPU) I do not want the visualisation to run in real time, rather the simulation should output a video file after it is finished.
(i'm not look for awesome AAA video game quality graphics, in addition the movement code will take up enough CPU time so I don't want to further slow the application down by adding heavy weight rendering code)
I think there are three ways of producing such a video:
Write raw pixel information to BMPs and stitch them together - I have done this in 2D but I don't know how this would work in 3D.....
Use an offline analogue of OpenGL/Direct3D, rendering to a buffer instead of the screen.
Write some sort of telemetry data to a file, indicating each agents' position, deformation etc for each time interval and then after the simulation has finished use it as input to a OpenGL/Direct3D program.
This problem MUST have been solved before - there's plenty of visualisation in HPC
In summary: How does one easily render to a video in an offline manner (very basic graphics not toy story - I just need 3D blobs) without impacting performance in a big way?
My idea would be to store the different states/positions of the vertices as single frames of a vertex animation in a suitable file format. A suitable format would be COLLADA, which is a intermediate format for 3D scenes based on XML, thus it can be easily parsed and written with general purpose XML libraries. There are also special purpose libraries for COLLADA like COLLADA DOM and pycollada. The COLLADA file containing the vertex animation could then be rendered directly to a video file, with the rendering software of your choice (3D Studio Max, Blender, Maya ...)