For my project i have to capture the legs of the body tracking, while a person is sitting. Especially the knees and feet. I’ve made it so far to draw my choosen joints in C#.
BUT if I put the Kinect under a table and the person is sitting in front of the table, the choosen joints start to flickering. I found out that the Kinect always tries to capture the whole body of the person. But the table prevents the Kinect of capturing the torso and the rest of the body. So my choosen joints for the legs can not be tracked correct.
My question is, how can I modify the BodyFrameReader to read or catupture only the legs, instead of the whole body? I am not talking about to draw my choosen joints. The BodyFrameReader of the BodyTracker shall only capture my joints. I use C#.
Can someone help and give me a code example? Thanks a lot
Here my code snippet
kinectSensor = KinectSensor.GetDefault();
bodyFrameReader = kinectSensor.BodyFrameSource.OpenReader();
kinectSensor.Open();
if (bodyFrameReader != null) {
bodyFrameReader.FrameArrived += Reader_FrameArrived;
private void Reader_FrameArrived(object sender, BodyFrameArrivedEventArgs e) {
bool dataReceived = false;
using (BodyFrame bodyFrame = e.FrameReference.AcquireFrame()) {
if (bodyFrame != null) {
if (bodies == null) {
bodies = new Body[bodyFrame.BodyCount];
}
Related
Even the official documentation has borderline insane recommendations to solve what is probably one of the most common UI/3D interaction issues:
If I click while the cursor is over a UI button, both the button (via the graphics raycaster) and the 3D world (via the physics raycaster) will receive the event.
The official manual:
https://docs.unity3d.com/Packages/com.unity.inputsystem#1.2/manual/UISupport.html#handling-ambiguities-for-pointer-type-input essentially says "how about you design your game so you don't need 3D and UI at the same time?".
I cannot believe this is not a solved problem. But everything I've tried failed. EventSystem.current.currentSelectedGameObject is sticky, not hover. PointerData is protected and thus not accessible (and one guy offered a workaround via deriving your own class from Standalone Input Module to get around that, but that workaround apparently doesn't work anymore). The old IsPointerOverGameObject throws a warning if you query it in the callback and is always true if you query it in Update().
That's all just mental. Please someone tell me there's a simple, obvious solution to this common, trivial problem that I'm just missing. The graphics raycaster certainly stores somewhere if it's over a UI element, right? Please?
I've looked into this a fair bit and in the end, the easiest solution seems to be to do what the manual says and put it in the Update function.
bool pointerOverUI = false;
void Update()
{
pointerOverUI = EventSystem.current.IsPointerOverGameObject();
}
Your frustration is well founded: there are NO examples of making UI work with NewInput which I've found. I can share a more robust version of the Raycaster workaround, from Youtube:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.EventSystems;
using UnityEngine.InputSystem;
using UnityEngine.UI;
/* Danndx 2021 (youtube.com/danndx)
From video: youtu.be/7h1cnGggY2M
thanks - delete me! :) */
public class SCR_UiInteraction : MonoBehaviour
{
public GameObject ui_canvas;
GraphicRaycaster ui_raycaster;
PointerEventData click_data;
List<RaycastResult> click_results;
void Start()
{
ui_raycaster = ui_canvas.GetComponent<GraphicRaycaster>();
click_data = new PointerEventData(EventSystem.current);
click_results = new List<RaycastResult>();
}
void Update()
{
// use isPressed if you wish to ray cast every frame:
//if(Mouse.current.leftButton.isPressed)
// use wasReleasedThisFrame if you wish to ray cast just once per click:
if(Mouse.current.leftButton.wasReleasedThisFrame)
{
GetUiElementsClicked();
}
}
void GetUiElementsClicked()
{
/** Get all the UI elements clicked, using the current mouse position and raycasting. **/
click_data.position = Mouse.current.position.ReadValue();
click_results.Clear();
ui_raycaster.Raycast(click_data, click_results);
foreach(RaycastResult result in click_results)
{
GameObject ui_element = result.gameObject;
Debug.Log(ui_element.name);
}
}
}
So, just drop into my "Menusscript.cs"?
But as a pattern, this is terrible for separating UI concerns. I'm currently rewiring EVERY separately-concerned PointerEventData click I had already working, and my question is, Why? I can't even find how it's supposed to work: to your point there IS no official guide at all around clicking UI, and it does NOT just drop-on-top.
Anyway, I haven't found anything yet which makes new input work easily on UI, and definitely not found how I'm going to sensibly separate Menuclicks from Activityclicks while keeping game & ui assemblies separate.
Good luck to us all.
Unity documentation for this issue with regard to Unity.InputSystem can be found at https://docs.unity3d.com/Packages/com.unity.inputsystem#1.3/manual/UISupport.html#handling-ambiguities-for-pointer-type-input.
IsPointerOverGameObject() can always return true if the extent of your canvas covers the camera's entire field of view.
For clarity, here is the solution which I found worked best (accumulated from several other posts across the web).
Attach this script to your UI Canvas object:
public class CanvasHitDetector : MonoBehaviour {
private GraphicRaycaster _graphicRaycaster;
private void Start()
{
// This instance is needed to compare between UI interactions and
// game interactions with the mouse.
_graphicRaycaster = GetComponent<GraphicRaycaster>();
}
public bool IsPointerOverUI()
{
// Obtain the current mouse position.
var mousePosition = Mouse.current.position.ReadValue();
// Create a pointer event data structure with the current mouse position.
var pointerEventData = new PointerEventData(EventSystem.current);
pointerEventData.position = mousePosition;
// Use the GraphicRaycaster instance to determine how many UI items
// the pointer event hits. If this value is greater-than zero, skip
// further processing.
var results = new List<RaycastResult>();
_graphicRaycaster.Raycast(pointerEventData, results);
return results.Count > 0;
}
}
In class containing the method which is handling the mouse clicks, obtain the reference to the Canvas UI either using GameObject.Find() or a public exposed variable, and call IsPointerOverUI() to filter clicks when over UI.
Reply to #Milad Qasemi's answer
From the docs you have attached in your answer, I have tried the following to check if the user clicked on a UI element or not.
// gets called in the Update method
if(Input.GetMouseButton(0) {
int layerMask = 1 << 5;
// raycast in the UI layer
RaycastHit2D hit = Physics2D.Raycast(Camera.main.ScreenToWorldPoint(Input.mousePosition), Vector2.zero, Mathf.Infinity, layerMask);
// if the ray hit any UI element, return
// don't handle player movement
if (hit.collider) { return; }
Debug.Log("Touched not on UI");
playerController.HandlePlayerMovement(x);
}
The raycast doesn't seem to detect collisions on UI elements. Below is a picture of the Graphics Raycaster component of the Canvas:
Reply to #Lowelltech
Your solution worked for me except that instead of Mouse I used Touchscreen
// Obtain the current touch position.
var pointerPosition = Touchscreen.current.position.ReadValue();
An InputSytem is a way to receive new inputs provided by Unity. You can't use existing scripts there, and you'll run into problems like the original questioner. Answers with code like "if(Input.GetMouseButton(0)" are invalid because they use the old system.
I am struggling to find a good tutorial or informations that would allow me to select multiple objects in 3D in a user friendly manner.
So far, the best tutorial I found is this one : https://sharpcoderblog.com/blog/unity-3d-rts-style-unit-selection. The tutorial works by using the transform.position of the selectable objects and checking if it within the user's selection.
What I wish is to have the user be able to select a unit even if it is only partially within the user's selection such as most RTS games do ( both in 2D and 3D ).
One possibility would be to create a temporary mesh using the camera's clipping distances and the user's selection and then check for collisions but I was not able to find any tutorials using this method nor do I know if it is the best approach to the subject.
If I understand correctly you want to
somehow start a selection
collect every object that was "hit" during the collection
somehow end the collection
Couldn't you simply use raycasting? I will assume simple mouse input for now but you could basically port this to whatever input you have.
// Just a little helper class for an event in the Inspector you can add listeners to
[SerializeField]
public class SelectionEvent : UnityEvent<HashSet<GameObject>> { }
public class SelectionController : MonoBehaviour
{
// Adjust via the Inspector and select layers that shall be selectable
[SerializeField] private LayerMask includeLayers;
// Add your custom callbacks here either via code or the Inspector
public SelectionEvent OnSelectionChanged;
// Collects the current selection
private HashSet<GameObject> selection = new HashSet<GameObject>();
// Stores the current Coroutine controlling he selection process
private Coroutine selectionRoutine;
// If possible already reference via the Inspector
[SerializeField] private Camera _mainCamera;
// Otherwise get it once on runtime
private void Awake ()
{
if(!_mainCamera) _mainCamera = Camera.main;
}
// Depending on how exactly you want to start and stop the selection
private void Update()
{
if(Input.GetMouseButtonDown(0))
{
StartSelection();
}
if(Input.GetMouseButtonUp(0))
{
EndSelection();
}
}
public void StartSelection()
{
// if there is already a selection running you don't wanr to start another one
if(selectionRoutine != null) return;
selectionRoutine = StartCoroutine (SelectionRoutine ());
}
public void EndSelection()
{
// If there is no selection running then you can't end one
if(selectionRoutine == null) return;
StopCoroutine (selectionRoutine);
selectionRoutine = null;
// Inform all listeners about the new selection
OnSelectionChanged.Invoke(new HashSet<GameObject>(selection);
}
private IEnumerator SelectionRoutine()
{
// Start with an empty selection
selection.Clear();
// This is ok in a Coroutine as long as you yield somewhere within it
while(true)
{
// Get the ray shooting forward from the camera at the mouse position
// for other inputs simply replace this according to your needs
var ray = _mainCamera.ScreenPointToRay(Input.mousePosition);
// Check if you hit any object
if(Physics.Raycast(ray, out var hit, layerMask = includeLayers ))
{
// If so Add it once to your selection
if(!selection.Contains(hit.gameObject)) selection.Add(hit.gameObject);
}
// IMPORTANT: Tells Unity to "pause" here, render this frame
// and continue from here in the next frame
// (without this your app would freeze in an endless loop!)
yield return null;
}
}
}
Ofcourse you could do it directly in Update in this example but I wanted to provide it in a way where you can easily exchange the input method according to your needs ;)
From UX side you additionally might want to call a second event like OnSelectionPreviewUpdate or something like this every time you add a new object to the selection in order to be able to e.g. visualize the selection outcome.
I might have understood this wrong and it sounds like you rather wanted to get everything inside of a drawn shape.
This is slightly more complex but here would be my idea for that:
Have a dummy selection Rigidbody object that by default is disabled and does nothing
don't even have a renderer on it but a mesh filter and mesh collider
while you "draw" create a mesh based on the input
then use Rigidbody.SweepTestAll in order to check if you hit anything with it
Typed on smartphone but I hope the idea gets clear
I think I would try to create a PolygonCollider2D because it is quite simple comparing to creating a mesh. You can set its path (outline) by giving it 2D points like location of your pointer/mouse. Use the SetPath method for it. You can then use one of its methods to check if another point in space overlaps with that collider shape.
While the PolygonCollider2D interacts with 2D components you can still use its Collider2D.OverlapPoint method to check positions/bounds of your 3D objects after translating it to 2D space.
You can also use its CreateMesh method to create a mesh for drawing your selection area on the screen.
You can read more about the PolygonCollider2D here.
Hope it makes sens and hope it helps.
I am trying to build a application which includes using leap motion interaction engine to move the object in unity 3d. However, i also need to find out which fingers are touching the object when interacting with object in unity. Is there anyway i can do that? Thanks in advance!!
Strictly speaking, the Grasping logic of the Interaction Engine has to check this very thing in order to initiate or release grasps, but it doesn't have a friendly API for accessing this information.
A more convenient way to express this, even though it's not the most efficient way, would be to detect when a hand is intersecting with the interaction object and checking the distance between each fingertip and the object.
All InteractionControllers that are intersecting with a given InteractionBehaviour can be accessed via its contactingControllers property; using the Query library included in Leap tools for Unity, you can convert a bunch of Interaction Controller references to a bunch of Leap Hands without too much effort, and then perform the check:
using Leap.Unity;
using Leap.Unity.Interaction;
using Leap.Unity.Query;
using UnityEngine;
public class QueryFingertips : MonoBehaviour {
public InteractionBehaviour intObj;
private Collider[] _collidersBuffer = new Collider[16];
private float _fingertipRadius = 0.01f; // 1 cm
void FixedUpdate() {
foreach (var contactingHand in intObj.contactingControllers
.Query()
.Select(controller => controller.intHand)
.Where(intHand => intHand != null)
.Select(intHand => intHand.leapHand)) {
foreach (var finger in contactingHand.Fingers) {
var fingertipPosition = finger.TipPosition.ToVector3();
// If the distance from the fingertip and the object is less
// than the 'fingertip radius', the fingertip is touching the object.
if (intObj.GetHoverDistance(fingertipPosition) < _fingertipRadius) {
Debug.Log("Found collision for fingertip: " + finger.Type);
}
}
}
}
}
I've created an arm with a custom pivot in Unity which is essentially supposed to point wherever the mouse is pointing, regardless of the orientation of the player. Now, this arm looks weird when pointed to the side opposite the one it was drawn at, so I use SpriteRenderer.flipY = true to flip the sprite and make it look normal. I also have a weapon at the end of the arm, which is mostly fine as well. Now the problem is that I have a "FirePoint" at the end of the barrel of the weapon, and when the sprite gets flipped the position of it doesn't change, which affects particles and shooting position. Essentially, all that has to happen is that the Y position of the FirePoint needs to become negative, but Unity seems to think that I want the position change to be global, whereas I just want it to be local so that it can work with whatever rotation the arm is at. I've attempted this:
if (rotZ > 40 || rotZ < -40) {
rend.flipY = true;
firePoint.position = new Vector3(firePoint.position.x, firePoint.position.y * -1, firePoint.position.z);
} else {
rend.flipY = false;
firePoint.position = new Vector3(firePoint.position.x, firePoint.position.y * -1, firePoint.position.z);
}
But this works on a global basis rather than the local one that I need. Any help would be much appreciated, and I hope that I've provided enough information for this to reach a conclusive result. Please notify me should you need anything more. Thank you in advance, and have a nice day!
You can use RotateAround() to get desired behaviour instead of flipping stuff around. Here is sample code:
public class ExampleClass : MonoBehaviour
{
public Transform pivotTransform; // need to assign in inspector
void Update()
{
transform.RotateAround(pivotTransform.position, Vector3.up, 20 * Time.deltaTime);
}
}
I have set up Unity navigation meshes (four planes), navigation agent (sphere) and set up automatic and manual off mesh links. It should now jump between meshes. It does jump between meshes, but it does that in straight lines.
In other words, when agent comes to an edge, instead of actually jumping up (like off mesh link is drawn) it just moves straight in line but a bit faster. I tried moving one plane higher than others, but sphere still was jumping in straight line.
Is it supposed to be like this? Is it possible to set up navigation to jump by some curve? Or should I try to implement that myself?
I came by this question, and had to dig through the Unity sample. I just hope to make it easier for people by extracting the important bits.
To apply your own animation/transition across a navmesh link, you need to tell Unity that you will handle all offmesh link traversal, then add code that regularly checks to see if the agent is on an offmesh link. Finally, when the transition is complete, you need to tell Unity you've moved the agent, and resume normal navmesh behaviour.
The way you handle link logic is up to you. You can just go in a straight line, have a spinning wormhole, whatever. For jump, unity traverses the link using animation progress as the lerp argument, this works pretty nicely. (if you're doing looping or more complex animations, this doesn't work so well)
The important unity bits are:
_navAgent.autoTraverseOffMeshLink = false; //in Start()
_navAgent.currentOffMeshLinkData; //the link data - this contains start and end points, etc
_navAgent.CompleteOffMeshLink(); //Tell unity we have traversed the link (do this when you've moved the transform to the end point)
_navAgent.Resume(); //Resume normal navmesh behaviour
Now a simple jump sample...
using UnityEngine;
[RequireComponent(typeof(NavMeshAgent))]
public class NavMeshAnimator : MonoBehaviour
{
private NavMeshAgent _navAgent;
private bool _traversingLink;
private OffMeshLinkData _currLink;
void Start()
{
// Cache the nav agent and tell unity we will handle link traversal
_navAgent = GetComponent<NavMeshAgent>();
_navAgent.autoTraverseOffMeshLink = false;
}
void Update()
{
//don't do anything if the navagent is disabled
if (!_navAgent.enabled) return;
if (_navAgent.isOnOffMeshLink)
{
if (!_traversingLink)
{
//This is done only once. The animation's progress will determine link traversal.
animation.CrossFade("Jump", 0.1f, PlayMode.StopAll);
//cache current link
_currLink = _navAgent.currentOffMeshLinkData;
//start traversing
_traversingLink = true;
}
//lerp from link start to link end in time to animation
var tlerp = animation["Jump"].normalizedTime;
//straight line from startlink to endlink
var newPos = Vector3.Lerp(_currLink.startPos, _currLink.endPos, tlerp);
//add the 'hop'
newPos.y += 2f * Mathf.Sin(Mathf.PI * tlerp);
//Update transform position
transform.position = newPos;
// when the animation is stopped, we've reached the other side. Don't use looping animations with this control setup
if (!animation.isPlaying)
{
//make sure the player is right on the end link
transform.position = _currLink.endPos;
//internal logic reset
_traversingLink = false;
//Tell unity we have traversed the link
_navAgent.CompleteOffMeshLink();
//Resume normal navmesh behaviour
_navAgent.Resume();
}
}
else
{
//...update walk/idle animations appropriately ...etc
Its recommended to solve your problems via animation. Just create a Jump animation for your object, and play it at the correct time.
The position is relative, so if you increase the Y-position in your animation it will look like the object is jumping.
This is also how the Unity sample is working, with the soldiers running around.
Not sure what version of unity you are using but you could also try this, I know it works just fine in 4:
string linkType = GetComponent<NavMeshAgent>().currentOffMeshLinkData.linkType.ToString();
if(linkType == "LinkTypeJumpAcross"){
Debug.Log ("Yeah im in the jump already ;)");
}
also just some extra bumf for you, its best to use a proxy and follow the a navAgent game object:
Something like:
AIMan = this.transform.position;
AI_Proxy.transform.position = AIMan;
And also be sure to use:
AI_Proxy.animation["ProxyJump"].blendMode = AnimationBlendMode.Additive;
If you are using the in built unity animation!
K, that's my good deed for this week.
Fix position in update()
if (agent.isOnOffMeshLink)
{
transform.position = new Vector3(transform.position.x, 0f, transform.position.z);
}