I am trying to use a KLT tracker for human tracking in a CCTV footage. The people are very close to the CCTV. I noticed that some time people change the orientation of the heads and also the frame rate is slightly slow. I have read from Rodrigues et al. paper Section 3.4 that the:
"This simple procedure (KLT tracking procedure) is extremely robust and can establish matches between head detections where the head HAS NOT BEEN continuously detected continuously detected due to pose variation or partial occlusions due to other members of the crowd".
Paper can be found in this link : Rodriguez et al.
1). I understood that the KLT tracker is robust to pose variations and occlusions. Am I right?
I was trying to track one single person in footage till now by using the MATLAB KLT as in :
MATLAB KLT
However, the points were not being found after JUST 3 frames.
2). Can someone explain why this is happening or else a better solution to this. Maybe using a particle/Kalman filter should be better?
I do not recommend using a KLT tracker for close CCTV cameras due to the following reasons:
1. CCTV frame rate is typically low, so people change their appearance significantly between frames
2. Since the camera is close to the people, they also change their appearance over time due to perspective effects (e.g. face can be seen when person is far from camera, but as he/she gets closer, only the top of the head is seen).
3. Due to closeness, people also significantly change scale and aspect ratio, which is a challenge for some head detectors.
KLT only works well when the neighborhood of the pixel, including both foreground and background, remains similar. The above properties make this less likely for most pixels. I can only recommend KLT as an additional motion based hint for tracking, as a vector of field of part motions.
Most single person trackers do not adapt well to scale change. I suggest you start with some state of the art tracker, like Struck (C++ code by Sam Hare available here), and modify the search routine to work with scale change.
KLT by itself only works for short-term tracking. The problem is that you lose points because of tracking errors, 3D rotation, occlusion, or objects leaving the field of view. For long-term tracking you need some way of replenishing the points. In the multiple face tracking example the new points are acquired by periodically re-detecting the faces.
Your particular case sounds a little strange. You should not be losing all the points after just 3 frames. If this happens than either the object is moving too fast, or your frame rate is too low.
Related
I have a problem on some Android devices with the particle system's rendering (weirdly enough the problem seems occur on devices with higher capabilities). The problem occurs when mesh based particles (Renderer/Renderer Mode/Mesh) are being rendered. It seems like the meshes that are being spewed out and slowly shrunk with time are being reduced ("reverse-tessellated") which results in a nasty visual effect. Does anyone know what might be the cause of this?
UPDATE: One thing that I've noticed is that with time - the longer the gameplay - this problem is getting worse.
UPDATE: What I've tried is to make one particle system bigger (around x5 times) in order to check if it will have any effect on it's rasterization. Normally particles are sized down from 1 to 0 based on their life-time. What I've noticed, after sizing them up, is that the problem does not occur anymore.
UPDATE: Visualisation of the problem:
Properly rendered:
Improperly rendered:
I was able to track the issue down. It turned out to be a problem within a toon shader I wrote while a go. All the things I've noticed are valid but unfortunately it took some time to put me on the right track.
Taken from Unity's documentation
The half and fixed types only become relevant when targeting mobile
GPUs, where these types primarily exist for power (and sometimes
performance) constraints. Keep in mind that you need to test your
shaders on mobile to see whether or not you are running into
precision/numerical issues.
Unfortunately for me, as it cost me quite some time, I've used half3/4 values with POSITION semantics which caused some numerical precision issues on some Android devices (in this case the particles were getting smaller and smaller - size 0 to be exact). As a general rule, from what I've read in Unity's documentation, float3/4 should always be preferred in conjunction with POSITION semantics.
In my 3D animation in AnyLogic, if I zoom out, the quality decreases, when I zoom in, it becomes fine. Example shown in these images:
vs
Are there settings that can prevent this quality decrease from happening?
I got a reply from AnyLogic support as follows:
Right now, we are working to improve the performance of 2D and 3D
animation at run-time. In particular, in the latest version, the
level of details changes while zooming in 3D window, i.e. objects that
are further away have less detail than closer objects. On your
screenshot, 3D objects look unacceptably bad at a relatively short
distance and you can do nothing with this. However, as far as I know,
there will be an option to tune the distance where the quality
decreases in the next update.
I created an Object to Play Animation through the HelloAR Example of ARCore. Then he covered Camera with his hand and caused a tracking loss.
And if you shine the space again, the object you create will return, but the Animation will start from the beginning.
If space is recognized again after the tracking loss occurs, sometimes the object is returned but not returned. Is there a way of distinguishing?
If you recognize space again after a tracking loss occurs, why does Animation start all over again when the object returns? Are you deleting and recreating the object?
ARCore uses a techniques called Visual Inertial Odometry. It is a hybrid techniques which combine computer vision and sensor fusion.
So what VIO does is it combines data extracted from feature points(corners, blobs, edges, etc) with data acquired from mobile device IMU unit. It is crucial in ARCore you know the position of your device. Because you estimate every trackable position based on this information(triangulation using device pose).
Also another aspect is ARCore builds a sparse map of the environment while you move in the room. So those extracted feature points are stored in the memory based on a confidence level and used later to localize device.
At last, what happens when tracking is lost is you can not extract feature points due to a while wall for example. When you can not extract feature points you can not localize the device. Therefore, device does not know where it is in this Sparse map i mentioned above. Sometimes you recover because you go back to the places which are already scanned and kept in this Sparse map.
Now for your questions:
If you anchor your objects. Your objects will return but there can be drifts because ARCore can accumulate errors during this process especially if you move during device tracking is lost. So probably they return but they are not at the same physical position anymore because of the drifts.
As in animation restarting since those anchors can not be tracked they deactivated. Also since you anchor your objects they are child of the anchor so your objects are deactivated as well. That is why your animation restart.
You can test both issues using instant preview and see what happens to anchors when you lose tracking. Good luck!
Is there a decent eye tracking package to replace the mouse for code editing?
I want to free up the mouse, but keep using my keyboard for editing code.
Having done some research on it, I concluded that proper eye tracking hardware is expensive. Using a webcam or high resolution video camera seems to be the most viable option.
Unfortunately, image-based tracking (as opposed to infra-red tracking) restricts the accuracy, and so not all features might be practical.
Desired eye-tracking IDE features:
Page scrolling
Tab selection
Setting cursor position
Selecting gaze-focused text with keyboard
A similar question recommends Opengazer for webcams, but I am particularly interested in speeding up basic text-editing. Any recommendations are appreciated, especially if you have experience with eye tracking and practical use cases.
The kind of accuracy you're looking for is pretty difficult to achieve (Since text tends to be pretty small).
IR tracking is actually pretty easy to accomplish. A few IR LEDs and an IR camera (which is really just a normal camera with different filters) and your pupil lights up (This can be done with under $100, more if you want a better camera though).
It's the head tracking that might be more of an issue.
You end up with quite a few degrees of freedoms that you need to track and your inaccuracies will just build up.
I'm pretty sure there is no out-of-the-box solution for problem, but on eyewriter.org there are really nice instructions how to build your own eye-tracker. It's accurate enought to let someone "draw" graphities using only his eyes. so it should be possible to convert the eye-movements into mouse-events.
It can be done reasonably accurately (al la this article on how people read code) but i've never seen a commercial product that does what you're asking for
Maybe take a look at Emotiv's headsets, they use thought patterns to perform tasks. They're designed for games but you can probably repurpose it for normal tasks
Re text cursor placement, Lightning (While I have not worked on this particular feature, I have previously contributed to the Text 2.0 project as a student) which is described in this paper:
Universal eye-tracking based text cursor warping
will place the text cursor at the most salient target in the neighborhood of the gaze position reported by the eye tracker.
However, you need a Tobii eye tracker that supports the TET API. You might want to contact Tobii to verify that the Tobii X2-30 eye tracker which costs < $10k is compatible.
Just use vim. Do more with the keyboard, less with the mouse.
Personally I had an issue always having to reach for a normal mouse, looked at various option eyetracking/voice/touchscreen and ended up changing the keyboard to a IBM Trackpoint end result being my hands never leave the Keyboard and my typing speed and accuracy improved due to not having to reposition my right hand.
Eye Tribe has a $99 consumer-level eye tracker that is available now.
“Using a webcam or high resolution video camera seems to be the most
viable option.”
Eye Tribe is a spinoff of Gaze Group, a research group located at the IT University of Copenaghen. The people of Gaze Group developed the open-source ITU GazeTracker software, which allows people to turn low-cost webcams into eye trackers.
http://www.gazegroup.org/downloads
Upon looking at the “downloads” section for Gaze Group, it seems that there are already some eye tracking applications to do some basic actions.
melhosseiny mentioned the Text 2.0 framework for creating eye tracking apps using HTML, CSS and JavaScript, and the Universal eye-tracking based text cursor warping feature for placing the text cursor at the most salient target.
Eye Tribe has its own SDK, but those things above could help if they work with Eye Tribe.
I'm working on locating an iPhone device in 3D space.
I can use lat/long to detect physical location, I can use the magnetometer to figure out the direction they're facing, and I might be able to use the accelerometer to figure out how their device is oriented, but I can't figure out a way to get height of the device off the floor.
Specifically, I need to know if the user is squatting down, or raising their hand toward the ceiling (a different of about 2 meters/6 feet).
I posted a more detailed description of what I'm trying to do on my blog: http://pushplay.net/blog_detail.php?id=36
I would love any suggestions as to how to even fake this sort of info. I really want the sort of interactivity and movement that would require ducking and bobbing, versus just letting someone sit back and angle the phone -- kind of the way people can "cheat" playing with a Wii...
The closest I could see you getting to what you're looking for is using the accelerometer/magnetometer as an inertial tracker. You'd have to calibrate the user's initial position on startup to a "base" position, then continuously sample the sensors on a background thread to build a movement model. This post talks about boosting the default sample rate of the accelerometer functions so that you can get a pretty fine-grained picture of the user's movements.
I'm not sure this will solve your concern about people simply angling the device to produce the desired action, but you will have to strike a balance between being too strict in interpreting movements and allowing for differences in movement
The CoreLocation stuff gives you elevation aswell as lat/long, so you could potentially use that although there are some significant problems with this:
Won't work well indoors (not a problem for Sat Nav, is a problem for games)
Your users would have to "calibrate" (probably by placing the phone on the floor) each location they use!
In fact, you'd need to start keeping a list of "previously calibrated locations"... which could vary hugely just in one house (eg multiple rooms and floors). Could get in the way of the game.
Can't be used on moving transport (tranes, planes, automobiles... even walking) because elevation changing so frequently.
Therefore I'd have thought that using the accelerometer as a proxy for height is a substantially more preferable route than determining absolute elevation.
I am not intimately familiar with the iphone. But it might require a hardware add-on. (which you probably don't want to do). After thinking on this the only way I know how is through light or more specific laser. You shoot out a laser on the floor and record the time it takes to get back. It's actually not a lot to put this hardware together and I am sure the iphone has connections for peripherals. Unless osmeone can trump me, I say ther eis no way to do that with an image.