User Tracking: Building game for crowded/heavy traffic area. Have trouble with USER LOST shutting down game.

[jblatta]

I am building a game for a tradeshow that will have heavy traffic.
The game is single player. When a user is playing and someone walks
in then out of view in the background the NEW USER/ USER LOST events
are fired. This seems to kill that ONISkeletonEvent.USER_TRACKING
data for the active user or make it shutter back and forth. What am I
doing wrong? Why is a random background person passing through and
firing the USER LOST event killing the data for the active user?

Also when an active user leaves and another person makes the POSE it
will work for a few secs before the USER LOST is fired for the first
user that left which brings the game it a halt.

I looked at the NISkeletons.mxml and it also has issues with a user
leaving and another joining before the first user is "LOST".

This game needs to go out end of week and I really need help finding a
solution for this. I am going to talk to work about dropping a good
size donation to the project for this help.

[AMB]

Are you developing on the Mac or PC side? One thing I've noticed for
trade show installations is placing an opaque wall behind the current
active user. Blocking out any other users from entering into the
playing field, it would make a nice addition to your physical space.
Aside from that you can update the MaxDepthValue inside the
GlobalDefaults.ini, it's measured in meters. By default it's set to
10000, so you can change that to a less amount perhaps.

On a PC it's located here: c:\Program Files\Prime Sense\Sensor\Data
\GlobalDefaults.ini
On a Mac here: /usr/etc/primesense/GlobalDefaults.ini

These are separate files as part of the OpenNI and NITE framework
installs so it should effect your game as well. On the ActionScript
side I've notice the latencies as well from time to time with the
NEW_USER and USER_LOST events, and I've still come to the conclusion
that it's coming on the OpenNI/NITE side with it being buggy from time
to time. Maybe as an alternative to listening to the NEW_USER and
USER_LOST events you could just create your own active user manager,
meaning if a user is stalled for say over 10 seconds then delay the
game. Without seeing any demos or code samples it's hard to fully see
what your doing. Any chance you can share it with us? You can send an
email to me at mosa...@gmail.com if you'd like to keep it more
private, not a problem.

Hope this helps, AMB

-------------

[jblatta]

Here is the solution I used. My game was running in an air app, when
the user was lost I restarted the entire app. I mainly reset the app
due to a memory leak in away3D. The main focus here is the user
filtering.

package
{
import adobe.utils.ProductManager;

import flash.desktop.NativeApplication;
import flash.display.MovieClip;
import flash.display.NativeWindow;
import flash.display.Sprite;
import flash.display.StageAlign;
import flash.display.StageDisplayState;
import flash.display.StageScaleMode;
import flash.events.Event;
import flash.events.TimerEvent;
import flash.system.System;
import flash.utils.Timer;

import org.as3openni.AS3OpenNI;
import org.as3openni.events.AS3OpenNIEvent;
import org.as3openni.faast.events.FAASTEvent;
import org.as3openni.nite.events.NiteGestureEvent;
import org.as3openni.objects.NiPoint2D;
import org.as3openni.objects.NiPoint3D;
import org.as3openni.objects.NiSkeleton;
import org.as3openni.openni.events.ONISkeletonEvent;
import org.as3openni.openni.events.ONIUserTrackingEvent;
import org.as3openni.util.math.NiPoint3DUtil;

public class filterTest extends Sprite
{

[SWF( frameRate="24", backgroundColor="#000000", width="1920",
height="1080")]

public var as3OpenNI:AS3OpenNI;
public var skeletonData:NiSkeleton = new NiSkeleton();

private var dupCheckArray:Array = new Array();
private var dupValues:Array = new Array();
private var lostUserTimer:Timer = new Timer(1000, 5);
private var zone:Object;
private var zoneEmpty:Boolean;
private var userListernersAdded:Boolean = false;
private var warningBox:WarningBox;


public function filterTest()
{
// Setup stage scaling
stage.scaleMode = StageScaleMode.NO_SCALE;
stage.align = StageAlign.TOP_LEFT;

this.addEventListener(Event.ADDED_TO_STAGE, initKinect);

// Setup load timer and Warning box. The warning box was just a
movieclip that overlays on top of the screen and tells the user they
have 5 secs to step back in the box before it resets.
warningBox = new WarningBox();
warningBox.visible = false;
this.addChild(warningBox);

// Lost user listener
lostUserTimer.addEventListener(TimerEvent.TIMER_COMPLETE,
userLost);
lostUserTimer.addEventListener(TimerEvent.TIMER,
function(e:TimerEvent):void{
warningBox.resetting.text = "resetting in " +
(lostUserTimer.repeatCount - lostUserTimer.currentCount);
warningBox.visible = true;
trace("resetting in " + (lostUserTimer.repeatCount -
lostUserTimer.currentCount));
});

// setup user zone the kinect will use to filter from
zone = createZone(5, 3.25, 3.75);

}

// This will shutdown and restart an air app. I had to do this
because my project was using an alpha version of away3D and I had a
memory leak so this restarted the whole thing. Only took about 5 secs
for it to restart and kinect to be ready for a user.
public function restartApp():void
{
shutdownKinect();
var mgr:ProductManager = new ProductManager("airappinstaller");
mgr.launch("-launch " +
NativeApplication.nativeApplication.applicationID + " " +
NativeApplication.nativeApplication.publisherID);
NativeApplication.nativeApplication.exit();
}


public function initKinect(e:Event=null):void
{
trace("initKinect");
as3OpenNI = new AS3OpenNI();
as3OpenNI.addEventListener(AS3OpenNIEvent.READY, onReady);
as3OpenNI.binaryPath = "bin/AS3OpenNI_Alpha_1.3.0";
as3OpenNI.debug = true;
as3OpenNI.traceLog = true;
as3OpenNI.userTracking = true;
as3OpenNI.depthMap = true;
as3OpenNI.init();
addUserTrackingListeners();
}


protected function onReady(event:AS3OpenNIEvent):void
{
trace("Ready...");
this.addEventListener(Event.ENTER_FRAME, onRender);
this.as3OpenNI.removeEventListener(AS3OpenNIEvent.READY, onReady);
}

protected function onRender(event:Event):void
{
if(as3OpenNI.isReady())
{
as3OpenNI.getDepthBuffer();
as3OpenNI.getUserTrackingBuffer();
as3OpenNI.getSkeletonsBuffer();
}
}

public function shutdownKinect():void
{
trace("SHUTDOWN KINECT");
removeEventListener(Event.ENTER_FRAME, onRender);
removeUserTrackingListeners();
as3OpenNI.bridge.exit(true);
}

private function addUserTrackingListeners():void
{

as3OpenNI.addEventListener(ONIUserTrackingEvent.USER_TRACKING_POSE_DETECTED,
userPoseDetected);

as3OpenNI.addEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_START,
userCalibrationStarted);

as3OpenNI.addEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_COMPLETE,
userCalibrationComplete);

as3OpenNI.addEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_FAILED,
userCalibrationFailed);
as3OpenNI.addEventListener(ONISkeletonEvent.USER_TRACKING,
skeleton);
}

private function removeUserTrackingListeners():void
{

as3OpenNI.removeEventListener(ONIUserTrackingEvent.USER_TRACKING_POSE_DETECTED,
userPoseDetected);

as3OpenNI.removeEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_START,
userCalibrationStarted);

as3OpenNI.removeEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_COMPLETE,
userCalibrationComplete);

as3OpenNI.removeEventListener(ONIUserTrackingEvent.USER_TRACKING_USER_CALIBRATION_FAILED,
userCalibrationFailed);
as3OpenNI.removeEventListener(ONISkeletonEvent.USER_TRACKING,
skeleton);
}

// When use is lost we restart the app
private function userLost(e:TimerEvent=null):void
{
trace("LOST USER!!!")
zoneEmpty = true;
restartApp();
}

protected function userPoseDetected(event:ONIUserTrackingEvent):void
{
trace("Pose Detected");
}

protected function
userCalibrationStarted(event:ONIUserTrackingEvent):void
{
trace("Calibration Started");
}

protected function
userCalibrationComplete(event:ONIUserTrackingEvent=null):void
{
trace("Calibration Complete");
}

protected function
userCalibrationFailed(event:ONIUserTrackingEvent):void
{
trace('Calibration Failed for User: ' + event.user);
}

// Get Skeleton Data. We use some filtering based on the user zone
to figure out who the active user is and ignore lost or stalled users.
protected function skeleton(event:ONISkeletonEvent):void
{
// We use the torso as the point we check to see if the user is in
the box
var userLoc:NiPoint3D = event.skeleton.torso;

// See if user is in the user zone
if(checkZone(userLoc)){
var fresh:Boolean = true;
for(var d:int = 0; d < dupValues.length; d++){
if(dupValues[d] == userLoc.pointZ){
fresh = false;
}
}

// if user data is from an active user and not stalled/ghosted
data
if(fresh == true){
if(zoneEmpty == true){
zoneEmpty = false;
userCalibrationComplete();
}

skeletonData = event.skeleton;
//////////////////////////////////////////////
// active user found, execute code here
//////////////////////////////////////////////

// keep alive timer that hides the warning box until user is
lost.
warningBox.visible = false;
lostUserTimer.reset();
lostUserTimer.start();
}
}

// add torso z data to dup check array
dupCheckArray.unshift(userLoc.pointZ);

// trim array to 10
dupCheckArray = dupCheckArray.splice(0,12);

// check for dups
var tempValues:Array = countValues(dupCheckArray);
dupValues = new Array();
for(var i:int = 0; i < tempValues.length; i++)
{
if(tempValues[i][1] > 1){
dupValues.push(tempValues[i][0]);
}
}

}

// Define User Zone and convert from feet to mm
private function createZone(dist:Number, width:Number,
depth:Number):Object
{
var z:Object = new Object();
z.distance = convertMM(dist);
z.width = convertMM(width);
z.depth = convertMM(depth);
return z;
}

// Convert Feet to MM
private function convertMM(n:Number):Number
{
return n * 304.8;
}

// Check if active user is in the defined user zone
private function checkZone(p:NiPoint3D):Boolean
{

if(p.pointZ > (zone.distance - (zone.depth/2)) &&
p.pointZ < (zone.distance + (zone.depth/2)) &&
p.pointX > -(zone.width/2) &&
p.pointX < (zone.width/2)
){
return true;
}else{
return false;
}
}

// This function looks at an array of values and checks for
duplicates and gives you a count of how many so you can find the stuck/
stalled data
private function countValues (a:Array) : Array
{
var i:int, j:int = -1, c:Object, o:Object = {}, r:Array = [],
v:Number, n:Number = a.length;
for(i = 0;i < n;++i) (c = o[v = a[i]]) ? ++c[1] : r[int(++j)] =
o[v] = [v,1];
return r;
}
}
}