C# HMM Gesture Recognition usng Kinect


I'm working on a solution to do gesture recognition using the Kinect sensor. Now I'm using Accord .NET to train the HMM. I have a dataset with saved gestures. This dataset has 11 gestures and each one has 32 frames with 18 points saved.

So I have a (double [12] [32,18]) input dataset and a (int[12]) output dataset, but when i do: double error = teacher.Run(inputSequences, output), it gives me this : "Specified argument was out of the range of valid values."

Does anyone knows how to solve this? Should treat the dataset before using it o on the hmm teacher or the dataset is ok like this?


The problem might have been that the teaching algorithm expects the training sequences to be in the form double[12][32][18], rather than double[12][32,18]. The training data should be a collection of sequences of multivariate points. It should also be necessary to note that, if you have 11 possible classes of gestures, the integer labels given in the int[12] array should be comprised of values between 0 and 10 only.

Thus if you have 12 gesture samples, each containing 32 frames, and each frame is a vector of 18 points, you should be feeding the teacher with a double[12][32][18] array containing the observations and a int[12] array containing the expected class labels.

The example below, extracted from the HiddenMarkovClassifierLearning documentation page should help to give an idea how the vectors should be organized!

// Create a Continuous density Hidden Markov Model Sequence Classifier 
// to detect a multivariate sequence and the same sequence backwards. 

double[][][] sequences = new double[][][]
    new double[][] 
        // This is the first  sequence with label = 0 
        new double[] { 0, 1 },
        new double[] { 1, 2 },
        new double[] { 2, 3 },
        new double[] { 3, 4 },
        new double[] { 4, 5 },

    new double[][]
            // This is the second sequence with label = 1 
        new double[] { 4,  3 },
        new double[] { 3,  2 },
        new double[] { 2,  1 },
        new double[] { 1,  0 },
        new double[] { 0, -1 },

// Labels for the sequences 
int[] labels = { 0, 1 };

In the above code, we have set the problem for 2 sequences of observations, where each sequence containing 5 observations, and in which each observations is comprised of 2 values. As you can see, this is a double[2][5][2] array. The array of class labels is given by a int[2], containing only values ranging from 0 to 1.

Now, to make the example more complete, we can continue creating and training the model using the following code:

var initialDensity = new MultivariateNormalDistribution(2);

// Creates a sequence classifier containing 2 hidden Markov Models with 2 states 
// and an underlying multivariate mixture of Normal distributions as density. 
var classifier = new HiddenMarkovClassifier
By : Cesar

This video can help you solving your question :)
By: admin