3 - Add Source Pins

Kinect MoCap Animation in After Effects — Part 3: Building the Puppet Rigging Template


This tutorial is now obsolete. Check out the new KinectToPin website for the latest version of the software and how to use it — it’s dramatically easier now.


Kinect MoCap Animation in After Effects

Tutorial links: Part 1: Getting Started | Part 2: Motion Capture with KinectToPin | Part 3: Building the Puppet Rigging Template | Part 4: Rigging a Digital Puppet

More info: A better way to control the puppet’s head | FAQ and Installation Guide

Project files: Click here to download the After Effects project (CS5+).

Quick note: the text is just transcripts of the videos, so you can read or watch as you prefer.


Welcome back! So now that you have your tracking data recorded, it’s time to build a character to apply it to. This is where things get a bit complicated.

Template Setup

We’re going to start by building a reusable template that you can use as the basis for any Kinect-controlled puppet. If you want to skip this section and download your own version of the template to start with, go for it, but I think knowing how it all works actually makes rigging easier.


1. Create a 640×480 24fps comp

Start by creating a new comp. Let’s call it “Puppet Template.” It has to match the dimensions and framerate of the Kinect, so: 640 by 480, 24 frames per second. Don’t worry, you’ll still be able to composite your puppet in HD later on.

I’ve added a reference image to help explain what’s going on: the Kinect tracking pins are numbered 1 to 15, starting with the left foot and ending with the head.


2. The Source layer

Now, we’re going to set up the layer that will hold the tracking data. Create a null or a new solid, and label it “Source.” It also needs to be 640×480. You can hit Ctrl + Shift + Y to resize it.

Select the puppet tool, and add 15 points to this layer, in roughly the shape of the Kinect source data.

They don’t need to be perfect, but you can use the reference image as a template if you want. Now, you aren’t going to be animating this layer directly: these pins are where you’re going to paste your tracking data so the puppet pieces can access it. That is, you’re going to use them as a source of information to control another set of pins on different layers.

It’s good to have a sample track to test your rig with.

To add your mocap data, open up its text file, copy the whole file, then select all the pins and hit paste.

Kinect tracking data can be very choppy, so we need to smooth it out, or all of our characters will look as if they’ve had far too much coffee. After Effects actually has a nondestructive smoothing function built into the program, but it’s only accessible as an expression. I’ve been using this software for more than a decade now and had no idea it was there, so I need to thank Roland Kahlenberg on CreativeCow for pointing me in the right direction on this, as it’s what really makes the process workable.

So let’s smooth things out a bit. Head down to the timeline, and hit U to show all keyframes. Hold alt and click the stopwatch to create an expression. It defaults to self-referencing, so that snippet of script just means “use these keyframes here to control this point.” You’re not going to delete that bit, just add a little piece of code on the end that will make the Kinect data play nice. Type .smooth(.5,15) .

You have to do this for all 15 pins, but you only have to do it this once.

You can take a look at the smoothed out path by clicking “show post expression graph” then switching to the Graph Editor.

You can really see the difference it makes! The smooth method works by averaging out the motion over time. The first number is the amount of time averaged — .5 means half a second. And the second value is the number of samples over that time. Make sure it’s an odd number so that the frame you’re looking at gets factored in. The bigger the number, the smoother the motion, but it may slow down your machine, and if you average things too much you’ll start to get into creepy mocap jello territory. I find .5 and 15 work pretty well for me, but you can adjust the numbers to suit your own designs.


3. Adding Invert Pins Option

Now, sometimes the data comes in with left and right inverted. The next thing we’re going to build gives us a quick method of flipping the point sources so that when that happens you just have to check a box instead of rewriting all your expressions.

Create a new layer, and call it Invert Pins. Add a Checkbox expression control (Effect -> Expression Controls -> Checkbox Control).

You can rename that if you want. I’m calling mine “Invert Pins?”.

We’re going to write conditional expressions that refer to it — that means if this box is checked, they’ll get the tracking data from one side, and if it’s unchecked, they’ll get it from the other.


4. Creating control layers

The next step is to create the control layers for the different body parts. You could just have all the expressions on one layer, but separating them gives you a little flexibility and makes it easier to find the body part you’re looking for when you’re connecting the pins on your rig to the controls.

Start by creating another null. Set the anchor point to 0 by selecting the layer, pressing a, then typing 0. If you need to shift a whole arm or leg over once you have things rigged, you’ll be able do it by moving these layers.

Let’s start with the left leg. Rename the layer leftLeg.

Create an expression point control (Effect -> Expression Controls -> Point Control), and call it leftAnkle.

The next bit is easier if you hit U to show all the points on your source layer so you can grab them with the pickwhip.

Enable expressions, and clear out the default one. Now it’s time for that conditional expression I mentioned earlier. We’re setting this layer up to change its content based on the Invert Pins checkbox.

Here’s how to add the expression. The bits in bold you should type exactly as they are, the bits in italics are instructions, and the comments after the //s are just to help explain what’s going on if you’re new to conditional expressions.

if( [Pickwhip the Invert Pins checkbox control] == false) // That’s saying “if the checkbox isn’t checked”…
fromWorld( [Pickwhip Source pin 1’s position] ) // …grab the position of the left ankle pin and convert its coordinates to World layer space…
}else{        // …otherwise…
fromworld( [Either pickwhip pin 4’s position, or just copy and paste the pin 1 info and change the 1 to a 4] ) //  …get the position of the other ankle in World layer space and use that instead.

The final expression should look like this:

if( thisComp.layer("InvertPins").effect("Invert Pins?")("Checkbox") == false) {
 fromWorld(thisComp.layer("Source").effect("Puppet").arap.mesh("Mesh 1").deform("Puppet Pin 1").position)
 fromWorld(thisComp.layer("Source").effect("Puppet").arap.mesh("Mesh 1").deform("Puppet Pin 4").position)

When you rig your puppet layers, your character’s left ankle is going to connect to this layer.

Now let’s set up the rest of the body parts. Here’s where that skeletal diagram comes in really handy: since you already have the basic expression format, all you have to do to set up the rest of the arm and leg controls is change the pin numbers.

Duplicate the leftAnkle expression point control by selecting it in the Effects palette and pressing control or command + D. Rename it leftKnee. If you had 1 and 4, now you need 2 and 5.

Do it all again for the leftHip, and this time change the numbers to 3 and 6.

Then, duplicate the whole leftLeg layer, and rename it rightLeg.

Press EE to show all expressions, rename them to rightAnkle, rightKnee and rightHip, and swap the numbers around for each of them. 1 and 4 becomes 4 and 1, 2 and 5 becomes 5 and 2, 3 and 6 becomes 6 and 3.

That’s both legs set up. Now it’s time for the arms. Duplicate the leftLeg layer again, and this time call it leftArm.

Rename the controls to leftWrist, leftElbow and leftShoulder, and, again, switch out the numbers. For the wrist, the pins are 7 and 10, for the elbow 8 and 11, and for the shoulders 9 and 12.

Duplicate the leftArm layer, and rename it to rightArm.

Rename the controls rightWrist, rightElbow and rightShoulder, and reverse the numbers for the opposite side of the body.

So now you have all four limbs. Time for the spine, waist, neck and head. These don’t flip left and right like the arms and legs, so it’s going to work a little differently.

Create a new null layer, set the anchor point to 0, and call this one Spine.

Create a new expression point control (Effect -> Expression Controls -> Point Control) and call this one “waist.” This one’s a little easier.

Type fromWorld( [Pickwhip pin 13’s position] )

The final expression should look like this:

 fromWorld(thisComp.layer("Source").effect("Puppet").arap.mesh("Mesh 1").deform("Puppet Pin 13").position)

All this does is grab the waist point’s location and change the numbers to World coordinates.

Next, duplicate the waist point control, rename it “neck” and change the pin number to 14.

Duplicate it again. Make this one “head” and change 14 to 15.

I like to color code the control layers, green for the left arm and leg and red for right, with a purple layer for the spine and head.

Whew. You now have a reusable puppet template! Now is a good time to save!

In the next part, we’re finally going to connect an actual puppet to this.

<iframe width=”580″ height=”325″ src=”http://www.youtube.com/embed/NZxLEXNn0ow” frameborder=”0″ allowfullscreen></iframe>

8 thoughts on “Kinect MoCap Animation in After Effects — Part 3: Building the Puppet Rigging Template”

  1. I’m one of those guys that never post comments. And now that I finally run across something so awesomly good that I just have to send proverbial – and virtual – kisses and hugs – I can’t find the words! Either the cat got my tongue or I’m still temporarily brain-numbed by the entire scenario…. Wow.

Leave a Reply to Victoria Cancel reply