In this lesson we’re looking at baking an animation. That is using the ThreeJS animation classes to move through an animation. Then for intervals in the animation getting the transformed vertex positions and storing them to a buffer. Then in our vertex shader we’ll use the transformed positions.

Here is what I mean. The two birds are a single mesh and the wing motion is via a pre-stored buffer not using an AnimationMixer. You might wonder why. The answer is because we can! This is a series of lessons introducing what you can do with TSL and a flock of birds provides a perfect example of what’s possible. Using instancing every bird in the flock would beat their wings in synchronisation. Which is not what happens in real life.

As usual you can press the JS button to see all the code. Notice in the FlockGeometry class we now have two extra attributes: uv and vertexID. UV is needed to position the texture map and vertexID gives a unique integer value to every vertex. We’ll use this index to access the correct vertex position data from the baked animation, vertex buffer.

When we load the sparrow GLB file. We store the entire parsed file as the app variable boid.

In the initStorage function we now have a time attributeArray.

The next function in the code is bakeAnimation. This receives one parameter a gltf instance. The hierarchy of the gltf.scene file is like this.

Layout of the gltf scene

Notice that the SkinnedMesh is scene.children[0].children[0]. The file we’re loading was created in Blender, which uses a Z-up coordinate space. When exporting as a GLB there is an option for Y-up. This simply makes all content into the child of a transform that rotates from Z-up to Y-up. That’s why the SkinnedMesh is nested inside another child.

Once we have a reference to the skinnedMesh. We convert the geometry to non-indexed and set the skinnedMesh geometry to this non-indexed version. We need to assign the gltf scene to the threejs scene instance, but we don’t want it to be visible. The assignment ensures the matrix calculations are processed. 

function bakeAnimation( gltf ){
  const skinnedMesh = gltf.scene.children[0].children[0];
  
  const geometry = skinnedMesh.geometry.toNonIndexed();
  skinnedMesh.geometry = geometry;
  
  gltf.scene.visible = false;
  scene.add( gltf.scene );

We create an instance of an AnimationMixer and grab the first animation clip in the gltf animations array. We create an AnimationAction using the clipAction method of the AnimationMixer. Then set the action playing. We create two constants:

  • interval: this is the time in seconds between each grabbed animation pose.
  • frameCount: this is the number of frames in the animation we’re grabbing based on the interval time. If you’re wondering double tilde (~~) does the same thing as Math.floor. 
  mixer = new THREE.AnimationMixer( gltf.scene );
  const clip = gltf.animations[0];
  const action = mixer.clipAction( clip );
  action.play();
  const interval = 1/15;
  const frameCount = ~~(clip.duration/interval) + 1;

Now we get the position BufferAttribute. Set a vertexCount constant and create a typed array that has vertexCount times frameCount times 3 elements. We set up a app scope object animInfo that has duration, interval, vertexCount and frame Count properties. Finally in the initialisation we create a Vector3 instance.

  let posAttr = geometry.getAttribute( "position" );
  const vertexCount = posAttr.count;
  const vertexArray = new Float32Array( vertexCount * frameCount * 3);
  animInfo = { duration:clip.duration, interval, vertexCount, frameCount };

  const skinned = new THREE.Vector3();

Next we enter a loop for frameCount times. We set the mixer time and render the scene. We set an offset value into the vertexArray that will move along each update of f by vertexCount times 3. The multiplication by 3 is because a position has an itemSize of 3. A Vector3 having a x, y and z component. Now we set skinned Vector3 to the position stored in the posAttr, the modelled location. A skinned mesh has a applyBoneTransform method, it takes a vertex index as the first parameter and a position vector as the second parameter. We also need to apply the skinnedMesh world matrix transform to the vector. Now we can stored the skinned values in the vertexArray.

  for (let f = 0; f<frameCount; f++ ){
    mixer.setTime( f * interval );
    renderer.render( scene, camera );
    const offset = f * vertexCount * 3;
    for (let i=0; i<vertexCount; i++){
       skinned.set( 
         posAttr.getX(i), 
         posAttr.getY(i), 
         posAttr.getZ(i)
       );
      skinnedMesh.applyBoneTransform(i,  skinned);
      skinned.applyMatrix4( skinnedMesh.matrixWorld );
      vertexArray[offset + i*3 + 0] = skinned.x;
      vertexArray[offset + i*3 + 1] = skinned.y;
      vertexArray[offset + i*3 + 2] = skinned.z;
    }
  }

At this stage the vertexArray has the vertex positions stored in blocks of vertexCount. The first block starts at 0, the second at vertexCount, the third at vertexCount * 2 etc. Now we can remove the gltf scene from the ThreeJS scene instance. We create an attributeArray from the vertexArray and return it.

scene.remove( gltf.scene );
  
  const vertexStorage = attributeArray(vertexArray, "vec3").label(
    "vertexStorage"
  );
  vertexStorage.setPBO(true);
  
  return vertexStorage;
}

Now we have a bakeAnimation function, in the tsl function we call it and create a reference to the returned vertexStorage buffer. Then we create various uniforms. Since these never change at runtime they don’t really need to be uniforms. We could instead in our tsl functions use their JavaScript equivalents. 

function tsl() {
  const vertexStorage = bakeAnimation( boid );
  const [ positionStorage, timeStorage ] = initStorage();

  deltaTime = uniform( float() );
  const duration = uniform( animInfo.duration );
  const interval = uniform( animInfo.interval );
  const frameCount = uniform( animInfo.frameCount );
  const vertexCount = uniform( animInfo.vertexCount );
  const useStorage = uniform( uint(1) );

Which brings us to the updates to the flockVertexTSL function, the vertex shader for our flock of birds with flapping wings. As well as sourcing the instance ID we also source the vertexID. The timeStorage buffer stores the animation time for each boid instance, we reference this as frame. The frameIndex is the integer version of the frame value. The nextIndex is simply frameIndex add one. We get the fractional value of frame, storing it as delta. That is the value after the decimal. For 2.31, fract would give 0.31. If nextIndex is greater or equal to frameCount then we set its value as 0, the animation is looping back to the start. 

const flockVertexTSL = Fn(() => {
    const instanceID = attribute("instanceID").toVar();
    const vertexID = attribute("vertexID").toVar();
    const frame = timeStorage.element( instanceID ).div( interval );
    const frameIndex = uint( frame ).toVar();
    const nextIndex = uint( frameIndex.add(1 )).toVar();
    const delta = fract(frame).toVar();
    If( nextIndex.greaterThanEqual(frameCount), () => {
      nextIndex.assign( 0 );
    })

We get the position of the vertex from the vertexStorage buffer at frameIndex and nextIndex. Storing these as pos1 and pos2. We use the delta value to blend between these two values, storing the result as pos. There is a gui that allows the user to switch back to using positionLocal rather than the vertexStorage buffer for the vertex position. We multiply by the usual matrices to transform from modelled position to homogeneous clip space.

    vertexID.addAssign( vertexCount.mul( frameIndex ));
    const pos1 = vertexStorage.element(vertexID).toVar();
    vertexID.assign( attribute("vertexID").add(vertexCount.mul(nextIndex)));
    const pos2 = vertexStorage.element(vertexID).toVar();
    const pos = mix( pos1, pos2, delta);
    const position = select( useStorage.greaterThan(0), pos, positionLocal ); 
    const finalVert = modelWorldMatrix.mul(position).add(positionStorage.element(instanceID));

    return cameraProjectionMatrix.mul(cameraViewMatrix).mul(finalVert);
  });

It just remains to use a Compute Shader to update the time stored for each boid. We add deltaTime that is updated in the render loop. If that means the value has exceeded duration then we subtract duration.

computeTime = Fn( () => {
    const instanceTime = timeStorage.element( instanceIndex );
    instanceTime.addAssign( deltaTime );
    If( instanceTime.greaterThan( duration ), () => {
      instanceTime.subAssign( duration );
    })
  })().compute(BOIDS);

And there we have two flapping birds. Time to put the flocking code and the baked animation together to create the flock of birds.

Previous
Next

If you find these articles useful, perhaps you can buy me a coffee by clicking the button below. It might encourage me to write more.

Signup to my mailing list.