For Senior Project at school, I was responsible for modeling and animating 3D fishes. The app that we were using required me to export the models into FBX file. I learned the hard way that exporting to FBX is a HUGE pain in the butt, and hopefully this article can smooth things out for anyone working with Maya & FBX.

Thing to keep in mind when exporting to FBX

• Deformer animations simply do not work. I’ve browsed through many, many forums looking for a solution but could not find an answer. What happens is that once exported to FBX, only the first frame will have the deformers applied, but not for any frames after that. This means that for the rest of the animation, the object being deformed will maintain the same shape from the first frame.
• Blend shapes work! You just have to re-arrange the Input Orders (explained later)
• Constraints may not alway work. I used parent constraints for my eyes and some of the controls, but they would do odd things when exported. For example, even though they’d work perfectly as an .mb file, in the FBX file there would be an offset on the objects that were constrained, so they wouldn’t be in the right positions.
• You have to delete non-deformer history after you smooth your mesh. Otherwise, even when your model is exported, it will still be unsmoothed.
• Make sure all your normals are facing the right way. To see normals, go to Display > Polygons > Face Normals. Not having the normals pointing the right way will mess up many things in the app/game, primarily the lighting.

Below, I’ve outlined the steps to properly export an FBX file. There is also a supplementary video that may help you get started. At this point, you should have the character modeled, textured, the joints and controls set up, and the mesh for any blend shapes created.

The proper steps for exporting to FBX

1. Smooth-bind the mesh to your rig.

Make sure you have the Animation menu open. Select the root joint, then select the mesh. Go to Skin > Bind Skin > Smooth Bind. Now, when you move your joints, your mesh should move as well. Don’t smooth just yet! Keep working in smooth-preview mode for now (hit 3 on your keyboard).

2. Weight paint the non-smoothed mesh.

You should now go ahead and weight-paint your model. I won’t go over the details on how to weight paint here, but I found this tutorial by unpicked1 very useful. It’s easier if you weight-paint before you smooth, since there are less vertices to worry about.

You only have to weight paint one side of your character. You can use the Mirror Skin Weight tool later to copy the weights from one side to another. When you use this tool, make sure your character is in its bind pose.

3. Animate the model.

Use your rig to animate the model. Walk, jump, swim… go nuts!

4. Smooth the mesh

Switch to the Polygons menu. Select the mesh, and go to Mesh > Smooth. If you have any meshes for blend shapes, go ahead and smooth them too. You may be tempted to export to FBX right away, but if you do now, your FBX model’s mesh will not be smoothed. Once you smooth it, you may have to weight-paint certain areas that may not be moving as they should.

5. Delete Non-Deformer History.

Go to Edit > Delete by Type > Options. Make sure that “All non-deformer history” is selected (Fig. 4).

Select the main mesh (not the blend shapes). Go to Edit > Delete by Type. This deletes the smooth node from the mesh, so now when you export it, it will be smooth in the FBX as well! If you did a simple Delete History, it would have deleted the smooth, but all weight-paint information and animations will be lost.

6. Create blend shape.

Switch to the Animation menu. Select the blend shape mesh, Shift+Select the main mesh, then go to Create Deformers > Blend Shape. The main mesh should now have a new attribute for the blend shape. When this new attribute gets closer to 1, the main mesh becomes more and more similar to the blend shape. Notice how in Fig.5, the goldfish’s mouth is now open like the blend shape.

7. Change Input Orders

The problem right now is that the blend shape is now prioritized over the animations. This means that if the value for the blend shape attribute is set to 1, the mesh will take on the blend shape for the duration of timeline instead of doing the animations. To fix this, we must change the input orders of the main mesh. Right click and hold on the main mesh, and go to Inputs > All Inputs (Fig. 6).

You will now see all the inputs of that mesh. Middle-click and drag the Blend Shape node on top of the Tweak node (when you middle-click over the Blend Shape node, you should see a plus sign). See Fig.7.

Now, even with the blend shape attribute set to 1, the model will retain its animation, PLUS have the blend shape applied.

8. Include blend shapes into animation

You can no go ahead and keyframe the blend shapes into your animation. For me, I used the fish’s blend shape to animate the opening/closing of the fish’s mouth (Fig. 8).

9. Check export options & export

Go to File > Export Selection… > Options. Under file type, select “FBX export”. Click the “Edit preset” button to open the FBX export options. In the options for the FBX export, look over the following:

• Under “Animation”, make sure the Animation checkbox is checked.
• Under “Bake Animation”, make sure the Bake Animation checkbox is checked. Also, set the Start and End values to the duration of your animation.
• Under “Deformed Models”, make sure all checkboxes are checked

Once this is done. select your model or the group containing your model. Go to File > Export Selection…. Export the FBX model to the directory of your choice. You should now have a fully-working FBX model.

I made a supplementary video that guides you through this as well. You can check it out here.

Conclusion

Anyway, those were my findings while working with FBX, and I couldn’t find information on this anywhere else.

Hope it helps!

*UPDATE* Caught embarrassing mistake. Thanks Grant!