As a newbie when I joint animation 7 years back, I thought lip sync is easiest thing to do and still many who join new have that notion but the reality, we get most of corrections in lip-sync only because of no training in this area.
I earlier just used to do Lip-sync straight ahead in single pass and check if it appearing good or not. Lip-sync done without planning or proper knowledge always eats up lots of our production time in doing it and re doing because of lots of corrections. I realized importance of it and started studying from different sources like animation survival kit, cartoon animation Preston Blair, and many more articles from internet… it helped me to gain more knowledge about lip-sync. In the process of discovery after 3 years in the animation I got chance to learn about phonetics in studio training and I discovered easiest way to lip-sync for me by gaining little understanding about phonetics which I will be sharing in this article. Let me add preface here that this article is from my learnings, observations and experiences in animation. This is not the only way to do lip-sync but this made my understanding clear for lip-sync so thought of sharing it.We never animate letters in lip sync. Imagine in a shot if you make shapes for all letters, it will give you very poppy animation. So we sync on sound we hear. Lip sync is not about animating all the letters which are spoken out but finding out main sound to hit our lip shapes, as per the accent. Character in the movie may have different lip shape all over movie, different characters make different lips shapes even if the dialogue is same, which depends on their accent, mood, attitude etc. to understand it let’s start with generic shapes. With practice we can develop more understanding over this subject.Little bit knowledge for phonemes is required to develop better understanding in lip sync animation.This article focuses solely on how to do lip sync with the sound in animation. Before jumping directly into the animation let’s do some pre-study.
Given below is the chart for the Vowel sounds
This chart is made after studying phonetics. It is necessary to study it little more by self, so we can get good grip over understanding over this subject. But here we are not going deep in to phonetics since this article only includes the technicalities of Lip sync.
Given below is the another chart for the Consonant sounds
Keep in mind the lip shapes which are shown in the chart are ‘generic’ shapes which is to describe and to increase our understanding for lip sync animation. Lips shape in animation varies as per the tempo of the sound, physic, characteristics and attitude of the character, feeling of the character in the context of the scene, etc.I like to follow steps even while lip-syncing. So let me show you my process of lip sync by breaking it down in four phases with an example.
Guidelines for animating lip-sync:
- Avoid static mouth; don’t hold a shape for too long. Use secondary action where ever it is required.
- Consonants like f, v, m, b, p hold them for two frame, so they get registration from the audience.
- Upper teeth do not move. Jaw can move up & down and also tilts left & right.
- Keep the transition smooth from one shape to another. Try to blend shapes nicely from one to other.
- Put your keys 2 frames before the sound, otherwise it won’t sync properly. Human see first then hear.
- Keep contrast in mind; don’t make same shape for the same word over and over again. We can speak and smile, speak and frown, speak and yawn at the same time.
- Use the cheeks and the nose while syncing. The cheeks and nose are the links from mouth animation to eye and brow animation. By keeping your nose and cheeks in the action you tie together the entire face of the character, creating a far more believable character who can act.
- Play with the shapes; exaggerate by understanding the tempo of the sound track.There is very nice example given in the Animation survival video kit, where the characters says, ‘Elephant juice,’ ‘Elevator shoes,’ ‘colorful,’ ‘I love you.’ using the same set of mouth shapes for all. Try speaking these by yourself in the mirror.Let’s start with our example, Transcript:
Phase 1: Breaking up the sound:Here I like to breakdown sounds of the dialogues in to the shapes on the paper as per the charts given above. Our target is to hit those shapes first which are given in the chart.Next immediate step is to transfer this idea on to the digital puppet (3d character). I make these shapes in my character in the shot. This step I call blocking and that’s why I like to see it in step mode only.So this stage includes
- Listen to the sound track in order to catch the vowel and consonant sound given in the chart. Then breaking down it on to a sheet of paper
- Finding and inserting Opening & closing, wide and narrow mouth shapes
- Blocking of main vowels sounds and consonants sound as per the sound waves on the time line
- Remember the guideline of putting your key 2 frames before the actual sound appear. (Human see first then hear.)
At this stage my timeline looks like this:
Very few keys here, only where the sound is at pick. Lets see how it looks in Playblast
Phase 2: Making supportive Shapes
Here I like to add more lip shapes which supports the main shape which are in the blocking, the starting and ending shapes of the words as per the sound I hear. Keep in mind this shape should not dominate the shape which are in phase one because those are the main once which audience need to register.So this stage includes,
- Try Adding personality in lips (if style of animation require or asked by supervisor, else don’t over do, keep it simple.)
- Syncing with expression (this means animation of cheeks and lower lead of the eyes with lips corner whenever needed.)
- Asymmetry (In this example we are doing semi realistic animation, so no need to exaggerate much)Also remember that in only law jaw can move up and down or sideways, because upper jaw is attached to the skull and it will only move with the head movement.at this stage my timeline looks like this:
few more keys added
Phase 3: Straight aheadSince our basic structure is ready till now so in this step I go straight ahead and add all lips shapes require to complete my animation. All holds and moving holds are to be added between the speech. Over shoot and over laps of the shapes are to be added and inbetweened in this process. Also establish relationship of lips with cheeks, nose, lower eye lids keeping flexibility of the rig.at this stage my timeline looks like this:much more keys added. Almost after every 2 or 3 frames there is a key. This is the only reason why new animators make lip sync mistakes because timeline is to messed up with so many keys. But in this case our steps are well defined and followed with accurate step which reduces the chances of mistake and with practice one can master over it.
Phase 4: PolishingThe final step where all touchups and tweaking to be done as per the need. Majority times I keep my graph in linear mode only because every frame is keyed but sometimes I smooth out the tangents, that is completely depends on our preference that how would we like to work keeping camera angle, deadline and complexity of the shot in mind. If rig is advance or working in big budget project muscle animation is required.
lets see our timeline now:
Volume of lips also majorly depends on the energy and the movement in the body. Possibilities are unlimited in animation, because it is said that animation is never finish. We can keep adding things on to our animation as per requirement and the style of animation. In our example I have kept it semi realistic. Lastly, while animating face one thing is important to keep in mind that whole face is one and it should not appear like only one part of the face is moving. Everything should move in connection with each other. If character is chubby the lips corner will affect the cheeks and even the nose and lower eye lid. We can add squash and stretch in lower part of the face and also add jaw movement to exaggerate and make it look cartoonish. Here in this article we have only covered up the lip sync area but in the acting shot we have to add emotions as well where character is happy or said it will affect the shapes in our lip sync in over all dialogue.
Keep one thing in mind, ‘Nothing moves alone, everything affects each other one way or other, more or less- in the world and in the animation and even on the face.’Since this was a short and easy dialogue we can go straight ahead animating it too, but for the long or complex one I always advice to plan it out and build your work flow to save time, energy and efforts.I would like to complete this article here may be in next article we will go deeper with the dialogue animation and expression.Reference:
- Font Size
- Reading Mode