John's Corner

Fix lip sync issues in video files

This tip covers how to remove delays between video and audio in a video file.

Video files can sometimes have an annoying delay between the video and audio, typically seen as a delay between the audio and the lip movement in the video. Some video containers (eg. mkv) allow this delay to be corrected within the container. However, this isn't possible with mp4 files. This tip should work with all types of video files.

The recommended tools for this tip are FFmpeg, VLC Media Player and Audacity VLC allows you to play the video and work out the delay. FFmpeg allows you to extract the video and audio into separate files (this could also be done with Xmedia Recode but FFmpeg is quicker). Audacity allows you to timeshift the audio back or forward by the required delay then FFmpeg recombines the audio and video to complete the job.

1. Start by loading your file into VLC. This will allow you to work out the exact delay between the audio and video.
2. Slide the video to a place where someone is talking. Carefully observe the video to get a sense of whether the audio is leading or trailing the video and how noticeable it is. This may take some practice.
3. Go to the VLC Tools menu and select Track Synchronization.
4. Adjust the Audio Track Synchronization control. Start with 0.100s (delayed) where the audio is leading the video or -0.100 (hastened) where the audio is trailing the video.
5. Replay the video and check the lip sync. Re-adjust Audio Track Synchronization each time halving the amount of change, until the audio and video sync is acceptable. Note this value.
6. Use FFmpeg to split the video and audio to separate files. Open a Command Prompt in the folder where FFmpeg is located and copy your video file to this folder. Use the following commands. Change source.mp4 in each command to match the name of your video file:
   ffmpeg -i source.mp4 -c:v copy video.mp4
   ffmpeg -i source.mp4 -c:a mp3 -b:a 256k audio.mp3
   The mp3 audio is required because Audacity, used in the next step, doesn't read aac files. The audio is output at 256kbits/sec to minimise degradation to the audio quality.
7. Load audio.mp3 into Audacity.
8. The correction should be made at the very start of the audio file where there is almost certainly silence. Select a section of audio as long as the delay you have determined above. It's easiest to type the values directly into the "Start and End of Selection" boxes.
9. If the delay is negative, hit Ctrl-X to delete the selection. If the delay is positive, hit Ctrl-C to copy the selection, then hit Ctrl-V twice to overlay it back then add it. Export the file to mp3.
10. Re-join the video and audio files, converting the audio to acc in the output file:
    ffmpeg -i video.mp4 -i audio.mp3 -c:v copy -c:a aac output.mp4
11. Test the final file in your favourite player or drop onto a web page.