Edit: Okay, it seems the issue is my files are VBR audio instead of CBR audio.
From a little extra research I think I know how to fix the video.
I've noticed lately on certain .avi files, not all, WMP always plays with the audio about 1-3 seconds out of sync. It's horrible to hear someone talking long before they appear on screen. No issues with .wmv files.
Now originally I thought it was the file(s) with the issue, but I noticed two things after playing around with them.
1) They all checked as having no audio delay in AVI-Mux.
2) VLC media player plays all the files with no audio sync problems.
Codecs:
VLC is using Xvid for video and mpga for it's audio.
WMP is using Nero Video Decoder for video and AC3Filter for audio.
Does anyone know how I could fix this? Is it possible to change the audio codec used in WMP?