Thank you everyone. I now feel much better. It hasn't helped that he is starting to get shy around people he doesn't know. Yesterday I left him with a new lady and he started crying right away. However, he had such a good time when it came time to leave my wife couldn't get his attention. He was practically yelling with joy.
Thanks again.