I would be more worried about emotional and mental abuse on them right now. He sounds similiar to my ex-hubby.
Belittling tones, remarks to make you feel stupid, treating you (Hundove) as if you were 2 and have no clue.
After a period of time this kind of abuse can lead to harder problems to deal with. I just came out of my "shell" within the past 2 to 3 years. Mostly after I became medicated to help with my bi-polar. (that's a different subject) I was allowing her father to walk all over me. He would take liberties to plan his "visitations" without consulting me over the dates. We had agreed to alternative most of the other holidays. Majority of the time, he walked over me and decided what holidays he was taking and I allowed it.
Have a talk with your lawyer. Lay everything out. You need something on how your going to do holidays, summers, school, etc.
Is he going to take him every other weekend? Every weekend? Are you going to have Thanksgiving with him this year and he gets next year? Things like this aren't thought about usually til they hit you in the face. Personally, when it happened to me, I just mostly stood there like a drooling idiot and let the ex have his way.
Now, it's different. I have the courage to tell him how I feel, think, etc. He no longer tells me that it will be this day and that's that. I have had to be a downright b**ch at him to get through to him that I was no longer going to allow myself to be a doormat. We have it situated now that I only speak to him over the phone when it's necessary. That's in regards to health issues, school problems, etc.
Becoming a doormat to him, will let him believe that he can always do whatever he wants and get away with it. Use reverse pysch on your son. If you notice a bruise that you don't know how happened, try asking if he fell on the sidewalk/slide or something to that nature. When you want to ask him about his visit with Dad, lead him in asking. "Wow, I bet Dad took you to somewhere great! Where did you go? Would you like to go there again? We might take a trip."
It's all a matter of how you ask them the questions. Sneaky, yes. But sometimes kids just plain forget what they've done and need alittle nudge remembering. You'll do great. You have a world here that will try to guide you with advice that you can use how you see fit.