I've been drilling with the KZ adapter and as long as you have the chest strap cinched tight and connector-to-weapon-doohickey-strap properly secured, I don't see how that could even happen. It isn't loose enough to get close to the charging handle except when I'm doing a transition to a sidearm and only briefly as I lower it. The connector piece is simply too long to allow the charging handle near it.
My setup consists of an all nylon single point with side-release buckles on both the connector-doohickey and the main chest strap. It's ghetto.
Only issue I ever have is when running... sometimes the buttstock will wiggle inside the chest strap over the top. My mag-boobs act like a sling shelf.
What kinda sling / plate setup are you running? I wanna see. Does it have an extension coming off the main body at all? I think that is crucial.
...
I missed you, Walter. I laid awake at times... wondering if still have that emergency Uzi wrapped in brown paper. I need to borrow it.