I would point my finger at the float or the chain that is pulled when you push the handle. In ether case they can both be adjusted. If the float is the issue the toilet is overflowing water because it is hitting the overflow point before the float stops it from filling, if it can't be adjusted, just bend the rod down
In the case of the chain it is too taught so it is keeping the flap in the bottom of the tank slightly open, let out a few links so it gets a good seal.