I had the same kind of the experience, and it took me 7 years to get rid of the man. I hoped to give a complete family to my son,I hoped he could change one day, I hoped he will understand, and I hoped....., but I was wrong, completed wrong even his father told me that " I willing to die if he could changes". His father gave up on him too.
Thank god, I made a right decision to get rid of this man even I get not even a penny of childcare from him.