6 year live in boyfriend

My boyfriend and I have lived together for 6+ years. His ex wife is a ALPHA beeyotch who has meddled in our lives since the start of the relationship. For 5 of the 6 years his 3 children came to our home and manipulated, made impossible purposely(with mom's blessings), and behaved badly. Recently his wife got angry again, as she does cyclicly(manic/depressive with really bad pms) and promised the children they would never have to return to my house again. Mind you I have a child who had come to know these children as her siblings. Since she made the rule I decided that the kids were never to be welcome in my home again. The boyfriend resents this decision and wishes I would resolve the issues with the ex. He however I should add in all the time we have been together, or while he was with her never stood up to her(much of the reason she gets away with this stuff). So here we are and I have asked him to move and want to make sure I am right. Pretty sure I am....