In 10.0 there was an assert to ensure that there were semi
sync clients before removing one, but it was removed in 10.1.
This patch adds the assertion back.