Do you use the latest development build 3.10d?
I ask, because I can't reproduce this behaviour with 3.10d. If I press "Remove" on the selected entries, they will be removed. And as expected, they reappear after a restart of Mp3tag (because they still exist in the choosen directory).