I've got 736 files loaded into mp3tag. I fished making my changes then did select all, remove. The file listing is empty. I quit the application then restart it and all those files that I removed are now back. I'm not exactly sure why that keeps happening.
"Removing" is not "Deleting". Removing just removes them from the MP3Tag listing but does not physically delete the files.
To delete them completely from storage, you need to press the "Option" button, then the context menu changes from "Remove" to "Delete".
Or: If you want MP3Tag to re-open "empty" you need to go Settings > Starting folder > None.
Setting it to None did what I was looking for.