I am using v3.07 and I have noticed that when I close the program and open it again, all the files I had open in the last session show up again.
This is not a bug, it's a feature:
If start from this directory is enabled, Mp3tag loads this (possibly empty) directory instead of the last used directory.