Is there a way to remove silence at the end of files

MP3tag manages the padding according to its needs - and removes padding for mp3 files when re-writing the files.

Your observation that the playing time of a file changes after updating tags (only they lead to padding) should be an alarm signal in respect to the integrity of the files as any decent player does not add the tag data to the payload data.
See here for tools to check files: