Tags not saving correctly

This may be a dumb question,
But I have a general question about tag saving.
I have approximately 16,500 files, and when I have files all properly organized, sometimes the tags will come up with different punctuation, so for example, if I have 1 folder of Red Hot Chili Peppers, i see them on my mp3 player as 4 different files with completely different files in all of them and it will show up something like this..

Red hot chili peppers
Red Hot chili peppers
Red Hot Chili peppers, etc, so even though they are all in the same folder, organized properly, I am getting several folders with different grammar,

Its not a huge deal, but does cause some frustration when searching for a specific song.

So my question is, when I'm finished editing them to all have the same title, artist, etc, and save all my edits, when I turn my mp3 player back on, it looks as though I've done nothing.

Is there something I'm doing wrong or missing, or didn't do right?
any help or suggestions would be awesome!

What are you talking about?
The tags inside the file or the filename?
What does MP3tag show?
What does the mp3 player display? The tags or the filename?
How does the mp3 player perform its updates?