Make sure you're writing ID3v2.3, not ID3v2.4, which is not supported in Windows Explorer.
upgraded to v2.71
All album art showing on all thumbnails.
I doubt that changing to a different MP3tag version really solves a WIndows Explorer problem.
Either you did something else in the meantime or you did not tell the whole story.