Ok, so I initially put an album art for Album A using the copy/paste method in Windows Media Player. This album art works everywhere, in every music app. By using MP3tag, I can see each track is tagged with this album art.
But now, I realize that I do not like the Album Art at all, and want to use an alternative album cover. I go to MP3tag, remove the album cover (and click save to be sure), and then I add the alternative album cover.
When I do this, and look at the album folder in File Explorer in Large Icons view mode, I can see that the new updated album art is displayed on each track.
However, there are some issues. On both Windows Media Player and Groove Music, the old album tag shows, not the new one. Also, in File Explorer, if I look at the album folder image, the old cover photo shows, not this new one.
WMP has a cache for album art.
see %appdatalocal%\Microsoft\Media Player\Grafikcache" and delete the folder LocalMls
Also: set all folder.jpg and albumartsmall.jpg files to visible, non-system:
(best in a command shell: open file explorer, navigate to the top-most folder, Shift-right click for a context menu and select "Open command shell here". In the command shell enter the following:
attrib -h -s /s *.jpg
and delete them:
del /s folder.jpg
del /s albumartsmall.jpg
Treat all thumbs.db in the same way.
Close the command shell)
Then load your files into MP3tag and export the album art with an action to the name
folder.jpg