Slow access because ...
-
network connections are the almost the slowest way to access a lot of files,
-
other processes like indexers, players, anti-virus programs could block the files for some time,
-
padding is not big enough so the files have to be rewritten,
-
Windows Defender slows down the access, see e.g.
Windows Defender - Speed of loading files in MP3Tag -
File sizes vary (e.g. is bigger for some files than for others) and together with insufficient padding lead to re-writing the big files - which takes longer.
These would be the knobs I would twiddle and tweak.