No matter what I do it keeps on reappearing once I return to or refresh the folder. I even tried deleting the old art, saving and then adding the new album art to once again save and it still happened 
Which program shows you the old art?
You could check in Windows Explorer for hidden folder.jpg and albumartsmall.jpg files and delete them.
If you use a player with its own database, update the database and delete any graphics cache.
And: in MP3tag check which tag version there are in the file. If you also see APE tags in addition to the ID3 tags, then get rid of the APE tags.
I think I fixed the issue. MusicBee said the folder.jpg was not writable so I went to properties and found out that both the cover.jpg and the folder.jpg were blocked for some reason. I unblocked them and copy pasted all of the files and put them into a new folder. After adding the new folder to the problem has stopped. Thank you for your help.