To find the files without embedded covers in MP3tag use the filter
%_covers% MISSING
You then may use an action of the type "Import cover from file" with the
Format string: *.jpg
to get any file that may reside in the file's folder as cover - if you have folder.jpg then most likely that one will be taken.
Toggle the filter to find those files where there really is no embedded cover, not even after the import. I fear there is then no way around it but to look for the covers again and really embed them.
If you do not trust MP3tag, then try the tool "Mp3diags" as linked in this thread:
This tool has an option to show files without embedded cover - if there are any files with that criterion.