[F] 2.44f may write tag data to the wrong files

gvm · November 4, 2009, 11:16pm

Before I start to detail the problems I've run into with 2.44f, I'd like to thank Florian for responding quickly to the Unicode numeric strings issue and releasing 2.44f in a very short period of time.

The bug I've come across in 2.44f is very serious, and has resulted in about 100 corrupted files (fortunately, all restorable from backup).

I don't have a fully reproducible example of the sequence of events yet, but it looks like such a high-impact issue that I thought I'd better report it as soon as possible.

I've been making heavy use of 2.44f over the last couple of days in order to correct about 2500 MP3s with Unicode numeric strings (generated by MP3Tag over the last year or so). 2.44f is the first recent development build to use the correct ISO-8859-1 strings for this format.

Reviewing my files after having completed the task, I've found several "blocks" of files where each file has had the alphabetically preceding (or following) file's tags written over the correct tags. I did back up all of my MP3s just before starting this operation, so my data isn't permanently lost, but a selective restore is still going to be painful.

Here's a "trivial" illustration of the sort of thing that I'm seeing:

File1    Tag1
File2    Tag2
File3    Tag4
File4    Tag5
File5    Tag6
File6    Tag6

In this example, the tags from files 4-6 have been written into files 3-5.

As I said, I don't have reproducible proof that the problem I've encountered was caused by 2.44f, but there's some significant circumstantial evidence.

In order to rewrite the Unicode tags, I constructed an Action Group that clears and then restores the relevant tags. This is done one-at-a-time, and it only affects a few tags, so there's no reason to assume that this is the source of the problem. In some cases, however, this wasn't sufficient to correct the problem (Microsoft's recent "compliance enforcement project" seems to include rejecting files with empty tags, amongst other things). If my Action Group didn't make the file playable, my next action would be to do a "cut all tags" (ctrl-x) and then "paste tags" [Edit: Although I originally said "paste" here, I would always use "undo" if any other action had taken place after the "cut"]. This appears to successfully force MP3Tag to completely rewrite all the tags. This is where the "tag shifting" problem seems to arise, however.

I haven't managed to reproduce the full effect yet, but I have managed to reproduce a set of that result in a corrupted screen display, showing the wrong tags being assigned to files, but they aren't actually written. This is behaviour that is new with 2.44f - I've tested it against 2.44, and that doesn't do this.

This is the set of actions required to show the screen corruption problem:

Create a group of test files named 0.mp3, 1.mp3, 2.mp3, etc. up to say 9.mp3 .
Remove all tags from these files.
Create a title field in each file, each containing just the number of the file (0.mp3 has title "0", etc).
Start a clean editing session on the group of files using MP3Tag 2.44f.
Sort the screen display by file name. The Title tags should be in the same sequence, 0-9.
Select files 5-9.
Cut the tags (Ctrl-X).
Without de-selecting the 5 selected files, re-sort the screen display by file name in *descending* order (just click on the "file name" header). The top 5 lines of the display will be (correctly) highlighted, with no data in them.
Undo the "Tag Cut" operation (Ctrl-Z).
The screen display now becomes corrupted - the top 5 lines remain highlighted and blank, and the pasted data appears in the bottom 5 lines, overwriting the existing data for files 0-4. Instead of replacing the tag data into the files that it originated from, it has been assigned to the files that now occupy the *screen position* that the data came from. This behaviour is different to that of 2.44, which correctly places the data in the top 5 rows, in reversed sequence.
Highlight any line. The data displayed on the line reverts to the correct data.

Performing the same sequence using version 2.44 shows the expected correct behaviour - the tag data is shown to be restored to the correct files (in their new screen positions).

This example does just show a screen corruption, and it's not quite the same "cut and paste" operation that I was doing on my "live" data, but it indicates (to me, at least) that there is a problem in this area (parts of MP3Tag not knowing quite where the data should be coming from or going to) that has newly arisen with version 2.44f - enough for me to believe that it wasn't just "finger trouble" that has shifted my data around (I could have believed it if it was just one block of data, but there are several, and it's a consistent "shift-by-1" which would be difficult to do accidentally).

It's my belief that some variant on this sequence of operations, with a single inserted line, or possibly a re-sorting as a result of the "cut" operation, and a slightly different cut/paste/undo(?) sequence has resulted in "shifted" data being written to the files, as well as just display corruption. I know that this is rather speculative at the moment, but the consequences of this type of bug can be very unpleasant.

Until this is resolved, I'd strongly suggest that anyone using 2.44f should keep backup copies of every file opened by MP3Tag (not just ones that are expected to be written).

I'll try to get more concrete information tomorrow.

Regards,

gvm

gvm · November 5, 2009, 6:28am

I've now been able to reliably reproduce the problem. The sequence of actions involves more than one Cut-Undo operation. I'll post a detailed sequence in a short while.

I'd definitely advise anyone using 2.44f to stop doing so until Florian has commented on this issue.

gvm

dano · November 5, 2009, 7:17am

I just want to add that it should be sufficient to use "File > Save Tag" on your files to change the encoding.

gvm · November 5, 2009, 7:44am

OK, it's definite - 2.44f can and will write tag data to the wrong files.

This is the set of actions required to show this file corruption problem:

Create a group of 10 test files named 0.mp3, 1.mp3, 2.mp3, etc. up to 9.mp3. The content of these files is unimportant.
Remove all tags from these files.
Create a title field in each file, each containing just the number of the file (0.mp3 has title "0", 1.mp3 has title "1", etc).
Start a clean editing session on the group of files using MP3Tag 2.44f.
Sort the screen display by ascending file name. The Title tags should appear in the same sequence, 0-9.
Select files 5-9.
Cut the tags (Ctrl-X).
Without de-selecting the 5 selected files, re-sort the screen display by file name in descending order (just click on the "file name" header). The top 5 lines of the display will be (correctly) highlighted, with no data in them.
Undo the "Tag Cut" operation (Ctrl-Z).
The screen display now becomes corrupted - the top 5 lines remain highlighted and blank, and the pasted data appears in the bottom 5 lines, overwriting the existing data for files 0-4. Instead of replacing the tag data into the files that it originated from, it has been assigned to the entries that now occupy the *screen position* that the data came from. This behaviour is different to that of 2.44, which correctly places the data in the top 5 rows, in (correctly) reversed sequence. At this point, the 2.44f screen display does not correctly represent the content of the actual files, but no incorrect data has yet been written to the files.
Highlight 4.mp3 . The Title displayed on the selected line is reloaded from the file 4.mp3 and reverts from the incorrectly-displayed "5" to the correct data, "4".
Hold down "Shift" and click 0.mp3 (the bottom line) to select the files 4.mp3, 3.mp3, 3.mp3, 1.mp3 and 0.mp3. Note that this action does not cause the displayed information for the four newly-selected files (3.mp3 - 0.mp3) to be re-loaded and corrected.
Cut these tags (Ctrl-X). At this point, no title information is being displayed at all.
Re-sort the display by file name in ascending order (click on the "file name" header again). The files 0.mp3 to 4.mp3 remain highlighted and are now in the top 5 lines of the display.
Undo the "Tag Cut" operation again (Ctrl-Z). The "Title" fields of files 0.mp3 to 4.mp3 now displayed (in ascending order of file name) are "9", "8", "7", "6", "4". This data has been written to the files, which are now corrupted.
To demonstrate that the files are damaged, close MP3Tag and re-open the same set of files.
The "Title" tags of files 0.mp3 to 9.mp3 are now "9", "8", "7", "6", "4", "5", "6", "7", "8", "9".

Performing the same sequence using version 2.44 shows the "expected" correct behaviour - no screen or file corruption occurs.

Other sequences involving combinations of "cut-sort-undo" and "shift-select" (or "control-select") operations using 2.44f have similar effects.

The sequence shown above may appear complex (and possibly unlikely), but similar events have happened to me in practice several times over the last few days, resulting in corruption of a couple of hundred files (after editing several thousand).

It looks to me as though the problem arises from a combination of:

MP3Tag's behaviour on "Undo" (that it rebuilds the tags from the in-memory representation, rather than restoring a binary copy of the original tags) and
MP3Tag's in-memory representation of the tags is becoming inconsistent with the files as a result of the cut-sort-undo sequence.

Under most circumstances, point 1 isn't critically important because whenever a line of screen data is touched by the user, it is usually re-loaded from the file. The cut-sort-undo inconsistency only becomes significant when a "shift-select" operation is used to select corrupted on-screen data, because it is not reloaded at this point. When the second "undo" operation id performed, the corrupted in-memory data is used to re-create a new representation of the "undone" files, rather than restoring their tags as "binary images".

As to what should be done to address this issue, it's clear that the cut-sort-undo inconsistency ought to be fixed, but it's not so obvious about the "rebuild from memory" behaviour of "undo". The current behaviour has the advantage that it writes a "clean" tag (and this is exactly what I was exploiting to "launder" my tags when I encountered the problem). On the other hand, users may expect "undo" to return their files back to their exact previous state, which it doesn't do. This should, at least, be documented.

Until this issue is resolved, I'd strongly suggest that 2.44f should be avoided.

Over to you, Florian (sorry to be the bearer of bad news again!)

Regards,

gvm

Florian · November 5, 2009, 8:26am

Thanks a lot for the detailed and helpful report.

Can you please try this unofficial version:
http://download.mp3tag.de/support/F08CC336...gv244fsetup.exe

Thanks!

Florian

gvm · November 5, 2009, 10:46am

Interesting - I'd like to continue this, but my response is heading off-topic for this bug report, so I've posted it here.

Regards,

gvm

gvm · November 5, 2009, 10:54am

Will do. Will take a short while to config up a clean Virtual Machine.

What has changed, so I know what to look/probe for?

Regards,

gvm

Florian · November 5, 2009, 12:54pm

Well, I hopefully fixed the bug you're describing. Besides that, this should also be fixed now.

poster · November 5, 2009, 1:55pm

Yes, it is.

gvm · November 5, 2009, 2:25pm

Ah - OK. I was angling for a description of the underlying problem, so I could attack it more effectively. However, just from stabbing in the dark at it...

This version does indeed seem to correct the this bug as described above!

Unfortunately it brings to light yet more "unexpected" behaviour (though it's low-priority, pre-existing in earlier versions and quite possible to live with). Here goes...

When cutting/pasting/undoing entire lines of tag data, some "undo" actions are grouped together in a non-intuitive way (and different to the grouping that occurs when cutting/pasting/undoing individual fields). Try the following sequences on the 10 test files from the previous example, (where "Action(N)" means perform Action on line N):

Cut(0)-Cut(1)-Cut(2)-Undo - The Undo appears to undo the last cut (line 2) only. Each subsequent Undo action undoes the cuts in reverse sequence. Superficially, this appears to be as expected.
Cut(0)-Cut(1)-Cut(2)-Undo-Paste(3) - Pastes line 2 into line 3*. This is not what would be expected - if the Cut(2) had been undone, then the Paste(3) should have pasted line 1 into line 3, it should not have pasted line 2 into line 3. It looks here as though the Undo has transmuted the Cut(2) into a Copy(2), rather than fully undoing it.
Cut(0)-Cut(1)-Cut(2)-Paste(3)-Paste(4)-Paste(5)-Undo - Undoes all of the Paste actions in a single operation and transmutes the Cut(2) into a Copy(2) as in the previous example. Subsequent Undo actions transmute the preceding cuts, one at a time as before. Expected action would be that a sequence of Undo actions would have undone the Paste actions one at a time, followed by undoing the Cuts, one at a time.

These aren't a big deal, but possibly something to be aware of when doing complex cut-paste sequences (particularly where the previous cut and paste points has been scrolled off-screen), and particularly when pasting one line of tags repeatedly onto a set of files - here it's important that if you do a single Undo, you undo all of the Pastes in this sequence. It also differs from the behaviour when cutting and pasting data into individual tags - in this case the behaviour is to unroll one keystroke at a time, as intuitively "expected".

I don't really think this is worth pursuing further unless the fix is trivial - I only mention it because of the context, and I probably wouldn't have bothered to post this as a "stand-alone" bug.

I'll bet that you wish you hadn't asked, now.

Thanks for getting the main problem fixed quickly (again).

Regards,

gvm

[* - Edit - corrected "obvious" typo- line 3 not line 4]

Oblio · November 6, 2009, 4:25am

Almost. With new unofficial 2.44f the selection state is not preserved when sorting on column with filter active.

poster · November 7, 2009, 1:51pm

With this unofficial version tagging from external sources does not work properly anymore.

Sometimes there is only the list of search results and if you choose one and get to "Adjust Tag Information" all that is displayed are the local files in the right windows.

With Amazon.com and going on to "Adjust Tag Information" it says "Sorry no entries are matching your search".

Florian · November 7, 2009, 1:59pm

The original issue is now fixed with the current Development Build.

Can you please try with the new version?

poster · November 7, 2009, 2:17pm

Works again. Thank you.

Only discogs still says "Error connecting to server". I did not mention this in my report, because I thought that it is a temporary problem with discogs.
Is it?

Florian · November 7, 2009, 3:21pm

I think it's a temporary problem (or a problem caused by the specific query string) -- it's working fine here.

Thanks for testing!

Florian · December 28, 2018, 2:03pm

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.