How can I select ID3V2.3 and UTF8

Hi there, I am looking for some help in the subject.
Normally I use "ID3v2.4 UTF8", that fits most of my applications. One of them would call though for upt to v2.3 only, and not v2.4. If I choose "ID3v2.3 UTF16" in Mp3tag, then it naturaly alters my special accented characters. UTF16 is not really an option for me, because I would have to edit/correct thousands of tracks when moving from UTF8 to UTF16. I don't see any advantage of using UTF16 anyhow, or any reasoning why UTF8 would not work with v2.3
Therefore, I am looking for an option of "ID3v2.3 UTF8", if that is possible.
Thank you

ID3v2.3 UTF8 is not allowed by the ID3v2 tagging standard.

But this should not be a problem for you because utf16 can store the same characters as utf8, it is just a different encoding.
If you switch to utf16 you shoudn't loose any accented characters, there must be another problem.

"ID3v2.3 UTF8 is not allowed by the ID3v2 tagging standard"

OK, thanks, I can live with it, if that's the standard (although I could not find any reference to where it is explicitly written like that).

Nevertheless, here is what I experience and cannot explain:

Using the exact same string containing those special chars, typed in from keyboard, the results

  1. Mp3tag write v2.4/UTF8 - player does not read v2.4 tag only the v1.1 tag (with that 30 chars limitation)
  2. Mp3tag write v2.3/UTF16 - player reads the v2.3 tag. but special characters screwed
  3. tag resulted by 2., edit in WMP by entering original string - player reads v2.3 and chars are ok
  4. Mp3tag write v2.3/ISO-8859-1 - player reads v2.3, and chars with double accents appear without any accents, rest are OK
  5. tag resulted by 4., edit in WMP by entering original string - player reads v2.3 and chars are ok again

What else should I look after then?

Here is what I am using:
OS - Windows XP SP3 - all necessary code page conversion tables installed, most importantly (1250 and 1252
player - DWJukebox v3.4.1 (wincab)
tag editor - Mp3tag v2p99a and WMP 9.00

Any direction is appreciated.

Exactly, if any editor converts v4 into v3, which may be lossy, it also needs to convert UTF-8 to UTF-16, which is always a lossless transformation.

Now you made it Chinese for me :slight_smile:
Please translate it to "basic foreign English" so I can try to pick it up..

And if you have any idea on what I wrote one post above, why it is happening?

special characters screwed

How exactly?

I made a series of screenshots according to my 1...5 steps above, hopefully this shows what happens.
Please note, that on the player screen those 2-2 chars (OUou) appearing with tilde (~) and hat (^) instead of double accents (") are font related within the player. The rest comes thru though:

0 - spec. chars as they appear in winword
0_spec_chars
0 - spec. chars as they appear in Mp3tag
0_spec_chars_inMp3tag

Ah... OK, then I split then the rest into separate posts due to new user 3 pcs/post limit

contd.:

1 - tag written by Mp3tag v2.4/UTF8 (note the 30 chars limit)
1_v2p4-UTF8_readv1p1only
2 - tag written by Mp3tag v2.3/UTF16 - total mess
2_v2p3-UTF16_read_v2p3
3 - tag edited in wmp with original string - all ok
3_WMPedit_v2p3_read_v2p3_OK

tbc.

contd.:
4 - tag writen by Mp3tag v2.3/ISO-8859-1 - double accents disappeared
4_2p3-ISO-8859-1_read_2p3_nodoubleaccents
5 - tag edited again in wmp with original string - all ok again
5_WMPedit_v2p3_read_v2p3_OK

end

Ok the player
doesn't read v2.4/UTF8 at all
doesn't understand v2.3/UTF16

so only option is v2.3/ISO-8859-1 but it seems the player doesn't use ISO-8859-1 but the system language/codepage instead
What's your OS language?
For example ISO-8859-1 and cp1250 are not fully compatible

The Id3v2 standard says to use always ISO-8859-1 but some programs in the past have used system codepage instead. An old problem if you use software that cannot handle unicode.

Yes, this is exactly where I got stuck.

OS language is English, keyboard is Hungarian, language for non-Unicode programs is Hungarian, installed codepages 1250 ... 1257, ISO 8859-1 ... -15, UTF7-8, practically all OEM-s and a few more.

Acive codepage is normally 852, but everything seems to be the same even if I change it to 1252 instead

So are you saying there is nowhere to go from here?

MP3diags (link see here:


)
has a function to convert the tag fields encoding to utf-16 assuming the local code page.
You could try that and see if it helps.

And, of course, there is an action im MP3tag to convert the current characterset to unicode:
http://help.mp3tag.de/options_format.html#codepage

I tried MP3diags - seems to be a nice program, and indeed is very useful overall.
I could not find any possibility though to convert v2.4 tags to v2.3 - what would be needed for my player.

Yes, that was my step 4. above as you see, and simply did not help.

OK, I found if I go to tag editor, and change a field then save, MP3diags saves the tag in v2p3/UTF16. So I am back to zero :frowning: .

As dano suggested above, it must have to do with some code page issue hardcoded in the player.

The only way now would be to markup the special characters while still being in V2.4 format e.g. with "==" around them - this should be achievable with a replace run in MP3tag.
Then you save the tags as V2.3 with all the losses that you already know, only this time, it should be possible to replace the specially marked characters with the correct version in V2.3 UTF-16.
WMP reads V2.3 UTF-16 correctly.

I am afraid this method does not deliver the result I need. Even if I replace the marked characters in v2.3/UTF16, as we stated above the player still would not understand it correctly. (Not mentioning that such a replacement cannot be automated, so not much of an option for hundreds of files)

Yes, that is correct.

Where the problem evolves though furhter is that WMP does not care if the file already had a v2.4 and a v1.1 tag, it simply creates a third tag in v2.3, that makes everything even worse.

If I remove all tags first and let WMP create its own and sole v2.3 tag, it writes - strange enough - with Latin 1 encoding. Windows and the player can read it nicely, but then it is again falling quite out of the standard

I can keep using it as a last resort to "save" 30+ long track/album information containing special chars. Not that "elegant way", but so far the only one that works (with all known drawbacks). I hoped somewhere that Mp3tag could solve this, but I understand the developer does not want to support non-standard features.

At the end of the day, if I were able fix the code page issue that prevents v2.3/ISO-8859-1 tags properly showing the double accented chars, that would be the ultimate solution.

But then this is not an Mp3tag support question. Thanks for all your help so far.

OK, I have reached the end of the road :slight_smile: .
Just checked the ISO-8859-1 standard here, and by definition there are missing characters (from several languages actually). In my case, exactly those four above. Period.

There is an old microsoft programm called AppLocale for WinXP which might be able to force Mp3tag to write a different codepage than ISO-8859-1

???
What about an action that replaces exactly these specially marked character sequences?

I would never use WMP as tagging program. As soon as you have user-defined fields in a file, tagging with WMP deletes them, the information is lost.
Also, it is never quite clear, when and if information is actually saved in the files or only in the WMP database.

Perhaps it has not become quite clear:
You load the files into MP3tag in all their V2.4 beauty.
You replace the 4 single characters with something like ==O, ==A etc.
You then save the V2.4 files with V2.3 tags. This should have lead to the loss of the special character information in the old days but now, as you have them marked with ==O etc., you can easily replace all of them with the single correct characters in UTF-16.
This conversion should be necessary only once as from now on you won't be using V2.4 any more.

Using unsupported software that is decades old, is the real problem, though.

http://dwjukebox.com/