Failing to handle ZWJ ZWNJ

[rajanna]

format tag field:
Field: _DIRECTORY
Format: %ALBUM%

1. Album (it has ZWNJ): "ರಾಜ್‌ಕುಮಾರ್"
2. Album (it has ZWJ): "ರ‍್ಯಾಂಕ್"

After the formatting is completed. The directory name will be

1. Directory (without ZWNJ): "ರಾಜ್ಕುಮಾರ್"
2. Directory (without ZWJ): "ರ್ಯಾಂಕ್"

Format function drops ZWNJ (u+200C) and ZWJ (u+200D). Both are invisible(non-printing) characters. Mac accepts them when I edit the folder directly. So, it is definitely the mp3tag, "format tag field" that is dropping the invisible(non-printing) characters.

This could be true for other invisible(non-printing) characters Non-breaking space NBSP U+00A0

Please rectify

[rajanna]

@Florian Please review this.

It is a bug, needs to be fixed.

[Florian]

I'm not sure yet, most users are not aware of those special characters and those of Unicode General Category Cc and Cf have caused a lot of hard-to-identify support issues in the past.

[rajanna]

They are not a strange characters, it’s part of standard Unicode set, OS should handle it just fine.

It is working fine in all tags, just the _DIRECTORY field it’s failing

[rajanna]

or At least is there a work around to this issue. I want to batch process folder names in unicode. It is possible they will have ZWJ, ZWNJ and I want to retain them

[rajanna]

@Florian can u please enable at least ZWJ and ZWNJ, it is fully supported in Mac OS filesystem. It is required many scripts in unicode.

[Florian]

I've added support for ZWJ and ZWNJ with Mp3tag for Mac v1.7.3.

[rajanna]

@Florian Thank you very much.

I greatly appreciate the changes

[rajanna]

Also, I tested the renaming files in Mac OS Ventura. There is no issue.