Filter Expression For Non-Ascii Characters

[mrralan]

Is it possible to search for all file names or folder names that contain non-ascii characters in them? Currently, I right-click a folder and get a command-prompt. Then I run...

powershell -Command "gci -recurse . | where {$_.Name -match '[^\u0000-\u007F]'}"

and this works. However, it would be nice to be able to get this info in MP3tag if possible.

[ohrenkino]

see e.g. here:

Filtering European ASCII characters

When I select the above only the first line is copied, and when I try to paste the second line onto the end of the first in the Mp3tag filter, it doesn't seem to work. Also, I'm sure that there's meant to be two commas between the z and the 0 - am I right? In any case, I tried that, and the best I could get was the titles with all numbers in them.

[mrralan]

Thanks for this. One example hard codes a bunch of characters to search for. Not what I am looking for. But this example, ARTIST MATCHES "[\xc0-\xff]"... is this doing the same thing as the powershell command?

[ohrenkino]

@mrralan

But this example, ARTIST MATCHES "[\xc0-\xff]"... is this doing the same thing as the powershell command?

Have you tried that expression?

[mrralan]

@mrralan

ARTIST MATCHES "[\xc0-\xff]"

Yes, but it is not finding all the ascii characters... for example, ö... and substituting [^\u0000-\u007F] does not yield the correct results.

[LyricsLover]

Did you try this syntax:

ARTIST MATCHES [\x{0000}-\x{007F}]

(I don't know what the range \u0000-\u007F should include, maybe you can explain what do you like to filter? Do you mean all this characters

U0000.pdf

inlcuding all the control chars?
Or do you mean filter everything except "a-zA-Z0-9äöüàéèÄÖÜ?")

[mrralan]

I have an mp3 player that cannot display certain non-ascii characters. I want to filter anything that has characters that are NOT in the list 0000 - 007f. Once I can identify which file names have these, I can replace them with the ascii equivalent. For example, I had a folder named, Eydie Gormé. I had to change it to Eydie Gorme. So the filter would just show me all the songs with non-ascii characters. The example ARTIST MATCHES "[\xc0-\xff]" found Eydie Gormé but it did not find another folder name with ö in it.

[ohrenkino]

Or see this thread:

Regular expression to filter path names with diacritics

Some software (in my case I'm thinking of MusicIP) cannot handle file and path names that contain diacritical marks (accents, umlauts, etc.: e.g., é, â, ü, ñ). But you may already have a lot of such paths in your music folders. Here is a filter that will find all these files: NOT %_path% MATCHES ^[a-zA-Z0-9\W_]*$

[mrralan]

NOT %path% MATCHES ^[a-zA-Z0-9\W]*$ is giving me the same results as ARTIST MATCHES "[\xc0-\xff]"... but searching the entire path.

It seems that ö is being found with "[\xc0-\xff]". It's just that I had that character in some TITLES so I had to change it to TITLE MATCHES "[\xc0-\xff]".

What is still not being detected is Apostrophe character. The single quote ' is in the basic latin block (0000-007F), the Apostrophe isn't. I need the search the find these also and then I can replace them with a single quote.

[dano]

* MATCHES [^\x{0000}-\x{007F}]

To check in path and all tags.

[ohrenkino]

@mrralan

What is still not being detected is Apostrophe character

Are you sure that it is the genuine apostrophe that you are looking for?
See e.g. here:

Apostrophe vs. Accent aigu

It depends on where you use this apostrophes. If you want to contribute for example to MusicBrainz, you should follow their rules: Use of basic ASCII punctuation characters such as ' and " is allowed, but typographically-correct punctuation (such as ’ for the English apostrophe) is preferred. Source: https://musicbrainz.org/doc/Style/Miscellaneous

[mrralan]

Yes, the Apostrophe is what I meant. Some of them use the accent in words like "wasn't" or "don't" and these don't work. The only acceptable characters are the ones that fall within the range '[^\u0000-\u007F]'}".

It looks like we have a winner with dano's expression! * MATCHES [^\x{0000}-\x{007F}] is working the way I want. I don't know why I couldn't get the results with LyricsLover's expression... it appears to use the same range syntax. Thanks everyone!

I use various filters frequently. Pulling them from history is okay until they roll off. Is there any way to store filters that are used a lot so they will always be there?

[LyricsLover]

@mrralan

I don't know why I couldn't get the results with LyricsLover's expression

I assume it is not the used unicode range that causes a problem.
With * in front of MATCHES you are looking in ALL tags.
My suggestion was only looking in the ARTIST tag.

So it seems that your apostrophe is somewhere else then in the ARTIST tag.

Update: (18:50 CET)
After reading the documentation again, I think the first ^ character in
[^\x{0000}-\x{007F}]
makes the difference, because it indicates a Negation.
Definition of Negation:

If the bracket-expression begins with the ^ character, then it matches the complement* of the characters it contains, for example [^a-c] matches any character that is not in the range a-c.

*As German speaker I had to translate the meaning of "complement".
Its some kind of "anything other than".

[LyricsLover]

@mrralan

Is there any way to store filters that are used a lot so they will always be there?

Yes, you can click on the small arrow-to-right on the right end of the filter input box

and then on "Manage History".