I see your point ...
and I really had to think some time whether your idea was so obvious that it is a pity that the function does not already work like it.
And then I remembered this thread:
The suggested format string to remove duplicates from genre was originally used for a different field.
And then I rememberd that the suggested filter expression to find the number of genres was originally used to determine the folder depth ...
So it is not always that the format strings are used only in 1 context.
Which means that we end up with the unavoidable request for an "option". Or perhaps a different sorting (first those for the current field, then all the others).