Mp3tag version 3.23 - json_select_many function (continued)

[rboss]

Hello again, Mp3Tag community.

I'll start by saying the following: my original post regarding the json_select_many function bug WAS indeed fixed as it had been originally discovered (and many thanks to Florian for his work and patience on the matter).

If you're interested you can read about it here:

Mp3tag version 3.22 - json_select_many function

Hello everyone, I think there is a bug with version 3.22, on the json_select_many function. I'll try to explain in detail: using my own script to parse json formatted data from iTunes API, on [ParserScriptAlbum] I use json_select_many to get the "contentAdvisoryRating" value for each track, which classifies it as explicit or not. It's important to note that the "contentAdvisoryRating" key is only present if that track is considered explicit and absent if it is not. In a simple example, the ra…

But, between work + personal commitments + fighting a cold I couldn't devote the proper time to test run my script against the new release until today, which was 2 days after the original thread expired and was closed. So I'm opening a new one regarding the same topic. And here's what I found:

bug ticket #62095 was fixed with the addition of and extra parameter to the json_select_many and json_select_many_count commands; which now (can) include result entries for non-existent items. This is useful for json-formatted data with omitted array values (read previous bug report -above- for details and examples).

However, using either
json_select_many "array" "key" "|" "|" 1000 1 or
json_select_many "array" "key" "|" "" 1000 1
both fail when the data has or includes the last key in the array with a missing value.

An example for clarity.
Let's say an album with 7 tracks has data with keys and values (album title IS "X", track title IS "Y", track number IS "N", etc.) for tracks 1,3,6 and 7. Tracks 2, 4 and 5 have some keys/values absent (this is verified to happen in iTunes API data; I'm unsure about others).

Outputting either json_select_many "array" "key" "|" "|" 1000 1 or json_select_many "array" "key" "|" "" 1000 1 to variable "Output1" produces the following debug line:

output["Output1"]= "value1||value3|||value6|value7"
>>> This is correct, desired, and interpreted properly in the 'Tag information' dialogue.

Now, the same data, with the last key (track 7 in this example) missing the desired key/value:
output["Output1"]= "value1||value3|||value6|"
>>> This is also correct and desired in the debug file; but it's not interpreted properly, because in the 'tag dialogue' window Track 7 shows the entire line of Output1 (and not just the value for the 7th key, which would be [blank]).

I know (from the previous bug submission) that this happens when there's a mismatch between the number of values an output array has vs. the expected number of values for the same output. From the above example (7 tracks), the output of Output1 should be interpreted as
output["Output1"]= "value1|-blank-|value3|-blank-|-blank-|value6|-blank-"
(7 keys matching 3 values + 4 "blank values" = 7 values >> 7/7 > correct)

but is interpreted as
output["Output1"]= "value1|-blank-|value3|-blank-|-blank-|value6|"
(7 keys matching 3 values + 3 blanks = 6 values >> 7/6 > error)

Further testing with the same data but using the command json_select_many "array" "key" "|" "|xxx" 1000 1 will either output (depending whether value7 is present or not)

output["Output1"]= "value1|-blank-|value3|-blank-|-blank-|value6|xxx" or
output["Output1"]= "value1|-blank-|value3|-blank-|-blank-|value6|xxxvalue7"
(7 keys matching 3 values + 3 blanks + xxx[value7] = 7 values >> 7/7 > interpreted correctly, but Track 7 would show "xxx" or "xxxvalue7", which is undesired).

This leads me to conclude that the revised json_select_many function output isn't properly interpreted when the last key in the respective array doesn't have a value. The command itself works as it should (as per the debug file's results), it's the interpretation of that output that causes incorrect values to appear.

Sorry for the lengthy exposition, but I hope I got my point across.

And many apologies to Florian for not being able to detect this behavior in time for the previous bug ticket. @Florian - Also, by the time you read this I'll be sending/have sent you a test script with example IDs for (hopefully) fixing this bug for good.

[Florian]

Many thanks for the detailed bug report. It's really so very helpful to have this detailed description accompanied by a script and data to test with. Thank you!

I've analyzed the issue and it's related to a certain — until now undocumented — requirement by Mp3tag's internal tokenizer which splits a string using | as delimiters into its separate parts. It always expects a trailing delimiter character and if not present, adds it by itself.

Something like
Explicit|Explicit|Explicit|Explicit
becomes
Explicit|Explicit|Explicit|Explicit|
internally and is split into the four parts.

If the last value is the empty string and the provided output is
Explicit|Explicit|Explicit|
the required trailing pipe symbol is already present and the string is split in three parts. The fourth track uses the original value, because there is no dedicated value for this track.

If you want to make sure, that a trailing empty element is always identified as such, you can simply use a trailing delimiter character in all cases, e.g., add
Say "|"
so the result would be
Explicit|Explicit|Explicit||

This string has the trailing delimiter character expected by the tokenizer and is split in four parts. The last part is the an empty element.

I should have added this to the documentation long ago, a negligence which I've now corrected:

For fields that have varying contents for tracks, e.g., TITLE, TRACK, or ISRC, Mp3tag expects the output buffer to contain the individual values separated and finalized by the pipe character |, e.g., Title 1|Title 2|Title 3|.

Thanks again for your detailed and helpful report!