Possible memory leak when addressing very large integers in 'json_select_many'

Bug Reports
1

Greetings to all,

I've been trying to get to the root cause of this for a few hours, so please forgive me if this bug title is misleading, as I'm by no means an expert, but here's what I found:

Today, when tagging some tracks, I got an integer value of "-2147483648" when extracting catalog id numbers from iTunes. As this is impossible, I checked the raw data and it was correct.

Looking at the debug file, the values were being parsed from JSON-formatted data, and using json_select_many.
This only happened on some files; and after some rounds of testing, the trend was that only when parsing integers (equal or) above the value "2147483648" would this issue manifest.

This particular number looked familiar, and converting "2147483648" to binary returns the first 32-bit integer (or "1000 0000 0000 0000 0000 0000 0000 0000").

To verify, I wrote myself a test script, including the following code:

...
[ParserScriptAlbum]=...
...
Use "{"array": [{"trackId":2147483645}, {"trackId":2147483646}, {"trackId":2147483647}, {"trackId":2147483648}, {"trackId":2147483649}, {"trackId":2147483650}]}"

json "ON" "current"
json_select_many "array" "trackId" "|" "" -1 1
OutputTo "TEST"
SayRest
...

that when run, returns

imagem
imagem355×355 2.22 KB

To me, this reads that up to the highest 31-bit integer (or up to 2147483647 decimal) everything is fine; on 2147483648 and above there seems to be some sort of overflow (?) which converts any integer of that range into -2147483648 (in binary this would be a 64-bit integer, with the first 32 bits set to "1" and the latter the same as the positive number. Which looks like (in my view) like an overflow (again, I'm no encoding expert).

But this issue seems to be limited to json_select_many; from the above demonstration array, using

json_select_array "array" 6
	json_select "trackId"
json_unselect_object
OutputTo "TEST0"
SayRest

does return '2147483650' in TEST0 as expected, so json_select is working as it should.

This particular test script was only verified on v3.35.1 stable; the original error parsing catalog ids has been verified as early as v3.34.1.

Can anyone duplicate and validate my findings, please?
And for any expert, could you provide a rationale (out of curiosity) as to why/how this happens?

Thank you.

2

Many thanks for the detailed report and the helpful example!

I can confirm your findings. It's not a memory leak, but an integer conversion/overflow issue in json_select_many.

Internally, JSON numbers are parsed in a way that can still represent values such as 2147483648 correctly. However, json_select_many converts numeric JSON values via a 32-bit integer path. Values above 2147483647 therefore overflow and end up as -2147483648. json_select uses a different, overflow-safe formatting path, which is why it returns the expected value.

I'll fix this so that json_select_many will use the same number formatting logic as json_select with the next development build.

Thanks again for tracking this down and for reporting!

1 Like