Version 2.89a - Win7 32Bit Memory Issues when scanning collections with Column of %_cover_width_% active

bug-fixed

#1

Florian,

Tonight, I was using the new %cover_art% and %cover_width% column display for purposes of finding files in my 106,000+ collection of flac files that did not have coverart.

Note that I am not using the new library database feature.
I was not loading all 106,000 files at one time. The most I was able to load was say 31,000 CD quality tracks at one time with no issues.

I did notice something weird that I had not seen prior to using this new feature and indeed ran into a few out of memory exceptions in the sflacscanner cpp file.

I start scan of 9850 High Resolution flac files with cover art (large file sizes for most coverart) and MP3 tag loads successfully. Upon completion, Private memory is at 649,948KB.

If I next drag a normal CD folder with 300x300 coverart into the MP3Tag window, the 11 tracks display as expected but system memory does not release. It stays at 659,948KB. It would appear that the only way I can release the memory is to exit the program. Minimizing and restoring the window has no effect.

I can be reached by email if additional information is required or you need more steps to reproduce.

Thanks,
Lyndel McGee


#2

I realized after I posted that I was not completely clear with the reproduction steps. Once memory grows to a large number, I continue loading groups of music files. What I have noticed is that on occasion, the memory will recliam 1-3MB (1,000KB). However, what I see more often than not is that the memory continues to grow until such time as I am out of memory. This behavior makes me think that a cover cache/hashtable/hashmap is not being cleared but as I don't have the source code, I cannot check this.


#3

Can you try the same with the Library enabled at "Options > Library"?


#4

Will be a few days but I will try and get back to you. I run a portable install so will revert to non-library when done.


#5

Florian,

Here are the results with 'Use Library' active and not specifying any folders (I have music in many places and multiple backup copies so I did not specify any specific locations). Note that using Library, I had no crashes and the memory sizes are below.

  1. MP3Tag starts initially and after loading approx 9800 Hi-Res tracks, the memory grew to 48,000KB. At this point, I activated 2 columns for display (cover_width and cover_type). (compare this earlier non-library-based result of "Upon completion, Private memory is at 649,948KB".

  2. Without closing MP3Tag, I then scanned a very small folder. The memory dropped back to 21,000KB.

  3. Again, without restarting, I scanned a folder with approx 30,000 Low-Res CD tracks with jackets no larger than 300x300 pixels. Memory grew to 84,000KB.

  4. After many one-off scans, I then scanned a High-Res music folder that has 34,494 files in it with many of these files having cover art that is >1MB in size. At the 60% mark, memory was 123,884KB and was vacillating a bit but slowly growing. At the end of the scan, the memory size is at 180,680KB and completed with no issues. Scan of this folder was not possible without Library Mode Active.

Based on these results, the 'Library Mode' appears to handle the volume better especially when cover column info is in use and the coverart can be large (>1MB in size).

I didn't expect a cover to take up memory once the information needed from it (width or type) was loaded from the file. Once info loaded, I thought that the cover memory could be freed and then read from the file as needed. That is why I opened the original report about a possible hashtable/map that could be leaking.

Let me know if you need something else on this one. But, I suspect you will close this out as working as designed. :slight_smile: with a recommendation to always use Library Mode.

Thanks,
Lyndel


#6

One final comment. Without restarting again, I scanned a very small folder Memory dropped from 180,680KB back down to 35,500KB as expected. Note that if Library Mode is not active, based on my initial report, I observed that memory was not reclaimed. One must exit and restart MP3Tag.


#7

Many thanks for the detailed description and for testing the issue. Glad that it's working as expected if the Library is enabled.

However, I'm also not satisfied with the memory footprint if the Library is disabled and just wanted to give you a short feedback: I did some analyzes and could track down the overhead to an in-memory database I'm using. It's not leaked memory, but memory that is only freed when the application is closed.

I'm currently working on an improved solution that frees this memory when the directory is changed. I'll keep you posted.


#8

I've fixed this with Mp3tag v2.89b. Thanks again for reporting!