Re: Adding native FLAC padding column and removal

Thanks but no thanks.

I didn't come here in search of alternate solution with dependencies. I came here to ask Florian what was the status of a BUILT-IN solution. A one-click no bullock solution that will work like in Foobar2000. Using outside tools is not a built-in solution; it's a workaround.

Here's the thing. I backup my settings and all that but, in the event I lose my backups and can't recover my setup, I'll have to reprogram this workaround and I ABSOLUTELY DETEST working with any kind of CLI or arguments. It needs to be dead simple for me to go that way.

Cheers.

Just an observation on my side:
a lot of suggestions for functions for which there is something that you call

have never been implemented.
So I think that until the day when your suggestion becomes a function that you like better you have to live with the "workarounds".

Look, 2 years ago I asked Florian for this and he said and I quote...

I came here asking about the bolded part. That is it.
I just didn't want to necro the old thread.

The topic is still on the to-do-list:

This post is from February 2023.
All I am saying: until it gets implemented you are stuck with that what is classified as workaround.

Some discussion about the empty padding in FLAC files after removing cover art has referenced this thread on the MusicBee forum. Seems the search continues for a way to recover some of this file space for those with larger libraries.

It's still not a thing almost a year later. I'm sad. I still have to drop the files in Foobar after tagging to remove the padding. I never got the command-line stuff to work in MP3tag.

If forum members show that something works like in this post:

and you say it does not work, it would be best if you showed us a similar screenshot of the tool definition so that others may spot what the problem may be.

1 Like

I've just released Mp3tag v3.23d with a new entry Utils → Optimize FLAC in the right-click context menu of FLAC files. This will remove the excess padding from the selected FLAC files.

To display existing padding, create a column with value %_tag_size_appended%. This reports the size of the padding for FLAC files, where %_tag_size_prepended% reports the size of the Vorbis Comments and embedded cover art.

1 Like

I have just tested the new function.

Until now I've displayed padding as $div(%_tag_size%,1024) to show the KiB value. Adding 2 more columns with %_tag_size_appended% and %_tag_size_prepended% yielded this result for a test album:
grafik

After using Utils → Optimize FLAC that becomes:
grafik
as expected.
So the function works.

A few things to note are:
As far as I can tell this step cannot be undone within mp3tag (Edit → Undo did not add the padding back).
Removing the padding entirely is pretty dumb in my opinion since that means any added information will lead to a rewrite of the file, slowing down editing to a crawl for negligible space savings.

I think this option would be far more useful if the user could set whether to remove the padding completely (now state) or if the padding should be set to a chosen value (if possible).

My converter of choice (dbpoweramp) handles it this way.
Instead of removing the padding entirely the padding is set to a fixed value of 4KiB if the metadata and embedded images do not blow it up past that point.

Which is why my workflow to remove excess padding is running all flac files through dbpoweramp to convert from flac to flac after I have removed embedded images etc.. That means the wasted 2MB padding per song from an embedded 3000x3000px jpg (for example) are reclaimed but a sensible padding of 4KiB remains. It also has the added benefit of ensuring that all files are encoded with a current version of flac with max compression and it also reveals files containing errors since the converter verifies the files.

So in summary: Removing the padding in mp3tag works, but I wouldn't recommend using it.

Thanks you so much, Florian! Better late than never. Once the next stable release hits, I'm updating for sure.

That's a YOU preference. I do not want any padding created from embedding image files. I much prefer having a single image files easy to change in the folder where my FLAC file(s) are. And adding a few seconds because I removed the padding for the padding for the tags to be recreated is no biggie.

The option is there for those like me.

That can also be an optional option. Don't force your opinion on others.

Padding is not special to image files., in flacs it is only a problem because of the possible notable waste of space as Mp3tag does not clear this space after deleting embedded artwork automatically and artwort is the only tag that really needs notable space. A 4k-padding-reserve makes always sense. The space for the usually used text-tags is vanishingly small. A typical music-flac-file of let's say 4 minutes could have a size of 25 MB. Who really cares in a file like this about an additional 4kb space which would be about 0,016 % additional space and often even does not make a difference because the file-system also saves in 4k-blocks.

If you clear the padding of flac-files with this new util of Mp3tag to zero any change you make to your tags with MP3tag will again create a padding of 4k, no matter whether you add an additional character or delete a character.

2 Likes

Are you sure? Because it really depends on how many FLAC files do you have.
If you change some thousands of FLAC files and they are saved on a NAS, you will have to wait several minutes or more because every file has to be completely rewritten due to the lack of free available (padding) space in your tags.

1 Like

Same. Which is why I remove any embedded images and then convert from flac to flac. The space taken up by the embedded image is reclaimed and only a tiny 4KiB padding remains for tag changes to retain the ability to change the tags within that 4KiB without having to rewrite the rest of the file.

That's what I wrote, is it not?

I don't feel like I've forced anything on anyone. I just stated why I have the opinion concerning padding that I do. I handle 100s of thousands of songs with several users and a multitude of software that accesses my music. Keeping my library performant is paramount to me. But I also like keeping things tidy and hate wasting space. Thus: No embedded images, BUT a 4KiB padding.
Do it your way if you prefer it. Doesn't really bother me if you like waiting on every metadata change.

I know that! I was talking about the padding added by the image files only, not the basic 4096 bytes for the tags. That 4096 bytes is normal and is not the padding I want to remove at all.

Yes? I'm not mass removing the padding (incl. the padding for tags) for a thousand song most of the time. It's usually by album (I'm looking at you, Bandcamp!). The re-adding of the 4096 bytes of padding for tags does really only take an extra second or two. On top of that, my files are also on a NAS. The files being on my local M2 SSD or the HDDs in my NAS really doesn't make a difference in the time it takes to re-add the 4096 bytes padding.

If, for you, it takes more time than that, I suspect you are using a old computer or something? I remember doing the same with my very old Core i7 950 and it didn't take that much longer. (I now have a Ryzen 9 5900X)

To me, it read like "this option shouldn't exist because..."
Keeping the basic 4096 bytes padding would only save a second or two at best per file (again, talking on an album basis - not massive batch editing).

@Florian Thank you for adding this new utility to remove unused padding in FLAC files after removing or adjusting large artwork. This is very helpful in cases where previously this remaining padding kept excessive file sizes for no other reason.

From the flurry of posts following your announcement of this addition, it appears there is already some heated discussion about whether removing 100% of the padding, or if leaving the standard 4k intact is preferred. So perhaps a feature request to enhance this new utility: have an option to remove all of the padding, or keep 4k in reserve.

1 Like

I think that keeping a 4 KiB padding would be the sensible option for most users:

  • An overhead of 4 KiB (at most) isn't much, even for a large number of files, especially with the storage capacities of TBs we have nowadays.
  • The much faster file updating outweights the tiny storage overhead.
  • Ah, and SSD wear (although this issue is supposed to be nonexistent).
  • The format just has been designed to use padding, as the metadata is at the beginning.

For users who want to squeeze every possible byte (as I was before), they still have the possibility to achieve it on their side.

Also, keep in mind the feature built in Mp3tag is only for convenience. I mean that as manually reducing the padding is sometimes required, it should at least be easily doable. It is required as a workaround because removing embedded images leaves an huge padding. In a perfect world, we shouldn't even have to care about manually reducing the padding.

1 Like

Thank you all for your feedback! I didn't expect such heated discussion for such a tiny feature, but apparently I should have put more thought into this before releasing. Thankfully, we're still in beta stage.

I'm very hesitant to add yet another configuration option. I know it's great to be able to configure the exact size of the padding, but it will result in just more configuration options — there are also other tag formats with padding :grimacing:

The discussion above revealed an inconsistency wrt. the new optimization feature: while writing tags, Mp3tag adds 4KB of padding in case the file needs to be rewritten. This should also be — and it's what most feedback here shows — the default when optimizing the padding. So I've just released Mp3tag v3.23e, which does exactly that: if you choose Utils → Optimize FLAC from the right-click context menu of FLAC files, the files will have 4096 bytes of padding after optimization.

I want to address this quickly: removing padding from FLAC after, e.g., removing embedded cover art would require rewriting the file if a new cover is added. I've tried to not be smart about this, because there are many different workflows, which might be disturbed by such automatic behavior. I think you meant your comment not as a suggestion, but wanted to quickly explain for people who are wondering about that.

And maybe there is also a silent group which was very happy with the complete removal of the padding. Please raise your voice so that I know, but also please use the workaround via metaflac for the time being.

4 Likes

I just understood this padding is not empty space added to reach the desired size (the usual meaning of padding), but an overhead having the configured size (+ 4 bytes). Refs the --add-padding operation.

About removing then adding again an embedded picture, the user should expect such operation to be costly. Hey, we are talking about embedding an image (and possibly large) into an audio file. So it would be better to reduce the padding when removing an image, rather than keeping the padding just in expection of this specific use case.

About the padding size, do we really need as much as 4 KiB? Its purpose is to avoid rewriting the entire file when just adding a few characters, or a few fields. Even with 512 bytes, there is plenty of space for several fields… (beware: such discussion about the padding size would certainly become endless, better find a way to settle it)

About this:

In a perfect world, we shouldn't even have to care about manually reducing the padding.

I just meant that, generally speaking, users should have to care only about business logic and data, not implementation details. The "padding case" here is a good example of this.

I think that value might stem from the fact that many file systems have a default block size of 4K. Meaning if you add 512 bytes or 4K is inconsequential as it will "consume" or "waste" one more block anyways. I don't have a source for that tho, it's just what I always thought.

That does sound more consistent, thanks!

Personally, the more option the better. The padding left behind should at least be a bit configurable - like a drop down box: No padding at all (0KB - will end up being rewritten the minute something is added/modified), 4KB and 8KB.

I've seen 8KB is some files and I actually don't know if those files are old or or the tool that was used to encode the files.

Alternatively, a checkbox that says "Leave 4KB of padding when optimizing padding in FLAC files."

I'd love to get the automatic behavior as well which would also take the above option into consideration. A simple checkbox would also fix this: "Automatically trim the padding of FLAC files when removing embedded artwork."

These two options would fit very well in Options -> Tags -> Advanced.

I'd say, for the default behavior, it should be to automatically trim the padding down to 4KB when removing artwork. My mindset is that most users will not know about FLAC padding not being automatically adjusted when removing artwork. That's because, for any other format, it's done automatically. But not FLAC.

Addendum: That goes without saying that the manual trim option should always stay because not all the files will have been handled with MP3tag beforehand.