Importing values from generated XML file to matching tracks

So I've been applying ReplayGain using foobar2000 to most of the recent music I've purchased, and have been looking into applying it to older albums, too.

However I dislike the way it modifies the Date Modified timestamps, since I use the Mp3Tag option to preserve the original timestamps and would like ReplayGain tags to be treated similarly.

Thus started my exploring alternative options using Mp3Tag and CLI tools :wink:

So I found a command-line ReplayGain scanning program called BS1770GAIN over on Sourceforge that allows you to import from a file/directory and output ReplayGain results to file.

  1. I use the following command to scan a directory, from a custom Mp3Tag Tools context menu item:
    bs1770gain "$cutRight(%_workingpath%,1)" --replaygain -p --unit db --xml -f "C:\Users\LaurenBacall\AppData\Local\Temp\RG-Results.xml"
    

    This sets ReplayGain as the type of scan, units as dB, and output format as XML. Also placed it in a consistent temp directory so I don't have to delete any left-over files later. The $cutRight() function is used since the program doesn't like the trailing backslash.

  2. I then import that file into a custom tag on all the selected files using an 'Import from File' action.

It's here that I'm stuck. Since I need the whole directory to be scanned to get the Album Peak and Album Gain results all the tracks are listed in the RG-Results.xml output file, with the Album Peak/Gain values listed at the end. What can be done to match the individual files and values in the XML to those selected in Mp3Tag and apply them as tags?

Example output from the file:

<bs1770gain>
  <album folder="Test">
	<track total="12" number="1" file="Todd Terje - It's Album Time - 01 - Intro (It's Album Time).flac">
	  <integrated lufs="-13.14" lu="-4.86" />
	  <sample-peak spfs="-0.14" factor="0.983947" />
	</track>
	<track total="12" number="2" file="Todd Terje - It's Album Time - 02 - Leisure Suit Preben.flac">
	  <integrated lufs="-11.07" lu="-6.93" />
	  <sample-peak spfs="-0.14" factor="0.984466" />
	</track>
	<track total="12" number="3" file="Todd Terje - It's Album Time - 03 - Preben Goes To Acapulco.flac">
	  <integrated lufs="-12.31" lu="-5.69" />
	  <sample-peak spfs="-0.14" factor="0.984100" />
	</track>
	<track total="12" number="4" file="Todd Terje - It's Album Time - 04 - Svensk Sås.flac">
	  <integrated lufs="-13.46" lu="-4.54" />
	  <sample-peak spfs="-0.14" factor="0.983764" />
	</track>
	<track total="12" number="5" file="Todd Terje - It's Album Time - 05 - Strandbar.flac">
	  <integrated lufs="-12.26" lu="-5.74" />
	  <sample-peak spfs="-0.14" factor="0.984283" />
	</track>
	<track total="12" number="6" file="Todd Terje - It's Album Time - 06 - Delorian Dynamite.flac">
	  <integrated lufs="-12.32" lu="-5.68" />
	  <sample-peak spfs="-0.15" factor="0.982971" />
	</track>
	<track total="12" number="7" file="Todd Terje - It's Album Time - 07 - Johnny and Mary (feat. Bryan Ferry).flac">
	  <integrated lufs="-11.32" lu="-6.68" />
	  <sample-peak spfs="-0.15" factor="0.983306" />
	</track>
	<track total="12" number="8" file="Todd Terje - It's Album Time - 08 - Alfonso Muskedunder.flac">
	  <integrated lufs="-9.95" lu="-8.05" />
	  <sample-peak spfs="-0.14" factor="0.983795" />
	</track>
	<track total="12" number="9" file="Todd Terje - It's Album Time - 09 - Swing Star (Part I).flac">
	  <integrated lufs="-13.74" lu="-4.26" />
	  <sample-peak spfs="-0.14" factor="0.983489" />
	</track>
	<track total="12" number="10" file="Todd Terje - It's Album Time - 10 - Swing Star (Part II).flac">
	  <integrated lufs="-13.45" lu="-4.55" />
	  <sample-peak spfs="-0.14" factor="0.983703" />
	</track>
	<track total="12" number="11" file="Todd Terje - It's Album Time - 11 - Oh Joy.flac">
	  <integrated lufs="-13.73" lu="-4.27" />
	  <sample-peak spfs="-0.13" factor="0.985412" />
	</track>
	<track total="12" number="12" file="Todd Terje - It's Album Time - 12 - Inspector Norse.flac">
	  <integrated lufs="-12.03" lu="-5.97" />
	  <sample-peak spfs="-0.14" factor="0.983947" />
	</track>
	<summary total="12">
	  <integrated lufs="-12.24" lu="-5.76" />
	  <sample-peak spfs="-0.13" factor="0.985412" />
	</summary>
  </album>
</bs1770gain>

And the ReplayGain values extracted from the above:

Album Gain:

<integrated lufs="-12.24" lu="<b>-5.76</b>" />
<sample-peak spfs="-0.13" factor="0.985412" />

Album Peak:

<integrated lufs="-12.24" lu="-5.76" />
<sample-peak spfs="-0.13" factor="<b>0.985412</b>" />

Track Gain (for track #01):

<integrated lufs="-13.14" lu="<b>-4.86</b>" />
<sample-peak spfs="-0.14" factor="0.983947" />

Track Peak (for track #01):

<integrated lufs="-13.14" lu="-4.86" />
<sample-peak spfs="-0.14" factor="<b>0.983947</b>" />

Isn't there any possibility given by the rg tool to write the wanted values directly into the files by command line syntax?
http://bs1770gain.sourceforge.net/#cli

Otherwise ...
... once the text of the tool's output log has been stored into a tag-field, e. g. named as BS1770GAIN, ...
then some proper made regular expressions can extract the wanted values from the text stream, ...
e. g. ...

$regexp(%BS1770GAIN%,'^.*<summary total=.*<integrated lufs=.*\slu="(.+?)".*$','$1')

... returns the string value ... '-5.76'

DD.20160222.1100.CET

There is, but two things:

  1. It makes copies of the files rather than applying the tags to the original files.
  2. Doing so obviously modifies the file timestamps since they are new files.

Since I was looking for a way to avoid changing the Date Modified timestamp I sought an Mp3Tag joint solution, since it has the option to preserve timestamps Options>Tags>Preserve file modification time when saving tags.

Thanks for a working example. Tried some regex earlier but didn't have success targeting the value.

How could the values from the single tracks be matched to the files? Some kind of conditional $if() filtering?

Hmm, yes, maybe do ask Peter Belkner to implement a switch for saving the last modified file datetime.

DD.20160222.1124.CET

Will get in touch and see what he says.

Does regular expression have a way of targeting a specific line number? I was thinking, since the line numbers remain consistent in the output (eg: track 1 always has the Gain value on line 4, while track 2 always has the value on line 8, increasing by 4 lines each track) there might be a way of making an Action that looks for the value depending on the track number.

Maybe use the track number ...

$regexp(%BS1770GAIN%,'^.*'$num(%TRACK%,1)'".*<integrated lufs=.*\slu="(.+?)".*$','$1')

Note: This scripting expression does not work as intended, because the first parameter of the function $regexp will be not updated resp. will be not evaluated for each file within a selection of files.

DD.20160222.1141.CET, DD.20160222.1353.CET

Tested and seems to return the same result for each file (from the same last instance of the <integrated lufs line). Also tried changing it to grab the peak value but it returned the same value, though I wouldn't be surprised if I missed something when changing it for the latter :rolleyes:

Emailed the dev and hopefully he sees it. Also have tried talking to Peter Pawlowski (foobar2000 dev) if I recall correctly and he said fb2k wouldn't support preserving the date modified timestamp since it's not correct :expressionless:

Note: If a Mp3tag regular expression does not match, then the result string is the same as the entire input string.
You can suppress this bad result by using an $if clause, which checks for such a result, and can deliver a different result.

If the track number does not fit to identify an existing file in relation to the numbering within the report text file, then try another unique value, e. g. the filename.

I have to rebuild your environment to say anything more.

DD.20160222.1231.CET

I made a bad assumption about scripting within the $regexp function.
Although it is possible to insert a Mp3tag variable into the first parameter of the function $regexp, ...
such embedded Mp3tag variable will be not updated for each underlying file in a selection of files.

A serial modification of the idea can work this way, ...
within an actionsgroup prepare a bunch of similar actions like following example ...

LU <== $if($eql(1,$num(%TRACK%,1)),$regexp(%BS1770GAIN%,'^.+?<track total=.+?\snumber="1".+?<integrated lufs=.+?\slu="(.+?)".+?$','$1'),%LU%) LU <== $if($eql(2,$num(%TRACK%,1)),$regexp(%BS1770GAIN%,'^.+?<track total=.+?\snumber="2".+?<integrated lufs=.+?\slu="(.+?)".+?$','$1'),%LU%) ... LU <== $if($eql(20,$num(%TRACK%,1)),$regexp(%BS1770GAIN%,'^.+?<track total=.+?\snumber="20".+?<integrated lufs=.+?\slu="(.+?)".+?$','$1'),%LU%)

DD.20160222.1402.CET

^ Thanks for the help, DetLevD, always appreciated. Didn't see your edit until now, but while I was about to go to sleep it struck me how I could accomplish it with only minimal regex!

Basically the idea was:

  1. Find the first appearance of the filename string using $strstr()
  2. Cut everything before it using $cutLeft()
  3. Find the first appearance of , and since all the previous text has been removed it will be the first closing tag of the track we want.
  4. Use regular expression to target the value within the remaining text.

Steps 3-4 were combined in my action, so it's not too long overall:


I've attached the Action to this post for anyone interested in a way of adding ReplayGain without changing the file Modified timestamps. Just change the path to suit for the Import file action.

ReplayGain___Apply_from_file.mta (851 Bytes)


ReplayGain___Apply_from_file.mta (851 Bytes)

The only thing that I should look at adding is a way to disable the writing of tags if the filenames don't match, since it would be nice to tell whether the XML is current or old if you're performing this repetitive Action.

At the moment the Action fills in the ReplayGain tags even if the XML is from a different directory/album (though the values match perfectly if it's from the matching directory). I'll try to see if I can figure something out later.

Edit:

Hmm. The command line tool also scans sub-directories, which if contain alternate tracks and different names causes issues. Was generating an XML file for one album and it had an 'OGG' sub-directory with transcoded files and since the filenames didn't match the FLAC versions in the parent directory the Action added the wrong values.

Edit 2:

It looks like certain Unicode are being modified when imported from the XML file, making the formatting choke. Example of trying to match %filename_ext%:


Original filename: Jay Scarlett & Onur Engin Present: Maverick Sessions Two - 04 - Stevo - Pad Pusher.flac

The above Unicode is a replacement colon character which Mp3Tag displays correctly when displaying %filename_ext% by itself but when looking at the imported XML the character has been replaced with three other characters [1]:


The original XML file however preserves the character:


Possibly a bug with Mp3Tag? I have several other Unicode filename replacements that I use, too. Will have to test what it does to them.

[1] :




Tested importing an XML with the following Unicode characters that I use for filename replacement:

∕
:
?
⏐

And Mp3Tag converted the characters to the following:

∕
:
?
⏐

What is going on.

Since my core problem was about preserving the original Date Modified timestamps of my collection I stumbled upon a solution while posting in another thread, thanks to JJ Johnson and DetlevD!

First I add the original date modified timestamp to a custom tag, then use DetlevD's export script to generate and execute a VBS script that restores the original Data Modified timestamps from the custom tags. Works great.