[WS] RateYourMusic

User_name · October 22, 2022, 2:44pm

It's much better with the first index window for artists. It's a great job. Thanks a lot.

I saw this, I don't know if it's intentional: line 150, there is no space after the semicolon separator, while there is one on the other script, ";" instead of "; ".

nathbot · October 22, 2022, 3:11pm

Not intentional! I added the missing space and updated the zip above.
Well spotted

User_name · October 24, 2022, 11:24pm

There is an error with the code I gave if there is no secondary genre on the album page.
I started over and copied the code that was already there.
Here's what it looks like: https://paste.myst.rs/i61pejs7
I think the Descriptors line is there all the time, even if it's empty. So no need for an "if" for this one.

nathbot · November 6, 2022, 12:12pm

Update:
RateYourMusic 2022-11-06

It integrates the fix from User_name on separate RYM primary/secondary genre tags,
fixes an issue I encountered with an album where the "Released" date had no link tag on the year,
and fixes a bug when getting artists in some Various Artists releases.

etownman1985 · December 2, 2022, 3:08am

I was wondering if it is possible to have the genre field just display the first genre if that is possible.

waterproof · January 28, 2023, 1:47pm

@nathbot Thanks for your work on the script !

I can't get it to work, though.

If I search the album I get : ERROR(L96): unknown command 'killtag'
If I search the artist data, I get : ERROR(L39/findinline): text not found.

nathbot · January 28, 2023, 2:40pm

Hi @waterproof ,
that's odd... so please don't mind me asking some obvious questions (beyond "Have you tried turning it off and on?" which unfortunately can't apply here).

Do you have the latest version of the script? (see my last message with version 2022-11-06 above) I'm asking because there is no killtag command on line 96... I'm not 100% sure the line number is reliable in error messages but who knows.
Which version of Mp3tag do you have? "Killtag" is definitely an existing command (cf Developing Tag Sources – Mp3tag Documentation) but perhaps it was added in a version more recent than yours... I had version 3.18 when I worked on the script in November, now I got 3.19 and it's still working for me.
If 1) and 2) check out, do you mind saying which album you searched for so I can try to reproduce the issue? With all the different types of albums, I may have missed a case during my last fixes.

Florian · January 28, 2023, 2:42pm

It's pobably from the Mac version. I've just checked and killtag is not (yet) implemented there.

waterproof · January 28, 2023, 3:03pm

That is it. Any plans to implement it in the near future ?

@nathbot is "Killtag" necessary to fetch genre tags from RYM or could I edit the script to make it work ?

Florian · January 28, 2023, 3:13pm

Yes, I'll add it with the next release now that I know it's used out in the wild.

nathbot · January 28, 2023, 4:10pm

Thanks @Florian!
In the meantime @waterproof, I tried using RegexReplace as an alternative. I did it quick and dirty so it might not be full-proof, but if it unblocks you until the next Mac update... try replacing the whole source code with this:

Click to expand code

# Mp3tag (v2.40+) parsing for RateYourMusic
# created by dano
#
# This file should be in your sources dir. 
# On Windows XP it's C:\Documents and Settings\*username*\Application Data\Mp3tag\data\sources
#	
#	-> DO NOT COPY/MODIFY AND REDISTRIBUTE THIS WITHOUT PERMISSION <- 
#
# update for RYM 2022 by nathbot
# v5.0
# v5.1
# add RYM primary genres, secondary genres, description; fix for Various Artists
# 2022-11-06
# Fix details import when Released date has a year with no link
# Integrate fix from User_name on separate RYM primary/secondary genre tags
# Fix bug when getting artists in Various Artists releases

[Name]=RateYourMusic
[BasedOn]=rateyourmusic.com
[IndexUrl]=http://rateyourmusic.com/httprequest?searchterm=%s&type=l&page=1&action=Search&rym_ajax_req=1&request_token=
[AlbumUrl]=http://rateyourmusic.com
[WordSeperator]=+
[IndexFormat]=%Cover%|%Artist%|%_url%|%Album%|%Year%|%Type%|%Publisher%|%Description%
[SearchBy]=%album%
[Encoding]=url-utf-8

[ParserScriptIndex]=...
# ###################################################################
#					I  N  D  E  X
# ###################################################################
# debug "on" "debug_RateYourMusic_search.out" "10"
# debugwriteinput "RateYourMusic_search.html"


findline "<table><tr class=\"infobox\">" 
do	
	# Cover
	findinline "<table><tr class=\"infobox\">" 
	moveline 1
	findinline "<td class=\"page_search_img_cell\">"
	if "<div"
		moveline 2 1
		findinline "data-src=\"" 1 1
		if "//"
			say "Yes"
			moveline 4 1
		endif
	else
		moveline 2 1
	endif
	say "|"
    
	# Artist
	# Always get artist name in latin characters
	RegexpReplace "(<a [^\<\>]+? class=\"artist\">)([^\<\>]+?<span class="subtext">\[)(.+?)(\]</span>)" "\1\3"
	# In case there are two artists, merge them into one name
	RegexpReplace "(class=\"artist\">[^\<\>]+?)</a> / <a [^\<\>]+? class=\"artist\">([^\<\>]+?</a>)" "\1 / \2"

	findinline "class=\"artist\">"
	sayuntil "</a"
	say "|"
	
	# URL
	GoToChar 1
	findinline "<i><a"
	findinline "href=\""
	sayuntil "\""
	say "|"
	
	# Album
	GoToChar 1
  findinline "searchpage\">"
	sayuntil "<"
	say "|"			
	
	# Year
	moveline 2 1
	findline "width:4em;\">"
	findinline ">"
	sayuntil "<"
	say "|"
	
	# Type
	moveline 2 1
	findinline ">"
	sayuntil "<"
	say "|"
	
	
	## Publisher	
	moveline 2 1
	sayregexp "(?<=\"label\">)[^<]+(?=<)" ", " 
	say "|"
	
	## Desc

	moveline 1 1
	findinline "</td>" 1
	if "<tr><td colspan=\"5\">"
		RegexpReplace "</?b[^>]*>" ""
		RegexpReplace "</?span[^>]*>" ""
		RegexpReplace "</?a[^>]*>" ""
		gotochar 1
		findinline "colspan=\"5\">" 1 1
		sayuntil "</td"
	endif

	saynewline
	
	moveline 1 1
	moveline -1 1
	findinline "</table></td></tr></table></td></tr></table>" 1 1
	# unspace
while " <br /><table>" 100

[ParserScriptAlbum]=...
# ###################################################################
#					A  L  B  U  M
# ###################################################################
# debug "on" "debug_albumdetails.out" "10"
# debugwriteinput "RateYourMusic_albumdetails.html"


# #######################
# Album
# #######################
outputto "Album"
findline "<div class=\"release_page\"  itemprop=\"mainEntity\" itemscope itemtype=\"http://schema.org/MusicAlbum\">"
findline "itemprop=\"name\""
findinline "<meta content=\""
sayuntil "\""

# #######################
# RateYourMusic Album URL
# #######################
outputto "RateYourMusic Album URL"
say "http://rateyourmusic.com"
findline "itemprop=\"url\""
findinline "<meta content=\""
sayuntil "\""

# #######################
# RateYourMusic Cover
# #######################
findline "alt=\"Cover art for "
moveline -1 1
findinline "src=\""
outputto "Coverurl"
say "http:"
sayuntil "\""

# #######################
# RateYourMusic Artist URL
# #######################
outputto "RateYourMusic Artist URL"
say "http://rateyourmusic.com"
gotoline 1
findline "itemtype=\"http://schema.org/MusicGroup\">"
findinline "itemprop=\"name\" href=\""
sayuntil "\""


gotoline 1
findline "<table class=\"album_info\">"

# #######################
# Year
# #######################
findline "<th class=\"info_hdr\">Released"
RegexpReplace "</?b[^>]*>" ""
RegexpReplace "</?a[^>]*>" ""
findinline "<td colspan="2">"
findInLine " "
findInLine " "
OutputTo "YEAR"
SayNextNumber

# #######################
# Full date
# #######################
findline "<th class=\"info_hdr\">Released" 1
# ## Remove tags in date
RegexpReplace "</?a[^>]*>" ""
RegexpReplace "</?b[^>]*>" ""
# ### change order to year-month-day
Replace "   " " "
RegexpReplace "([0-9]+) ([A-Za-z]+) ([0-9]+)" "\3-\2-\1"
# ### replace month string by numbers
Replace "January" "01"
Replace "February" "02"
Replace "March" "03"
Replace "April" "04"
Replace "May" "05"
Replace "June" "06"
Replace "July" "07"
Replace "August" "08"
Replace "September" "09"
Replace "October" "10"
Replace "November" "11"
Replace "December" "12"
# ## Add leading zero to days 1-9
RegexpReplace "([0-9]{4}-[0-9]{2}-)([0-9][\s]*<)" "\10\2"
OutputTo "FULLDATE"
findinline "<td colspan=\"2\">" 1
SayUntil "</td>"

# #######################
# Genre(s)
# #######################
gotoline 1
findline "class=\"release_genres\""
# ## Primary genres
findline "<span class=\"release_pri_genres\">"
OutputTo "GENRE"
sayregexp "(?<=/\">)[^<]+(?=</a)" "; "
# ## Secondary genres
findline "<br />"
findinline "<br />"
if "<span class=\"release_sec_genres\">"
	findline "<span class=\"release_sec_genres\">"
	findinline "<span class=\"release_sec_genres\">"
	if "<a "
		say "; "
		sayregexp "(?<=/\">)[^<]+(?=</a)" "; "
	endif
endif

# #######################
# Separate RYM Primary Genre tag (courtesy of User_name)
# #######################
gotoline 1
findline "class=\"release_genres\""
findline "<span class=\"release_pri_genres\">"
OutputTo "RATEYOURMUSIC PRIMARY GENRE"
# Note from nathbot: here the ";" separator can be changed to whatever you prefer. For "\\", enter "\\\\".
sayregexp "(?<=/\">)[^<]+(?=</a)" ";"

# #######################
# Separate RYM Secondary Genre tag (courtesy of User_name)
# #######################
findline "<br />"
findinline "<br />"
if "<span class=\"release_sec_genres\">"
	OutputTo "RATEYOURMUSIC SECONDARY GENRE"
	findline "<span class=\"release_sec_genres\">"
	# Note from nathbot: here the ";" separator can be changed to whatever you prefer. For "\\", enter "\\\\".
	sayregexp "(?<=/\">)[^<]+(?=</a)" ";"
endif

# #######################
# Descriptors (courtesy of User_name)
# #######################
gotoline 1
OutputTo "RATEYOURMUSIC DESCRIPTORS"
findline "<span class=\"release_pri_descriptors\">"
Replace "<span class=\"release_pri_descriptors\">" ""
# Note from nathbot: here the ";" separator can be changed to whatever you prefer. For "\\", enter "\\\\".
Replace ",  " ";"
SayUntil "</span>"

gotoline 1
findline "<table class=\"album_info\">"
findline "<span itemprop=\"byArtist\""

# Artist and tracks

# Always get artist name in latin characters
RegexpReplace "(<a [^\<\>]+? class=\"artist\">)([^\<\>]+?<span class="subtext">\[)(.+?)(\]</span>)" "\1\3"

findinline "class=\"artist\""
movechar 1
if "Various Artists"

	############################
	# Various Artists
	############################

	outputto "ALBUMARTIST"
	sayuntil "</a"

	# Tracks
	gotoline 1
	findline "section_main_info section_outer"
	findline "show-for-small"
	MoveLine 1
	findline "show-for-small"
	MoveLine 1
	findline "show-for-small"
	findInLine "show-for-small"
	if "\"><div class=\"section_tracklisting\">"
		findline "section_tracklisting"
		findline "id=\"tracks_mobile\""
		JoinUntil "track_preview_tracks_mobile"

		# Always get artist name in latin characters
		RegexpReplace "(<a [^\<\>]+? class=\"artist\">)([^\<\>]+?<span class="subtext">\[)(.+?)(\]</span>)" "\1\3"

		# Clean up spaces between tags
		RegexpReplace "[\s\t]+<" "<"
		RegexpReplace ">[\s\t]+" ">"
		# Get rid of "disc one", "disc two"... lines
		RegexpReplace "<span class=\"rendered_text\"><strong class=\"rymfmt\">Disc[^\<\>]+?</strong></span>" ""
		# Remove tables
		RegexpReplace "<table[^\<\>]+?>.+?</table>" ""
		# Multiple artists on the track: join them with ;
		RegexpReplace "</a>(&amp;|and|,)<a [^\<\>]+? class=\"artist\">" "; "
		RegexpReplace "</a>/<a [^\<\>]+? class=\"artist\">" "; "

		# Remove some unused stuff
		RegexpReplace "<div class="credits_roles">[^\<\>]+?</div>" ""

		GoToChar 1

		# Track artist

		outputto "Artist"
		
		do

			findInLine "<span class=\"rendered_text\">"
			if "<a"
				findInLine "class="\artist\">"
				SayUntil "</a>"
				movechar 4
				if "&amp;"
					Say " "
					SayUntil " - "
				endif
			else
				# No artist
				Say "No Artist"
			endif
			Say "|"
			findInLine "</li>"
			if "<li class=\"track\">"
			else
				findInLine "</li>" 1 1
			endif

		while "<li class=\"track\">"


		GoToChar 1
		# Erase any artist name before we get track names
		RegexpReplace "(<span class=\"rendered_text\">)<a[^\<\>]*? class=\"artist\">[^\<\>]+?</a>([^\<\>]+?</span>)" "\1\2"
		RegexpReplace "(<span class=\"rendered_text\">)&amp;[^\-]+?- " "\1"
		RegexpReplace "(<span class=\"rendered_text\">)- ([^\<\>]+?</span>)" "\1\2"

		# Track title
		outputto "Tracks"
		
		do

			findInLine "\"rendered_text\">" 1
			SayUntil "</span>"

			# Is there a Featuring?
			findInLine "<div style=\"clear:both;\"></div></div>"

			if "</li>"
				# No credits
			else
				findInLine "<ul class=\"credits\">"
				if "<li class=\"featured_credit\"><b>feat.</b>"
					Say " (feat. "
					findInLine "class=\"artist\">"
					SayUntil "</a>"
					Say ")"
				endif
				findInLine "</ul>"
			endif

			Say "|"
			findInLine "</li>"

		while "<li class=\"track\">"
	else
		# no tracks
	endif

else
	outputto "ALBUMARTIST"
	############################
	# Not Various Artists
	############################

	# Album artist

	# In case there are two artists, merge them into one name for Album Artist
	gotochar 1
	RegexpReplace "(class=\"artist\">[^\<\>]+?)</a> &amp; <a [^\<\>]+? class=\"artist\">([^\<\>]+?</a>)" "\1 / \2"
	RegexpReplace "(class=\"artist\">[^\<\>]+?)</a> / <a [^\<\>]+? class=\"artist\">([^\<\>]+?</a>)" "\1 / \2"
	
	# Write tag
	# findinline "class=\"artist\">"
	outputto "ALBUMARTIST"
	# sayuntil "</a>"
	# sayregexp "(?<=class=\"artist\">)(.*?)(?=</a>)" ";"
	
	do
		findInLine "class=\"artist\">"
		sayuntil "</a>"
		findInLine "</a>"
		if "<br />"
			say ";"
		endif
	while "<br />"

	# Tracks
	############
	gotoline 1
	findline "section_main_info section_outer"
	findline "show-for-small"
	MoveLine 1
	findline "show-for-small"
	MoveLine 1
	findline "show-for-small"
	findInLine "show-for-small"
	if "\"><div class=\"section_tracklisting\">"
		findline "section_tracklisting"
		findline "id=\"tracks_mobile\""
		JoinUntil "track_preview_tracks_mobile"

		# Always get artist name in latin characters
		RegexpReplace "(<a [^\<\>]+? class=\"artist\">)([^\<\>]+?<span class="subtext">\[)(.+?)(\]</span>)" "\1\3"

		# Clean up spaces between tags
		RegexpReplace "[\s\t]+<" "<"
		RegexpReplace ">[\s\t]+" ">"
		# Get rid of "disc one", "disc two"... lines
		RegexpReplace "<li class=\"track\"><div class=\"tracklist_line\" style=\"width:100%;\"><span class=\"tracklist_num\"></span><span class=\"tracklist_title\"><span><span class=\"rendered_text\"><b>[^\<\>]+?</b></span></span><span class=\"tracklist_duration\" data-inseconds=\"0\"></span></span><div style=\"clear:both;\"></div></div></li>" ""
		# Remove tables
		RegexpReplace "<table[^\<\>]+?>.+?</table>" ""
		# Multiple artists on the track: join them with ;
		RegexpReplace "</a>&amp;<a [^\<\>]+? class=\"artist\">" "; "
		RegexpReplace "</a>/<a [^\<\>]+? class=\"artist\">" "; "

		# See if tracks have individual artist (like in a split with 2 artists)
		findinline "<span class=\"rendered_text\">"
		if "<a title=\"[Artist"
			outputto "Artist"
			RegexpReplace "<span class=\"rendered_text\">([^\>\<].+?)</span>" "<span class=\"rendered_text\"><a class=\"artist\">No artist</a> - \1</span>"
			RegexpReplace "(<span class=\"rendered_text\"><a [^\<\>]+?) (class=\"artist\">)([^\<\>]+?<span class="subtext">\[)?(.+?)(\]</span>)?" "\1 class=\"main artist\">\4"
			gotochar 1
			findinline "<li class=\"track\">"
			do
				findInLine "class=\"main artist\">"
				sayuntil "</a>"
				findInLine "</li>"
				say "|"
			while "<li class=\"track\">"
			# sayregexp "(?<=class=\"main artist\">)(.*?)(?=</a>)" "|"
		else
			outputto "Artist"
			sayOutput "ALBUMARTIST"
		endif

		GoToChar 1
		# Erase any artist name before we get track names
		RegexpReplace "(<span class=\"rendered_text\">)<a[^\<\>]*? class=\"main artist\">[^\<\>]+?</a>- ([^\<\>]+?</span>)" "\1\2"

		# Track title
		outputto "Tracks"

		do

			findInLine "class=\"rendered_text\">"
			SayUntil "</span>"

			# Is there a Featuring?
			findInLine "<div style=\"clear:both;\"></div></div>"

			if "</li>"
				# No credits
			else
				findInLine "<ul class=\"credits\">"
				if "<li class=\"featured_credit\"><b>feat.</b>"
					Say " (feat. "
					findInLine "class=\"artist\">"
					SayUntil "</a>"
					Say ")"
				endif
				findInLine "</ul>"
			endif

			Say "|"
			findInLine "</li>"

		while "<li class=\"track\">"
		
	else
		# no tracks
	endif
endif

waterproof · January 30, 2023, 9:45am

Thanks @nathbot !

Now the request goes through with some albums, but most of the time I get : ERROR(L59/findinline): text not found. When the request is activated, I almost never get the right album in the list of suggestions, though.

Florian · February 3, 2023, 11:14am

While working on adding support for the KillTag command on macOS, I've noticed a small issue with the script:

findInLine "class="\artist\">" should be
findInLine "class=\"artist\">"

Florian · February 6, 2023, 10:19am

This is now fixed with Mp3tag for Mac v1.7.4. Thanks for pointing!

chthonic · March 17, 2023, 9:07pm

I'm getting this error when I try to find artists with Japanese characters in them. I tried with only english letters and it works fine. It searches the artist, finds the artist, but gives out this error when selecting the artist to get the metadata from.

Screenshot 2023-03-17 140530

nathbot · March 18, 2023, 11:57am

Update: [RateYourMusic 2023-03-18]
(Dropbox - RateYourMusic_20230318.zip - Simplify your life)

@waterproof Today I've been getting 0 results on all my album searches with a "text not found" error on line 59 like you. Here is an update that fixes it, let me know if it's still not OK for you.

@chthonic I can reproduce your problem, it's clearly an encoding issue, even though the debug info for the search results does show the URL correctly encoded for Japanese results:
Yes|Mao Denda|/release/single/傳田真央/one-last-kiss/|One Last Kiss|2001|CD||
(not exactly the URL you were trying to get, but I don't think it matters)

I'm afraid I don't know how to fix it...

@Florian Can you help, by any chance?

Florian · March 22, 2023, 4:40pm

Yes, it's an encoding issue that seems to be special to this site. The problem is, that the fragment /release/single/傳田真央/one-last-kiss/ is not part of a valid URL.

Mp3tag expects the invalid parts of a URL (the Japanese characters in this case) to be percent-encoded at this stage. I'm not sure how to handle this yet — without breaking other things.

I've also noticed that the issue I've mentioned above is still present with this version of the script and that it produces an error if the text in L447 isn't found.

zeka · May 25, 2023, 11:20am

Hey there, I'm super new to regex. I've been trying to edit this script all evening to no avail.

Is there a way to edit the script so that it doesn't fetch secondary genres and to only fetch the first primary genre listed?

Thanks in advance!

ohrenkino · May 25, 2023, 12:36pm

If you do not succeed with regular expressions in the script ...
You could create an action to remove duplicate fields after you have used the script:

User_name · May 25, 2023, 2:43pm

I just did this to get only the first primary genre.
I haven't tested it much, and I don't know much about scripting, so there may be errors.

Click to expand code

# #######################
# First RYM Primary Genre tag
# #######################
gotoline 1
findline "class=\"release_genres\""
findline "<span class=\"release_pri_genres\">"
OutputTo "RATEYOURMUSIC FIRST PRIMARY GENRE"
findinline "<span class=\"release_pri_genres\">"
if "<a  class=\"genre\""
	findinline "a  class=\"genre\""
	findinline "\">"
	sayuntil "</a>"
endif