[WS] Beatport.com by stevehero [Release, Single Track & Artwork tagging]

iamyou · July 12, 2023, 3:01pm

It's as if Beatport followed Musk's stance of restricting 'scrapers' of its data/information,
and so Beatport also figured out that track length information was unique enough
to match song titles and names with.
Then, they decided to bury track length even further into the code.

Cuchi · July 12, 2023, 4:55pm

Hi dude,

When can you help us with the script update and how can we support you?

bitskream · July 12, 2023, 5:06pm

It looks like the reorganized info is now more efficiently packed into a JSON data structure, all on a single line. That means the original scripts cannot parse it, though it does look like all the info is there. My regex knowledge is insufficient to mod the scripts to extract just the JSON record, as extracting the info from that should be relatively straightforward. Track length info is still present, and even Camelot Key info is present now, so the previous script code to convert key format will become redundant.

Wannastayunercover · July 12, 2023, 7:12pm

Looking at it ATM, I guess steve should amend his code pretty quickly since json support in mp3tag is straightforward and code should be lighter. Btw I saw a "score" section in data could be usefull to carry it on for automation tools.

stevehero · July 12, 2023, 10:29pm

A link has always been in the starting post

Most of the code used the JSON parsing methods available where it could. I'll have to have a closer look. Hopefully, they've added more data and whatnot to the JSON. I'll try to get working on it as soon as I can.

DaveSumm · July 13, 2023, 12:28am

Beers coming your way if you manage to fix this. Even more beers if you can figure out how to search within a publisher (I mentioned it earlier)

Cuchi · July 13, 2023, 3:27am

Donation sent for a coffee

stevehero · July 13, 2023, 6:04pm

Big coffee that. Thank you. Will get started later on. I'm going to change that donation link to revolut as PayPal skim around 13% off which is bad for both you and I.

So if you feel like making a donation, hold off for a bit until I change that.

Wannastayunercover · July 13, 2023, 6:32pm

Steve , tell me how do you reach object queries I'm stuck at this point.

Started with :
regexpreplace ".application/json">({.)</script><iframe.*" "$1"

json "ON" "current"

json_select_object "props"
json_select_object "pageProps"
json_select_object "dehydratedState"
json_select_object "queries"

I've forgotten asterisks wouldn't display, they are in the code

stevehero · July 13, 2023, 7:23pm

Regex needs a bit of work.

Wannastayunercover · July 13, 2023, 8:23pm

whatever body too short they say

Cuchi · July 13, 2023, 8:47pm

Also consider USDT TRC20 to be able to donate since I do not have access to REVOLUT

stevehero · July 13, 2023, 8:51pm

This'll get you going.

# Remove non JSON from HTML, leaving only valid JSON
regexpreplace "<!DOCTYPE\s+.+?json\">" "" # Remove before JSON
regexpreplace "</script><noscript>.+?</html>" "" # Remove after JSON

then paste the output after this in the output file into https://jsonlint.com/. It will format the JSON so you can make out what's, what. Hint queries is an array. You need to use the right method, not forgetting to give it an index.

You'll see how I did in the the script when I have it done. But work with this info. If I show you fully you may not learn as well.

Will probably not be today or tomorrow I get this done. Off work, car to get serviced, golf and movie are planned.

Anyway:

houselanguage · July 13, 2023, 8:59pm

Take your time Big B! Patiently waiting for the new update like most of us! Thanks!

owlcatz · July 13, 2023, 9:05pm

Thanks again Steve no worries waiting here. Appreciate your responsiveness!

Wannastayunercover · July 13, 2023, 10:27pm

on its way too

stevehero · July 14, 2023, 12:00am

95% done but I have to sleep.

The Camalot key is available in the JSON which is a nice have along with the ISRC and UPC.

bitskream · July 14, 2023, 12:14am

Excellent! Share your Revolut details when you get a minute!

thesuperintendent · July 14, 2023, 9:08am

Awesome work again @stevehero

Wannastayunercover · July 14, 2023, 9:26pm

for the ones who may want to modify the already existring script :

here the json part for track search

[ParserScriptIndex]=...

debug "on"  "c:\BEATPORT track search debug index.html" 20
DebugWriteInput "c:\debug-input.out"

findline "<!DOCTYPE html>"
findinline "<!DOCTYPE html>"
joinuntil "</body>"
replace "|" "$verticalBar()"

regexpreplace ".*application\/json\">({.*)<\/script><noscript><iframe.*" "$1"

json "ON" "current"

json_select_object "props"
json_select_object "pageProps"
json_select_object "dehydratedState"

json_select_array "queries" 1
json_select_object "state"
json_select_object "data"

json_foreach "data"
json_select "release_date" # RELEASED
sayrest
say "|"
json_select_object "release" # RELEASE
json_select "release_name"
sayrest
say "|"
json_unselect_object

json_select "length" # LENGTH
sayrest
say "|"

json_select "track_id" # ID
sayrest
say "|"

json_select "track_name" # TITLE
sayrest
say " ("
json_select "mix_name"
sayrest
say ")"
say "|"

json_select_many "artists" "artist_name" ", "  # artists
sayrest
say "|"

json_select_many "artists" "artist_name" ", "  # remixers
sayrest
say "|"

json_select_object "label" # LABEL
json_select "label_name"
sayrest
json_unselect_object
say "|"

json_select_many "genre" "genre_name" " ● " # GENRES/S
sayrest
say "|"

sayrest
say "https://www.beatport.com/track/"  # _URL
say "-/"
json_select "track_id"

sayrest
saynewline
json_foreach_end
`
and here's the one for track direct `
regexpreplace ".*application\/json\">({.*)<\/script><noscript><iframe.*" "$1"

json "ON" "current"

json_select_object "props"
json_select_object "pageProps"
json_select_object "dehydratedState"
json_foreach  "queries"
json_select_object "state"
json_select_object "data"
json_select_object "release"
json_select "id"
ifnot ""
outputto "BEATPORT_RELEASE_ID"
sayrest

outputto "ALBUM"
json_select "name"
sayrest

outputto "COVERURL"
json_select_object "image"
json_select "uri"
json_unselect_object
sayrest

outputto "PUBLISHER"
json_select_object "label"
json_select "name"
json_unselect_object
sayrest

json_unselect_object

outputto "BEATPORT_TRACK_ID"
json_select "id"
sayrest

outputto "BEATPORT_TRACK_URL"
say "https://www.beatport.com/track/"
say "-/"
json_select "id"
sayrest

outputto "ARTIST"
json_select_many "artists" "name" ", "
sayrest

outputto "MIXARTIST"
json_select_many "remixers" "name" ", "
sayrest

outputto "Length" # shows track length when only single track available, convenient to check time comparison
json_select "length"
sayrest

outputto "BPM" 
json_select "bpm"
sayrest

outputto "CATALOG #"
json_select "catalog_number"
sayrest

outputto "GENRE"
json_select_object "genre"
json_select "name"
sayrest
json_unselect_object

outputto "INITIALKEY"
json_select_object "key"
json_select "camelot_number"
sayrest
json_select "camelot_letter"
sayrest
outputto "CHORD"
json_select "name"
sayrest
json_unselect_object


outputto "ISRC"
json_select "isrc"
sayrest

outputto "TITLE"
json_select "name"
sayrest
json_select "mix_name"
say " ("
sayrest
say ")"
outputto "YEAR"
json_select "publish_date"
sayrest

endif


json_unselect_object
json_unselect_object
json_unselect_object
json_foreach_end

json_unselect_object
json_unselect_object
json_unselect_object
json "OFF"

Got the release search and release details code done as well but since I use almost no more original code neither tab order lists I won't publish the whole code. fix the necessary parts.

Searches seem to be faster than before, would say 40% faster