I'm modifying dano's vgmdb.net script so that I can import the Japanese song titles. The charset according to the html is UTF-8. The debug output file that's generated is in UTF-16 little-endian. I need to parse the html as UTF-8 but as far as I can tell there is no way of doing so. I tried using a lot of options for the [Encoding], but the help file seems to suggest it only applies for the interpretation of the URL.
Here's the code modified to hone in on the problem:
[Name]=VGMdb
[BasedOn]=VGMdb.net
[SearchBy]=http://vgmdb.net/album/5411
[Encoding]=utf-8
[ParserScriptAlbum]=...
# ###################################################################
# A L B U M
# ###################################################################
debug "on" "C:\zDebug\debug_VGMdb_B.html" "10"
debugwriteinput "C:\zDebug\VGMdb_B.html"
# Tracks
FindLine "id=\"tl9143\""
findline "class=\"smallfont\"><span class=\"label\">"
do
# outputto "Tracks"
OutputTo "TitleJP"
moveline 1
findinline "width=\"100%\">"
sayuntil "</td>"
say "|"
outputto "_Length"
moveline 1
sayregexp "(?<=<span class=\"time\">)[\d:]+(?=</span>)" ""
say "|"
findline "class=\"smallfont\"><span class=\"label\">" 1 1
while "<td "
Here's the output when reaching a Japanese title (note that the first title for this particular album is correctly given in English):
=========
Script-Line : 39
Command : sayuntil
Parameter 1 : ><
Output : >Overture|星ããšç¥<
Line and position:
==========
The output at this point instead should be Overture|星くず祭. Is there a way to set the text encoding for the parser?