Recent extensions to websource scripts like the %placeholder% syntax made it possible, to write code that use output buffers like "real" variables. Beside the reuse of their values in other strings
they can be logical variables, where empty means false and having any content means true, and that are used with the commands ifoutput and ifnotoutput.
they can be pointer variables, when holding the name of another output buffer in %placeholder%, with sayoutput, set, outputto, ifoutput and ifnotoutputcommands.
they can build an (associative) array, by using a name with fixed and variable parts, e.g. "Array_%Index%".
This means that we can write code, that abstracts from actual values and can be used in the body of a loop. Sadly to say but in my opinion there currently is no appropriate looping command available.
The do...while command is not really useable, because it can't be nested, occupies the input buffer and is very complicated to get it right. The json_foreach command can't be used, because we don't have a JSON object here (and if we are already in json mode we can't switch back and forth between different json objects). But json_foreach is a good example of what would be useful here.
The suggested foreach command looks like this:
foreach "index" "value-list" "separator"
# body
foreach_end
The first parameter "index" iterates through the strings in second parameter "value-list", separated by the third.
It should be allowed to nest foreach loops und the input buffer should not be changed by the loop.
The following example shows what actual is possible. (It builds a combination of three filter components for a regular exression.)
# Definiton of an array
set "Filter_Composer" "Written[ -]By|Composed By|Music By|Songwriter"
set "Filter_Lyricist" "Lyrics By|Words By"
set "Filter_Arranger" "Arranged By|Adapted By"
# Building a filter consisting of above components
set "Join"
set "Filter_Any_Role"
outputto "Filter_Any_Role"
# Unrolled loop
set "Temp_Name" "Composer"
ifoutput "Filter_%Temp_Name%"
sayoutput "Join"
sayoutput "Filter_%Temp_Name%"
set "Join" "|"
endif
set "Temp_Name" "Lyricist"
ifoutput "Filter_%Temp_Name%"
sayoutput "Join"
sayoutput "Filter_%Temp_Name%"
set "Join" "|"
endif
set "Temp_Name" "Arranger"
ifoutput "Filter_%Temp_Name%"
sayoutput "Join"
sayoutput "Filter_%Temp_Name%"
set "Join" "|"
endif
And that's how a foreach command could improve the above unrolled loop:
I currently have to copy and paste over 100 lines of code for every tag field I want to support in my discogs websource script. With the proposed foreach loop available the script could be up to one third shorter and faster, and it could be more flexible be adapted to specific needs.
The foreach command would of course be equally useful for me, if the first parameter "index" is left out and the values are emitted instead on the current input. This form could even be more useful for other cases, where operations with the string have to be carried out. Important for me is only that foreach can be nested and doesn't block the input buffer like the do..while command does for correct working. My example code would read then
I’m a bit divided: your first implementation looks interesting as a way of a command creating a dynamic variable (“index” or %index%) that goes through a set list of desired values which IS currently possible with json_foreach despite needing more setup with an array converted to JSON before-hand. Would this variable persist after the command or only be available during that loop?
It actually does solve part of a problem that json_foreach greatly suffers of not being able to output a loop’s currently selected item from the “value-list” as it’s accessible from that “index” variable. We’re still not able to output singular values from an array fed into json_select_array without reliance on an invariable index position or without formatting values as “key:value” for a json_foreach to fetch via json_select key or creating temporary buffers that saves/removes each value as they are used up through a loop.
Your second implementation is simpler yet more flexible as an alternative way of making a fixed loop just by removing the requirement of JSON which could be handy for HTML-based sources. More flexible because it doesn’t require setting up a variable that may/may not be required. Maybe too simple, though…?
The biggest factor is how json_foreach does block off access to other parts of the response that aren’t currently inside the loop. It is possible to circumvent if these JSON objects are isolated into temporary buffers to be called on when needed with Use → json “on” “current” but it does get messy with regex. Using your proposals would allow calling whatever, wherever, whenever.
I do love new variations and combinations of existing commands that allow broader levels of control, especially JSON-less versions that can then have more purpose. I’m just unsure if a JSON-less version of this command is completely necessary when existing commands could instead be given more functionality to eradicate or at least circumvent their flaws. It might still be easier development-wise to make a clone of an existing command to be fair.
Also, would it be possible to use SayRegexp in your script as a solution? It could help you filter all the desired values in one go from your response per filter, just again using a json_foreach and temp buffers to scroll through each filter value. I used this in my TMDB#Results.inc script for getting all the cast, directors etc. from the credits.
Because this would be the first command with this side effect, I think it has no chance to be implemented in this form by the developer @Florian and therefore I will prefer the second, simpler and also more "traditional" way of emitting the next value from the list of values on the current input.
I plan to use the proposed foreach command inside of up to two outer json_foreach loops, so I doubt that it is possible to switch with json "on" "current" to a self constructed json object without disturbing the outer json loops.
The structure of my script with unrolled loop currently look like this
json_foreach "tracklist"
json_foreach "extraartists"
set "Index" "Composer"
.... block with many lines
set "Index" "Lyricist"
.... same block as above
set "Index" "Arranger"
.... same block as above
json_foreach_end
json_foreach_end
And with the new foreach loop it would look as follows
json_foreach "tracklist"
json_foreach "extraartists"
foreach "Composer,Lyricist,Arranger" ","
.... block with many lines
foreach_end
json_foreach_end
json_foreach_end
This is only an example of what can be done with the %placeholder% syntax and it is no real problem I'm working on. It should show how to write script parts that can be used inside loops.
Thanks to @arb comments, I'm now using a workaround for the suggested foreach command. This workaround converts the list of strings into an json array, switches with json "on" "current" to this json object and then it loops with json_foreach through the list of strings. This works at the innermost level of enclosing json_foreach loops without flaws, and until all outer loops are finished at the json root level.
To restart json now on the original input from the web source a backup of this data has to be made at the begin of the script. With this backup the json processing can be resumed and continued at the json root level. This illustrates the following example code with a named json array:
# a backup of input is necessary
outputto "Backup_of_Input"
sayrest
json "on" "current"
json_foreach "outer_array"
# use a new selfmade json array to control a json_foreach loop
set "json_object" "{"array":[{"key":"value one"},{"key":"second value"},{"key":"and so on"}]}"
use "%json_object%"
json "on" "current"
json_foreach "array"
json_select "key"
json_foreach_end
json_foreach_end
# down at the root level return back to the original input
use "%Backup_of_Input%"
json "on" "current"
json_foreach "another_array"
json_foreach_end
For my project this workaround fits very well, but if someone else has the need for a json-less foreach loop, which is not located at the innermost nesting level of surrounding json_foreach loops, it probably cannot be used.
The suggested json-less loop foreach ... foreach_end inherits a further problem, because it should not overlap with loops json_foreach ... json_foreach_end. To solve this conflict, the commands json_foreach_end and foreach_end could be thought of being equal and both end the other foreach commands also. Alternatively the new command could be named json_foreach_fake and then use json_foreach_end to stop the iteration.
