Discussion: Standards in relay over-ride scripts

Relay over-ride scripts end up having naming conflicts with other relay over-ride scripts that function on the same page. I've been considering this, and there is a simple solution which might save you (the script writer) time in the future. It is fairly simple, and would make importing the re-named script into a new "shell" script a lot less time consuming.

This will be a reference for myself and others to use. Contributions/suggestions are welcomed.


  • [li]avoid global variable declarations where possible.[/li]
    [li]Place all code which does something to the source in a function, and pass the source to the function as a parameter. Return the modified source.[/li]
    [li]Try to use function names that will be unique to your script[/li]
    [li]Place all calls to your created functions inside void main()[/li]
    [li]Avoid top level commands[/li]

Here is an example. It uses my Link back to manage store at the top of the store log page script as an example. Strangely I wasn't thinking about this when I wrote the script, but it follows the ideas.

storelog.ash
PHP:
buffer addtoplink(string source)
{
buffer results;
results.append(source);
results.replace_string(
  "<td style=\"padding: 5px; border: 1px solid blue;\"><center><table>",
  "<td style=\"padding: 5px; border: 1px solid blue;\"><center><table><tr><td><center><a href=\"managestore.php\">Back to Your Store</a></center></td></tr><tr><td></td></tr>"
  );

return results;
}

void main()
{
addtoplink(visit_url()).write();
}

Now when another script is written to do something else on the same page, rename the file to storelogtoplink.ash, then write a new storelog.ash file as follows:

Code:
import storelogtoplink.ash;
//import new script here

void main()
{
buffer source;
source.append(visit_url());

//code to call new script function can be here
source = addtoplink(source);
//or it can be here

source.write();
}

Because void main() is not used in an imported script, we need to do nothing to the original script (unless it conflicts in some way with what the new script is doing). Not having top level commands means that we are less likely to have a command executed before it's time. Avoiding global variables means we won't end up with variable naming conflicts. The use of unique names ahead of time will reduce the chances of a naming conflict in function names. (naming all your functions which make changes to the source "modify_source" is not such a good idea.

Comments, suggestions, Edits by other moderators/admins, and any improvements to this concept are not only welcomed, but will be enjoyed by me as it will make what I try to do in the future better.
 

holatuwol

Developer
A Different Implementation?

Would Greasemonkey-style annotations be useful?

In other words, rather than relying on the name of the script, maybe add an additional capability where KoLmafia scans through the initial comment blocks for something along the lines of @include and @exclude to see whether or not the script should be injected, and scans all the .ash scripts sitting in the /relay folder each time it encounters a KoL page. This might slow things down (slightly) because it's no longer a quick lookup, but it may be useful in the short term for testing?
 

raorn

Member
Why search for scripts, if you can use "all"? Just execute relay/PAGENAME/*.ash.

Furthermore, mafia can do all the dirty work herself, passing current page buffer to scripts and writing output when all scripts are done.
 

holatuwol

Developer
The idea would be to make it so that instead of making a set number of scripts work on one page, make it so every script can run on every page.

Suppose, for example, you wanted to write a simple script which imitated the color KoL images which ran everywhere, rather than strictly on Firefox. You could write hundreds of scripts which all contained the same exact code, or you could have a single script applied globally.

I didn't think people would use it that way, but it's entirely possible that this "inject in more than one place" idea may be better, as it's far more flexible.
 
Sorry I didn't get back sooner. I have been offline for a while because my ISP wanted to raise the price of my internet service to a silly level.

My thoughts are that rather than having Kolmafia search for a script every time it encounters a kol page, Kolmafia should already know if there is one. This would avoid the slower hardware access. I also see a possible problem with the order of script execution. That problem will become more obvious as more relay over-ride scripts are shared. There are several ways to handle both issues, but they all boil down to kolmafia keeping a list.


  • [li]Method 1: change all relay over-ride scripts from .ash to .rly, and have a register method. The register method would contain info about what pages the script should run on. When the script is manually ran, kolmafia would read only the register method and ignore the rest of the script until encountering the page. This would require a file be saved by kolmafia for re-loading at next run, which would contain a list of registered scripts, and how they are registered. That file could then be manually edited by the user if the order the scripts run needs modified or a script needs removed.[/li]

    [li]Method 2: kolmafia could search the relay directory at start-up, and read the info as indicated in Holatuwol's post above, and store the info in memory. Kolmafia would then check it's stored info for a script which should run. This method is less desirable because when adding a new script, it would require a restart. Also, this method leaves the order of execution issue unsolved.[/li]

    [li]Method 3: have a manually created/edited file in the relay folder (call it relay.inf during this post) and when Kolmafia starts, load the file if it exists. This file would tell Kolmafia how to handle relay over-rides. This method would allow the removal of the option to enable Relay over-ride scripts. If the file exists, then Relay over-rides are on, if not then it's off.
    Relay.inf would follow a format similar to this: (horribly similar to windows .inf files)

    • [li]{all}[/li]
      [li]myscript4[/li]
      [li][/li]
      [li]{storelog}[/li]
      [li]myscript1[/li]
      [li]myscript2[/li]
      [li][/li]
      [li]{adventure}[/li]
      [li]myscript1[/li]
      [li]myscript3[/li]

    this would also require a restart.
    [/li]

I personally prefer method 1, or a variation of it.

All these methods leave open the question of non-standard kol pages. One of the first things I did when relay over-rides was in it's infancy was to create my own page called BottomMenu.html. It contains some of the links I use the most for faster access. I have considered adding to that page using the newer form of relay over-rides where I can create the links based on variables like my inventory status. That brings to light the question of creating custom java scripts, but that is a whole new topic. I might start a topic later on that explaining in detail.
 

holatuwol

Developer
If I follow a Greasemonkey implementation model, method 1 = method 2, except method 1 uses register() and method 2 uses script comments.
 
In general, method 1 = method 2, as you said except that method 1 uses register() and method 2 uses script comments. That is where the big difference is though. The Register method would allow adding a new relay over-ride script without restarting kolmafia. Since kolmafia would only have to load 1 file at startup, the list of registered scripts, kolmafia should start faster using method 1.

I know a lot of users of Greasemonkey are not aware of this, but if you go to manage user script, and click-hold a script name, and drag it up or down, this changes the order the script appears in the list. It's noted below the list. It does more than that though, it changes the order of execution too. That's the important thing. Here is an example of where that might apply:

This pseudo code will insert the word hi at the beginning of every table:
script 1:
PHP:
document.replacestring( "<table>", "<table>hi" );

This pseudo code will insert the word bye at the first table row of every table:
script 2:
PHP:
document.replacestring( "<table><tr>", "<table><tr>bye" );

script 1 breaks script 2 if script 1 runs before script 2. If script 2 runs first, it will be fine, and script 1 will not be hurt either.

Adding a Greasemonkey like interface with the movement feature would work, but I don't feel it appropriate to ask that you do so much extra work adding that part of the interface because I think it is probably going to be more work than is needed. I do think some way of changing the order of execution is needed though.

The code I used above is probably not using proper methods of changing html files, but it is the easiest way that I have found to achieve the simple things like injecting a link into a page. You will find this method actually applied here.
 

holatuwol

Developer
I can always implement a 'register' command which does exactly what method 1 does (except in a way that's less cryptic than a new file extension) which avoids the adding scripts weakness you speak of. Using a Greasemonkey storage model avoids the ordering weakness (I need not have the UI -- any script injection which doesn't know in advance what it needs will cache this data in some file).

The thing is, worrying too much about the whole 'oh, two scripts will break each other' problem feels counterintuitive. Ideally, whenever I'm looking for new ideas for the relay browser, I would implement some of the changes you guys post here in addition to some of my own. Should I be discouraged from making changes to the core because they might break the scripts you guys are writing? I don't think so.

One of the reasons I think people avoid writing these scripts is that KoLmafia's relay browser HTML is a moving target, which is a good thing for KoLmafia but a bad thing for script writers. You're operating in that environment, so giving you the illusion that you run in a sandbox doesn't make much sense.
 
[quote author=holatuwol link=topic=1538.msg7522#msg7522 date=1205005015]
The thing is, worrying too much about the whole 'oh, two scripts will break each other' problem feels counterintuitive. [/quote]

The very root of this topic...the first post is about making 2 or more scripts work on the same page. In order for 2 or more scripts to work on the same page, they must not break each other. I'm lost ???

[quote author=holatuwol link=topic=1538.msg7522#msg7522 date=1205005015]
Ideally, whenever I'm looking for new ideas for the relay browser, I would implement some of the changes you guys post here in addition to some of my own. Should I be discouraged from making changes to the core because they might break the scripts you guys are writing? I don't think so.
[/quote]

No, not "I don't think so", just no. Doing so would severely limit your ability to upgrade Kolmafia. That concept would be counterintuitive. That is on the same level as: If Intel refused to make a change to the instruction set in a new model of processor because it would break Windows, that would be counterintuitive. Think of Kolmafia as the processor, and the script as Windows.

If Microsoft realized that a change to Excel broke Word, and didn't fix it (or couldn't), Microsoft Office's sales would plummet very quickly as people started switching to Corel Office, or Open Office, or some other office suite. Both Excel and Word run in Windows, so now think of Kolmafia as Windows, and the scripts as Excel and Word.
 

holatuwol

Developer
[quote author=efilnikufecin link=topic=1538.msg7526#msg7526 date=1205042825]
The very root of this topic...the first post is about making 2 or more scripts work on the same page. In order for 2 or more scripts to work on the same page, they must not break each other. I'm lost
[/quote]

In the ideal case, the HTML that KoLmafia presents won't be a static target. Therefore, trying to meticulously plan out a mechanism to protect you from each other when I'm the most likely person to break everything you're writing feels counterintuitive.
 

raorn

Member
When talking about "several scripts on one page", don't forget about visit_url() function.

If I understand correctly, KoLmafia calls relay script before executing request to KoL server. This is what visit_url() does. However, URL parameters still accessible to script.

Look at "disco combos" or "El Vibrato" scripts (yes, I'm talking about fight.php overrides again). It adds form with non-standard actions which is checked before executing visit_url() and may generate some more requests (and it's results are returned instead of visit_url()). What would happen if this request will be passed to game server? We have two-and-a-half "parts" here:

1. check, if we must do Something Different before posting request to server
1.5. do Something Different and return result as page buffer OR execute visit_url()
2. do it's main job with page buffer passed as argument and return modified page buffer

So, KoLmafia should call each script at least two times:

boolean check_pre() - for each script to see if it wishes to do some additional actions
buffer do_pre() - if any script returns true in check_pre() (first non-false script) OR visit_url() if all scripts return false
buffer main(buffer) - for each script, passing modified page to next script

"First non-false script" means that any two scripts can't trigger additional actions at the same time on same conditions. Script writes should take care of it.
 
Hmm, what would happen if the script I wrote for the degrassi knoll general store were to have the additional items it adds to the store passed to the server? Nothing interesting, but that concept does add a new list of problems. This takes us right back where I was in the first post. The difference would be in the wrapper script.

[quote author=raorn link=topic=1538.msg7529#msg7529 date=1205075182]
So, KoLmafia should call each script at least two times:

boolean check_pre() - for each script to see if it wishes to do some additional actions
buffer do_pre() - if any script returns true in check_pre() (first non-false script) OR visit_url() if all scripts return false
buffer main(buffer) - for each script, passing modified page to next script
[/quote]

I disagree with that concept. I actually think that trying to work with that model would lead to more trouble than it is worth when we could easily make a shell script based on the needs of the merging of the 2 scripts.

When you look at it from that perspective, it does become more counterintuitive. When you narrow your perspective to just adding a few easier to get to links, it is less so.

This leaves me with the thought that a cli or menu command to add or remove scripts from the pool, and a single file to load at startup which tells Kolmafia whats what would be the best plan. New question: with the concept of adding and removing scripts via the file at startup, and other commands while running would it be more feasible to keep the script constantly in memory unless the script is removed? That idea would seemingly also cut back on the performance loss of relay scripting.
 

raorn

Member
[quote author=efilnikufecin link=topic=1538.msg7531#msg7531 date=1205081394]
I disagree with that concept. I actually think that trying to work with that model would lead to more trouble than it is worth when we could easily make a shell script based on the needs of the merging of the 2 scripts.[/quote]
In this case only autoloading of "wrapper" scripts makes sense.

New question: with the concept of adding and removing scripts via the file at startup, and other commands while running would it be more feasible to keep the script constantly in memory unless the script is removed?
Can you check mtime of a script and reload it if it's changed?
 
[quote author=raorn link=topic=1538.msg7543#msg7543 date=1205145980]
Can you check mtime of a script and reload it if it's changed?
[/quote]

I believe that is how it is, that it is checked.

[quote author=holatuwol link=topic=1538.msg7551#msg7551 date=1205173658]
All parsed ASH scripts are kept in memory.
[/quote]

I should have been more clear, if Kolmafia does check to see if the script has been modified, then eliminating that step might be an option that could increase the performance of relay over-ride scripts. While debugging a script in progress, when modified the user could just re-execute the method to add the script to the pool. I don't know if this would be enough of a gain to constitute the effort though.

OT (sort-of) if Kolmafia checks the modified date of the script to determine if it should be re-loaded, then instead of locking the file, could Kolmafia instead open the file share-aware? Kolmafia technically is share aware (if that is the case), and it would mean that when an exception occurs during parsing of a script, the file would not be locked, and the user/writer would not be forced to shutdown and restart Kolmafia. Normally when I encounter an error in a script I am writing, it is a parse error like having 6 opening ( and only 5 closing ).
 

raorn

Member
[quote author=raorn link=topic=1538.msg7529#msg7529 date=1205075182]
So, KoLmafia should call each script at least two times:

boolean check_pre() - for each script to see if it wishes to do some additional actions
buffer do_pre() - if any script returns true in check_pre() (first non-false script) OR visit_url() if all scripts return false
buffer main(buffer) - for each script, passing modified page to next script
[/quote]

ASH implementation. Main disadvantage is that I need to edit script to add something and check for namespace intersections. Things could be much easier if KoLmafia can do this job automatically.

Code:
import </relay/fight_buttons.ash>;
import </relay/fight_info.ash>;
//import </relay/fight_XXX.ash>;

void main()
{
	buffer page;

	if (buttons_check()) {
		page = buttons_preaction();
	} else if (info_check()) {
		page = info_preaction();
	//} else if (XXX_check()) {
	//	page = XXX_preaction();
	} else {
		page = visit_url();
	}

	page = buttons_main(page);
	page = info_main(page);
	//page = XXX_main(page);

	page.to_string().write();
}

// vim: set ft=javascript ts=4:

P.S. Posting this here for archiving purposes.
[quote author=efilnikufecin link=topic=1538.msg7531#msg7531 date=1205081394]I disagree with that concept. I actually think that trying to work with that model would lead to more trouble than it is worth when we could easily make a shell script based on the needs of the merging of the 2 scripts.[/quote]

What do you think about these functions:

void import(string) - acts as import <>;
value call(string, ...) - calls function by name passed as string argument.

So I could write something like this:

Code:
string modules = get_property("somepageRelayModules").split_string("\\s*,\\s*");

foreach m in modules
  import("/relay/somepage_" + m + ".ash");

void main()
{
  foreach m in modules {
    if (call(m + "_can_run"))
      call(m + "_main");
}

There could be some problems implementing call() function since it may return any type and can have exactly 1 or more arguments of any type.
 

raorn

Member
[quote author=holatuwol link=topic=1538.msg7220#msg7220 date=1203146869]
The idea would be to make it so that instead of making a set number of scripts work on one page, make it so every script can run on every page.[/quote]

In this case I would like to know current page. Like, "fight.php", "choiceadv.php", "sewer.php", etc... In short, I would need something like RelayRequest.getPath() in ASH library.
 
[quote author=raorn link=topic=1538.msg7577#msg7577 date=1205329215]
There could be some problems implementing call() function since it may return any type and can have exactly 1 or more arguments of any type.
[/quote]

The quote above may very well say it all.

[quote author=raorn link=topic=1538.msg7661#msg7661 date=1206053538]
In this case I would like to know current page. Like, "fight.php", "choiceadv.php", "sewer.php", etc... In short, I would need something like RelayRequest.getPath() in ASH library.
[/quote]

I second that notion. I occasionally create my own pages, and in order to link to such pages I have to modify another page. This currently requires 2 (or more) scripts to achieve. With the concept model, both pages could be handled in one script.
 

raorn

Member
[quote author=efilnikufecin link=topic=1538.msg7531#msg7531 date=1205081394]I disagree with that concept. I actually think that trying to work with that model would lead to more trouble than it is worth when we could easily make a shell script based on the needs of the merging of the 2 scripts.[/quote]

Making merging shell-script requires lot of hand-working and manual editing. Why do it myself, if machine can do it easier? I've been working with this model for a while and this is what I have in mind:

1. One script may be executed on different pages. So it must tell Mafia on which pages it would run.
2. Furthermore, script would like to know, for which page it is being called. Mafia should pass page name to script.
3. More than one script may be executed on a given page. So, script doesn't need to call visit_url(), but accept page text as an argument.
4. Page, on which script is working, doesn't need to exist if script renders page by itself. In this case Mafia should not call visit_url(), but use script-provided functions (see my script_settings.ash).
5. Before sending request to game server, script may want to check how would this request be sent. In this case script would like to make different request(s) to game server (see fight_disco_combos.ash) or modify it's internal settings (fight_buttons.ash).

With suggested implementation (greasemonkey-style) it's unclear who and when should call visit_url() and how can one avoid calling visit_url() when it's not needed. It's a bit unclear, what would happen if one script would like to make some throw_item() or use_skill() requests, based on form_field()s.

So I'm back to 2-call suggestion:

For each registered script call "preaction" function with "page name" argument that may return page text until some script returns anything. If no script returns page text, call visit_url(), otherwise use page text returned from first successful script. Then for each script call "render" function (main()) with "page name" and "page text" arguments, collect modified page and pass it to next script.

I really, Really, REALLY need 2 calls to script. Please, think about it. Relay override is very powerful and useful feature and I don't want it to be nerfed.
 

dangerpin

Member
Nerfed? I thought a nerf was a change that decreased the power or effectiveness of something, usually something that was overpowered to begin with. Can you nerf something by deciding not to make it more powerful? It doesn't seem to fit with the meaning of the word to me.
 
Top