item_search.ash -- Find groups of items with a single query.

[philmasterplus]

Item Search V0.3

Warning: The latest release introduces large changes.
When you update this script, you need to download both item_search.ash and PUtils.ash. Also, you can remove item_search_keywords.txt from your (Mafia Root)\data directory, since it is no longer used.


What is this?

A script library that will help you search for multiple items by keywords and categories. As a script writer, I often found a need to find items that fell into a specific category or matched certain keywords. However, using foreach loops to check each item for every search was messy. This script is the solution.

Warning: many of the functions in this script deal with large maps, and thus can be very slow depending on your computer's performance.

This script is still in its development stages. Various features are subject to sudden changes. If you are unsure whether updating this script would cause problems with another script, do NOT download the latest version.


For Users

item_search.ash requires zlib.ash and PUtils.ash, the latter of which is included along at the moment. PUtils.ash contains various miscellaneous functions, some of which are used by item_search.ash.

How a Text Query Search Works

The keywords map contains various keywords that items can be classified into, as well as its name and initials. For a full list of available keywords returned by each classifiers, type the following command into the CLI:

> item_search.ash showkeywords

The search() function handles three special characters: the vertical bar ("|"), the whitespace (" "), and the exclamation mark ("!"), interpreted in that order.

1. Whitespace (" "): Denotes an AND-search. For example, knob potion will return all potions with the words knob in them.
2. Vertical bar ("|"): Denotes an OR-search. brimstone accessory|hat will return a list of all hats and Bad Moon rewards that are accessories. brimstone accessory|brimstone hat will return all Bad Moon rewards that are accessories or hats.
3. Exclamation mark ("!"): Denotes a NOT-search. !accessory !hat brimstone will return all Bad Moon rewards that are neither accessories nor hats.

Did I tell you that brimstone will also match the Brimstone Chicken Sandwich? If you don't like it, you can add equipment or !consumable to exclude the Sandwich.

To test your query using the default classifier settings, type the following into the CLI:

> item_search.ash [text query]

For Script Authors

To use this script, enter import <item_search.ash> at the top of your script. As said above, it requires ZLib and PUtils.ash (included below). It uses no saved configuration, one or two text files loaded from zarqon's Map Manager, and several KoLmafia internal data files.

The functionality of item_search.ash is wrapped around the record item_search, which contains all the information needed to perform a search:

record item_search
{
    int [item] prices;                   #Mall prices are stored here.
    string [item] keywords;
    boolean [item] current_selection;    #The last search results are stored here. By default, includes all items in the keywords map.
};

In nearly all cases, you will use only one instance of item_search in your script.

Classifier Functions

As of v0.2, item_search.ash supports classifier functions. These functions are used to generate keywords for each item in build_keywords(). The list of default classifier functions is stored in the boolean [string] item_search_classifiers, where the function name is stored. Should you wish to deactivate one of the classifiers (to speed up build_keywords()), you can assing a boolean value of false to the map, or just remove the particular function from the map. To view the default list of classifier functions and their availability by default, see item_search_classifiers.txt.

Note: the classifier function classify_potion_effects() is disabled by default (marked false) because searching for effect names yielded undesireable results (E.g. swim matches the shark cartilage. Guess why.). You can reactivate it by using

item_search_classifiers[ "classify_potion_effects" ] = true;

in your script before running create_item_search() or add().

You can also create custom classifier functions and add them to the item_search_classifiers map so as to support custom keywords. A classifier function must be of the form:

string classify_something( item i );

It must return a whitespace-separated list of keywords, followed by a single whitespace.

Search Functions (at last!)

The following is a list of available functions that can be used to create and perform a search. Most of the search-related functions return the item_search record, so you can chain multiple operations using "Java-like" function calls.

item_search create_item_search( boolean [item] item_list )

A "constructor" for item_search that generates keywords for each item in item_list. You would usually call this only once at the beginning of your script.
Most of the time, you would want to use $items[] for the item_list. However, if your script deals with a specific list of items, you can use that list instead to drastically reduce search time.​

item_search add( item_search ims , item i , int default_price )

Implemented in v0.2, this function adds a new item to the item_search record, automatically adding the keyword and using the default price. It does not add the item to the current_selection map. To do so, you will need to use reset().​

item_search update_prices( item_search ims , float price_age )

Updates prices for all items in item_search.current_selection, using historical prices younger than price_age. Will not modify the original price if historical_price() or mall_price() returns 0 (item does not exist in mall).​

item_search update_prices_all( item_search ims , float price_age )

Same as above, except that it updates prices for all items in the keywords map instead.​

item_search search( item_search ims , string query )

The flower of item_search.ash, this function searches for all items that match the given query from current_selection, removing any that fails to match.
See "For Users" for explanation on formatting a text query.​

item_search search_mall_price( item_search ims , int min_p , int max_p )

Finds all items within a certain mall price range (inclusive). Untradeable items have a fixed price( usually 0 ).
Warning: does not check whether an item's price is valid. See update_prices() for manually updating mall prices.​

item_search search_autosell_price( item_search ims , int min_p , int max_p )

Finds all items within a certain autosell price range (inclusive), using autosell_price(). Remember, autosell_price() may not work the way you'd expect.​

item_search search_custom( item_search ims , string func_name )

Implemented in v0.2, this function receives a custom function and uses it to narrow the search. The passed function must be of the form:

boolean func_name( item i , string keywords , int mall_price )

Same as above, it does not guarantee that the passed mall price is valid.​

item_search reset( item_search ims )

Resets current_selection to all items stored in the keywords map.​

If all this sounds weird and woozy, here is a sample code to show how awesome this can be:

item_search my_search = create_item_search( $items[] );
foreach it in my_search.search( "familiar larva" )
    .update_prices( 1 ).search_mall_price( 100000 , 100000000 )
    .current_selection
    print( it + " is an expensive familiar, probably an IOTM." );
my_search.reset().search( "tiny plastic crimbo" ).update_prices( 1 );
foreach it in my_search.current_selection
    print( it + " is available at " + my_search.prices[ it ] + " in the mall." );

Thanks To

• zarqon: For his wonderful work on zlib.ash
• Bale, heeheehee, StDoodle: Help on various ASH functions and quirks.

Change Log

2010-07-08 v0.1: Initial release
2010-07-12 v0.2: Completely restructured how search works. Added classifier functions. Added negative search (!). Deprecated item_search_keywords.txt and added item_search_classfiers.txt. Added optional support for potion effects, adjustable via classfiers.
2010-07-14 v0.3: Slight speed improvements. Added classify_rare_items(). classify_consumables(), classify_packages(), and classify_rare_items() now require a call to activate_functionality( string functionality ) at the start of a script.

 

[slyz]

Just a quick fix and a suggestion that I should have posted earlier, in the other thread:

- shouldn't line 51 be

if ( !i.is_displayable() ) traverse_keywords( keyword_list , "quest item" );

This could be the bug that cost you your Brimstone items.

- It wouldn't be hard to add this in a script that imports item_search.ash, but it could be handy:

item_search search_custom( item_search ims , string function )
{
	foreach it in ims.current_selection
		if ( ! (call boolean function(it))  )
			remove ims.current_selection[ it ];
	return ims;
}

so we could do things like this:

import "smashlib.ash";
import "item_search.ash";

item_search search_custom( item_search ims , string function )
{
	foreach it in ims.current_selection
		if ( ! (call boolean function(it))  )
			remove ims.current_selection[ it ];
	return ims;
}
float smash_value( item it ) {
	int value = 0 ;
	if ( !have_skill($skill[Pulverize]) || !is_smashable(it) )
		return value ;
	float [item] smash_yield = get_smash_yield(it);
	foreach yield in smash_yield
		value = value + autosell_price(yield.to_item()) * smash_yield[yield];
	return value;
}
boolean should_pulverize( item it ) {
	return ( ( item_amount(it) > 0 ) && is_smashable(it) && ( autosell_price(it) < smash_value(it) ) ) ;
}

item_search pulv_search = create_item_search( $items[] ); 

print( "Items to pulverize:" ); 
foreach it in pulv_search.search( "giftable" ).search_custom( "should_pulverize" ).current_selection
    print( it + " - autosell: " + autosell_price(it) + " - smash: " + smash_value(it));

- It would also be nice to have an overloaded form of create_item_search() that accepts get_inventory():

item_search create_item_search( int [item] item_map ) {
	boolean [item] item_list ;
	foreach it in item_map
		item_list[ it ] = true;
	return create_item_search(item_list);
}

As you can see, I've been having fun with this, thanks a lot!

 

[philmasterplus]

Just a quick fix and a suggestion that I should have posted earlier, in the other thread:

- shouldn't line 51 be

if ( !i.is_displayable() ) traverse_keywords( keyword_list , "quest item" );

This could be the bug that cost you your Brimstone items.

- It wouldn't be hard to add this in a script that imports item_search.ash, but it could be handy:

item_search search_custom( item_search ims , string function )
{
	foreach it in ims.current_selection
		if ( ! (call boolean function(it))  )
			remove ims.current_selection[ it ];
	return ims;
}

Thank you, glad to hear you're having fun with it. Both changes implemented in v0.2. Incidentally, the Brimstone-smashing happened before I implemented item_search.ash (which explains my motivation).

Edit: Major Update to v0.2!

• Keyword creation now uses string concatenation instead of maps
• Added classifier functions, which are slightly slower but much more extensible and powerful.
• Fixed the is_displayable() bug (wiped the entire code, really) while writing classify_tradeability().
• Added add() and search_custom(), as requested.

 

[philmasterplus]

v0.3 released!

Changes to item_search.ash:

• Keyword generation now uses buffers. Should be slightly faster now.
• Added classify_rare_items(), which will tell you whether an item is an IOTM, semi-rare, ultra-rare, or a clan dungeon boss loot (really, anything that smashes into epic wads). One step closer to the IOTM catalog project!
• classify_consumables(), classify_packages(), and classify_rare_items() require a call to activate_functionality( string functionality ) before using any of them. Each function needs to have a certain functionality activated:
  • classify_consumables(): "spleen_item_information"
  • classify_packages(): "gift_packages"
  • classify_rare_items(): "rare_item_information"
  Multiple functionalities can be activated at once:
  

  activate_functionality( "spleen_item_information, gift_packages, rare_item_information" );

Summary:

> item_search iotm weapon

Starting up a test run of item_search.ash...
Executing your query...
$item[ Spooky Putty ball ] matched.
$item[ Spooky Putty snake ] matched.
$item[ Staff of Queso Escusado ] matched.
$item[ bottle-rocket crossbow ] matched.
$item[ flaming juggler's balls ] matched.
$item[ haiku katana ] matched.
$item[ ice sickle ] matched.
$item[ origami riding crop ] matched.
$item[ stinky cheese sword ] matched.
Search finished.

 

[slyz]

Just a little addition to classify_equipment() for weapons: add "melee" or "ranged" as keywords.

string classify_equipment( item i )
{
	string s = i.to_slot().to_string();
	switch ( s )
	{
	case "hat":
	case "pants":
	case "shirt":
	case "familiar": break;
	case "acc1": s = "accessory"; break;
	case "off-hand": if ( i.item_type() == "shield" ) s += " shield"; break;
	case "weapon":
		if ( i.weapon_type() == $stat[ Moxie ] ) s += " ranged";
		else s += " melee";
		s += " " + i.weapon_hands() + "-handed " + i.item_type();
		break;
	default: return "";
	}
	return s + " equipment ";
}

I tested this with this bit of code:

import <item_search.ash>

print( "Ranged weapons" );
item_search my_ims = create_item_search( $items[] );
foreach it in my_ims.search( "ranged" ).current_selection
	print( it );

 

[slyz]

Here is a little tool I use to decide which items to pulverize in-run (those which yield more meat than their autosell value when smashed). It simply prints in the gCLI a list of the items that you can pulverize for profit.

It could be extended to be a relay_ script from which you could pulverize selected items directly.

I made this to try out philmasterplus' library, and maybe posting it will renew interest in it albeit his apparent disappearance.

If people do decide that item_search.ash is worth using, I can use my Minion powers to maintain it if needed.