Add the ability to filter monsters

.github/workflows/regenerate.yml

@@ -33,6 +33,7 @@ jobs:
           python generateEquipment.py
           python generateEquipmentAliases.py
           python generateMonsters.py
+          python generateMonsterAliases.py
       - name: Create pull request
         run: |
           git config user.name github-actions

CONTRIBUTING.md

@@ -22,7 +22,7 @@ By contributing to this repository, you agree that your code will be available u
 ### Project structure
 The web app is contained inside the `src/app` directory. This project uses [Next.js 13's app routing structure](https://nextjs.org/docs). We opt to use TailwindCSS heavily in this project rather than writing CSS, but there is a `src/styles/global.css` file containing some styling.
 
-The `src/lib` directory contains the "core" code for the calculator itself. This code is heavily based on [some psuedocode](https://oldschool.runescape.wiki/w/RuneScape:Sandbox/combat_pseudocode) written collaboratively by the community.
+The `src/lib` directory contains the "core" code for the calculator itself. This code is heavily based on [some pseudocode](https://oldschool.runescape.wiki/w/RuneScape:Sandbox/combat_pseudocode) written collaboratively by the community.
 
 ### JS/TS style guide
 We lint the project using [`eslint`](https://eslint.org/). We tend to follow the [Airbnb style guide](https://github.com/airbnb/javascript) for JavaScript closely. Any JS/TS code that you write should adhere to this style guide. There are several eslint rules that are turned off or reconfigured because they are not beneficial for readability of our code, you can check [.eslintrc.cjs](/.eslintrc.cjs) for the disabled rules.
@@ -32,16 +32,18 @@ Most JavaScript IDEs will detect eslint and warn you accordingly of any problems
 You can also run `yarn lint`, which will check for any issues with your code.
 
 ### Scripts
-The `scripts` directory contains several Python 3 scripts that are used for generating the dataset required by this appliocation.
+The `scripts` directory contains several Python 3 scripts that are used for generating the dataset required by this application.
 
 * `generateEquipment.py` fetches applicable equipment from the OSRS Wiki and saves the output as JSON. It also downloads each equipment image to the local directory.
+* `generateEquipmentAliases.py` fetches applicable equipment from the OSRS wiki and generates a typescript file of item id aliases. These are used to deduplicate the equipment search.
 * `generateMonsters.py` fetches monsters from the OSRS Wiki and saves the output as JSON. It also downloads each NPC image to the local directory.
+* `generateMonsterAliases.py` parses the JSON output of `generateMonsters.py` to create a typescript file of monster id aliases. These are used to deduplicate the monster search.
 
 Where possible, we prefer serving images direct from the web app instead of the wiki for a few reasons. The main reason is that because the wiki can be edited by users, it is very easy for a user editing the wiki to break the functionality of this app by renaming or changing a file.
 
 ### Running locally
 1. Install dependencies with `yarn` (our package manager of choice).
-2. Run the development Next.js server running `yarn dev`.
+2. Run the development Next.js server by running `yarn dev`.
 3. Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
 
 Attempting to find local RuneLite instances can print a lot of failed WebSocket connection messages to the browser console. Adding a filter of `-WebSocket` will filter out these messages.

Dockerfile

@@ -15,7 +15,7 @@ RUN --mount=type=cache,target=/root/.cache/pip pip install -r requirements.txt
 
 ### Util for regenerating /cdn on dev machines via docker
 FROM scraper-base AS scraper-dev
-CMD ["sh", "-c", "python generateEquipment.py && python generateMonsters.py && python generateEquipmentAliases.py"]
+CMD ["sh", "-c", "python generateEquipment.py && python generateMonsters.py && python generateMonsterAliases.py && python generateEquipmentAliases.py"]
 
 ### Used below in release docker image
 FROM scraper-base AS scraper-image
@@ -27,6 +27,7 @@ ADD ./cdn /srv/cdn
 RUN python generateEquipment.py
 RUN python generateEquipmentAliases.py
 RUN python generateMonsters.py
+RUN python generateMonsterAliases.py
 
 
 

scripts/generateMonsterAliases.py

@@ -0,0 +1,129 @@
+"""
+    Script to generate NPC ID aliases for common cosmetic variants.
+    This script shouldn't be used to overwrite src/lib/MonsterAliases.ts entirely, but can instead be used as
+    a way of bootstrapping that file.
+
+    Written for Python 3.9.
+"""
+from collections import namedtuple
+import re
+import json
+import unittest
+
+INPUT_FILE_NAME = './cdn/json/monsters.json'
+OUTPUT_FILE_NAME = './src/lib/MonsterAliases.ts'
+
+data = {}
+
+dataJs = """/**
+ * A map of base NPC ID -> variant NPC IDs for NPCs that are identical in function.
+ */
+
+const monsterAliases = {"""
+
+MonsterAliases = namedtuple('MonsterAliases', ['base_name', 'base_version', 'alias_ids'])
+
+monsters = []
+with open(INPUT_FILE_NAME, 'r') as f:
+    monsters = json.load(f)
+
+def handle_base_variant(variant_monster, base_name, base_version):
+    global data
+    global monsters
+    base_variant = next((x for x in monsters if x['name'] == base_name and x['version'] == base_version), None)
+    if base_variant and base_variant['id'] != variant_monster['id']:
+        data.setdefault(base_variant['id'], MonsterAliases(base_name, base_variant['version'], [])).alias_ids.append(variant_monster['id'])
+
+def get_monster_id(name, version):
+    global monsters
+    return next((x for x in monsters if x['name'] == name and x['version'] == version), None)['id']
+
+def test_baby_blue_dragon(data):
+    assert get_monster_id('Baby blue dragon', '2') in data[get_monster_id('Baby blue dragon', '1')].alias_ids, "Baby blue dragon variant missing"
+
+def test_blue_dragon(data):
+    assert get_monster_id('Blue dragon', 'Task only, 1') in data[get_monster_id('Blue dragon', '1')].alias_ids, "Blue dragon task only variants missing"
+
+def test_blue_dragon_tapoyauik(data):
+    assert get_monster_id('Blue dragon', 'Ruins of Tapoyauik, 1') not in data[get_monster_id('Blue dragon', '1')].alias_ids, "Ruins of Tapoyauik blue dragons merged"
+
+def test_zemouregal_armoured_zombie_melee(data):
+    assert get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Melee, 5') in data[get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Melee, 1')].alias_ids, "Armoured zombie (Zemouregal's Fort) melee variants missing"
+
+def test_zemouregal_armoured_zombie_ranged(data):
+    assert get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Ranged, 5') in data[get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Ranged, 1')].alias_ids, "Armoured zombie (Zemouregal's Fort) ranged variants missing"
+
+def test_zemouregal_armoured_zombie_merged(data):
+    assert get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Ranged, 5') not in data[get_monster_id('Armoured zombie (Zemouregal\'s Fort)', 'Melee, 1')].alias_ids, "Armoured zombie (Zemouregal's Fort) melee variants merged with ranged"
+
+def run_tests(data):
+    test_suite = unittest.TestSuite()
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_baby_blue_dragon(data)))
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_blue_dragon(data)))
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_blue_dragon_tapoyauik(data)))
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_zemouregal_armoured_zombie_melee(data)))
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_zemouregal_armoured_zombie_ranged(data)))
+    test_suite.addTest(unittest.FunctionTestCase(lambda: test_zemouregal_armoured_zombie_merged(data)))
+    test_runner = unittest.TextTestRunner()
+    test_runner.run(test_suite)
+
+def main():
+    global dataJs
+    global monsters
+    for monster in monsters:
+        name_match = re.match(r"^(Armoured zombie \(Zemouregal's Fort\)|(Baby (blue|green|red) dragon)|Cave goblin miner|(Red|Blue) dragon|Cultist|Emissary (Acolyte|Ascended|Chosen|Conjurer)|Farmer|Guard\((Burthorpe|Desert Mining Camp|Port Sarim Jail|Prifddinas|Varlamore)\)|Jelly|Jungle horror|Kalphite (Guardian|Soldier)|Knight of (Ardougne|Varlamore)|Man|Paladin|Penance Runner|Tormented Demon|Skeleton \((Barrows|Forthos Ruin|Lucien's camp|Melzar's Maze|Stronghold of Security|Wilderness Agility Course)\)|Werewolf|Woman|Zombie \(Entrana Dungeon\)|Zombie pirate \((Braindeath|Harmony) Island\)|Zombie swab)$", monster['name'])
+
+        plain_variant_match = re.match(r"^\d+$", monster['version'])
+        pennance_runner_match = re.match(r"^Wave ([1-6]|7/10|[8-9])$", monster['version'])
+        werewolf_match = re.match(r"^(Alexis|Boris|Eduard|Galina|Georgy|Imre|Irina|Joseph|Ksenia|Lev|Liliya|Milla|Nikita|Nikolai|Sofiya|Svetlana|Vera|Yadviga|Yuri|Zoja)$", monster['version'])
+        melee_match = re.match(r"^Melee, \d$", monster['version'])
+        ranged_match = re.match(r"^Ranged, \d$", monster['version'])
+        task_only_match = re.match(r"^Task only, \d+$", monster['version'])
+        eye_color_match = re.match(r"^(Blue|Green|Pink|Red|Yellow) eyes$", monster['version'])
+        jelly_match = re.match(r"^(Regular|Grey|Tan|Olive|Dark|Green)( \(w\))?$", monster['version'])
+        farmer_match = re.match(r"^(Green shirt, (long|short) sleeves|Khaki shirt, (bracers|no bracers)|Straw hat, (black|blonde|brown) hair)$", monster['version'])
+        ruins_of_tapoyauik_match = re.match(r"^Ruins of Tapoyauik, \d$", monster['version'])
+        barrows_skeleton_match = re.match(r"^(Unarmed( round shield)?|Armed round shield|Square shield)$", monster['version'])
+        woman_match = re.match(r"^(Purple|Brown|Red|Varrock|Shayzien)$", monster['version'])
+        kalphite_guardian_match = re.match(r"^(Upper|Lower) level", monster['version'])
+
+        if not name_match:
+            continue
+        if plain_variant_match:
+            handle_base_variant(monster, monster['name'], '1')
+        elif pennance_runner_match:
+            handle_base_variant(monster, monster['name'], 'Wave 1')
+        elif werewolf_match:
+            handle_base_variant(monster, monster['name'], 'Alexis')
+        elif melee_match:
+            handle_base_variant(monster, monster['name'], 'Melee, 1')
+        elif ranged_match:
+            handle_base_variant(monster, monster['name'], 'Ranged, 1')
+        elif task_only_match:
+            handle_base_variant(monster, monster['name'], '1')
+        elif eye_color_match:
+            handle_base_variant(monster, monster['name'], 'Blue eyes')
+        elif jelly_match:
+            handle_base_variant(monster, monster['name'], 'Regular')
+        elif farmer_match:
+            handle_base_variant(monster, monster['name'], 'Green shirt, long sleeves')
+        elif ruins_of_tapoyauik_match:
+            handle_base_variant(monster, monster['name'], 'Ruins of Tapoyauik, 1')
+        elif barrows_skeleton_match:
+            handle_base_variant(monster, monster['name'], 'Unarmed')
+        elif woman_match:
+            handle_base_variant(monster, monster['name'], 'Purple')
+        elif kalphite_guardian_match:
+            handle_base_variant(monster, monster['name'], '')
+
+    for k, v in sorted(data.items(), key=lambda item: item[1].base_name):
+        dataJs += '\n  %s: %s, // %s' % (k, v.alias_ids, v.base_name)
+
+    dataJs += '\n};\nexport default monsterAliases;\n'
+
+    run_tests(data)
+
+    with open(OUTPUT_FILE_NAME, 'w') as f:
+        print('Saving to Typescript at file: ' + OUTPUT_FILE_NAME)
+        f.write(dataJs)
+main()

src/app/components/monster/MonsterSelect.tsx

@@ -5,6 +5,7 @@ import { observer } from 'mobx-react-lite';
 import { Monster } from '@/types/Monster';
 import { CUSTOM_MONSTER_BASE } from '@/lib/Monsters';
 import { IconPencilPlus } from '@tabler/icons-react';
+import monsterAliases from '@/lib/MonsterAliases';
 import Combobox from '../generic/Combobox';
 
 interface MonsterOption {
@@ -44,9 +45,36 @@ const MonsterSelect: React.FC = observer(() => {
       resetAfterSelect
       blurAfterSelect
       customFilter={(items, iv) => {
-        if (!iv) return items;
-        // When searching, don't show the custom monster option in the results
-        return items.filter((i) => i.value !== -1);
+        const remainingVariantGroups: { [k: number]: number[] } = {};
+        const remainingVariantMemberships: { [k: number]: number } = {}; // reverse map
+
+        for (const monster of items) {
+          if (monster.value === -1) continue;
+          const mId = monster.monster.id;
+          if (mId === undefined) continue;
+          for (const [base, vars] of Object.entries(monsterAliases)) {
+            const baseId = parseInt(base);
+            if (baseId === mId || vars.includes(mId)) {
+              remainingVariantGroups[baseId] = remainingVariantGroups[baseId] ? [...remainingVariantGroups[baseId], mId] : [mId];
+              remainingVariantMemberships[mId] = baseId;
+            }
+          }
+        }
+        return items.filter((mOpt) => {
+          // If there is a search query do not show custom monster
+          if (mOpt.value === -1) return !iv;
+          const mId = mOpt.monster.id;
+          if (mId === undefined) return true;
+          const baseId: number | undefined = remainingVariantMemberships[mId];
+          if (baseId === mId) return true;
+          if (baseId !== undefined) {
+            const group = remainingVariantGroups[baseId];
+            if (group.includes(mId)) {
+              return group.indexOf(mId) === 0 && !items.find((o) => o.monster.id === baseId);
+            }
+          }
+          return true;
+        });
       }}
       onSelectedItemChange={(item) => {
         if (item) {

src/lib/MonsterAliases.ts

@@ -0,0 +1,42 @@
+/**
+ * A map of base NPC ID -> variant NPC IDs for NPCs that are identical in function.
+ */
+
+const monsterAliases = {
+  14113: [14114, 14115, 14116, 14117], // Armoured zombie (Zemouregal's Fort)
+  14118: [14119, 14120, 14121, 14122], // Armoured zombie (Zemouregal's Fort)
+  241: [242, 243], // Baby blue dragon
+  14105: [14106], // Baby blue dragon
+  5194: [5872, 5873], // Baby green dragon
+  244: [245, 246], // Baby red dragon
+  265: [266, 267, 268, 269, 5878, 5879, 5880, 5881, 5882], // Blue dragon
+  14103: [14104], // Blue dragon
+  5330: [5331, 5332, 5333, 5336, 5337, 5338, 5339], // Cave goblin miner
+  12918: [12919, 12920, 12921, 12922, 12923], // Cultist
+  13763: [13732, 13732, 13732, 13732], // Emissary Acolyte
+  13706: [13707, 13708, 13709, 13710, 13711, 13712, 13713], // Emissary Ascended
+  13738: [13739, 13740, 13741, 13742, 13743], // Emissary Chosen
+  13777: [13778], // Emissary Conjurer
+  11919: [3243, 3244, 11920, 11921, 11918, 3114], // Farmer
+  437: [441, 11245, 442, 438, 11242, 440, 11244, 11241, 439, 11243], // Jelly
+  1045: [1044, 1046, 1042, 1043], // Jungle horror
+  960: [962, 959], // Kalphite Guardian
+  13114: [13115, 13116, 13117, 13118, 13119], // Knight of Varlamore
+  3293: [11930, 11931, 11932, 11933], // Paladin
+  1669: [5749, 5750, 5751, 5752, 5753, 5754, 5755, 5756], // Penance Runner
+  247: [248, 249, 250, 251], // Red dragon
+  1685: [1688, 1687, 1686], // Skeleton (Barrows)
+  10717: [10718, 10719, 10720, 10721], // Skeleton (Forthos Ruin)
+  13476: [13477, 13478, 13479], // Skeleton (Lucien's camp)
+  3972: [3973, 3974], // Skeleton (Melzar's Maze)
+  2521: [2522, 2523, 2520, 2524, 2525, 2526], // Skeleton (Stronghold of Security)
+  13495: [13496, 13497, 13498, 13499, 13500, 13501], // Skeleton (Wilderness Agility Course)
+  13599: [13600, 13601], // Tormented Demon
+  2603: [2593, 2598, 2605, 2600, 2594, 2602, 2596, 2607, 2599, 2612, 2604, 2609, 2597, 2606, 2601, 2610, 2608, 2595, 2611], // Werewolf
+  3111: [3112, 3113, 11053, 3015], // Woman
+  64: [65, 66, 67, 68], // Zombie (Entrana Dungeon)
+  613: [614, 615, 616, 617, 618], // Zombie pirate (Braindeath Island)
+  563: [575, 576, 577, 578, 579, 580, 581, 582, 584, 586, 565, 588, 590, 592, 593, 595, 597, 599, 567, 568, 569, 571, 572, 573, 574], // Zombie pirate (Harmony Island)
+  619: [620, 621, 622, 623, 624], // Zombie swab
+};
+export default monsterAliases;