Describe the feature you'd like
Most scrapers are URL scrapers, and the way they work is by matching whatever URL is in the field for any scene/performer/gallery/image to a pattern declared in the scraper.
The problem arises when two or more scrapers can scrape the same pattern, a situation that can occur when scrapers are consolidated into larger scrapers that can cover multiple domains because they use the same CMS or belong to the same network. A recent example would be the new MojoHost scraper introduced in CommunityScrapers 3bdeead that moved patterns from over a dozen scrapers into one big scraper which will be easier for us maintainers to keep updated.
Users who have the old FFCSH scraper installed can install the new MojoHost scraper and suddenly their scraping of fantasyflipflop.com becomes non-deterministic because sometimes Stash will use the new MojoHost scraper and sometimes it will use the old FFCSH scraper. Scraper definitions are stored in maps, and the iteration order of Golang maps is not guaranteed (and is indeed random), which means the scraper used can and will vary.
Describe the benefits this would bring to existing users
This is incredibly confusing for users so I think we should warn them when two or more of their scrapers share URL patterns. I think most common cause of this problem is the aforementioned updating/consolidation of scrapers so we should suggest to users that the most probable correct resolution for this problem is to uninstall the older package, but not to do it automatically.
Is there an existing way to achieve this goal?
A plugin could in theory iterate over all scrapers and their patterns and run this check in the browser, but in my opinion this should be in the core of Stash because it's not their fault that I love tinkering with the CommunityScrapers repo.
Have you searched for an existing open/closed issue?
Additional context
No response
Describe the feature you'd like
Most scrapers are URL scrapers, and the way they work is by matching whatever URL is in the field for any scene/performer/gallery/image to a pattern declared in the scraper.
The problem arises when two or more scrapers can scrape the same pattern, a situation that can occur when scrapers are consolidated into larger scrapers that can cover multiple domains because they use the same CMS or belong to the same network. A recent example would be the new MojoHost scraper introduced in CommunityScrapers 3bdeead that moved patterns from over a dozen scrapers into one big scraper which will be easier for us maintainers to keep updated.
Users who have the old FFCSH scraper installed can install the new MojoHost scraper and suddenly their scraping of fantasyflipflop.com becomes non-deterministic because sometimes Stash will use the new MojoHost scraper and sometimes it will use the old FFCSH scraper. Scraper definitions are stored in maps, and the iteration order of Golang maps is not guaranteed (and is indeed random), which means the scraper used can and will vary.
Describe the benefits this would bring to existing users
This is incredibly confusing for users so I think we should warn them when two or more of their scrapers share URL patterns. I think most common cause of this problem is the aforementioned updating/consolidation of scrapers so we should suggest to users that the most probable correct resolution for this problem is to uninstall the older package, but not to do it automatically.
Is there an existing way to achieve this goal?
A plugin could in theory iterate over all scrapers and their patterns and run this check in the browser, but in my opinion this should be in the core of Stash because it's not their fault that I love tinkering with the CommunityScrapers repo.
Have you searched for an existing open/closed issue?
Additional context
No response