Better configuration of browsers

[dgtlmoon]

@opicron this is a huge PR... its going to take quite a few weeks I think, it might even get rolled into a new major major release instead like 1.0.1 along with #3996

Upgrade Path Testing Checklist

1. Migration update_31 — the core risk

• Fresh install (no existing datastore): verify browser_profile='system' is the default for new watches, tags have no browser_profile set.
• Old datastore with fetch_backend='html_requests' on watches/tags: should migrate to browser_profile='direct_http_requests', fetch_backend='system'.
• Old datastore with fetch_backend='html_webdriver' on watches: should migrate to browser_profile='browser_chromeplaywright'.
• Old datastore with fetch_backend='extra_browser_' on watches: extra_browsers entries should become named profiles in browser_profiles, and the watch should point to the new machine name.
• System-wide settings.application.fetch_backend: should map to settings.application.browser_profile correctly.
• Tags with overrides_watch=True and explicit fetch_backend: should have browser_profile set and fetch_backend reset to 'system'.
• Re-run safety: running update_31 twice should be a no-op (all paths check for existing browser_profile).
• fetch_backend='system'/missing on watch: should produce None machine name (stay on chain resolution).

2. Content fetcher registry — name changes

• Plugin fetchers: any external plugin returning ('html_cloakbrowser', cls) from register_content_fetcher will not be found by the new get_fetcher() calls. Only ('cloakbrowser', cls) works now.
• CloakBrowser plugin specifically (changedetection.io-cloak-browser): verify its register_content_fetcher returns 'cloakbrowser' (already fixed in this branch) and it appears correctly in the browser profile list.
• Custom third-party plugins using html_ prefix in their hook return: they will silently disappear from the fetcher registry. Need a compatibility warning or broader fallback.
• available_fetchers() list: confirm all four builtins (requests, playwright, selenium, puppeteer) appear correctly in the browser profile dropdown on /settings/browsers.
• get_fetcher('requests') called from base.py fallback path: verify fallback to requests still works when an unknown fetcher name is specified.

3. browser_profile field on Watch — None vs 'system'

• Watch created before migration with browser_profile=None: after update_31, browser_profile should be a machine name string or None (meaning system). The effective_browser_profile property must not crash.
• Watch created after migration: default is 'system' sentinel string — verify resolve_browser_profile handles this correctly and falls through to the global setting.
• effective_browser_profile property: requires _datastore to be a full ChangeDetectionStore (not datastore.data). Any place that creates a Watch without the full store will get a fallback — verify it falls back gracefully.
• Edit form browser_profile radio: None → 'system' mapping in edit.py must correctly pre-select the right radio for existing watches.

4. default_ua settings — key rename (html_requests → requests)

• Settings page POST with old form data: verify requests-default_ua-html_requests from a cached/bookmarked form submission doesn't silently corrupt the UA setting (old key is simply ignored by WTForms — UA reverts to default).
• Reading existing datastore settings.requests.default_ua: the key stored on disk may still be html_requests. Verify the settings page reads and saves back with the new key requests without losing the value. Check store/init.py for any key migration.
• Playwright/Selenium/Puppeteer UA fields: only rendered when PLAYWRIGHT_DRIVER_URL or WEBDRIVER_URL is set — verify the conditional in DefaultUAInputForm still works in env without those vars.

5. data-visible-for JS visibility (edit page)

• Webdriver delay / JS execute fieldset: now visible for fetch_backend=playwright|selenium|puppeteer|cloakbrowser. Test with each browser type selected — must appear. Must not appear for requests.
• Request body/method fieldset: now fetch_backend=requests only. Test it shows for requests profiles, hides for browser profiles.
• ignore_status_codes fieldset: must show for all fetcher types. Test with several selections.
• System profile selection: browserProfileFetcherMap['system'] falls back to default_system_fetch_backend — test on a clean install (no PLAYWRIGHT_DRIVER_URL) that selecting "system" shows the requests fieldset.

6. API backwards compatibility

• GET /api/v1/watch/: fetch_backend field is now stripped from response. Any API client reading watch.fetch_backend will get null/undefined. Check API clients / integrations.
• PUT /api/v1/watch/ with fetch_backend in body: should be silently ignored (WTForms Optional). Verify it doesn't 400.
• GET /api/v1/tags: fetch_backend also stripped from tag objects.
• Import via XLSX/CSV: browser_profile defaults to 'system' for imported watches (test already updated).

7. Browser profiles UI (/settings/browsers)

• Edit button → /<machine_name>/edit GET route: verify it correctly pre-populates the form for built-in profiles as well as custom ones.
• Saving a custom profile: verify the machine name is stable across edits (name → slugified machine name must match the key in browser_profiles dict).
• Deleting a custom profile: watches pointing to that machine name will have a dangling browser_profile — verify effective_browser_profile falls back gracefully (should log warning, fall to system default).
• Icons in the profile table: fetcher_status_icons filter on profile.get_fetcher_class_name() — verify it resolves correctly for all four builtins and for CloakBrowser.

8. Fetcher capabilities / fetcher_supports_screenshots

• Watch overview icon: CloakBrowser icon must appear when a watch uses a CloakBrowser profile. Verify effective_browser_profile resolves correctly from the full datastore.
• Diff/preview {% if fetcher_supports_screenshots %}: screenshot tab should appear for playwright/selenium/puppeteer profiles, not for requests.
• base.py browser_steps override: if a watch has browser steps and the fetcher doesn't have supports_browser_steps, it should override to playwright. Verify this with selenium/requests profiles.

[opicron]

This is very exciting, please take your time.

[matbgn]

🤯🤯🤯 Merged?!! We do not have the same conception of weekSSS!

...again some magic happened