Skip to content

feat: run entity detection fully inside the browser #1268

Description

@tobocop2

BLOCKED until huggingface/candle#3751 and safetensors/safetensors#818 merge and release. The solution is ready in those PRs; no draft PR here until they ship.

What is the proposed feature?

The pure-Rust entity detection landing in #1263 runs natively today. This issue is about running it fully inside the browser: the model loads into the page and detects entities locally, with no server round-trip. The plumbing exists and was verified in a real browser; what blocks it is two bug fixes in third-party libraries.

Why it was deferred

#1263 ships the native path only. No configuration of the wasm feature could compile: on a host target it reached tokio::task::block_in_place without the tokio-runtime feature, and on wasm32 it needs the NerBackend ?Send trait change.

What landing it takes

  1. The NerBackend ?Send-on-wasm32 split, landing with feat(xberg-wasm): OCR results include each line's coordinates, and JS apps can plug in their own OCR and NER #1259.
  2. A ner-candle-wasm feature (["ner", "dep:xberg-gliner-candle"], no tokio) whose detect path calls inference directly instead of through block_in_place, with a wasm32 compile check in CI.
  3. The two upstream dependency fixes: Fix DeBERTa-v2 models failing to run at F16 huggingface/candle#3751 (fixes candle#3750) and fix: large tensors failing to load on 32-bit targets safetensors/safetensors#818 (fixes safetensors#817).

The upstream blockers, briefly

candle#3750 (fix up as candle#3751): debertav2 hardcodes F32 scalars, so F16 forward fails, and F16 is what makes a DeBERTa-v2-scale model fit wasm32 linear memory. safetensors#817 (fix up as safetensors#818): metadata validation spuriously overflows on 32-bit targets once a tensor passes 2^32 bits. Until both merge and release, the wasm path only works against patched builds, which we do not ship.

Already verified end to end

@jamon8888 ran the full path in a real browser in #1253: model load through the streaming F32-to-F16 loader, correct entity detection through CandleBackend::from_bytes.

Attribution

Designed and implemented by @jamon8888 in #1253. Filed separately so this work survives #1262 closing when #1263 merges.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions