You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The pure-Rust entity detection landing in #1263 runs natively today. This issue is about running it fully inside the browser: the model loads into the page and detects entities locally, with no server round-trip. The plumbing exists and was verified in a real browser; what blocks it is two bug fixes in third-party libraries.
Why it was deferred
#1263 ships the native path only. No configuration of the wasm feature could compile: on a host target it reached tokio::task::block_in_place without the tokio-runtime feature, and on wasm32 it needs the NerBackend?Send trait change.
A ner-candle-wasm feature (["ner", "dep:xberg-gliner-candle"], no tokio) whose detect path calls inference directly instead of through block_in_place, with a wasm32 compile check in CI.
candle#3750 (fix up as candle#3751): debertav2 hardcodes F32 scalars, so F16 forward fails, and F16 is what makes a DeBERTa-v2-scale model fit wasm32 linear memory. safetensors#817 (fix up as safetensors#818): metadata validation spuriously overflows on 32-bit targets once a tensor passes 2^32 bits. Until both merge and release, the wasm path only works against patched builds, which we do not ship.
Already verified end to end
@jamon8888 ran the full path in a real browser in #1253: model load through the streaming F32-to-F16 loader, correct entity detection through CandleBackend::from_bytes.
Attribution
Designed and implemented by @jamon8888 in #1253. Filed separately so this work survives #1262 closing when #1263 merges.
BLOCKED until huggingface/candle#3751 and safetensors/safetensors#818 merge and release. The solution is ready in those PRs; no draft PR here until they ship.
What is the proposed feature?
The pure-Rust entity detection landing in #1263 runs natively today. This issue is about running it fully inside the browser: the model loads into the page and detects entities locally, with no server round-trip. The plumbing exists and was verified in a real browser; what blocks it is two bug fixes in third-party libraries.
Why it was deferred
#1263 ships the native path only. No configuration of the wasm feature could compile: on a host target it reached
tokio::task::block_in_placewithout thetokio-runtimefeature, and on wasm32 it needs theNerBackend?Sendtrait change.What landing it takes
NerBackend?Send-on-wasm32 split, landing with feat(xberg-wasm): OCR results include each line's coordinates, and JS apps can plug in their own OCR and NER #1259.ner-candle-wasmfeature (["ner", "dep:xberg-gliner-candle"], no tokio) whosedetectpath calls inference directly instead of throughblock_in_place, with a wasm32 compile check in CI.The upstream blockers, briefly
candle#3750 (fix up as candle#3751):
debertav2hardcodes F32 scalars, so F16 forward fails, and F16 is what makes a DeBERTa-v2-scale model fit wasm32 linear memory. safetensors#817 (fix up as safetensors#818): metadata validation spuriously overflows on 32-bit targets once a tensor passes 2^32 bits. Until both merge and release, the wasm path only works against patched builds, which we do not ship.Already verified end to end
@jamon8888 ran the full path in a real browser in #1253: model load through the streaming F32-to-F16 loader, correct entity detection through
CandleBackend::from_bytes.Attribution
Designed and implemented by @jamon8888 in #1253. Filed separately so this work survives #1262 closing when #1263 merges.