You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Urdu has ~51M speakers in India (2011 census; far more worldwide) and is an official language in several Indian states, but OpenMed has no ur pack. Urdu is right-to-left in Perso-Arabic script — the Arabic block plus Urdu-specific letters ٹ ڈ ڑ ں ے ھ and the extended Arabic-Indic digits ۰-۹ (U+06F0-06F9, distinct from Arabic's ٠-٩) — so Aadhaar, phone, and PIN digit runs are LTR islands inside RTL text and naive spans corrupt bidi ordering; OpenMed already ships an Arabic (ar) pack whose RTL and digit-normalization learnings must be reused rather than reinvented.
Scope
Register 'ur' in SUPPORTED_LANGUAGES, LANGUAGE_NAMES, LANGUAGE_MODEL_PREFIX, and LANGUAGE_MONTH_NAMES (جنوری..دسمبر) in openmed/core/pii_i18n.py
Add _URDU_PII_PATTERNS to LANGUAGE_PII_PATTERNS following the existing _ARABIC_PII_PATTERNS approach: dates with Urdu month names, +91 mobiles, PIN codes with پن کوڈ context (Urdu-belt states: J&K 18xxxx-19xxxx, Telangana 50xxxx, UP 2xxxxx), Aadhaar 4-4-4 reusing validate_aadhaar, and name patterns with honorifics جناب / محترمہ / سید / بیگم / خان and the صاحب/صاحبہ suffix as context
Normalize both extended Arabic-Indic (۰-۹) and Arabic-Indic (٠-٩) digits to ASCII before numeric validators, and route ur-vs-ar using Urdu-specific letters (ے ٹ ڈ ڑ ں ھ) so Urdu text never falls into the ar pack
Guarantee bidi-safe redaction: spans over LTR digit islands inside RTL lines must not split bidi control characters or reorder logical text, reusing and extending the ar pack's RTL span tests
Add DEFAULT_PII_MODELS['ur'] reusing the multilingual privacy-filter model and register the pack via the plugin framework's one-call onboarding; wire LANG_TO_LOCALE['ur'] as a documented approximation (no Urdu Faker locale in Faker 40.x — verified) with an en_IN backend plus a small bundled Urdu name provider (given names + خان/سید/بیگم elements) in openmed/core/anonymizer/locales.py; NATIONAL_ID_PROVIDERS['ur'] = aadhaar
Author synthetic golden fixture openmed/eval/golden/fixtures/i18n/ur.jsonl (RTL notes with embedded LTR Aadhaar/phone runs, both digit systems, honorific-suffixed names, Urdu-English code-mixed lines) and extend the leakage gate and unit tests
Acceptance criteria
Residual leakage = 0 on the synthetic ur golden fixtures through the release-gate leakage check, including lines using extended Arabic-Indic digits
Offsets are grapheme-safe and bidi-safe: no combining mark, ligature, or bidi control is split, and redacted RTL lines round-trip with logical order preserved
Urdu fixtures route to the ur pack, never the ar pack, in routing tests keyed on Urdu-specific letters
Aadhaar, phone, and PIN values render detection-identical across ASCII, Arabic-Indic, and extended Arabic-Indic digits
tests/unit/core/test_locale_coherence.py passes with ur listed as a documented approximate locale emitting exactly one UserWarning
Out of scope
Pakistani document formats (CNIC) — Indian documents only in this pack
Nastaliq font rendering or PDF layout concerns (text-level only)
Training an Urdu-specific NER model
Devanagari-written Hindustani (that is the hi pack); real clinical text — synthetic fixtures only
Files
openmed/core/pii_i18n.py
openmed/core/anonymizer/locales.py
openmed/eval/golden/fixtures/i18n/ur.jsonl
openmed/eval/release_gates.py
tests/unit/core/test_locale_coherence.py
tests/unit/eval/test_golden_fixtures.py
Task: OM-694 · Milestone: v1.9 · Priority: P1 · Size: M
Depends on: OM-678 · Blocks: —
Roadmap: Chinese & Indian languages initiative
Spec: PLANS/V2/EXECUTION/tasks/OM-694.md
Summary
Urdu has ~51M speakers in India (2011 census; far more worldwide) and is an official language in several Indian states, but OpenMed has no ur pack. Urdu is right-to-left in Perso-Arabic script — the Arabic block plus Urdu-specific letters ٹ ڈ ڑ ں ے ھ and the extended Arabic-Indic digits ۰-۹ (U+06F0-06F9, distinct from Arabic's ٠-٩) — so Aadhaar, phone, and PIN digit runs are LTR islands inside RTL text and naive spans corrupt bidi ordering; OpenMed already ships an Arabic (ar) pack whose RTL and digit-normalization learnings must be reused rather than reinvented.
Scope
Acceptance criteria
Out of scope
Files
Task: OM-694 · Milestone: v1.9 · Priority: P1 · Size: M
Depends on: OM-678 · Blocks: —
Roadmap: Chinese & Indian languages initiative
Spec: PLANS/V2/EXECUTION/tasks/OM-694.md