ICU4X Datagen Troubleshooting - What data or markers am I missing?

[final-star-1]

I am using the npm package that lets me use ICU4X in TypeScript. Currently I am using icu4x-datagen to create my own provider. I am using version 2.1.0 for both the npm package and the datagen Rust crate.

This is what I run for the datagen script:

icu4x-datagen \
    --cldr-root ./cldr-48.0.0-json-full \
    --tzdb-root ./tzdata2025b \
    --locales full \
    --markers \
        DecimalSymbolsV1 \
        DecimalDigitsV1 \
        LocaleParentsV1 \
        LocaleLikelySubtagsLanguageV1 \
        LocaleLikelySubtagsScriptRegionV1 \
        LocaleLikelySubtagsExtendedV1 \
        LocaleScriptDirectionV1 \
        LocaleExemplarCharactersAuxiliaryV1 \
        LocaleExemplarCharactersIndexV1 \
        LocaleExemplarCharactersMainV1 \
        LocaleExemplarCharactersNumbersV1 \
        LocaleExemplarCharactersPunctuationV1 \
    --format blob \
    --overwrite \
    --out ./cldr_48_tzdb_2025b.postcard

This worked fine for the bare number formatting I was trying to do in my application, but specifically when I started using myLocale.setUnicodeExtension('nu', 'latn') I started to get missing data or identifiers:

ICU4X data error: Missing data for identifier (marker: DataMarkerId { hash: [116, 100, 109, 104, 49, 67, 69, 132] }, request: en-US/latn)

It is very difficult for me to follow the docs to know what markers or data I am missing... (maybe I just don't know rust enough?) But any hint at all would be greatly appreciated.

[Manishearth]

I bet this is one of the locale fallback markers. Let me have a look and figure out which one.

Can you share which ICU4X APIs you are using?

[Manishearth]

Oh, f9790c9#diff-bdc2297ee070f11fe31a711a8b75fb5c3fe47e005757059d0b949d99ae7d3206 also touched this code recently. Maybe this is a bug that is already fixed.

[final-star-1]

Here is my blob. I couldn't upload the blob directly on GitHub (they only accept certain file types) so I zipped it. Hope that doesn't cause issues.

cldr_48_tzdb_2025b.blob.zip

[Manishearth]

Hmm. Using the blob you have given, on the ICU4X 2.1 branch, and the Rust code I used above, does not throw any errors.

[final-star-1]

I am currently using icu version 2.1.0 from the npm repository. Are there perhaps any fixes on that branch that the npm package does not have?

[Manishearth]

I figured it out. It was not clear to me that your only problem was that you were seeing the warning, not any actual error.

[Manishearth]

Okay, I think a thing I didn't realize is that you're not actually getting an error, you're just seeing the error in the logs. The code is still succeeding, just printing an (incorrect) error along the way.

This was fixed in f9790c9; previously the code did not use allow_identifier_not_found.

This can be backported if you really need, but we usually don't consider printed warnings to be a huge deal.

[Manishearth]

Yeah. These warnings are usually disabled in most backends, except JS. I think the Logger type lets you disable them in prod if you want.

We'll be doing a 2.2 near the end of March which should have the fix you need.

[final-star-1]

I don't see much that I can do with the Logger API in TypeScript: https://icu4x.unicode.org/2_1/tsdoc/classes/Logger.html

Am I looking at the wrong place?

[Manishearth]

Ah, maybe we don't expose anything there. Never mind.

[sffc]

The logging is wired up here

https://github.com/unicode-org/icu4x/blob/main/ffi/npm/lib/diplomat-wasm.mjs

I'm not sure if it is configurable. If you are able to edit the file, you can remove the console logs.

[Manishearth]

We could in theory expose a way to reinitialize the logger with a different level in our logging interface