Skip to content

fix(provider): accept fractional-second created_at in model manifests - #676

Open
owizdom wants to merge 1 commit into
Layr-Labs:masterfrom
owizdom:fix/manifest-fractional-seconds
Open

fix(provider): accept fractional-second created_at in model manifests#676
owizdom wants to merge 1 commit into
Layr-Labs:masterfrom
owizdom:fix/manifest-fractional-seconds

Conversation

@owizdom

@owizdom owizdom commented Aug 23, 2026

Copy link
Copy Markdown

Summary

ModelCatalogClient.manifestDecoder decoded created_at with JSONDecoder's .iso8601 strategy, which resolves against whatever Foundation the host ships. On macOS 15.x that rejects fractional seconds, and the coordinator marshals created_at from a Go time.Time, so every live manifest carries them. Result: every model download failed at the manifest hop before a byte transferred. This parses the wire format explicitly instead, accepting both whole-second and fractional-second RFC 3339.

Linked issue

Closes #673

Before / after

Behavior:

flowchart TB
  subgraph Before["Before: every download aborts at the manifest hop"]
    direction TB
    A1["darkbloom start, pick a model"] --> B1["GET /v1/models/catalog"]
    B1 -->|"int epoch, decodes"| C1["GET /v1/models/catalog/manifest/:id"]
    C1 --> D1{"JSONDecoder .iso8601 meets<br/>created_at 2026-05-25T22:46:27.580497Z"}
    D1 -->|"macOS 15.x"| E1["reject: could not decode catalog response<br/>download aborts, 0 bytes transferred"]
    D1 -->|"macOS 26.x"| F1["accept: download proceeds"]
  end
  subgraph After["After: the parse no longer depends on the host"]
    direction TB
    A2["darkbloom start, pick a model"] --> C2["GET /v1/models/catalog/manifest/:id"]
    C2 --> D2{"explicit RFC 3339 parse"}
    D2 -->|"fractional or whole seconds"| F2["download proceeds on every macOS"]
    D2 -->|"anything else"| E2["DecodingError naming the value"]
  end
  Before ~~~ After
Loading

Code:

flowchart TB
  subgraph CodeBefore["Before: ModelCatalogClient.manifestDecoder"]
    direction TB
    X1["dateDecodingStrategy = .iso8601"] --> Y1["host Foundation decides which shapes parse"]
    Y1 --> Z1["macOS 15.x: whole seconds only"]
    Y1 --> Z2["macOS 26.x: whole + fractional"]
  end
  subgraph CodeAfter["After: ModelCatalogClient.manifestDecoder"]
    direction TB
    X2["dateDecodingStrategy = .custom"] --> P1["ISO8601FormatStyle(includingFractionalSeconds: true)"]
    P1 -->|"no match"| P2["ISO8601FormatStyle(includingFractionalSeconds: false)"]
    P1 -->|"match"| OK["Date"]
    P2 -->|"match"| OK
    P2 -->|"no match"| P3["DecodingError.dataCorrupted"]
  end
  CodeBefore ~~~ CodeAfter
Loading

All four call sites of the shared decoder pick this up: fetchManifest (coordinator manifest), ModelDownloader+Download.swift:77 (CDN manifest.json), SpecDecStore.swift:86 and SpecDecResolver.swift:346 (drafter manifests).

Test plan

  • swift test --filter ModelCatalogTests24 tests in the suite pass, 0 failures (6.271s). That is the 20 already on master plus the 4 added here.
  • swift build --target ProviderCore builds clean (147.7s), no warnings or errors in the changed file.
  • Compiles under Swift 6 language mode with strict concurrency; Date.ISO8601FormatStyle satisfies Sendable.

How the test run was obtained, since it is not the plain command: this machine has Command Line Tools without full Xcode, so swift test needs the swift-testing framework pointed at explicitly, and two test targets that are built entirely on XCTest (ProviderCoreFoundationTests, DarkbloomPublishTests) plus one XCTest file in ProviderCoreTests (JinjaSanitizationTests.swift) cannot link at all. Those were excluded locally and reverted afterwards; none is touched by this PR, and the suite under test ran unmodified:

swift test --filter ModelCatalogTests \
  -Xswiftc -F -Xswiftc /Library/Developer/CommandLineTools/Library/Developer/Frameworks \
  -Xlinker -F -Xlinker /Library/Developer/CommandLineTools/Library/Developer/Frameworks \
  -Xlinker -rpath -Xlinker /Library/Developer/CommandLineTools/Library/Developer/Frameworks \
  -Xlinker -rpath -Xlinker /Library/Developer/CommandLineTools/Library/Developer/usr/lib

Four new tests in ModelCatalogTests.swift:

  • fetchManifestDecodesFractionalSeconds walks the real HTTP path via RegistryURLProtocol, using the literal created_at production serves for gpt-oss-20b
  • manifestDecoderAcceptsEveryRFC3339NanoShape covers whole seconds and 1-9 fractional digits, both Z and +00:00
  • manifestDecoderPreservesSubMillisecondPrecision pins the microseconds
  • manifestDecoderRejectsNonRFC3339CreatedAt keeps the decoder from getting permissive

Confirmed against the live coordinator that all five manifests carry microsecond fractions (gpt-oss-20b at .580497Z, qwen3.6-35b-a3b-vl-mtp-mxfp8 at .184276Z, the three gemma-4-26b variants likewise), so on an affected host this is a total download outage rather than a flake.

Worth stating plainly: the regression test fails without the fix on macOS 15.4.1 and passes without it on macOS 26.2, because the pre-fix decoder already accepts both shapes there. That split is the defect. CI runs macos-latest, so it could not have caught this, and it will not fail on this PR's absence either.

Components touched

  • coordinator (Go)
  • provider (Rust, legacy)
  • provider-swift (Swift CLI)
  • console-ui (Next.js)
  • enclave (Swift)
  • infra / CI / release
  • docs

Protocol / interface changes

  • No protocol/interface changes

The wire format is untouched. This makes the client accept what the coordinator already emits.

Notes for reviewers

  • Why not ISO8601DateFormatter. [bug] Every model download fails to decode the manifest: coordinator emits fractional-second timestamps, client uses .iso8601 which rejects them (0.8.10) #673 suggests [.withInternetDateTime, .withFractionalSeconds], but that combination rejects input with no fraction, so it would swap a constant failure for an intermittent one. Go's RFC3339Nano strips trailing zeros, and darkbloom-publish writes CDN manifest.json through JSONEncoder.iso8601, which always truncates to whole seconds. Both shapes are live, so the decoder has to take both. Date.ISO8601FormatStyle is also a Sendable value type, which matters for a static decoder shared across concurrent downloads, and it keeps microseconds the formatter truncates to milliseconds (1779749187.580497 vs 1779749187.58).
  • Why the existing test missed it. fetchManifestDecodesRegistryRoute builds its fixture with JSONEncoder.iso8601 and Date(timeIntervalSince1970: 0), so it round-trips through the same library and only ever exercises the one shape that already worked. The new tests use literal wire bytes instead.
  • Happy to target a milestone if you tell me which release this should land in.

JSONDecoder's .iso8601 strategy resolves against whichever Foundation the
host ships, so what it accepts is a property of the provider's macOS version
rather than of the wire format. The coordinator marshals a manifest's
created_at from a Go time.Time, which RFC3339Nano-encodes with 1-9 fractional
digits; on macOS 15.x that fraction is rejected and every model download
aborted at the manifest hop before transferring a byte. All five live
manifests carry one.

Parse the two shapes that are actually on the wire instead. The fraction is
not always present: Go strips trailing zeros, and darkbloom-publish writes
the CDN manifest.json through JSONEncoder.iso8601, which always truncates to
whole seconds. Pinning the decoder to .withFractionalSeconds would therefore
trade a constant failure for an intermittent one.

Uses Date.ISO8601FormatStyle rather than ISO8601DateFormatter: it is a
Sendable value type, which the shared static decoder needs, and it keeps the
microseconds the formatter truncates to milliseconds.

Closes Layr-Labs#673
@vercel

vercel Bot commented Aug 23, 2026

Copy link
Copy Markdown

@owizdom is attempting to deploy a commit to the EigenLabs Team on Vercel.

A member of the Team first needs to authorize it.

@owizdom

owizdom commented Aug 23, 2026

Copy link
Copy Markdown
Author

Closing to keep my open PRs down to one. Leaving the finding here so #673 isn't lost:

JSONDecoder.dateDecodingStrategy = .iso8601 resolves against the HOST Foundation, so what it accepts depends on the machine's macOS version, not the wire format — 2026-05-25T22:46:27.580497Z is rejected on macOS 15.4.1 and accepted on 26.2.

The obvious fix regresses the other direction: ISO8601DateFormatter with [.withInternetDateTime, .withFractionalSeconds] rejects whole-second input. Both shapes are genuinely live — Go's RFC3339Nano strips trailing zeros, and darkbloom-publish writes CDN manifest.json via JSONEncoder.iso8601 (always whole seconds). Date.ISO8601FormatStyle handles both and keeps microseconds that ISO8601DateFormatter truncates to ms.

Happy to reopen if it's wanted.

@owizdom owizdom closed this Aug 23, 2026
@owizdom

owizdom commented Aug 23, 2026

Copy link
Copy Markdown
Author

Reopening — issue #673 is still open and this fixes it. Sorry for the churn.

@owizdom owizdom reopened this Aug 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant