Skip to content

Wrong byte order for replacement character in UTF-16 BE encoding_scheme #52

Description

@captainurist

When encoding to UTF-16 BE using basic_utf16_be<char> and an error occurs (requiring the replacement character U+FFFD), the replacement is output in little-endian byte order instead of big-endian.

Repro:

#include <ztd/text.hpp>
#include <span>
#include <iostream>
#include <iomanip>

int main() {
    // Truncated UTF-8 sequence (incomplete 3-byte sequence for '中')
    std::string input = "\xe4\xb8";  // Missing third byte
    std::span<const char> input_span(input.data(), input.size());

    // Transcode to UTF-16 BE - the truncated input causes a decode error,
    // which should insert replacement character U+FFFD
    auto result = ztd::text::transcode(input_span,
                                       ztd::text::compat_utf8,
                                       ztd::text::basic_utf16_be<char>{},
                                       ztd::text::replacement_handler);

    std::cout << "Result bytes: ";
    for (unsigned char b : result) {
        std::cout << std::hex << std::setfill('0') << std::setw(2) << (int)b << " ";
    }
    std::cout << std::endl;

    // Expected (U+FFFD in big-endian): ff fd
    // Actual (before fix):             fd ff  <-- WRONG! Native little-endian byte order

    return 0;
}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions