Skip to content

Implement utf8 standard library #253

Description

@davydog187

Problem

The utf8 standard library is not implemented. utf8.lua from the official suite is deferred entirely on local utf8 = require'utf8' (yields true from our require cache rather than the library table).

Scope

Lua 5.3 §6.5 — six surface functions plus one field:

  • utf8.char(...) — codepoints → UTF-8 string
  • utf8.codepoint(s [, i [, j]]) — UTF-8 string → codepoints
  • utf8.codes(s) — stateless iterator over (byte_pos, codepoint) pairs
  • utf8.len(s [, i [, j]]) — number of codepoints (or nil, byte_pos on invalid)
  • utf8.offset(s, n [, i]) — byte position of the n-th codepoint
  • utf8.charpattern — string constant: pattern matching one valid UTF-8 byte sequence

All operations are over bytes; Elixir's String.next_codepoint/1 handles validation for us.

Suggested approach

  1. New module lib/lua/vm/stdlib/utf8.ex implementing Lua.VM.Stdlib.Library. Follow the layout of lib/lua/vm/stdlib/math.ex.
  2. Register via install_library/2 in lib/lua/vm/stdlib.ex so require"utf8" resolves it from package.loaded.
  3. New test file test/lua/vm/utf8_test.exs covering the well-defined cases plus invalid-sequence handling.

Acceptance

  • mix test passes.
  • utf8.lua either passes outright or progresses to a well-defined later failure that we can triage separately. Remove its :all entry from test/lua53_skips.exs.

Metadata

Metadata

Assignees

No one assigned

    Labels

    direction:ADirection A: suite triage to 0.5.0kind:suiteLua 5.3 official test suite work

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions