Skip to content

Stream map expressions: datetime module not accessible for attribute access (in functions dict, not names) #3520

Description

@abrostoff

Bug Description

The datetime module is added to the simpleeval functions dict in CustomStreamMap.functions (mapper.py line 322), but it is not added to the names dict in _eval(). This means expressions like datetime.datetime.utcnow().isoformat() fail with:

simpleeval.AttributeDoesNotExist: Attribute 'datetime' does not exist in expression 'datetime.datetime.utcnow().isoformat()'

In simpleeval, entries in functions are only invocable as name(args) — they do not support attribute/dot access. Attribute access requires the entry to be in the names dict.

Expected Behavior

As documented in the Inline Stream Maps docs, the datetime module should be usable in expressions with attribute access:

stream_maps:
  mystream:
    loaded_at: "datetime.datetime.utcnow().isoformat()"
    create_year: "int(datetime.date.fromisoformat(create_date).year)"

The SDK's own type detection logic in _get_type() also explicitly expects these patterns to work:

# mapper.py lines 419-425
if expr.startswith(("datetime.date.", "datetime.date(")) or expr.endswith(".date()"):
    return th.DateType()

if expr.startswith(("datetime.datetime.", "datetime.datetime(")):
    return th.DateTimeType()

Suggested Fix

Add datetime to the names dict in _eval(), alongside record, config, etc.:

names["datetime"] = datetime

(And similarly for json if it has the same issue.)

Environment

  • singer-sdk version: 0.53.5 (latest)
  • Python version: 3.12

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions