Support Superset catalog, currency, folders, and format metadata#207
Conversation
Extend the Apache Superset dataset adapter to import/export fields that
previously had no Sidemantic mapping, preserving them under meta['superset']
for roundtrip fidelity:
- Dataset: catalog (multi-catalog qualifier, mapped into catalog.schema.table),
currency_code_column, folders (column/metric folder organization).
- Column: advanced_data_type, python_date_format, datetime_format.
- Metric: currency ({symbol, symbolPosition}), d3format (also mapped to the
Sidemantic format field), warning_text.
Adds a multi_catalog_revenue fixture and a dedicated metadata test module, and
extends the orders fixture to exercise the new fields through roundtrip.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 595f0cc691
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
When a dataset has a catalog but null schema, the two-part catalog.table reference was re-emitted with the catalog copied into schema, producing cat.cat.table on the next parse. Treat a preserved catalog with a two-part reference as catalog.table and keep schema null.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 807db72296
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Real Superset exports nest currency_code_column under extra rather than top-level. Parse now reads both locations (preferring top-level) and export emits it under extra as well, so dynamic currency formatting survives a Superset -> Sidemantic -> Superset roundtrip.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 68ee626c65
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
1 participant
What changed
Extends the Apache Superset dataset adapter (
sidemantic/adapters/superset.py) to import and export several dataset/column/metric fields that previously had no Sidemantic mapping. Fields without a first-class Sidemantic equivalent are preserved undermeta['superset']so they survive a Superset -> Sidemantic -> Superset roundtrip.Upstream features now supported
Verified against the Apache Superset v1 dataset import/export schemas (
ImportV1DatasetSchema,ImportV1ColumnSchema,ImportV1MetricSchema).catalog- multi-catalog dataset qualifier. Folded into the model'scatalog.schema.tablereference and preserved in meta.currency_code_column- per-row currency code column for currency formatting.folders- column/metric folder organization (nested{uuid, type, name, description, children}).advanced_data_type,python_date_format,datetime_format.currency({symbol, symbolPosition}) andwarning_text.d3format- mapped to the Sidemanticformatfield, and also preserved raw for exact roundtrip.Tests / fixtures
tests/fixtures/superset/multi_catalog_revenue.yamlexercising the 3-partcatalog.schema.tablequalifier plus all new metadata.tests/adapters/superset/test_metadata.pycovering parse, export, and roundtrip for every new field.tests/fixtures/superset/orders.yamlso the existing roundtrip tests also cover the new fields.Backward compatibility
Existing behavior is unchanged: 2-part
schema.tablereferences, virtual datasets, metric/column type mapping, and all prior fields keep working. Datasets without the new keys produce no extra metadata.Known limitations
currency,warning_text, and the dataset-levelcatalog/currency_code_column/foldersare pass-through metadata (preserved for roundtrip), not interpreted by Sidemantic's query engine.foldersreference columns/metrics by name/uuid as authored upstream; the adapter does not validate that referenced members still exist.